Documentation - Revision history

Antonio De Jesus Menchaca Martinez: /* Services */

2025-10-07T11:27:54Z

Services

← Older revision		Revision as of 11:27, 7 October 2025
Line 424:		Line 424:
	LETSENCRYPT_HOST: ${DOMAIN_NAME}		LETSENCRYPT_HOST: ${DOMAIN_NAME}
	LETSENCRYPT_EMAIL: ${LETSENCRYPT_EMAIL}		LETSENCRYPT_EMAIL: ${LETSENCRYPT_EMAIL}
	~~AWS_ACCESS_KEY_ID: zcby8I0PeG1uprpYO4KR~~
	~~AWS_SECRET_ACCESS_KEY: xyaCmOf86QWiyGM3L5BfKFv5WQxS70pjKKAbqQIN~~
	~~AWS_REGION: us-east-1~~
	~~AWS_ENDPOINT: http://minio:9000~~
	~~AWS_PATH_STYLE: <q>true</q>~~
	networks:		networks:
	- observatory_net		- observatory_net

Antonio De Jesus Menchaca Martinez: /* System Architecture */

2025-06-13T09:28:50Z

System Architecture

← Older revision		Revision as of 09:28, 13 June 2025
Line 48:		Line 48:

	* '''Data Consistency & Quality'''		* '''Data Consistency & Quality'''
	** ~~[https://unitapedia.univ-unita.eu/hop/~~ '''Apache HOP ETL''']: Ensures data from different sources ([https://unitapedia.univ-unita.eu/strapi/ Strapi], [https://unitapedia.univ-unita.eu/minio/ MinIO] CSVs) is validated, cleaned, and structured before landing in the datamart.		** '''Apache HOP ETL''': Ensures data from different sources ([https://unitapedia.univ-unita.eu/strapi/ Strapi], [https://unitapedia.univ-unita.eu/minio/ MinIO] CSVs) is validated, cleaned, and structured before landing in the datamart.
	** [https://www.mediawiki.org/wiki/Extension:Semantic_Media_Wiki ~~'''~~Semantic MediaWiki~~'''~~]~~: Allows~~ for structured data definitions and cross-referencing, ensuring consistent reporting across tasks and indicators.		** '''Semantic MediaWiki''':[https://www.mediawiki.org/wiki/Extension:Semantic_Media_Wiki Semantic MediaWiki] allows for structured data definitions and cross-referencing, ensuring consistent reporting across tasks and indicators.

	* '''Maintainability & Future Growth'''		* '''Maintainability & Future Growth'''

Antonio De Jesus Menchaca Martinez: /* System Architecture */

2025-06-13T09:27:55Z

System Architecture

← Older revision		Revision as of 09:27, 13 June 2025
Line 40:		Line 40:
	* '''Open-Source & Extensibility'''		* '''Open-Source & Extensibility'''
	** '''MediaWiki''': Chosen for its mature ecosystem (extensions like [https://www.mediawiki.org/wiki/Extension:Semantic_Media_Wiki Semantic MediaWiki], [https://www.mediawiki.org/wiki/Extension:External_Data External Data], [https://www.mediawiki.org/wiki/Extension:Page_Forms Page Forms]) and robust community support.		** '''MediaWiki''': Chosen for its mature ecosystem (extensions like [https://www.mediawiki.org/wiki/Extension:Semantic_Media_Wiki Semantic MediaWiki], [https://www.mediawiki.org/wiki/Extension:External_Data External Data], [https://www.mediawiki.org/wiki/Extension:Page_Forms Page Forms]) and robust community support.
	** ~~[https://unitapedia.univ-unita.eu/pga/~~ '''PostgreSQL''']: Offers advanced query capabilities, reliability, and easy integration with [https://unitapedia.univ-unita.eu/hop/ '''Apache HOP'''].		** '''PostgreSQL''': Offers advanced query capabilities, reliability, and easy integration with [https://unitapedia.univ-unita.eu/hop/ '''Apache HOP'''].
	** ~~[https://unitapedia.univ-unita.eu/minio/~~ '''MinIO''']: An open-source, S3-compatible object store that fits seamlessly into containerized deployments.		** '''MinIO''': An open-source, S3-compatible object store that fits seamlessly into containerized deployments.

	* '''Security & SSL'''		* '''Security & SSL'''

Antonio De Jesus Menchaca Martinez: /* Steps to Implement */

2025-06-13T09:22:20Z

Steps to Implement

← Older revision		Revision as of 09:22, 13 June 2025
Line 796:		Line 796:

	== Steps to Implement ==		== Steps to Implement ==
	# '''Docker Setup''':		* '''Docker Setup''':
	<ol>		<ol>
	<li>Configure <code>docker-compose.yml</code> to define containers for MediaWiki, [https://unitapedia.univ-unita.eu/strapi/ Strapi], [https://unitapedia.univ-unita.eu/pga/ PostgreSQL], [https://unitapedia.univ-unita.eu/hop/ Apache HOP], [https://unitapedia.univ-unita.eu/minio/ MinIO], and Nginx-proxy.</li>		<li>Configure <code>docker-compose.yml</code> to define containers for MediaWiki, [https://unitapedia.univ-unita.eu/strapi/ Strapi], [https://unitapedia.univ-unita.eu/pga/ PostgreSQL], [https://unitapedia.univ-unita.eu/hop/ Apache HOP], [https://unitapedia.univ-unita.eu/minio/ MinIO], and Nginx-proxy.</li>
Line 803:		Line 803:
	</ol>		</ol>

	# '''MediaWiki Deployment''':		* '''MediaWiki Deployment''':
	<ol>		<ol>
	<li>Install MediaWiki and configure <code>LocalSettings.php</code> with:		<li>Install MediaWiki and configure <code>LocalSettings.php</code> with:
Line 820:		Line 820:
	</ol>		</ol>

	# '''Strapi Setup''':		* '''Strapi Setup''':
	<ol>		<ol>
	<li>Build and deploy Strapi using the provided <code>Dockerfile</code>.</li>		<li>Build and deploy Strapi using the provided <code>Dockerfile</code>.</li>
Line 833:		Line 833:
	</ol>		</ol>

	# '''ETL Configuration''':		* '''ETL Configuration''':
	<ol>		<ol>
	<li>Configure [https://unitapedia.univ-unita.eu/hop/ Apache HOP] to set up ETL processes:		<li>Configure [https://unitapedia.univ-unita.eu/hop/ Apache HOP] to set up ETL processes:
Line 845:		Line 845:
	</ol>		</ol>

	# '''Ontology Integration''':		* '''Ontology Integration''':
	<ol>		<ol>
	<li>Implement ontology-based semantic relations in MediaWiki using [https://unitapedia.univ-unita.eu/index.php?title=Extension:Semantic_MediaWiki Semantic MediaWiki].</li>		<li>Implement ontology-based semantic relations in MediaWiki using [https://unitapedia.univ-unita.eu/index.php?title=Extension:Semantic_MediaWiki Semantic MediaWiki].</li>
Line 851:		Line 851:
	</ol>		</ol>

	# '''Nginx Reverse Proxy Configuration''':		* '''Nginx Reverse Proxy Configuration''':
	<ol>		<ol>
	<li>Configure Nginx-proxy to handle HTTPS, route traffic to containerized services:		<li>Configure Nginx-proxy to handle HTTPS, route traffic to containerized services:
Line 864:		Line 864:
	</ol>		</ol>

	# '''Integration Testing''':		* '''Integration Testing''':
	<ol>		<ol>
	<li>Test inter-service communication (e.g., MediaWiki querying PostgreSQL, Strapi APIs).</li>		<li>Test inter-service communication (e.g., MediaWiki querying PostgreSQL, Strapi APIs).</li>
Line 870:		Line 870:
	</ol>		</ol>

	# '''Deployment''':		* '''Deployment''':
	<ol>		<ol>
	<li>Deploy containers in a production environment with robust resource allocation.</li>		<li>Deploy containers in a production environment with robust resource allocation.</li>

Antonio De Jesus Menchaca Martinez: /* Data Architecture */

2025-06-13T09:21:08Z

Data Architecture

Show changes

Antonio De Jesus Menchaca Martinez: /* Data Sources */

2025-06-13T09:17:36Z

Data Sources

← Older revision		Revision as of 09:17, 13 June 2025
Line 570:		Line 570:

	=== Data Sources ===		=== Data Sources ===
	# ~~<q>~~Manually~~</q>~~		# '''Manually'''
	## [https://unitapedia.univ-unita.eu/strapi/ Strapi] Input Forms: Certain UNITA activities (e.g., task-based deliverables) require manual data entry. Authorized users in each UNITA office submit relevant metrics and progress updates through Strapi forms.		## [https://unitapedia.univ-unita.eu/strapi/ Strapi] Input Forms: Certain UNITA activities (e.g., task-based deliverables) require manual data entry. Authorized users in each UNITA office submit relevant metrics and progress updates through Strapi forms.
	# ~~<q>~~Semi-Automatically~~</q>~~		# '''Semi-Automatically'''
	## [https://unitapedia.univ-unita.eu/minio/ MinIO] Uploads (CSV): UNITA offices may upload internal data from local databases (e.g., student mobility records, research outputs, events) into MinIO buckets for automatic ingestion into the Datawarehouse.		## [https://unitapedia.univ-unita.eu/minio/ MinIO] Uploads (CSV): UNITA offices may upload internal data from local databases (e.g., student mobility records, research outputs, events) into MinIO buckets for automatic ingestion into the Datawarehouse.
	## Publicly Available Data Sets: Data from external APIs (e.g., Moodle, Erasmus+ portals) or open data repositories may be incorporated for broader impact analysis or benchmarking.		## Publicly Available Data Sets: Data from external APIs (e.g., Moodle, Erasmus+ portals) or open data repositories may be incorporated for broader impact analysis or benchmarking.

	=== Data Ingestion Methods ===		=== Data Ingestion Methods ===

Antonio De Jesus Menchaca Martinez: /* 4.6 Data Governance, Quality, and Security */

2025-06-13T09:06:56Z

4.6 Data Governance, Quality, and Security

← Older revision		Revision as of 09:06, 13 June 2025
Line 715:		Line 715:
	[[File:Graphs example.png\|thumb\|930px\|center\|Graphs Extension]]		[[File:Graphs example.png\|thumb\|930px\|center\|Graphs Extension]]

	== ~~4.6~~ Data Governance, Quality, and Security ==		== Data Governance, Quality, and Security ==

	=== ~~4.6.1~~ Data Quality Assurance ===		=== Data Quality Assurance ===

	==== Validation Rules ====		==== Validation Rules ====

Antonio De Jesus Menchaca Martinez: /* Validation Rules */

2025-06-13T09:06:20Z

Validation Rules

← Older revision		Revision as of 09:06, 13 June 2025
Line 721:		Line 721:
	==== Validation Rules ====		==== Validation Rules ====
	* '''Strapi Forms'''		* '''Strapi Forms'''
	** Field-Level Validation: Certain fields (e.g., numerical indicators, date fields) must conform to a predefined format or fall within acceptable ranges. If a value is invalid, the user is prompted to correct it before the data is stored.		** '''Field-Level Validation''': Certain fields (e.g., numerical indicators, date fields) must conform to a predefined format or fall within acceptable ranges. If a value is invalid, the user is prompted to correct it before the data is stored.
	** Review & Publish Step: Strapi includes a <q>Published</q> button that serves as an extra checkpoint. Users must explicitly publish each entry after reviewing it, ensuring that data is accurate and complete before it proceeds to the transformation process in [https://unitapedia.univ-unita.eu/hop/ Apache HOP].		** '''Review & Publish Step''': Strapi includes a <q>Published</q> button that serves as an extra checkpoint. Users must explicitly publish each entry after reviewing it, ensuring that data is accurate and complete before it proceeds to the transformation process in [https://unitapedia.univ-unita.eu/hop/ Apache HOP].

	* '''Apache HOP Transformations'''		* '''Apache HOP Transformations'''

Antonio De Jesus Menchaca Martinez: /* Datawarehouse Compartmentalization */

2025-06-13T09:05:12Z

Datawarehouse Compartmentalization

← Older revision		Revision as of 09:05, 13 June 2025
Line 776:		Line 776:
	* '''Additional Databases:'''		* '''Additional Databases:'''
	** The unita-data database contains metadata and configuration details used by MediaWiki.		** The unita-data database contains metadata and configuration details used by MediaWiki.
	** The unita-data-demo database is used for testing and proof-of-concept purposes.

	This multi-database, multi-schema design enables:		This multi-database, multi-schema design enables:

Antonio De Jesus Menchaca Martinez: /* Roles and Responsibilities */

2025-06-13T09:04:20Z

Roles and Responsibilities

← Older revision		Revision as of 09:04, 13 June 2025
Line 739:		Line 739:
	==== Roles and Responsibilities ====		==== Roles and Responsibilities ====
	* '''UNITA Office (CRUD on Alliance Level for Their University):'''		* '''UNITA Office (CRUD on Alliance Level for Their University):'''
	## Can create, read, update, and delete records for their institution’s indicators.		** Can create, read, update, and delete records for their institution’s indicators.
	## Responsible for ensuring that data submitted (e.g., participant counts, events) is accurate and timely.		** Responsible for ensuring that data submitted (e.g., participant counts, events) is accurate and timely.
	# '''Task Leader (Read-Only at Task Level):'''		* '''Task Leader (Read-Only at Task Level):'''
	## Can monitor and view data for all universities participating in their assigned task(s) (e.g., [https://unitapedia.univ-unita.eu/index.php/Education_and_research_%26_innovation_community T2.3]).		**Can monitor and view data for all universities participating in their assigned task(s) (e.g., [https://unitapedia.univ-unita.eu/index.php/Education_and_research_%26_innovation_community T2.3]).
	## Typically does not edit or delete data, but may request corrections or clarifications from the UNITA Offices.		** Typically does not edit or delete data, but may request corrections or clarifications from the UNITA Offices.
	# '''Project Manager (CRUD at All Levels):'''		* '''Project Manager (CRUD at All Levels):'''
	## Has the highest level of access, capable of modifying any indicator data across the alliance.		** Has the highest level of access, capable of modifying any indicator data across the alliance.
	## Primarily uses this access for oversight or emergency corrections, while day-to-day data entry is handled by UNITA Offices.		** Primarily uses this access for oversight or emergency corrections, while day-to-day data entry is handled by UNITA Offices.

	==== Version Control of Data Definitions ====		==== Version Control of Data Definitions ====