Data Architecture Viewpoint - Revision history

Ernesto Exposito at 14:13, 19 March 2026

2026-03-19T14:13:17Z

← Older revision		Revision as of 14:13, 19 March 2026
Line 1:		Line 1:
	The '''Data Architecture Viewpoint''' is also of technical nature, will be oriented to communicate about the data model used to operate the observatory. This viewpoint is intended for experts in data management of the alliance to observe how the integration of the different data sources of each university has been implemented and will also allow users less expert in data processing to make specific queries to observe specific indicators on certain conditions (reports on a particular instrument, on its deployment, on its evolution over time, etc.).The results of these consultation operations, if relevant, could later be integrated into one of the 4 initial viewpoints.		The '''Data Architecture Viewpoint''' is also of technical nature, will be oriented to communicate about the data model used to operate the observatory. This viewpoint is intended for experts in data management of the alliance to observe how the integration of the different data sources of each university has been implemented and will also allow users less expert in data processing to make specific queries to observe specific indicators on certain conditions (reports on a particular instrument, on its deployment, on its evolution over time, etc.).The results of these consultation operations, if relevant, could later be integrated into one of the 4 initial viewpoints.
			===PostgreSQL (Data Warehouse)===
			* '''Role''': Central repository storing structured data such as deliverables, indicators, and metrics.
			* '''Multi-Database Setup''':
			** ''strapi'': Contains raw input tables from [https://unitapedia.univ-unita.eu/strapi/ Strapi] forms.
			** ''datamart'': Holds transformed and processed data ready for MediaWiki queries.
			** ''unita-data'': Contain additional metadata or wiki configuration tables.
			* '''Administration''': Managed via pgAdmin for database operations (e.g., backups, user management).

			===Apache HOP (ETL and Reporting)===
			* '''Processes''':
			** '''Data Retrieval''': Fetches raw datasets from [https://unitapedia.univ-unita.eu/minio/ MinIO] buckets (CSV files) or [https://unitapedia.univ-unita.eu/strapi/ Strapi] tables in [https://unitapedia.univ-unita.eu/pga/ PostgreSQL].
			** '''Data Transformation''': Cleans and normalizes data, ensuring consistency (e.g., date formatting, numeric checks, selecting values).
			** '''Data Integration''': Loads validated data into the datamart schema for consumption by MediaWiki.
			* '''Scheduling & Monitoring''': Deployed [https://unitapedia.univ-unita.eu/hop/ Apache HOP] “Carte Server” allows scheduling of jobs and transformations, with logs for error handling and performance monitoring.


			[[File:Apache Service.jpg\|thumb\|930px\|center\|Apache HOP Service Architeture]]

			===MinIO (Object Storage)===
			* '''Role''': Stores raw data files (CSV, PDFs, images, etc.) uploaded by UNITA Offices.
			* '''Integration''': [https://unitapedia.univ-unita.eu/hop/ Apache HOP] connects to [https://unitapedia.univ-unita.eu/minio/ MinIO] using an S3-compatible interface, retrieving files for ETL processing.
			* '''Organization''': Multiple buckets can be created (e.g., “dev” for storing [https://unitapedia.univ-unita.eu/hop/ Apache HOP] transformations, indicators buckets to store CSV files coming from UNITA Offices).


			[[File:Minio Service.jpg\|thumb\|930px\|center\|MinIO Service Architeture]]

			=== Strapi (Middleware / Headless CMS) ===
			* '''Purpose''': Provides a user-friendly interface for UNITA Offices to manually input or update indicator data.
			* '''Data Flow''': Stores raw records in its own [https://unitapedia.univ-unita.eu/pga/ PostgreSQL]L database schema, which [https://unitapedia.univ-unita.eu/hop/ Apache HOP] then reads, transforms, and pushes into datamart.
			* '''APIs''': Exposes REST or GraphQL endpoints if needed for external integrations or advanced use cases.


			[[File:Strapi Service.jpg\|thumb\|930px\|center\|Strapi Service Architeture]]

	[[Category:Viewpoints]]		[[Category:Viewpoints]]

Antonio De Jesus Menchaca Martinez at 13:35, 27 May 2025

2025-05-27T13:35:27Z

← Older revision		Revision as of 13:35, 27 May 2025
Line 1:		Line 1:
	The '''Data Architecture Viewpoint''' is also of technical nature, will be oriented to communicate about the data model used to operate the observatory. This viewpoint is intended for experts in data management of the alliance to observe how the integration of the different data sources of each university has been implemented and will also allow users less expert in data processing to make specific queries to observe specific indicators on certain conditions (reports on a particular instrument, on its deployment, on its evolution over time, etc.).The results of these consultation operations, if relevant, could later be integrated into one of the 4 initial viewpoints.		The '''Data Architecture Viewpoint''' is also of technical nature, will be oriented to communicate about the data model used to operate the observatory. This viewpoint is intended for experts in data management of the alliance to observe how the integration of the different data sources of each university has been implemented and will also allow users less expert in data processing to make specific queries to observe specific indicators on certain conditions (reports on a particular instrument, on its deployment, on its evolution over time, etc.).The results of these consultation operations, if relevant, could later be integrated into one of the 4 initial viewpoints.
			[[Category:Viewpoints]]

Antonio De Jesus Menchaca Martinez: Created page with "The '''Data Architecture Viewpoint''' is also of technical nature, will be oriented to communicate about the data model used to operate the observatory. This viewpoint is intended for experts in data management of the alliance to observe how the integration of the different data sources of each university has been implemented and will also allow users less expert in data processing to make specific queries to observe specific indicators on certain conditions (reports on..."

2025-04-24T11:07:30Z

Created page with "The '''Data Architecture Viewpoint''' is also of technical nature, will be oriented to communicate about the data model used to operate the observatory. This viewpoint is intended for experts in data management of the alliance to observe how the integration of the different data sources of each university has been implemented and will also allow users less expert in data processing to make specific queries to observe specific indicators on certain conditions (reports on..."

New page

The '''Data Architecture Viewpoint''' is also of technical nature, will be oriented to communicate about the data model used to operate the observatory. This viewpoint is intended for experts in data management of the alliance to observe how the integration of the different data sources of each university has been implemented and will also allow users less expert in data processing to make specific queries to observe specific indicators on certain conditions (reports on a particular instrument, on its deployment, on its evolution over time, etc.).The results of these consultation operations, if relevant, could later be integrated into one of the 4 initial viewpoints.