Documentation

System Overview

The UNITApedia system is composed of two integrated main components designed to enhance data accessibility, transparency, and collaboration among UNITA members. It connects a shared data warehouse with a MediaWiki-based front-end, creating a dynamic and scalable ecosystem for data visualization, management, and analysis.

Shared Data Warehouse

Acts as the central repository for structured data such as deliverables, indicators, and progress metrics. Utilizes metadata, ontology, and semantic web technologies to provide a comprehensive, interconnected view of data collected across all UNITA members. Supports efficient data centralization, organization, and analysis, ensuring a unified understanding of the data ecosystem. Backed by PostgreSQL, enabling complex queries, scalability, and robust data storage. Alongside Apache HOP as an ETL to develop powerful data pipelines.

MediaWiki-Based Front-End Interface

Provides a user-friendly system for monitoring project progress, visualizing metrics, and assessing impact. Acts as the primary user interface, powered by extensions like External Data, Scribunto, and Semantic MediaWiki. Dynamically retrieves data through its API layer, integrating seamlessly with the data warehouse. Enhances decision-making and collaboration by providing stakeholders with real-time, actionable insights. Share and collaborate with other users to extend the UNITA knowledge-base.

Key Features

• Near real-time integrated data pipeline processus:
  • Utilizes robust APIs to fetch and display updated information from the PostgreSQL database.
  • Near-instantaneous process from data extraction to final result display on UNITApedia.
• User-Friendly Interface:
  • Built on MediaWiki, ensuring an intuitive experience for users of varying technical backgrounds.
  • Extensions like Page Forms and Semantic MediaWiki simplify data input, annotation, and querying.
• Open Source:
  • Designed with modularity and scalability in mind, allowing deployment across other UNITA members or similar institutions.
  • Supports customization to meet unique institutional needs while adhering to UNITA’s vision.
• Dynamic Queries:
  • Uses optimized prepared PostgreSQL statements and Lua scripting via MediaWiki extensions to deliver efficient and dynamic data visualization.
  • Allows advanced customization of data presentation formats based on user needs.
• Scalable Architecture:
  • Employs a Dockerized infrastructure for each subsystem (MediaWiki, Strapi, PostgreSQL, Apache HOP, etc.), ensuring modularity and scalability.
  • Supports efficient deployment, updates, and resource allocation.
• Enhanced Collaboration and Transparency:
  • Enables cross-institutional collaboration by centralizing data in the shared warehouse.
  • Provides stakeholders with real-time visualizations, ensuring informed decision-making and alignment with organizational goals.

System Architecture

This chapter provides an overview of the UNITApedia system architecture, highlighting the containerized design, data flows, and interactions between the various services. The architecture ensures scalability, maintainability, and security, while leveraging open-source technologies to facilitate collaboration and data accessibility across the UNITA alliance.

The following considerations shaped the UNITApedia architecture:

• Modularity & Scalability
  • Docker ensures each service is isolated, easily updated, and can be scaled independently if usage grows.
  • Clear separation of roles (Strapi for input, Apache HOP for ETL, MediaWiki for output) streamlines development and maintenance.

• Open-Source & Extensibility
  • MediaWiki: Chosen for its mature ecosystem (extensions like Semantic MediaWiki, External Data, Page Forms) and robust community support.
  • PostgreSQL: Offers advanced query capabilities, reliability, and easy integration with Apache HOP.
  • MinIO: An open-source, S3-compatible object store that fits seamlessly into containerized deployments.

• Security & SSL
  • Nginx-Proxy + ACME Companion: Provides automated certificate management and secure HTTPS connections, protecting data in transit.
  • Role-Based Access: Strapi enforces form-level permissions, while MediaWiki can be configured with namespace-based access for sensitive data.

• Data Consistency & Quality
  • Apache HOP ETL: Ensures data from different sources (Strapi, MinIO CSVs) is validated, cleaned, and structured before landing in the datamart.
  • Semantic MediaWiki: Allows for structured data definitions and cross-referencing, ensuring consistent reporting across tasks and indicators.

• Maintainability & Future Growth
  • Each service can be updated or replaced with minimal impact on the others, thanks to Docker’s container-based isolation.
  • The architecture can accommodate new data sources, additional tasks/indicators, or new alliances with minimal refactoring.

UNITApedia Global Architecture

Request Flow

1. User Interaction: A UNITA Office user or a Task Leader navigates to the UNITApedia URL.
2. Nginx-Proxy: Receives the request over HTTPS and routes it to the appropriate container (MediaWiki, Strapi, etc.).
3. Data Entry (Strapi): If the user is adding new indicator data, the form submission is stored in the Strapi database.
4. ETL (Apache HOP): On a scheduled or on-demand basis, Apache HOP retrieves the new entries from Strapi (or CSV files in MinIO), applies transformations, and loads them into the datamart.
5. MediaWiki Display: MediaWiki queries the datamart schema via the External Data extension to display up-to-date metrics on wiki pages or dashboards.
6. Administration: pgAdmin is used by database administrators for maintenance tasks, accessible behind the Nginx-Proxy with proper credentials.

UNITApedia Technological Architecture

High-Level Overview

UNITApedia is composed of several interconnected services running in Docker containers, orchestrated via Docker Compose. The main components are:

Nginx-Proxy Service

• Role: Acts as a reverse proxy, routing external HTTP/HTTPS requests to the appropriate backend service based on URL paths.
• Security: Integrates with the ACME Companion service for automatic SSL certificate management and renewal, ensuring secure connections via HTTPS.
• Endpoints: Forwards traffic to MediaWiki, Strapi, phpMyAdmin, pgAdmin, MinIO, Apache HOP, and any additional admin interfaces.

Nginx-proxy Service Architeture

MediaWiki Container

• Primary Role: Serves as the user-facing front-end, allowing UNITA stakeholders to view, edit, and query data related to alliance activities and indicators.
• Extensions:
  • External Data – Dynamically queries the PostgreSQL data warehouse (datamart) to display indicators, metrics, and other information.
  • Page Forms / Semantic MediaWiki – Enables structured data input and advanced semantic queries within the wiki.
  • Scribunto (Lua) – Allows for more advanced scripting and data manipulation.
  • PluggableAuth + OpenID Connect – Integrates SSO (e.g., Keycloak) for institutional login.
• Configuration: Managed via LocalSettings.php, which includes namespace definitions (e.g., DataSrc and Doc) and data source connections (prepared SQL statements).