Architecture
The SCARCITY core library follows a layered architecture with clear separation of concerns.
System Diagram
graph TD
App[Application Layer] --> FMI[FMI Service]
App --> Sim[Simulation]
App --> Meta[Meta-Learning]
subgraph "Core Runtime"
FMI --> Fed[Federation Layer]
Fed --> Eng[Engine Layer (MPIE)]
Eng --> Stream[Stream Processing]
Note[Runtime Bus & Telemetry] -.-> Eng
Note -.-> Fed
DRG[Resource Governor] -.-> Eng
DRG -.-> FMI
end
Stream --> Ops[Core Operators]
Component Interaction Flow
-
Data Ingestion Stream sources feed data windows to the engine via the
StreamSource. -
Path Exploration MPIE proposes and evaluates candidate paths using bandit algorithms (UCB).
-
Federation Successful paths are packaged into
PathPacksand shared across domains via the Federation Coordinator. -
Meta-Learning Cross-domain patterns are learned and applied to the global model prior.
-
Resource Management DRG monitors system resources (CPU, RAM) and throttles the Engine or FMI layer adaptation.
-
Simulation Agent-based models provide what-if analysis capabilities to predict system behavior.