VFS End-to-End Workflow Architecture v1.7

📊 Workflow Diagram (Vertical Stack)

graph TD subgraph "1. Source Layer" NAPI[NewsAPI.ai] OFF_SOC[Offline Social Monitoring - Admin] end subgraph "2. Pipeline Engine (Path A)" SCH[Cloud Scheduler - Daily 00:00] IS[Ingestion Job - Cloud Run] AUTH_FILT[Guardrails: Source Blocking] FS_PROP[Firestore: topic_proposals] NE[Neural Service - Gemini 2.5 Flash] SY[Symbolic Engine - LISP Macros] TAM[Thematic Anchor Mapping] FS_CL[Firestore: claims] end subgraph "3. Forensic Archive (Path B)" GCS_RAW[GCS: vfs-raw-ingest] BQ_LAKE[BigQuery: Lake Metadata] end subgraph "4. Analytics & Sustainability (Path C)" COSTS[Global LEDGER: resource_costs] BQS[BigQuery Sync Extension] BQ_V[BigQuery: Normalized Views] AS[vfs-analytics Service] FS_STATS[Firestore: entity_stats] end subgraph "5. Authority & Normalization (Path D)" AH[vfs-authority: Universal Hub] EP[Entity Profiles] FS_AUTH[Firestore: authority_entities] FS_TOP[Firestore: authority_topics] DNR[Display Name Resolution] end subgraph "6. Public Delivery (Path E)" PUI[Public UI: vfs.6109.blog] CAC[Canonical Consolidation] GBX[The Glass Box Transparency] METRIC[Live Metrics] end subgraph "7. Admin Oversight" TRIAGE[Live Triage] MONITOR[Deployment Monitor] OVERSIGHT[Resource Oversight] WATCH[Watchlist Management] end %% Ingestion Flow NAPI --> IS OFF_SOC -.->|Manual Guidance| TRIAGE SCH --> IS IS --> AUTH_FILT IS --> GCS_RAW IS --> FS_PROP %% Processing Flow FS_PROP --> NE SY --> NE NE --> AH AH --> FS_AUTH NE --> FS_PROP NE --> COSTS NE --> FS_CL FS_CL --> TAM TAM --> FS_TOP %% Analytics Flow FS_CL --> BQS FS_PROP --> BQS BQS --> BQ_V BQ_V --> AS AS --> FS_STATS %% Delivery Flow FS_PROP --> TRIAGE FS_STATS --> TRIAGE COSTS --> OVERSIGHT TRIAGE --> PUI WATCH --> FS_AUTH %% Public Flow PUI --> DNR DNR --> FS_AUTH DNR --> FS_TOP PUI --> CAC FS_CL --> PUI AS --> PUI COSTS --> GBX GBX --> METRIC PUI --> GBX

🏗️ Detailed Component Breakdown

1. The "Harvest" (Ingestion & Guardrails)

Primary Source: NewsAPI.ai for high-integrity news ingestion.

Offline Social Intelligence: Admin monitors social platforms offline to identify emerging sentiment trends and steer VFS toward relevant special reports.

Strategic Guardrails: Proactive filtering via authority_sources. Includes temporary blocking of sources that drift toward sensationalism or non-grounded reporting.

Shadow Archive: Immutable storage in COLDLINE GCS.

2. The "Brain" (Neural Orchestrator)

Models: Gemini 2.5 Flash (Neural) + Lisp Symbolic Engine (Constraint Enforcement).

Thematic Anchor Mapping: Maps granular claims to high-level Canonical Thematic Anchors for narrative structure.

Logical Friction: Identification of contradictions vs. historical fact store.

3. The "Lens" (Internal Oversight)

Standardized Dashboard: Unified tools for Triage, Resource Management, and Deployment Monitoring.

Manual Triggering: "Deep Dive" adjudications are triggered manually by Admin to refine automated requirements and ensure editorial precision.

Longitudinal Source Credibility: Future focus on gauging sensationalism drift in news sources. Sources prioritizing worldview sympathy over documented claims will be temporarily de-indexed.

4. The "Output" (Public Adjudicator)

Subscriber UI: Premium glassmorphic public record.

Display Name Resolution: Human-readable persona presentation.

Thematic Consolidation: Narrative-centric grouping of grounded claims.