NHS Integrated Care Board Microsoft Fabric Real-Time Patient Data: 8-Hour Lag to 20 Minutes

Numlytics built NHS Microsoft Fabric real-time patient data infrastructure for an NHS Integrated Care Board in the Midlands, migrating all 22 Azure Data Factory pipelines including 5 completely undocumented ones to Microsoft Fabric, implementing near-real-time ingestion via Fabric Eventstream connected to the EPIC HL7 FHIR interface, and reducing patient data lag from 8 hours to 20 minutes. 160 clinical and operational staff are now on self-service analytics, and ad-hoc reporting requests dropped 62% within 8 weeks of launch.

The Challenge: Clinical Teams Making Real-Time Decisions on Eight-Hour-Old Data

The ICB had invested seriously in its data infrastructure over three years, building 22 Azure Data Factory pipelines ingesting patient pathway data from EPIC, the regional NHS Spine, an estates management platform, a workforce scheduling tool, and a locally developed SQL Server data warehouse. As winter pressure activity intensified, the limitations became dangerous.

• Eight-hour patient data lag: All 22 pipelines ran on overnight batch schedules. By the time bed managers and pathway coordinators arrived, data in their Power BI dashboards was already 8 hours old and growing less useful throughout the day, a genuine operational risk during busy winter periods.
• Five completely undocumented pipelines: Built by a contractor who left in 2022 with no documentation and no monitoring. One had been failing silently for eleven days without detection, discovered only when a senior clinician noticed waiting time figures had not changed across two consecutive weekly reports.
• Four-week ad-hoc reporting backlog: The data team of five was overwhelmed with SQL query requests from clinical and operational teams. Directorate managers were routinely making workforce and resource decisions using unofficial Excel trackers rather than waiting for data team support.
• No real-time clinical operations capability: Bed managers had no live view of occupancy, discharge notifications, or admission status. All operational decisions were made on yesterday's numbers.

One undocumented pipeline had been failing silently for eleven days before anyone noticed. The ICB had been reporting incorrect performance data to the board during that period. The silent failure scenario had to be structurally eliminated, not patched.

The Numlytics Approach: Six Weeks of Discovery Before Committing to an Architecture

Numlytics designed this NHS Microsoft Fabric real-time patient data programme as a five-phase engagement, beginning with a complete audit of all 22 pipelines before any architecture decisions, and ending with self-service analytics for 160 clinical and operational staff.

1. 01
   Pipeline Audit and Risk Assessment (Weeks 1 to 6)

   A complete audit of all 22 ADF pipelines, which took longer than estimated because of the five undocumented ones. Reverse-engineering required SQL Server query tracing, Azure Monitor log analysis, and interviews with engineers who had worked alongside the original contractor. By week six, Numlytics had a complete dependency map, a failure mode analysis for each pipeline, and a prioritised migration plan. The five undocumented pipelines were designated for full rebuild rather than migration.

2. 02
   Microsoft Fabric Lakehouse Architecture: OneLake Medallion (Weeks 7 to 13)

   Numlytics designed and built the target Microsoft Fabric Lakehouse on OneLake using a three-tier Medallion structure. The Bronze layer handles raw ingest from all six source systems. The Silver layer applies clinical data quality rules, handles SNOMED code mapping and ICD-10 classification, and produces conformed patient pathway records. The Gold layer contains reporting-ready aggregates including waiting time calculations, bed occupancy rates, referral-to-treatment pathways, and workforce utilisation metrics.

3. 03
   Near-Real-Time Ingestion via Fabric Eventstream (Weeks 12 to 17)

   For the four highest-priority operational data streams including bed occupancy, A&E attendance, inpatient admissions, and discharge notifications, Numlytics implemented real-time ingestion using Fabric Eventstream connected to the EPIC HL7 FHIR interface. These streams now land in the Bronze Lakehouse layer within two minutes of the source event. Clinical staff see bed status changes, discharge completions, and new admissions in their dashboards within 20 minutes, compared to the previous 8-hour batch lag.

4. 04
   Pipeline Migration and Rebuild (Weeks 13 to 20)

   The 17 documented pipelines were migrated to Fabric Data Factory with improved scheduling, monitoring dashboards, and automated alerting for failures. The 5 undocumented pipelines were rebuilt from scratch using the dependency maps produced during discovery, with full unit testing and documentation before deployment. All 22 pipelines now have health dashboards in Fabric, failure alerts routed to the data team, and a monthly pipeline review process. The eleven-day silent failure scenario has been structurally eliminated.

5. 05
   Self-Service Analytics and Clinical Training (Weeks 20 to 24)

   Numlytics designed a governed self-service analytics layer on the Gold Lakehouse tier. Certified semantic models with directorate-level row-level security allow clinical and operational teams to explore data within their own domain without data team support. Eight half-day training workshops were delivered across clinical, commissioning, finance, and workforce teams. Copilot was enabled for the ICB senior leadership group using endorsed semantic models as the sole authorised data source.

Delivery Timeline

The Results

Technology Stack

Frequently Asked Questions