MLOps and Model Deployment Backbone for a Regional Hospital Network's Triage AI
Let's Connect
Overview
What we built
An 18-hospital network had piloted stroke and other time-critical triage AI models, but every deployment still depended on an engineer doing it by hand. We built the MLOps backbone that turned that manual effort into an automated, monitored pipeline.
In plain terms: the network's 18 hospitals had begun using AI models to help flag stroke and other time-critical findings on scans, but nothing about running those models was automated. Engineers copied each model onto individual servers by hand, nobody tracked which version was running where, and no monitoring watched a model once it went live. One pilot model quietly degraded for weeks before anyone noticed, and bringing a new hospital site online swallowed over a month of ad hoc IT effort every single time.
We built the MLOps and deployment backbone the network was missing: containerised model serving behind a central registry, CI/CD pipelines with automated validation gates against curated test sets, staged hospital-by-hospital rollouts, and live dashboards tracking drift, latency and alert volumes. Deployment lead time fell from over 6 weeks of manual work to under 3 days, drift on a triage model was caught and rolled back within 48 hours, and uptime held at 99.6% across all 18 hospitals in the first year, with alerts reaching the care team in under 90 seconds at the 95th percentile.
The Problem
Hand-deployed models, no monitoring
The network's 18 hospitals had piloted imaging triage models to flag stroke and other time-critical findings, and the pilots worked well enough clinically to keep expanding. What never grew up alongside them was the machinery for running those models safely. Every deployment was still a manual act: an engineer copying a model file onto an individual server, with no shared registry of what was actually live where.
That gap turned into a real clinical risk. One pilot model degraded silently for weeks before anyone noticed, because nothing was watching its behaviour once it went live. There was no monitoring layer to catch drift, no dashboard tracking latency or alert volumes, and no straightforward way to check whether the model reaching a given hospital was even the version anyone had approved.
Growth made the manual approach worse with every site added. Bringing AI triage to a new hospital meant repeating the same ad hoc IT effort from scratch, and each rollout consumed over a month of work before a single scan could be triaged there. With no rollback path beyond redoing the manual copy, even fixing a bad deployment meant starting the whole process again.
Manual, untracked deployment
Models were copied onto individual hospital servers by hand, with no central registry recording which version was running at which site.
No post-launch monitoring
Nothing watched a model once it went live, so drift or errors could run unnoticed until a clinician happened to spot something wrong.
Silent model degradation
One pilot model degraded for weeks without detection, undermining confidence in triage output at exactly the sites relying on it most.
Slow, repeated rollouts
Each new hospital site required over a month of ad hoc IT effort, repeating the same manual work rather than reusing a proven process.
What it was costing them
Every week a degraded model ran unnoticed was a week of triage output the network could not fully trust, and every new hospital site meant paying the same manual deployment cost again rather than building on the last rollout. Engineers spent their time on repetitive server-by-server copying instead of clinical safety work, while leadership had no dashboard to show regulators or clinicians that the models running were the models actually approved.
The Solution
Containerised MLOps deployment backbone
We built the network's MLOps and deployment backbone around containerised model serving behind a central registry, replacing the practice of copying files onto individual servers with one authoritative source of what should be running where. Every model version now deploys the same way, regardless of which hospital site it is heading to.
Around that registry we added CI/CD pipelines with automated validation gates against curated test sets, so a model has to clear defined checks before it can reach a hospital. Rollouts happen in a staged, hospital-by-hospital sequence rather than all at once, and live dashboards track drift, latency and alert volumes continuously once a model is live.
We closed the loop with audit and recovery. Every prediction is now logged, giving the network a durable record of what each model returned, when it returned it, and which version produced the result. Rolling back to any previous model version now takes a single command, instead of repeating the original manual deployment across every affected hospital server one at a time.
Key decisions
Central registry over local copies
A central model registry replaced the practice of copying files onto individual servers, giving the network one authoritative record of what runs where.
Validation gates before rollout
CI/CD pipelines run automated validation against curated test sets, so a model has to clear defined checks before reaching a single hospital.
Stage rollouts hospital by hospital
Deployments move through the network in a staged, hospital-by-hospital sequence rather than all at once, containing the impact of any issue.
Monitor drift and latency continuously
Live dashboards track drift, latency and alert volumes for every model in production, replacing the silence that let one pilot degrade unnoticed.
Make rollback a single command
Reverting to any previous model version now takes one command, turning what used to be a repeat deployment into a routine safeguard.
Measurable Impact
What changed after launch
The backbone changed how quickly and safely the network can move. Deployment lead time fell from over 6 weeks of manual work to under 3 days, and drift on a triage model was caught and rolled back within 48 hours of onset rather than running for weeks unnoticed.
Reliability became measurable rather than assumed. Model-serving uptime held at 99.6% across all 18 hospitals in the first year, and alerts reached the care team in under 90 seconds at the 95th percentile, giving clinicians a dependable signal instead of a system nobody was quite sure was still watching.
Deployment speed
Over 6 weeks of manual server-by-server work
Under 3 days through the automated pipeline
Drift detection
Silent degradation ran for weeks unnoticed
Caught and rolled back within 48 hours
Serving uptime
No monitoring once a model went live
99.6% uptime sustained across 18 hospitals
Alert latency
No tracked latency target existed
Under 90 seconds at the 95th percentile
Headline results
Model deployment lead time cut from over 6 weeks of manual work to under 3 days
Input drift on a triage model detected and rolled back within 48 hours of onset
99.6% model-serving uptime sustained across all 18 hospitals in the first year
Scan-to-care-team alert latency held under 90 seconds at the 95th percentile
Tech & Tools Used
What powered the build
Every tool below earned its place in this engagement. Here is the part each one played.
Python (FastAPI)
Serves the triage models behind the registry's APIs, handling incoming scan requests from each hospital and returning predictions for logging and monitoring.
PyTorch
Runs the triage models themselves, the same inference code validated in CI/CD before it is packaged for staged rollout to any hospital.
Docker
Packages every model version into a container, so the same artefact that passed validation is exactly what deploys to each hospital server.
Kubernetes
Orchestrates the containerised model-serving workloads across the network, restarting unhealthy instances and supporting the single-command rollback to any earlier version.
MLflow
Provides the central model registry, recording every version, its validation results and which hospitals it has been rolled out to.
Prometheus
Collects drift, latency and alert-volume metrics from every deployed model, feeding the live dashboards that replaced the old silence after go-live.
Grafana
Turns the Prometheus metrics into the dashboards engineers watch for drift and latency, the monitoring layer the network previously lacked entirely.
GitLab CI/CD
Runs the automated validation gates against curated test sets, blocking any model from reaching a hospital until it clears the defined checks.
PostgreSQL
Stores the prediction log and deployment history behind the audit trail, recording what each model returned and which version served it.
Orthanc (DICOM)
Handles the DICOM imaging traffic feeding the triage models, giving the pipeline a consistent way to receive scans from hospital scanners.
Ready to Build your Hospitals & Health Systems Business with MLOps & Model Deployment
Ask Byte
Ask Byte
Typically replies instantly
just Now
Hi! I'm OrganByte's assistant. How can I help you today?
AI-generated content may be incorrect
