Code vs. Pathogens: How NHS England is Using Graph Databases and Python to Eradicate Hepatitis C

Hey everyone, Alex here. Welcome back to another edition of Coding with Alex on sysseder.com. Today, we are taking a detour from our usual Kubernetes configs and Rust microservices to look at a massive real-world win for technology. You might have seen the headlines today: England is officially on track to become one of the first countries in the world to completely eliminate Hepatitis C.

Now, if you are a software engineer, your first instinct might be to think, "That's a medical breakthrough, Alex. What does that have to do with me?"

The answer is: everything. Behind this massive public health milestone isn't just pharmacology; it is a masterclass in data engineering, graph database design, and predictive Python pipelines. The NHS (National Health Service) didn't achieve this just by buying drugs—they achieved it by solving a massive, distributed "needle in a haystack" data problem to locate, track, and treat undiagnosed patients across a population of 56 million people.

Today, we're going under the hood to look at how we can use graph databases, fuzzy matching algorithms, and modern data pipelines to solve complex tracking problems—whether you're tracking a viral pathogen in a population or tracing microservice dependencies in a distributed cloud architecture.

The Engineering Challenge: The Invisible Entity Problem

To eliminate a disease like Hepatitis C, you have to treat the people who have it. Sounds simple, right? Here is the engineering catch: Hepatitis C is often asymptomatic for decades. Millions of people have it without knowing it. However, they leave a trail of "bread crumbs" across disparate, siloed database systems over thirty years: a positive antibody test in an offline lab system in 1998, a needle-exchange program record in a local charity SQL database in 2005, and a modern emergency room admission in 2021.

As developers, we face this exact problem all the time. We call it the Entity Resolution Problem. How do you know that "John Doe" born in 1978 in London is the same "J. Doe" who accessed a service in Manchester ten years later?

Relational databases (like PostgreSQL or MySQL) are fantastic for transactional data, but they fail miserably when you need to query highly interconnected, deeply nested, and fragmented relationships across billions of rows. Doing recursive JOIN operations across dozens of tables to find connections is a quick way to crash your database engine.

To solve this, modern medical data architecture has shifted toward Graph Databases (like Neo4j) combined with Python-based data-cleaning pipelines. Let's build a prototype of how this works.

Building the Graph: Modeling Nodes and Relationships

Instead of thinking of patients and clinics as rows in a table, we model them as a graph. In our graph model:

Nodes represent entities: Patient, Clinic, LabResult, and Alias.
Edges (Relationships) represent connections: TESTED_AT, HAS_RECORD, POTENTIAL_DUPLICATE, or SHARED_CONTACT.

Here is an architectural text-representation of what this graph looks like:

[Patient: A] --(POTENTIAL_MATCH)--> [Patient: B]
     |                                    |
(TESTED_AT)                          (TESTED_AT)
     v                                    v
[LabResult: Pos_1998]                [LabResult: Pos_2012]

By structuring our data this way, we can run graph traversal algorithms to find paths between disconnected data points that point to the same physical person. Let's look at how we can write a Cypher query (the SQL of graph databases) to find these hidden links.

Writing the Cypher Query to Identify High-Risk Chains

Imagine we want to query our database to find patients who have had a positive test in the past, have not been prescribed treatment, and have a high-risk relationship chain. In Neo4j, that query is incredibly clean and executes in milliseconds, compared to a massive, slow SQL multi-join:


// Find patients with a positive lab result who haven't had treatment
MATCH (p:Patient)-[:HAS_RESULT]->(r:LabResult {status: "Positive", disease: "HepC"})
WHERE NOT (p)-[:RECEIVED_TREATMENT]->(:Treatment)
MATCH (p)-[:LINKED_TO|SHARED_ADDRESS*1..2]-(other:Patient)
RETURN p.id AS PatientID, p.lastKnownPostcode AS Postcode, collect(other.id) AS Connections;

This query doesn't just look at the individual. It uses the *1..2 syntax to traverse one to two hops out to find other linked individuals (such as family members, housemates, or geographic hubs) to identify hot spots. Try doing that with a standard SQL self-join without burning your CPU to the ground!

The Python Pipeline: Record Linkage and Fuzzy Matching

Before we can even load this data into a graph database, we have to clean it. In public health—and in enterprise SaaS migration—data is messy. Names are misspelled, dates of birth are fat-fingered, and postcodes change.

To solve this, we can build a record linkage pipeline in Python using the Levenshtein distance algorithm (to measure string similarity) and the Double Metaphone phonetic algorithm (which groups names by how they sound, rather than how they are spelled). This ensures that "Stephen" and "Steven" are flagged as potential matches.

Here is a practical Python script using the jellyfish library to build a fuzzy-matching preprocessor for our database loader:


import jellyfish

def analyze_patient_records(record_a, record_b):
    """
    Compares two patient records and determines the probability of them being the same person.
    """
    # 1. Check phonetic match of last names
    sound_a = jellyfish.metaphone(record_a['last_name'])
    sound_b = jellyfish.metaphone(record_b['last_name'])
    
    # 2. Check Levenshtein distance of first names (0.0 to 1.0 similarity)
    first_name_sim = jellyfish.jaro_winkler_similarity(record_a['first_name'], record_b['first_name'])
    
    # 3. Check exact date of birth
    dob_match = record_a['dob'] == record_b['dob']
    
    # Calculate a matching score
    score = 0
    if sound_a == sound_b:
        score += 0.4
    if first_name_sim > 0.85:
        score += 0.3
    if dob_match:
        score += 0.3
        
    return score

# Example Usage
patient_1998 = {
    "first_name": "Jonathon",
    "last_name": "Smith",
    "dob": "1978-05-12"
}

patient_2015 = {
    "first_name": "John",
    "last_name": "Smyth",
    "dob": "1978-05-12"
}

match_probability = analyze_patient_records(patient_1998, patient_2015)
print(f"Match Probability: {match_probability * 100:.1f}%")
# Output: Match Probability: 100.0% (Phonetic last name match, high JW first name similarity, exact DOB)

By deploying these pipelines as serverless microservices (e.g., AWS Lambda or Google Cloud Functions) processing incoming data streams, the NHS and its tech partners were able to automatically merge duplicate profiles, creating a single, clean "golden record" for every individual in the system.

Architecting for Privacy: Zero Trust and Data Masking

I know what you're thinking: "Alex, this is highly sensitive medical data. How do you build a system like this without violating privacy laws like HIPAA or GDPR?"

This is where modern software architecture shines. You do not store Personally Identifiable Information (PII) directly in your analytical graph database. Instead, you use a Pseudonymization Architecture.

Here is how a secure, decoupled data pipeline is designed:

The Vault: An isolated, highly secure relational database stores the raw PII (Names, DOB, SSN/NHS Number) and generates a cryptographically secure, random UUID for each patient.
The Hash Pipeline: Before data is sent to the analytical engine or graph database, PII is stripped. Names are hashed using salted SHA-256 or replaced entirely by the UUID.
The Graph: The graph database only ever sees the UUIDs, relationship types, and timestamp data. It performs the complex math and connection tracing without ever knowing *who* the physical person is.
The Re-Identification Gateway: Only authorized clinicians, accessing the system through a secure API gateway with strict Role-Based Access Control (RBAC) and Multi-Factor Authentication (MFA), can map the UUID back to a real name when it is time to deliver treatment.

As developers, we should apply this "Zero Trust" architecture to our own systems. If you're building analytics platforms, user tracking, or telemetry pipelines, never pass raw email addresses or user names through your event streams. Use transient IDs and keep your PII strictly isolated.

Conclusion: The Ultimate Proof of Concept

England's march toward eliminating Hepatitis C is proof of what happens when world-class medicine meets modern, scalable software engineering. It shows that our skills as developers aren't just for building checkout carts, social media feeds, or B2B SaaS widgets. The algorithms we write, the database schemas we design, and the pipelines we build have the power to save lives at a national scale.

Next time you're profiling a database query, refactoring a fuzzy matching algorithm, or setting up a secure API gateway, remember: the architecture decisions you make today are the foundation for the massive, society-changing systems of tomorrow.

What's your take?

Have you had to tackle complex entity resolution or graph processing in your own projects? What tools did you use to scale it? Let me know in the comments below, or share this article with your team's lead architect!

Until next time, keep coding, keep optimizing, and keep building things that matter.