The clock the tree was missing.

Health states.

Start with the states. A Markov model describes a disease as a handful of distinct health states — the situations a patient can be in that matter clinically and economically. For a relapsing condition, three states are enough to tell the story:

• In remission — the disease is controlled.
• Relapsed — the disease is active, costing more and worth less in quality of life.
• Dead — the end state.

Two rules govern the set of states, and they're the same discipline we met at the chance nodes of a tree. The states must be mutually exclusive — a patient is in exactly one at any moment, never two at once — and collectively exhaustive — between them they cover every situation a patient could be in. At any instant, every patient in the modelled cohort sits in one, and only one, of these boxes.

That's the model's entire world: three boxes. Everything the disease does, it does by moving patients between them.

Cycles: time in equal steps.

Now add the clock. A Markov model chops time into a sequence of equal-length periods called cycles — a month, a quarter, a year, whatever suits the disease. The model advances one cycle at a time, and within each cycle patients may move from the state they're in to another.

Cycle length is a genuine modelling choice, not a detail. It has to be short enough to catch the disease's rhythm: if patients can relapse and recover within weeks, an annual cycle is blind to it. But shorter cycles mean more computation and raise a subtle timing question — do patients move at the start of a cycle, the end, or somewhere in between? — which introduces a small but real bias. That bias has a standard fix, the half-cycle correction, and it's important enough to get its own lesson next. For now, hold the picture: time moves in equal ticks, and each tick is a chance to move.

Transition probabilities.

What governs the movement is a set of transition probabilities: for each state, the probability of moving to each other state in a single cycle. Collect them and you have the engine of the whole model — the transition matrix.

For our three-state example, per yearly cycle:

• From Remission: relapse with probability 0.15, die with probability 0.02, or stay in remission with probability 0.83.
• From Relapsed: recover to remission with probability 0.40, die with probability 0.10, or stay relapsed with probability 0.50.
• From Dead: stay dead, with probability 1.

Two things deserve attention. First, each row sums to 1 — including the probability of staying put. From any state a patient must go somewhere next cycle, even if "somewhere" is the same box; those exhaustive options have to account for the whole of the probability. Second, Dead is an absorbing state: once a patient enters it, they never leave. Every lifetime model needs one, because it's the drain that eventually empties the cohort and lets the model finish.

Notice the one move a tree could never make: from Relapsed back to Remission. Patients returning to a state they've already occupied — that's the whole reason this structure exists.

The Markov assumption: no memory.

Here is the idea at the centre of the model, the one every assessor has to understand cold. In a Markov model, a patient's transition probabilities depend only on the state they're in right now — not on how they got there, and not on how long they've been there. The model has no memory. This is the Markov assumption, and it's often called memorylessness.

Follow what it implies. Two patients both sitting in "Relapsed" — one who arrived this cycle, one who's been relapsed for eight years — face the identical probability of dying or recovering next cycle. The model cannot tell them apart, because the only thing it knows about a patient is which box they're in.

Memorylessness is what keeps the model small and tractable: three boxes and a matrix describe an entire lifetime, no matter how long. But it's also the model's central fiction, because real diseases remember. The risk of death is far higher in the first month after major surgery than years later; the chance of a further relapse often rises with the number of prior relapses. A single state with a single transition probability flattens all of that into one average.

When that flattening matters, modellers reach for a standard patch: tunnel states. Instead of one "post-surgery" box, you build a short chain — month 1 → month 2 → month 3 — that patients pass through in strict order, each with its own transition probabilities. The chain gives the model a limited, deliberate memory of time since an event. Tunnel states are the telltale sign of a modeller who noticed the assumption straining and did something honest about it.

Run the cohort.

Below is our three-state model. Press Step cycle to advance one year at a time and watch a cohort — everyone starting in remission — flow between the states and drain, slowly, into death. The relapse slider changes how fast patients leave remission. Watch two things build up as the cohort runs: total cost and total QALYs, per patient.

Every bar is the previous bar run once through the same matrix — that's the entire model, one rule applied over and over. Push the relapse slider up and watch the cohort sicken faster, the QALYs accrue more slowly, and the cost climb: the lifetime numbers are nothing but the trace, weighted by what each state costs and is worth, added up cycle by cycle.

(Costs and QALYs here are accrued from each cycle's opening state occupancy — i.e. assuming patients spend the whole cycle in the state they started it in. That approximation is exactly what the next lesson corrects.)

From the trace to ΔC and ΔE.

This is the moment the whole module has been building toward. Back in Module 7, every ICER and every net benefit ran on two numbers — ΔCost and ΔEffect over a lifetime — and we simply assumed them. Here is where they're actually made.

Give each state a per-cycle payoff: a cost and a utility.

• Remission: £2,000 and 0.9 QALYs per year.
• Relapsed: £12,000 and 0.5 QALYs per year.
• Dead: £0 and 0.

Now, each cycle, do the same weighted averaging you did in a decision tree — but on the state occupancy. Multiply the share of the cohort in each state by that state's cost, and by its utility, and sum. Take a cycle where the cohort sits 60% in remission and 40% relapsed:

Do that every cycle, add the results across the whole horizon, and you have lifetime cost and lifetime QALYs — for one arm. Run the model again with the new treatment's transition probabilities and payoffs, and you have the other. The difference between the two arms is ΔCost and ΔEffect. That is the number M7 divided into an ICER.

One refinement, met properly in Module 5: costs and QALYs far in the future are discounted — worth less in today's terms — so later cycles contribute a little less than earlier ones. The mechanics don't change; each cycle's contribution is just scaled down before summing.

Now you.

When Markov fits, and when it strains.

A Markov model is the right tool when three things hold:

• The disease is chronic and recurring. Patients move between states over a long horizon, sometimes returning to states they've left — precisely what a tree can't do.
• The states are few and clinically distinct. You can describe the disease with a small number of boxes that differ in cost and quality of life. Parsimony is a virtue: every extra state needs its own transition probabilities, and those have to come from somewhere.
• Transitions are roughly memoryless. What happens next depends mainly on the current state, not on history — or, where it doesn't, the history-dependence is contained enough to patch with a few tunnel states.

It strains when the disease has strong memory — when time-since-event or accumulated history drives risk in ways that would need a forest of tunnel states to capture — or when the relevant "states" are really a continuum. For those cases, two alternatives wait in the wings. Partitioned survival models sidestep transition probabilities altogether by deriving state occupancy directly from survival curves — the dominant approach in oncology, and the next lesson but one. And discrete event simulation drops the fixed-cycle, memoryless frame entirely, modelling individual patients with their own histories and event times. Both trade Markov's transparency for realism, and knowing when that trade is worth it is a core modelling judgement.

What is the problem, and the standard fix?

Why this matters for HTA

The Markov model is the workhorse of health economic evaluation — which means most of the modelling disputes you'll ever referee happen inside one. A few reflexes pay off repeatedly:

• Interrogate the state structure first. Do the states capture the clinically and economically important situations, and no more? Missing a costly state understates cost; inventing states no data can populate manufactures false precision. The structure is a claim about the disease before it's a calculation.
• Hunt for hidden memory. Ask where the disease's risks depend on history or time-in-state, then check whether the model ignored it or handled it with tunnel states. An unaddressed memory violation quietly biases the whole result — usually in a predictable direction.
• Trace every transition probability to its source. Each number in the matrix is an evidence claim. The ones that move the answer most — typically progression and mortality — deserve the hardest look, and are the natural targets for the sensitivity analysis of Module 9.

A Markov model is a whole life compressed into a few boxes and one rule, applied again and again. Its genius is how much it captures with so little — and your job is to find the place where "so little" left something important out.

Markov models, in one breath.

• A Markov model describes a disease as a small set of mutually exclusive health states, with patients moving between them each cycle according to transition probabilities (each row of the matrix sums to 1; Dead is absorbing).
• Its defining constraint is memorylessness: where a patient goes next depends only on their current state, never on history or time-in-state. Powerful, and often the model's central fiction — patched with tunnel states where memory matters.
• Run a cohort through the matrix cycle by cycle to get the Markov trace; weight each cycle's state occupancy by state costs and utilities, sum (and discount) across the horizon, and you have lifetime cost and QALYs — the ΔCost and ΔEffect the ICER needed.
• It fits chronic, recurring disease with few distinct states; it strains under strong history-dependence, where partitioned survival or discrete event simulation may serve better.

The tree ran once and stopped. Markov takes the same weighted averaging and puts it on a loop — and out of that loop, at last, come the lifetime numbers every cost-effectiveness verdict rests on.

There's one honest simplification we glossed over: we accrued each cycle's costs and QALYs as if patients spent the entire cycle in the state they started it in. They don't — they move throughout. Correcting that small, systematic timing error is the half-cycle correction, and it's next.