A young doctor sits facing a patient with diffuse symptoms. In the system before her, an algorithm has already sorted journal data, suggested differential diagnoses and ranked probabilities. The system is probably doing something right: it processes more data faster than she could have done alone, and it does not overlook the blood test value she might have dismissed. But something else is also happening, something that concerns her and not the patient: the questions she does not need to ask are precisely the questions she would have learned something from asking, and they are also the questions that might have led her to conclusions the data does not already contain.

A healthcare system under pressure

It is worth acknowledging the situation as it actually is. Norwegian hospitals operate under a pressure that has been building for years: growing waiting lists, staff shortages, administrative burdens eating into clinical time. In this situation it is not merely legitimate but necessary to adopt technology that can shorten waiting lists, reduce error margins in image interpretation, ease administrative burdens and distribute resources more precisely. Where routine tasks take time away from clinical presence, automation can free up room for what actually requires a human being. No one is served by romanticising the inefficiency of a system that is already failing patients, and technological resistance must not become a disguised conservatism.

The argument that follows is therefore not that the current system is good enough and should be protected from change, but that in the rush to repair what is broken, we risk breaking something else without noticing.

When decision support shapes attention

In clinical encounters, much of what makes healthcare good is difficult to capture in indicators: the continuity of a relationship over time, the trust that leads a patient to say what she actually feels, the clinician's ability to read something in a situation that is not documented anywhere. These things are real, they have clinical significance, and they are systematically underrepresented in the data on which systems are trained. This is not because they are unimportant, but because they are difficult to measure, and what is difficult to measure rarely ends up in training data.

The more interesting point, however, is not that the unmeasurable is underrepresented, for this is nothing new. What is new is the mechanism: as decision support systems gradually shape what clinicians direct their attention towards, clinical culture shifts. The systems point attention towards what they are built to see, and what they are not built to see recedes into the background. No one has decided that the measurable is most important, but the infrastructure has done so, because attention is a limited resource, and it is led where the tools direct it.

Judgement as something formed

Clinical judgement is a capacity that is formed through practice, not a property one possesses. The ability to ask the right question at the right time, to feel the unease of a vague symptom description and let that unease lead further, to read what is unspoken in a consultation room: this ability is the result of years of accumulated experience, of having been wrong and understood why, of apprenticeship in the broadest sense. The central question becomes not only what is overlooked in the moment, but what erodes over time.

Craft is the natural parallel. What is transferred from master to apprentice is not technique alone, but a way of seeing and sensing the material. This way of seeing cannot be taught directly; it is formed by the apprentice having to try, fail, become uncertain, and find the way forward. It is this process, standing in what one does not understand long enough for understanding to grow, that constitutes productive friction.

In most craft traditions this has been understood: the introduction of tools that remove friction has been handled with care, because the friction itself is part of the training. Medical education retains cadaver dissection despite advanced simulation tools. The point is not to reject the tools, but to design the learning pathway around them, with awareness of what is lost when the process changes.

The double loss

Here lie structural, not merely technical, challenges. When AI systems take over an increasing share of diagnostic work, a double loss occurs. The first is the loss in the moment: the dimension of the clinical encounter that the system does not see. The second is the loss over time: that the conditions which form judgement gradually disappear. A new generation of clinicians develops capacities in response to the tasks they actually perform. When the tasks change in character, the formation changes with them. The loss is significant, but absent from the reckoning. And what makes the loss difficult to detect is that the competence that remains is not easily distinguished from the competence that earlier conditions produced. A clinician formed with decision support appears capable within the frameworks she operates in. It is only when those frameworks deviate from the normal, when the system fails, or when a situation is genuinely novel, that the difference in formation becomes apparent.

It is easy to object that automating diagnostic work will free the doctor to spend more time on precisely the relational, judgement-dependent tasks. The objection has merit, and the promise of freed-up time is real, but it rests on two assumptions that are rarely made explicit. The first is that the tasks being removed were merely time-consuming, and not also formative, that the doctor learned nothing from asking the questions herself. The second is that the attention formed in encounter with a system that has already evaluated and sorted is of a different character than the attention formed without such a system. A clinician who grows up with decision support is trained to evaluate proposals rather than to make assessments herself, and that forms a different kind of clinical eye: less exploratory, perhaps less willing to ask the question the system has not asked. A clinician with time to spare because the system has asked the questions for her is not the same clinician formed by having asked the questions herself.

What kind of clinicians we want to form

The question of how much should be automated is therefore not merely a technical question about the precision of systems, but a question about what healthcare is for. If the answer includes that patients should be met as whole human beings by clinicians who bear responsibility for the assessments they make, then that commits us to something regarding which capacities must be formed and maintained, and which conditions must be protected for them to be formed.

This means that decisions about what to automate are more than technical procurement decisions. They are decisions about what kind of professionals we want in ten, twenty and thirty years, and what kind of profession they are to belong to. These decisions require a conversation conducted between more parties than those who build and sell the technology: it belongs to the educational institutions, the health trusts, the professional associations and public debate. But those who build the tools also have a responsibility to think beyond the next product version, because the tools they design shape the learning conditions of those who will use them.

The young doctor in front of the screen does not need to choose between the system and her own judgement. But someone, her educational institution, the hospital at which she works, the supplier who built the system, must have thought through where her productive friction is to come from as more and more of it is removed. The aim of automation is right: to free up time and attention for what requires a human being. But that requires us to ask, for each task and in each domain, not only whether this can be done by a machine, but also what the human being becomes by doing it herself, and whether it is immaterial that the human being is no longer formed by it.

Read an extended version of the article here