Twin pillars of ML disappointment¶

In the great and creaking city of Ankh-Morpork, success depends on luck, a tolerance for interesting smells, and the ability to keep a straight face while the whole system declares that everything is going exactly to plan. Our model has none of these. It has two pillars instead, Data and Requirements, both as reliable as a guild oath sworn after midnight, each holding the thing up with the enthusiasm of a wet cardboard box.

Between them, they ensure the model will be flawed if it is lucky, scandalous if it is not, and profoundly misunderstood by the people who commissioned it either way.

Garbage in, gospel out¶

In theory, data is the fuel that powers the clever contraptions. In reality, it behaves more like the damp kindling the Night Watch uses when attempting to start the annual solstice bonfire. Everyone gathers round, insists it will work, swears they have done this before, and then watches the thing smoulder resentfully while producing nothing but smoke and disappointment.

The eternal whinge¶

Data scientists have a single truly reliable setting. It is the one where they explain, with the desperation of a man who has seen the edge of the world and found it insufficiently documented, that there is not enough data. Ten million samples? A mere whiff. More data is always needed. More data is the answer to every question. More data is the oxygen of their existence, and there is always someone, apparently, standing on the hose.

There are, to be fair, moments of genuine famine. Training a model on three JPEGs and an optimistic shrug is unlikely to succeed unless the target domain is children’s fridge art. More often the problem is not a lack of data but a lack of useful data.

A warehouse full of inputs can still amount to a drought. Does the dataset represent humanity in all its chaotic, contradictory glory, or does it look more like a damp pile of Tesco receipts and mislabelled cat photos? Ten thousand shots of the same blurry stapler teach a model very little beyond the unfortunate truth about the procurement department. A billion rows of missing values and timestamps set to the Unix epoch teach it only despair.

Quantity counts, but not as much as the people chanting for more seem to believe, especially when the last time they inspected a histogram was sometime before The Patrician outlawed unlicensed fortune-telling.

Data quality, where hope goes to die¶

Data quality is the noble, shimmering ideal that everyone mentions and no one recognises. Ask for a definition and the answers range from “not horrible” to “the thing that did not crash the pipeline last week”.

It fails in three reliably different ways.

Consistency¶

Consistency is the hope that data will behave sensibly, like a decently trained dog. What usually turns up is a cat that has learned the art of irony. One column says “United Kingdom”. Another says “UK”. A third says “U.K.”. A fourth offers “Europe, sort of”, as though the field itself were having an existential crisis. Attempt a join across that lot and watch a soul leave the building.

Correctness¶

Correct data is meant to reflect reality. Instead, it often reflects an intern’s dreams during the final hour of a shift. “Customer Age: 217” turns up, and the choice is between an immortal customer and a mug landing on a keyboard. Meanwhile, the fraud model is busy awarding a youth savings account to a lich.

Completeness¶

Completeness is the cheerful fiction that a dataset contains information rather than an art installation composed mainly of absence. Eighty per cent missing values is fine, says someone. Just impute the mean. Now everyone has exactly average income, average height, and the same number of cats. Useful for modelling Lego figures. Less so for people.

Cleaning data is tedious, thankless, and absolutely necessary. Much like sweeping the streets of Ankh-Morpork: nobody praises the work, and skipping it makes everything unpleasant in ways that accumulate quietly and smell alarming.

The art of lowering expectations¶

If gathering requirements feels like group therapy held in a broom cupboard, that is because it is. Stakeholders describe an AI system that is ethical, transparent, unbiased, fast, accurate, and cheap. Two of those are available. Two is generous.

Requirements documents begin life filled with hope, then slowly become graveyards of abandoned ambition. There is always a line that reads “must handle edge cases”. It will not. There is always “must be explainable”. It will not be, unless the explanation involves three wizards and a chalk circle. There is usually something about avoiding bias. The details follow, along with a stiff drink.

Once in a while someone produces a 73-page PDF. Nobody reads it until six months after deployment, when the system has done something outrageous and everyone is trying to work out who approved it.

Success lies not in fulfilling the vision but in managing expectations early enough that the outrage arrives in smaller, more manageable bursts.

The supplier’s requirements document¶

Since 2025 the most ambitious requirements have been written by the supplier. In January 2025 OpenAI’s chief executive wrote that the company was now confident it knew how to build AGI, and by June that the takeoff had started.

Anthropic’s had already promised a country of geniuses in a datacenter, smarter than a Nobel laureate in most fields, possibly as early as 2026, and in January 2026 put it as little as one to two years away, with half of entry-level white-collar jobs displaced within one to five. In September 2026 OpenAI’s president told an a16z podcast that “AGI has turned out to be less of a point in time and more of this sort of fuzzy spectrum.”

Earlier that month the ARC Prize, whose newest puzzles humans solved at 100 per cent and frontier models at 0.51 in March 2026, reported the new model at 62.7 per cent in September with its standard harness, and 99.9 with its provider-adapter harness. It wrote in the same post that it was not claiming the system was AGI and that the test does not represent the open-endedness of the real world.

The International AI Safety Report of January 2025, commissioned by the UK government, records that its experts consider loss of control anything from implausible to likely. Geoffrey Hinton used his Nobel banquet speech to say nobody had any idea whether control could be kept, and more than seventy-five thousand people have signed a call to prohibit superintelligence until there is consensus it can be done safely.

On jobs, economists at the European Central Bank found half of surveyed workers using AI at work in 2026, up from a quarter two years earlier, saving about eight per cent of their time, and no sign of it reducing employment at firm level.

Nine days earlier the same bank’s economists had written that the AI rally had lifted valuations to levels last seen in the dot-com bubble and that a correction was likely. Managing expectations downward was hard enough when the customer wrote them.

Neither pillar keeps to itself. What arrives downstream is a model that discriminates without ever being told to and cannot account for itself when asked, in a world that has since written laws about both.

Ethics and bias: the minefield¶

Every team insists it does not discriminate. Meanwhile, the model is redlining half the city with the efficiency of a Guild accountant on deadline. Bias seldom hides outside the dataset. Mostly it is the dataset. It arrives baked in, like raisins in a fruitcake nobody wanted but everybody receives, year after year.

Common practice is to remove the obviously sensitive columns and consider the matter closed. Race, gender, the usual suspects. Never mind that fields such as postcode and forename are quietly carrying the same information with the determination of a mule that has not been fed in three days.

A system that subjects “Jamal from East Ham” to a lengthy security review while fast-tracking “Sebastian from Surrey” into preferential customer support has something amiss with it, and it is not just the ambient fog.

The Dutch student-finance agency managed it with three fields. Its algorithm for catching students who claimed the away-from-home grant while living with their parents scored them on type of education, the distance between the two addresses and age. Some 21,500 students were selected and checked between 2013 and 2022. In November 2024 the Dutch data protection authority found no objective justification for any of the three, and the minister concluded that students with a migration background had been indirectly discriminated against.

The authority’s chair put the general case plainly: an algorithm with selection criteria distinguishes between groups of people by definition, and that distinction has to be justified, precisely, every time.

The traditional remedy is a fairness audit, which in many organisations means skimming a Medium article titled “Ethical AI for Busy People” shortly before the quarterly meeting. Bias is a mirror, and few care for the reflection.

Explainability: the corporate fig leaf¶

Explainable AI is the respectable cloak a model wears to meet regulators. It looks sincere, it sounds thoughtful, and it conceals how little of what the thing does in a given case anyone can recover.

Feature 147 and its deviation from baseline can be discussed at length. Users nod politely, then go looking for legal advice. SHAP values and LIME plots are genuinely clever, but they answer the wrong question. The applicant refused a mortgage wants to know what to do differently next time, and receives a ranked list of contributions from variables they have never heard of.

In February 2025 the Court of Justice of the European Union described what that applicant is owed, in a case about an Austrian mobile contract worth ten euros a month. The controller has to describe the procedure and principles it applied, in a form that lets the person see which of their data was used and how. It can be appropriate to tell them how far a change in that data would have changed the result. Handing over the algorithm does not count.

Where the controller pleads trade secrets, the secret goes to the supervisory authority or the court, which then decides how much the person gets to see.

In meetings, executives pretend to understand the diagrams, engineers pretend to enjoy producing them, and regulators pretend to find them reassuring. It is all terribly civil and answers nobody’s actual question.

An explanation requiring a whiteboard, three hours, and a sworn oath from the High Priest of Quantitative Mysticism is not an explanation. It is performance art.

Regulations: the compliance theatre¶

Onto the stage shuffle the great chorus of acronyms. GDPR, HIPAA, the AI Act, and the rest. Their purpose is to ensure that when something important gets broken by machine learning, it is at least broken in a documented fashion.

There will be a debate about whether log files count as personal data. They do. There will be an argument about whether storing them in the cloud is legal. It depends. There will be the consoling thought that nobody audits these things until a newspaper becomes involved.

Compliance is not the same thing as being ethical. It is about being demonstrably correct on paper. It resembles keeping a diary, not for posterity, but in case the Thieves’ Guild asks questions.

The golden rule is simple. Document everything. Not because it improves the system, but because the regulator arrives eventually, and a sufficiently thick paper trail will soak up a surprising quantity of tears.

The EU AI Act entered into force on 1 August 2024, which means it is no longer a threat on the horizon but a fact of life, like damp and unreliable contractors. It operates on a risk-based tiered system, the logic being that not all machine learning is equally capable of ruining someone’s afternoon.

At the top sit the prohibited practices: social scoring, whether the scoring is done by a government or by a company, real-time remote biometric identification in publicly accessible spaces, where the ban falls on law enforcement uses and carries narrow exceptions that lawyers are already widening, and systems designed to manipulate people through subliminal techniques. These are banned outright, which has not stopped several organisations from discovering that what they built falls uncomfortably close to the line.

Below that are high-risk systems, which arrive by two routes. One is a list of standalone uses: credit scoring, recruitment, law enforcement, education, critical infrastructure. The other covers AI built into products already regulated for safety, medical devices among them. The distinction is dry, and it decides which deadline applies. A model deciding who gets a loan, who gets an interview, or who gets flagged at the border is in this category whether anyone planned it that way or not. The obligations run to transparency, human oversight, data governance, accuracy and robustness, documented to a standard that satisfies auditors who were not present when any of the decisions were made and will not be impressed by a notebook.

Fines for prohibited practices reach 35 million euros or seven per cent of global annual turnover, whichever is greater, with the lower of the two applying to smaller companies. For most other violations the ceiling is fifteen million euros or three per cent. The European AI Office oversees the general-purpose model rules and, since July 2026, the systems their makers build on them and the AI inside the largest online platforms, while enforcement of the rest falls to national authorities, staffed throughout by people who have read the regulation rather more carefully than the average practitioner.

The compliance timeline has been phased, and then re-phased. Prohibitions applied from February 2025 and the obligations on general-purpose models from August 2025. The high-risk deadlines proved less durable. The Digital Omnibus, signed on 8 July 2026 and published in the Official Journal on 24 July as Regulation (EU) 2026/1744, entered into force on 27 July, six days before the original high-risk date would have applied. The obligations for the high-risk uses listed in Annex III now apply from 2 December 2027, and those for systems in products covered by Annex I from 2 August 2028.

This has produced the familiar spectacle of organisations simultaneously insisting they are fully compliant and quietly reclassifying their systems to avoid the high-risk category. A component in a regulated product that does nothing to keep anyone safe, whether it assists the user, tunes performance, automates a step, adds convenience or checks quality, is by its terms not a safety component, and so not high-risk. Consulting firms have never been busier. The paperwork has never been thicker.

In December 2023 the Court of Justice held that a credit score is itself an automated decision wherever the bank treats it as decisive, so the GDPR’s rule against significant decisions by machine alone reaches the scoring bureau.

The UK’s Data (Use and Access) Act, in force on this point since 5 February 2026, replaced the rule with a permission: a significant decision may be taken by automated processing alone, so long as safeguards let the person be told of the decision, make representations, obtain human intervention and contest it. The old prohibition survives where the decision rests on special-category data.

Between bad data and impossible promises, the model is already fragile before anyone lays a finger on it. Which is unfortunate, because a good many people are about to.