Taking AI into production

Name the decision or process the system will change, and the person accountable for that decision today. If no such person exists, nobody will take ownership of the system after the pilot.

Most teams frame a pilot around a use case, which describes what the system will do. A use case does not name anyone who is worse off if the system never exists. So it can attract broad support from people who carry no responsibility for the outcome.

The DTA readiness checklist asks a more useful question than a use case does. It asks whether there is "a principal Business Owner involved that can ensure proposed outcomes are validated".⁸ The scenario at the start of this paper shows what happens without one.

Research on adoption made the same point more than 20 years ago. Greenhalgh and colleagues reviewed how innovations spread through service organisations. They separate adoption by individuals from take-up by the organisation. Take-up depends on the organisation's capacity to absorb change, spare resources, dedicated funding and leadership.⁹ A proof of concept scope leaves all of those out.

In my engagements, those that reached production had that person named before the build started. Most of those that stopped had an enthusiastic sponsor and no such person.

Measure the current process before anything is built: volume, time, effort and error rate. Without those numbers, results measured after the system is introduced cannot be compared with the previous process.

This is the cheapest of the 5 decisions and the one most often skipped, because the cost comes early and the benefit comes much later.

The METR trial shows why. People's reports of how much time they saved can be the opposite of the measured result, and they are not aware of the difference. A pilot with no baseline has to ask users whether the system saved them time. That is the one measure the trial showed to be unreliable.

The DTA guidance asks for this. Principle 5 requires AI initiatives to have "defined success metrics and baselines, systematic evaluation methods and pathways to value".¹⁰ It does not say how to establish a baseline, and in my experience nobody does unless someone insists before the build starts.

Score whether the output is supported by its source, and separately whether it is worth acting on. A demonstration using inputs chosen by the builders does not show how the system performs on real work.

A grounded system finds material and then writes from it. The method here is for that kind of system. Score whether the output is supported by the material it cites, and separately whether it is worth acting on. The first can be checked by anyone who can read the source. The second needs someone who knows the subject. Keeping them apart shows which part failed, and the 2 failures have different fixes. AccuFind has published the method it uses for this, including a variant for exploratory work where no gold standard exists.¹¹ Other methods that keep the 2 questions apart will serve as well.

A general impression is not enough, because a model's performance varies between tasks in ways that cannot be predicted without testing. Dell'Acqua and colleagues ran a preregistered experiment with 758 consultants, in collaboration with Boston Consulting Group. On 18 tasks the model was capable of performing, people using AI completed 12.2 per cent more tasks and worked 25.1 per cent faster. On a task chosen to sit outside it, "subjects using AI were 19% less likely to produce correct solutions compared with those without AI".¹² That is one task in one experiment. It shows that there are tasks a model performs poorly, without identifying which tasks those are. That is why each use case has to be evaluated separately.

For work that depends on knowing the agency's documents, which covers most legal and policy work, buy a model and build retrieval around it. Fine-tuning is justified only in a few narrow cases.

Mistral's published method places building tailored models second of 4 steps, immediately after choosing the use cases.¹³ For a government agency working from its own documents, the model training in that step is usually the part to skip. The published evidence on how to give a model access to new factual material supports that.

The published comparison of the 2 approaches favours retrieval. Ovadia and colleagues found retrieval consistently outperformed continued pre-training for injecting new factual knowledge, and that fine-tuning on new facts can degrade performance.¹⁴ That is the finding most relevant to an agency whose material is legislation and policy.

Light fine-tuning is cheap. On narrow legal annotation tasks, a few hundred to a thousand labelled examples were usually enough to beat the frontier commercial models of the time.¹⁵ An agency that wants to try it on an open-weight model can afford to.

The cost that matters is the recurring obligation. A fine-tune is tied to a base model, and the vendor withdraws base models on its own schedule. OpenAI gives generally available models at least 6 months' notice, and it shut down a set of older models on 23 October 2026. It is also winding down self-serve fine-tuning. Organisations that had never fine-tuned could not start after 7 May 2026. From 6 January 2027, active customers will no longer be able to create new jobs.¹⁶ A vendor can withdraw the fine-tuning capability as well as the base model. Every base model change means redoing the tune, on top of the evaluation any deployment needs.

Open-weight models carry the same obligation for a different reason. The weights persist, so nobody can withdraw them. But a tuned model does not improve, while bought frontier models do, so an agency that wants current performance will need to re-tune periodically. Self-hosting adds hosting, patching and security assessment. Someone still has to maintain the evaluation set either way.

Privacy law applies as well. The Office of the Australian Information Commissioner treats training a model on personal information as a secondary use. That applies where training was not a primary purpose of collection. An entity that cannot establish the use is within reasonable expectations, and related to the original purpose, should seek consent, offer an opt-out, or both.¹⁷ That guidance applies under the Privacy Act. State agencies operate under their own privacy arrangements, which in South Australia are administrative rather than statutory, but the same question about secondary use arises.

The Commonwealth already provides bought models. GovAI, run by the Department of Finance, brokers models to Commonwealth agencies. Its chat app for Australian Public Service staff, GovAI Chat, ran an alpha trial from April to June 2026. A beta trial has run since July, and staff can use it if their agency has joined.¹⁸ A Commonwealth agency should check what it already has access to before building anything.

Identify the operating budget the service will sit in. Name the person who owns the benefit from a business-as-usual role, and the process change the benefit depends on. Settle this before the pilot puts anything in front of real users.

Most of the pilots I have seen stop did so because this decision had not been made.

A proof of concept is usually paid for from an innovation line, a central pool or a discretionary budget. A production service needs ongoing money, which needs a costed benefits case, which needs the baseline from Decision 2 and the evaluation from Decision 3. An agency that skipped those cannot build the case. In my engagements the reasons given for stopping were usually about confidence, and the underlying obstacle was usually money.

The DTA checklist asks about this twice. After the proof of concept, it asks whether adequate funding is available. Before scaling, it asks whether adequate funding has been sought and approved for the pathway to production, and whether there is provision for top-up funding.⁸ Whether an agency can answer depends on who was named in Decision 1.

The benefit also has to belong to someone who is not on the project. The Commonwealth's benefits management guidance is explicit. The benefit owner "must hold a business as usual role in the organisation (as opposed to a project delivery role)".¹⁹ That guidance was written for digital and ICT investment generally, not for AI.

The lag between a technology and its measured benefit is well documented. Brynjolfsson, Rock and Syverson argue that general purpose technologies deliver measured productivity only after a lagged wave of complementary investment in process, skills and organisation. In their view this lag has likely been the biggest contributor to the productivity paradox.²⁰

Two further costs belong in the production case and are usually missing. Where the work itself changes, consultation requirements under the agency's enterprise agreement or award apply before the change. And because vendors change and withdraw models on their own schedule, evaluation is a standing operating cost rather than a project activity.

The DTA's Scenario 6 shows a cost nobody budgeted for. A complaint triage model reached 85 per cent accuracy and could not be deployed. The model needed frequent training and updating. The agency's change management process required manual approvals for even minor code changes, and could not keep up. The lesson drawn is that AI development and IT modernisation "should be treated as a unified program".²¹