Resources

An AI assistant for practitioners working from statutory guidance

Practice guidance

Practitioners applying a thousand-page practice framework had no way to get a cited answer in the time they had. Structural analysis found 177 issues in the guidance itself, and a grounded assistant over the same corpus was scored across four rounds before anyone used it.

Checking a regulator’s published requirements against the law

Licensing and operational policy

An agency could not say which of its published licensing requirements came from the Act and which were its own. Two reviews found 290 issues in public-facing documents and 121 in operational policy, each placed as a comment inside the source file.

Updating an operational policy library for a new Act

Operational policy library, legislative transition

A new principal Act had passed and every policy in the library cited the one it replaced. A six-week remap split the mechanical from the substantive, and left the provisions where a wrong edit is worse than no edit to a person.

One safety plan, two work health and safety regimes

Work health and safety across two regimes

Work done in one country and directed from the other means two safety regimes apply at once, and they do not use the same words for the people who hold the duties. The operation is delivered by contractors rather than employees, which changes who holds which duty.

Evaluating grounded AI systems

Most AI pilots in government are judged on whether people liked them. This paper sets out a scoring method that keeps two questions apart, whether an output is right and whether it is worth acting on, including a variant for exploratory work where no gold standard can exist, and is plain about which numbers such an evaluation can and cannot report.

On Good Policy

What has to be true for an organisational policy document to do its job. Eleven evidence-based elements, from currency and lawful authority to workability and assumed knowledge, each with what failure looks like in practice and how to check for it. Ships with a companion appraisal schedule for running the checks across a real policy library.