Too much legacy spaghetti, not enough bread and butter policy

Sarah Ellison, Chief Technology Officer at Catalyst.

When Digitising Government Minister Paul Goldsmith was asked whether New Zealand would look to local or overseas technology to overhaul the public service, his response followed a familiar, disappointing script. Admitting he wasn’t aware of any local AI providers with the "scale of Claude or Copilot," he concluded that the state would simply "be making use of the best technology available."

This statement, emerging alongside plans to reshape the public sector workforce by embracing automated systems, reveals a profound misunderstanding of what is actually at stake. The real issue facing our public sector isn't just an unmanageable headcount problem; it is a clarity problem. By defaulting to international tech behemoths under the guise of efficiency, we are failing to leverage a significant opportunity to strengthen the economic benefit for New Zealand and our people.

Catalyst has been working with technology for nearly 30 years, and with artificial intelligence for around 16 of those. When you operate in this space for decades, you develop a reliable instinct for when technology is being used as a genuine tool, and when it is being used as a substitute for thinking. Right now, the national debate is doing the latter.

The policy hallucination

One of the well-known limitations of modern generative AI is hallucination which is the tendency to produce confident, fluent, entirely incorrect answers. The model fills the gap with whatever sounds right.

A version of this is currently happening in our policy conversation. Politicians driving these changes openly acknowledge they are not AI experts, yet the certainty with which AI is being positioned as a silver bullet for public sector duplication has the exact same quality as a hallucinated output: certain, specific, and with real-world consequences if it goes wrong.

AI is being treated as a single, catch-all category when it is not. The difference between a model that triages support requests and one that determines welfare eligibility, assesses credit scores, or drafts ministerial advice is a matter of kind, not degree. Treating AI as a catch-all leads to procurement decisions made at the wrong level of abstraction and governance frameworks that are too vague to be enforceable. Applying AI to a broken process does not fix the process; it merely speeds it up.

The structural causes: Legacy spaghetti and foreign worldviews

The current state of our public sector is largely the result of two systemic issues. First, we are fighting decades of "legacy spaghetti"- expensive, proprietary, closed software systems tangled across various agencies. Rushing to embed AI from a small number of powerful overseas providers without the right foundations is about to lock New Zealand into a new era of vendor dependency, at greater speed and scale.

Second, by defaulting to international platforms, we aren’t just purchasing software; we are exporting data sovereignty and importing foreign worldviews. Protecting New Zealand's interests as AI becomes embedded should be bread and butter policy. Global AI models carry trained assumptions about privacy, authority, and social relationships that can fundamentally misrepresent local values and kaupapa. They are unequipped to uphold our commitments to te Tiriti o Waitangi.

What we feed the machines matters. Passing sensitive citizen notes (such as health notes, welfare assessments, school records, land information) through offshore infrastructure governed by foreign law is a reckless risk. As my colleague Chris Cormack frequently observes: best practice for Māori data is best practice for all data. If an AI system is trained on data that doesn't understand the unique context of Aotearoa, it will simply fill that blank space with assumptions made in Silicon Valley.

Aotearoa has an opportunity to lead in building AI grounded in the rights and interests of communities - all of them. The Māori Data and AI Governance Framework from Te Kāhui Raraunga offers exactly that kind of foundation. It is not a Māori-only consideration. It is a model for how any society can approach AI with values, accountability, and care.

The remedy: Digital conservation islands

We do not need to build a resource-heavy Kiwi competitor to Microsoft or Google to win this race. Safety, control, and trust are the true measures of value. Scale is a lazy metric.

The government’s own Public Service AI Framework explicitly calls for "pathways to enable safe AI testing." Technology to support safe experimentation and adoption already exists in New Zealand in the form of open-source AI Safety Zones from Catalyst. These Zones are hosted on Catalyst Cloud which is Toitū net carbon-zero certified infrastructure, supporting climate obligations.

Think of these as digital conservation areas: contained, locally hosted environments where public sector agencies can experiment and innovate safely with proven open-source models. Data stays in Aotearoa, isn’t used to train foreign models, and never falls outside New Zealand's legal jurisdiction.

You cannot outsource judgement

When it comes to the most disruptive technology of our generation, we are missing the very basics of AI policy. This isn't a political debate about the size of the state. It is a fundamental question of what we are willing to trade in the pursuit of ‘efficiency’. While Australia has mandated a Chief AI Officer for every department and legally requires public risk assessments for high-risk applications, New Zealand is flying on purely opt-in, voluntary guidelines.

Efficiency does not mean moving fast in the dark. It means asking the right questions to solve the right problems and meeting those with the right tool for the right job. AI can definitely create efficiencies for the public sector, but only if decision-makers understand the root cause of the problem they are trying to solve. Instead, there should be a clear picture of the tools available (local and international), honesty about what the trade-offs for citizens and their data is, and the right policies in place to ensure we aren’t on the cusp of creating new legacy issues.

ENDS

This article has also been published on The Post.