What Every SES Executive Must Know Before Signing an AI Vendor Contract
The Australian Public Service AI Plan 2025 has moved quickly. Chief AI Officers are being appointed, GovAI procurement frameworks are taking shape, and departments are under real pressure to show progress on AI adoption. That pressure is landing on SES officers who are being asked to sign off on vendor contracts — often without the technical or legal background to spot what’s buried in the fine print.
This is not a theoretical risk. AI vendor agreements routinely contain clauses around data rights, model training, liability, and service continuity that can bind your agency for years. Getting this wrong doesn’t just create budget problems — it can compromise citizen data, expose your agency to reputational harm, and leave you locked into a vendor relationship you can’t exit cleanly.
Here’s what you need to understand before you sign.
The Data Rights Trap
The single most important question to ask about any AI vendor contract is: what happens to your agency’s data once it enters the vendor’s system?
Many AI platforms — particularly those built on large language models — have training clauses that allow the vendor to use data submitted through the platform to improve their underlying models. In a commercial context, that’s an inconvenience. In government, where that data may include sensitive policy deliberations, citizen records, or procurement intelligence, it’s a serious problem.
Look specifically for clauses that address three things. First, training and fine-tuning rights — can the vendor use your inputs to train or improve their model? This should be explicitly prohibited for government data, and the prohibition should appear in the contract itself, not just in a terms-of-service document that the vendor can update unilaterally.
Second, data residency — where is your data stored and processed? Under the APS AI Plan 2025, agencies are expected to maintain visibility over data sovereignty. If the contract says “global infrastructure,” that’s worth interrogating hard. Global infrastructure is not an answer; it’s a deflection.
Third, data retention and deletion — what happens to your data when the contract ends? Some vendors retain data for months or years after termination. You need a clear, enforceable deletion clause with a defined timeframe, not a vague commitment to follow industry best practice.
Before any contract goes to signature, your legal and data teams should produce a one-page summary of what data rights the vendor claims. If they can’t produce that summary, the contract isn’t ready to sign.
Lock-In Is a Procurement Risk, Not Just a Technical One
Vendor lock-in is usually framed as a technology problem — your developers chose a proprietary API and now migration is expensive. But for SES officers, lock-in is a procurement risk with budget, continuity, and sovereignty implications that deserve the same scrutiny as any major infrastructure commitment.
In AI contracts, lock-in can happen in several distinct ways. Model dependency is the first. If the AI system is trained on your agency’s data in a proprietary format, switching vendors may mean starting over from scratch — losing months of tuning and institutional knowledge embedded in the model. That’s not a sunk cost you can easily write off.
Output portability is the second. Some vendors don’t allow you to export the outputs, configurations, or trained parameters of the AI system you’ve built on their platform. You are renting the capability, not owning it — and when the contract ends or the vendor raises prices, you have very little leverage.
Integration depth is the third and often the most insidious. The more deeply an AI vendor’s tools are woven into your workflows — your case management system, your document processing, your public-facing services — the harder and more expensive exit becomes. By the time the contract is up for renewal, the practical cost of leaving may far exceed the cost of staying on unfavourable terms.
The APS procurement framework requires consideration of supplier dependency risk. But that guidance was written before AI-specific lock-in patterns were well understood. The practical test is simple: ask the vendor, in writing, what it would cost in time and money for your agency to migrate to a competitor. Their answer — or their reluctance to answer — tells you a great deal about the relationship you are entering.
Build an exit strategy into every AI contract from day one. Negotiate for data portability, model portability, and a transition assistance clause that requires the vendor to actively support migration if the relationship ends. A vendor confident in their product’s value will agree to this without hesitation. One that resists should prompt questions about why.
Liability Clauses Are Written for the Vendor, Not for You
Most AI vendor contracts contain liability limitations that are entirely standard from a commercial software perspective and entirely inadequate from a government service delivery perspective. Understanding the gap between those two positions is essential before you sign.
The typical structure is a liability cap set at the value of fees paid in the preceding twelve months, combined with broad exclusions for indirect loss, consequential loss, and loss of data. In practice, this means that if an AI system your agency relies on produces a significant error — say, incorrect benefit assessments processed at scale, or a security breach exposing citizen records — your financial recovery from the vendor may be a fraction of the actual harm caused.
Government agencies carry obligations that commercial customers do not. You cannot simply disclaim responsibility for service failures to citizens in the way a vendor can disclaim responsibility to you. That asymmetry needs to be reflected in the contract terms you negotiate.
Ask specifically about indemnification for AI-specific failure modes: model hallucination, biased outputs, and system unavailability during critical service windows. These are not hypothetical risks — they are documented patterns in deployed AI systems. If the vendor won’t negotiate meaningful accountability for these scenarios, that is a material risk your agency is absorbing without compensation.
What Good Governance Looks Like in Practice
Signing the contract is not the end of the executive’s responsibility — it’s the beginning of an ongoing oversight obligation. AI systems are not static software. They change, sometimes in ways the vendor may not proactively disclose. Model updates, changes to underlying infrastructure, and shifts in data handling policy can all affect what you thought you agreed to.
Build three things into the governance structure from the outset. First, require the vendor to notify you in writing before making any material change to the model, the data handling arrangements, or the terms of service — and specify that material changes require your agency’s written consent before taking effect.
Second, establish regular performance reviews that go beyond uptime metrics. For AI systems, governance reviews should include accuracy audits, bias assessments, and confirmation that the system is still performing as intended against the original use case. Systems that drift from their intended performance are a compliance risk, not just a quality problem.
Third, ensure there is a named executive accountable within your agency for the performance of each significant AI system. Diffuse accountability in AI deployments is one of the most consistent patterns preceding high-profile failures. Someone needs to own it — and that ownership should be visible, documented, and reviewed.
The Questions You Should Ask Before You Sign
The goal here is not to make SES officers into AI lawyers. It is to ensure that the right questions are being asked before commitments are made that are difficult to undo. The following are worth putting to any AI vendor, in writing, before negotiations conclude.
What data does the vendor collect, how long is it retained, and under what circumstances can it be used to train or improve models? What are the explicit data residency commitments, and are they contractually binding rather than policy-based? What would a migration to a competing platform cost, and will the vendor provide transition assistance? What are the liability limits for AI-specific failure modes, and are they negotiable? How will the agency be notified of material changes to the system or the contract terms?
A vendor that answers these questions clearly and without resistance is a vendor worth doing business with. The answers themselves matter — but so does the quality of the engagement. Transparency before the contract is signed is a reasonable proxy for the transparency you will experience throughout the relationship.
AI procurement is not fundamentally different from any other high-stakes government procurement. The principles are the same: understand what you are buying, protect the public interest, and ensure you can exit if the relationship stops working. What is different is that the failure modes are newer, less well understood, and often embedded in contract language written by vendors with far more AI experience than most government agencies currently have. Closing that gap is the work — and it starts well before the signature page.
The views expressed in this article are those of the author in a personal capacity and do not represent the views of any Australian Government agency, employer, or client. Data Mastery operates independently and is not affiliated with any government agency.