The launch party is over, the project team has moved to the next initiative, and the AI system now answers real customers. Three months later it starts giving odd answers, the vendor emails about a model shutdown, and the monthly bill has jumped. Someone has to deal with it. Often nobody has been told it is them.

In our last post we looked at why live AI does not always pay off. This one covers the less glamorous question that sits underneath: who operates the system, and what do they need to do every week to keep it working?

An AI system is not a normal application

A conventional application changes when you change it. An AI system changes when anything around it changes: the model behind the API, the data it reads, the prompts that steer it, the users who find new ways to ask questions, and the price per request.

A widely cited academic study offers clear evidence. Researchers asked GPT-4 the same question set in March and June 2023 and found its accuracy on identifying prime numbers fell from about 84% to about 51%, with no change on the customer’s side. The authors concluded that services built on large models need continuous monitoring. The study is older, but the lesson has only become more relevant as more systems depend on hosted models.

Name two owners, not one

Every live AI system needs a named business owner and a named technical owner.

• The business owner is accountable for whether the system is still doing its job: the accuracy that is acceptable, the cases it must never handle alone, and the call to switch it off.
• The technical owner is accountable for uptime, releases, monitoring, vendor changes, security patches and cost.

Write both names into the runbook with a deputy for each. The NIST AI Risk Management Framework playbook says the same thing in its own words: mechanisms must exist, with responsibilities assigned and understood, to supersede, disengage or deactivate AI systems that perform inconsistently with their intended use. If nobody has the authority to switch it off, nobody really owns it.

What drifts after launch

Plan for five kinds of change.

1. Data. Source documents, product catalogues and policies change. A retrieval system that was right in March quotes a withdrawn policy in October.
2. Model behaviour. Hosted models are updated, and fine-tuned models slowly meet inputs that differ from their training data.
3. Vendor models and APIs. Providers retire models on a schedule. Anthropic, for example, commits to at least 60 days’ notice for publicly released models, and on September 30, 2026 it announced the retirement of Claude Sonnet 4.5 for November 30. The same page shows the sampling parameters temperature, top_p and top_k being removed in API version 2026-10-21, after which requests that set them return an error. OpenAI’s deprecation page lists GPT-4o and GPT-4.1 for shutdown on January 14, 2027, and says preview models may get as little as about two weeks’ notice. Notice periods differ by vendor and by model type, so someone must read the notices.
4. Prompts and tools. Every edit to a prompt, a retrieval setting or a connected tool is a release. Treat it like one, with version control and testing.
5. Cost per task. Longer prompts, more retries and a newly added agent loop quietly raise the price of each completed task.

Monitor what matters

CPU and uptime graphs are necessary and nowhere near enough. Add four layers:

• Quality: a fixed evaluation set of real, labelled cases, run on a schedule and before every change. This is your regression test.
• Behaviour in production: override and escalation rates, user thumbs-down, refusals, and a weekly sample of outputs read by a person.
• Safety and security: blocked inputs, tool calls outside policy, and any sensitive data in outputs. Our guide to AI agent guardrails covers the controls behind these signals.
• Cost and latency: cost per completed task, not just per token, and response time at the slow end.

Set thresholds in advance. A number that triggers a review is useful. A dashboard nobody checks is not.

Have an incident path for AI failures

AI incidents rarely look like outages. The system is up and confidently wrong. Your runbook should define what counts as an incident (a wrong answer that reached a customer, a policy breach, a cost spike), who is paged, and how to contain it: a kill switch, a fallback to a human queue or to a previous model version, and a way to pull a bad document from the knowledge base.

The NIST playbook lists the pieces a post-deployment monitoring plan should include: user input channels, appeal and override, incident response, recovery, change management and decommissioning. Rehearse one scenario a quarter, such as “the vendor retires our model in 60 days” or “the system quoted the wrong price”.

Set a re-evaluation cadence

Pick a rhythm and put it in the calendar:

• Weekly: review samples, cost and escalation numbers.
• Monthly: re-run the full evaluation set, review vendor notices, check data sources for staleness.
• Quarterly: test at least one alternative or newer model on your evaluation set, review the roadmap, and rehearse an incident.
• On every vendor notice: start migration testing immediately, not in the last month.

If you serve or work in a Canadian federally regulated financial institution, this is becoming an expectation. OSFI’s Guideline E-23 on model risk management covers AI and machine learning models, including third-party models, takes effect on May 1, 2027, and expects defined standards for model monitoring and decommissioning.

What an AI support agreement must add

A standard application maintenance contract covers bug fixes, patching, uptime and a response-time table. Keep all of that. For AI, add:

• Quality targets, not just availability. An agreed evaluation set and a minimum score, reviewed on a schedule.
• Drift and vendor monitoring. Who watches deprecation notices and API changes, and how many days before a retirement the migration plan is due.
• Model and prompt change control. No change to a model, prompt or retrieval setting without a test run and a rollback option.
• Incident definitions for AI failures, with severity levels for wrong or unsafe output, not only downtime.
• Cost reporting and caps. Monthly cost per task, alerts on spikes, and a budget ceiling.
• Data responsibilities. Who maintains the knowledge base and who approves new sources.
• Re-evaluation and reporting. Scheduled model reviews, a written summary each month and a named service owner.
• Exit terms. Documentation, evaluation sets and prompts handed over in a usable form.

If a proposed contract reads like a website maintenance plan, it will not cover the ways your AI fails.

Keep your AI running with 247 Labs

247 Labs builds AI systems and keeps them healthy through DevOps and support services, from monitoring and release management to cost control. If you have AI in production and no clear owner, talk to our team about reviewing one live system: who owns it, what is monitored and what your support terms should say.