Tag: Sustainability

and no, not the carbon kind

Whether the AI system you shipped last quarter will still be running next year without being rebuilt from scratch.

When people write about sustainability and AI, they usually mean electricity. Datacentre load, water consumption, the emissions cost of a training run.

That’s a real conversation. It isn’t this one.

I want to talk about a different kind of sustainability: whether the AI system your team shipped last quarter will still be running next year without being rebuilt from scratch.

Because most of them won’t be.

Software decays. AI systems expire.

Traditional software ages gracefully. Dependencies drift, documentation goes stale, the odd library hits end-of-life — but the thing keeps running. Decay is slow, visible and negotiable.

AI systems don’t work like that. They fail discontinuously.

A model gets deprecated and your application stops working on a date set by someone else’s roadmap. A provider updates a checkpoint and your carefully tuned prompt quietly starts producing different output — not worse in any way that throws an error, just different enough to matter.

And occasionally the model simply disappears. In June, three days after Anthropic released Claude Fable 5, a US export control directive forced it offline for every customer worldwide. It came back on 1 July once the controls were lifted. Nineteen days is survivable. The point is that nobody in the dependency chain chose it — not the customers, not the vendor — and almost no architecture review had that failure mode on the list.

The uncomfortable truth is that most AI systems being built right now are built against a specific model, a specific prompt and a specific moment. They aren’t built to last. They’re built to demo.

Sustainable AI engineering is the discipline of building systems that survive the thing underneath them changing.

Here’s what that actually requires.

1. Your foundation has an expiry date

Cloud infrastructure gave us a decade of stability. An EC2 instance you provisioned in 2016 behaves the same way today. That stability quietly shaped how we build.

Foundation models offer nothing of the sort. The useful life of a frontier model is somewhere between six and eighteen months, and the decision to retire it isn’t yours.

This means model substitution has to be a design assumption, not a migration project. If swapping the model behind a feature requires a sprint of prompt rewriting and manual spot-checking, you don’t have a system. You have a dependency you can’t move.

The June suspension was the dramatic version of this problem. The mundane version — a deprecation notice with ninety days on it — will hit far more teams, far more often, and get far less attention.

2. Evals are the new regression suite

We have thirty years of collective muscle memory around testing deterministic software. Write the test, get a pass or a fail, gate the deploy.

None of that transfers cleanly. The same input produces different outputs. Correctness is often a judgement rather than an assertion. So a lot of teams have quietly stopped testing and started eyeballing.

Which means they have no way to answer the only question that matters when something changes: did that break anything?

The tempting shortcut is to reach for a similarity score and call it a test. That doesn’t hold either — I’ve written before about why BERTScore and cosine similarity aren’t enough, and the short version is that they will happily pass an answer that is fluent, plausible and wrong.

Without a real evaluation suite, every model upgrade is a leap of faith, every prompt tweak is unverified, and every regression is discovered by a customer. Eval debt behaves like technical debt, except it compounds faster and you can’t see the balance.

The teams building sustainably treat evals as infrastructure — versioned, automated, run in CI, and owned by someone.

3. Prompts and context are production code

In a lot of organisations, the prompt that drives a customer-facing feature lives in a Confluence page, a notebook cell, or a string literal someone edited on a Friday afternoon.

This is config on somebody’s laptop, wearing a new hat. We solved this problem once already and appear determined to relearn it.

Prompts, retrieval configurations, tool definitions and context assembly logic are the highest-leverage, highest-volatility parts of an AI system. They change behaviour more than the code around them. They deserve version control, code review, change history and a rollback path.

This is also the part that makes incidents survivable. As I covered in the AWS governance playbook, every interaction should be traceable back to the system prompt version, model version, guardrail version and knowledge source version that produced it. Without that, an investigation becomes archaeology.

If you can’t answer “what changed, when, and who approved it” for a prompt, you cannot operate that system responsibly — and you certainly can’t debug it six months from now.

4. Every AI feature needs a failure mode

Ask most teams what their AI feature does when the model is rate-limited, degraded, timing out or unavailable, and the honest answer is: it fails.

That’s not a resiliency posture. That’s a single point of failure with a language model attached.

Sustainable systems have somewhere to go when the primary path is unavailable:

• A fallback model, ideally from a different provider
• A deterministic path that’s worse but functional
• Graceful degradation — narrower scope, cached results, reduced capability
• A clean handoff to a human
• Circuit breakers, so a degraded model doesn’t quietly poison a thousand downstream decisions

We wouldn’t ship a payments integration with no retry logic and no fallback. We’re shipping AI features that way constantly, and the reason is that the failure modes are new enough that nobody’s asked the question yet.

5. Cost is a runtime property, not a line item

Traditional software has broadly fixed unit economics. AI systems don’t. Cost per interaction is a live variable that moves when the model changes, when the context window grows, when an agent decides to take twelve steps instead of three.

Plenty of AI systems get switched off despite working perfectly. They just cost more per transaction than the transaction is worth, and nobody modelled that until the invoice arrived.

If cost isn’t observable per feature, per user and per interaction — in something closer to real time than a monthly bill — it isn’t controllable. Token anomaly detection is the early warning; connecting that spend to a defensible outcome is what gets the system through its first budget review.

6. A system nobody owns isn’t sustainable

The barrier to building something impressive with AI has collapsed. That’s genuinely good. It also means organisations are accumulating agents, assistants and pipelines built quickly by people who have since moved on.

Who maintains the agent someone built in a weekend? Who gets paged when it starts making bad calls? Who decides whether it still needs to exist?

Ownership is the question that the agent control plane keeps running into from every direction — who owns the tools, the policies, the prompts, the incident response. It’s the least technical item on this list and probably the one that kills the most systems. Engineering rigour is irrelevant if nobody owns the outcome.

The throughline

None of this is exotic. It’s mostly the operational discipline we already apply to everything else, redirected at a substrate that changes far faster than we’re used to.

But notice what these six things have in common: model abstraction, evaluation infrastructure, prompt management, fallback routing, cost observability, ownership. Almost none of them are application concerns. They’re platform concerns — the same argument for a dedicated AI platform team, and the same argument I made about the AI operating model, arriving from a different direction.

Asking every product team to independently invent resiliency is how you end up with none.

The organisations pulling ahead over the next year won’t be the ones that shipped AI features fastest. They’ll be the ones that don’t have to rebuild them.

Is your AI stack built to survive the model underneath it changing — or would a deprecation notice mean starting over?