Where AI ideas go to die (and how to stop it)

I want to share my experience of the AI leadership conversations I have, recurringly, with clients. I sit in meetings where one side of the table wants to move faster. The other wants a set of policies signed off before anyone touches anything. Unfortunately, both are right, and that's where the problem seems to sit.

Lock AI down too tightly and the usage doesn't stop. It moves into personal accounts on private devices, where you can't see or measure it at all. The dreaded 'Shadow AI'. Blocking organisational AI usage doesn't remove the risk. That has a price: It creates depth charges: IBM's 2025 Cost of a Data Breach Report found that organisations with high levels of shadow AI paid around $670,000 (roughly £500,000) more per breach, and that 63% of organisations had no policies in place to manage AI or prevent shadow AI.

Open it up completely and you get the opposite problem: fifty enthusiastic pilots, no shared learning, and an unexpected bill or, more worryingly, a data incident somewhere along the way.

The answer isn't a better compromise. Governance and experimentation should grow together, with the rules scaling up as the stakes do. You don't have to choose between them.

Enthusiasm without governance scales risk, but fear without experimentation stops learning.

Govern by consequence, not by enthusiasm

I'm a rugby fan (Bath, since you ask; other clubs are available) and I'm in awe of one of their approaches when developing a new set-piece move. The coaches let players try all sorts of things on the training pitch, the promising ones get a run-out in a pre-season friendly, and only the moves that work under pressure make it into the match-day playbook, with every player drilled on their role and the calls nailed down. AI adoption can work the same way. The approach we use with clients splits AI work into three lanes, and the rules get stricter as an idea moves through them.

1. Sandbox: explore freely

Use approved tools only, with no client or personal data and no connection to live systems. Inside that boundary, let people play. At this stage, the goal is learning, not ROI, so keep the rules to one page. If people have to ask permission to try something here, your sandbox might well be too small.

2. Pilot: prove it

When an idea shows promise, it moves up a lane. Now it touches a real process with a small group of users, and light governance comes in: a named owner, a data review (anonymising sensitive data first often keeps the insight without the exposure), a human checking the outputs, and one or two success measures agreed upfront.

Ronny Kohavi, the former experimentation leader at Microsoft, Amazon and Airbnb, has shown that at Microsoft only around one-third of ideas actually improved the metrics they were designed to improve. So, most ideas should stop here, and that's a good outcome. A pilot that fails cheaply has done its job, leaving you free to develop the ones that push forward into production.

3. Production: scale it properly

Only proven use cases get this far, and this is where full governance earns its keep: security, monitoring, cost controls, integration with the systems you already run, and clear accountability when something goes wrong. It's also where most organisations find that the best answer combines well-built conventional software with AI deployed only where it adds real value. It isn't "AI everywhere!".

If you work in financial services or another regulated sector, much of this lane will feel familiar. The obligations you already have around operational risk, outsourcing and data protection apply here too, so you don't need to build a separate governance regime just for AI.

Why this works

It gives those essential, pioneering enthusiasts permission and your wary risk owners confidence. The question changes from "can we use AI?" to "which lane is this in?", which is far easier to answer. And it builds a pipeline. Instead of one big bet, you get a steady flow of small experiments, and the strongest rise to the top on evidence, not on whoever argued loudest.

It also tackles a blind spot. A breach is easy to picture and easy to put in a board paper. The cost of standing still isn't: slower decisions, signals in your data that nobody acted on, competitors quietly pulling ahead. It never appears on a risk register, but it's just as real. The better question isn't "is AI safe?" It's "is our whole approach, including the risk of standing still, one we can live with?"

Our view at NewOrbit is that governance shouldn't be a gate at the start of the road. It should be guardrails that get firmer and faster the further you go.

Our top tip – to get started this week

Define your sandbox: pick the tools, set the data rules and write the one page. Then tell your team they're free to experiment inside it. You'll be surprised how quickly the good ideas come through and how comfortable you can make the governance team, all at once.

If you're working out how to move your best AI experiments into production without losing control, I'd love to talk it through. Send me a message or book a call, and we can work out which of your ideas belong in which lane.