Two green checks and a bug

11 October 2026

How do you trust anything AI builds? The tools you would normally check it with are AI as well, and this week I watched two of them agree an app was fine when it wasn't. It started with something ordinary, a billing console I put up for a company called Northwind Traders.

Try it: restaged.dev/demos/northwind-billing. Sign in and take a payment.

Last time I showed an agent reaching the company over MCP. This is an ordinary web app reaching the same company instead. Same data, same world, different protocol. And that's the point. Underneath, it isn't talking to the real Auth0, Stripe and Slack, and it isn't talking to mocks either. It's talking to live twins of them that run the real APIs. When you sign in, the token really is checked against a twin of Auth0. When you take a payment, a real Stripe call goes to a twin of Stripe.

The Northwind Billing Console signed in as Hana Sato. A wire log lists each real HTTP call the server made to a twin: a Slack post to #new-orders, Stripe creating a payment intent and a customer, and Auth0 verifying the id_token against JWKS and exchanging the code for tokens. Below it, open orders with Charge and Decline buttons and the matching Slack message. — The console running on live twins. The wire log shows the actual calls behind one payment: Auth0 verifying the token, Stripe taking the charge, Slack posting the receipt.

The obvious question is why not just use the test environments Auth0 and Stripe already hand you. I asked myself the same thing, and I'm still not certain, but a few things pull me towards twins:

It comes up on its own. A test tenant and a test-mode account are each set up by hand, in their own dashboard. A twin stands up from a hostname, so an agent or a test run can get going without a person wiring it up first.
It's one joined-up company. The same customer sits in Auth0, Stripe and Slack at once, with data that lines up across them. The vendors' own sandboxes don't know about each other.
The data is mine to set and throw away. I choose what's in it, run a scenario, and reset. That suits short experiments and prototypes, where a leftover test tenant is just more to tidy up.

That left me with a question I couldn't put down. How do I know it's actually working? Not that it looks right on the screen. That the app and the twins underneath are both doing what I asked, and nothing I didn't.

So this week I spent my time on that question and tried a lot of ways to verify an app like this. Most of them went nowhere, which is how experiments usually go.

Everything is agentic now, so that's the route I took first. On one of these apps I had an AI code review read the source, and an end to end agentic tester drive the running app. Both came back green. The whole time, the app was handing out data it should have kept back. A record that was meant to be members only could be read by someone who was never allowed near it, because the check that locks it sat on the path the screen uses and a second way in never had it.

To be clear, none of this reached production. The app was never deployed, and I found the leak on a copy running on synthetic data, so nothing real was exposed and no one was affected. Being able to break it safely is the point of working this way.

What bothers me is why neither tool caught that. The agentic tester used the app the way a user does. It signed in and worked through the screens. It never prodded at the app like a tester would, let alone like someone trying to get in. The AI review read the code that was there and agreed it made sense. It never asked whether what was there was what I'd actually asked for.

Then I wondered whether a purpose-built security scanner would do better, so I pointed OWASP ZAP at it, no AI in the loop at all. It missed the same bug twice. Left to crawl the app on its own it never found the way in, because the way in isn't a link or a form on any page, and a crawler can only follow what is. So I handed it the exact endpoint and told it to attack it, and it ran its full set of checks, more than a hundred of them. The private record came straight back in the response, and it still raised nothing. A scanner looks for attack signatures. It has no idea who is meant to see what, so a broken access-control gap like this goes straight past it.

Two of them were AI, reviewing the code and driving the tests, which is AI checking AI the whole way down. The scanner wasn't AI and still had no answer, and I'm not sure a person would have caught this one either.

What I'd like from you

I've got some ideas, but I'd rather ask you straight:

How do you trust AI to check your work when it's AI all the way down?
What makes you sure a person would do any better at the review?
When every check comes back green, what do you actually trust?

Find me on LinkedIn.