Earlier this year I'd have told you AI is a force multiplier. I still think that's true. What I underestimated was how much it multiplies when nobody is checking.

At PEP Health we rebuilt our data warehouse with a much smaller team than we'd planned for. The team had gone from six people to two, and the old platform had been switched off early. AI coding tools were how we closed the gap. They wrote most of the models: staging, the Data Vault, the marts and the tests around them.

In one sense it worked. We had a running warehouse far sooner than two people should have been able to build one.

Then we started checking it.

What we found

The problems weren't the kind that fail a build. There were joins that looked right and dropped most of the rows for one source. Keys were generated two different ways in two different places. A date filter had been copied into a model where it didn't belong, and it capped every report downstream. Each one was plausible on its own, and each one was written confidently, often with a comment explaining why it was correct.

Most of my time since has gone on proving what the warehouse does and rebuilding the parts where it went off-book.

Being honest, I think we'd have reached the same place just as quickly if we'd sat down and written the core ourselves. The AI got us to an MVP faster. The product wasn't as viable as I wanted, and we've been paying back the time we saved at the start ever since.

What I'd do differently

I'd still use it. It's very good at the work around the edges: boilerplate, tests, documentation, migrations, reading code I didn't write, and tracking down why two numbers don't match. It does well at any task where the answer is easy to check.

The core model is different. That's where the business logic lives, and business logic is exactly what an LLM can't see. It doesn't know that two sources spell the same hospital differently, or that a date means one thing in one feed and something else in the next. It makes a reasonable guess and moves on, where a person would stop and ask someone.

So next time, people write the core models and the rules about what the data means. AI helps with everything around that, and anything it writes gets checked against real numbers, not just read.

The bigger worry

I'm sceptical that these tools will keep improving at the rate they have been. I'm more worried about how they're being used right now. A lot of teams treat AI as a blunt instrument, until every screw looks like a nail.

It's a very good tool, and for the right job it's brilliant. Use it for everything and you end up with a lot of work that looks finished and isn't.