Any tool that gets used for everything is overhyped. Right now that's AI and Databricks. A few years ago it was something else, and in a few years it'll be something new.
The pattern is the same every time. A tool gets popular, and people reach for it before they've properly looked at the problem they're trying to solve. Then they spend months bending it into something it isn't good at. Most of the platforms I've inherited had at least one of these: a tool chosen because it was new, now doing a job it wasn't designed for, with a layer of workarounds on top.
The engineers I'd most want on a team are usually the ones drawn to new technology, and I don't want to lose that curiosity. But a new tool never arrives on its own. It brings new failure modes, new deployment patterns, new things to monitor, and a smaller pool of people you can hire to run it. Sometimes that's worth it, because the tool lets you do something you genuinely couldn't before. Often it isn't.
Two questions
Before adopting anything, I ask what problem it solves that we can't already solve with what we have. Then I ask who's going to look after it in two years.
If the honest answer to the first is "nothing much, but it's nicer", that isn't enough. Adopting a tool takes a week. Living with it takes years. That's why I usually pick the boring option. Boring tools are ones the organisation already knows how to carry. You can hire for them, new starters can learn them, and when they break, someone on the internet has already hit the same problem.
The same goes for your own code
The second question applies to everything you build, not only what you buy. I once rebuilt a data pipeline that had grown into a single 18,000-line Python script. It did everything in one place: ingestion, transformation, validation, loading, error handling and retries. When something went wrong, finding where was a project in itself.
I replaced it with twenty scripts, each doing one job. Each one was almost boringly simple. You could read any of them in a few minutes, see exactly which stage had failed, and fix it without touching the others. New engineers were contributing within days instead of weeks.
Nobody writes an 18,000-line script on purpose. It builds up because people are trying to help. A function handles one more case. A script gets another branch because the deadline is tomorrow. Every one of those decisions made sense at the time, and the cost was paid later by someone else: a new starter, whoever was on call at midnight, or the original author six months on with no memory of why.
A platform only one person understands is a risk, however well it runs. I've inherited a few. At The National College, the whole job started from one.
So whether it's a tool or a function, the question I ask is the same: will this make sense to someone who wasn't here when it was chosen? Future you is a stakeholder. So is whoever replaces you.