Previous post: What Claude told me after looking at my career..
Ever since then, I’ve found myself turning the career question over again.
[ An era where AI drives the cost of development toward zero ]
(Supply) My core offering: “full-stack development, AI included.”
(Supply) My pitch as a candidate: “a DS who can actually build.”
(Demand) The market, meanwhile, is handing the work DS used to own over to other teams.
So what does it take to be the kind of DS — the kind of AI specialist, the kind of leader — who survives?
[ Taking stock of my weaknesses and strengths ]
- Strength: engineering
This one I’ve proven well enough. From standing systems up to running them, I’ve been across the whole range.
- The crux: product thinking
I’m fine at defining a problem and driving it forward — what’s missing is the step where I validate whether this plan is actually the right one.
Here are the three methods I’ve typically leaned on:
-
Interviews (though they’re full of bias, and there seems to be a gap between what people actually use and what they say they need. And lately I’ve come to feel that since it’s the people doing the hands-on work — not the people above them — who end up using the thing, it’s those practitioners I need to be interviewing.)
-
Log-data analysis of the relevant systems (the logging may have been built for a different purpose, and you only ever see what’s inside the existing screens)
-
Benchmarking (looking at comparable AX project case studies, conference talks, and so on)
Where I could do better:
-
In interviews, watch what people actually do, not just what they say
-
In log analysis, look beyond the system — at the Excel files, chat messages, emails, and manual workarounds happening outside it
-
Always, without exception, write the success metric down as a number
-
Weakness: measurement and iteration
There was very little of the measure-then-improve cycle — confirming the incremental lift a model actually delivered. In other words: I built the thing, and that was the end of it.
A/B tests, a framework for measuring incremental lift, properly constructed treatment and control groups — these are non-negotiable.
[ What does the AI job market look like right now? ]
In short:
-
DS postings that are purely domain modeling are getting more specialist (five years of nothing but recommender systems, five years of nothing but ad performance measurement…), while ML Engineer and AI Engineer postings — which bundle ops and development into the same role — are on the rise
-
DA roles, the measurement/experimentation/causality work, hold steady
-
Postings centered on LLM applications, evaluation, and AX have grown noticeably.
The executive who owned IT (AI) at my company got pushed out, the AI project went on hold, and I watched it drift with nowhere to land. What that showed me is this: AX isn’t where a company makes money — it’s where a company spends it. Which has me thinking hard about wanting to work somewhere closer to the algorithms that actually cut costs or bring in revenue, somewhere the output is unambiguous and arrives fast.
So yes, there are plenty of in-house AX postings — but it may just be a fashion.
Digital transformation, DT teams… the positions everyone was hiring for three years ago have all vanished. Because nobody could answer the question of how much they actually helped.
And with in-house AX platforms, proving impact turns out to be nearly impossible.
“136 hours of work time reclaimed per month!” — how do you prove you actually reclaimed it? In the end it comes down to “work got a bit easier” and “it’s fine, I guess.” (And honestly, ChatGPT and Gemini have raised expectations so high that it’s hard to keep up…)
So part of me simply feels this: I don’t want to build services that could just as easily not exist.
I want to fight to build the thing somebody genuinely needs — then get feedback on it, and fix it.
So what?
Fashions come and go, and there’s no right answer, so maybe the most correct thing is to stick it out doing what I actually want to do.
-
Own problems that produce real results, and solve them
-
Measure the real impact (KPIs)
(+) And if I can put my past AX experience to work on internal process automation too, all the better.
댓글