Earlier this year, I experimented with using LLMs for data analysis. The result was somewhat disappointing. The findings often felt unreliable, and it took me more time to check, understand, and modify the output than my usual workflow. That was surprising because LLMs look very powerful in other business tasks. I’ve seen our business development team use AI for market research, and product managers use it to draft documents, sketch quick mockups, and brainstorm product flows.
The deeper issue seems to be how data analysis works in practice. We rarely define the problem perfectly at the start. The real question becomes clearer only as we explore the data and iterate. An analyst who knows their own scripts can adapt quickly as things evolve, but with an LLM, you first have to interpret its logic, verify the results, and then adjust through conversation. It ends up feeling less like using a faster tool and more like managing a junior analyst. And if the manager still has to work like the analyst, the overall process doesn’t necessarily get more efficient.
Recently, I think I’ve finally made LLMs work in my daily data-analysis workflow. The key was not just asking better questions, but building the right toolkit and infrastructure around it. In other words, I had to shift from being the analyst myself to becoming the manager of a team of AI analysts.
First, I set up a key-metrics data mart. The obvious benefit is accuracy. It reduces the number of choices the model has to make. For example, it should not have to guess what goes into the denominator of a default rate, whether early repayments are included, or how we define rejection rate. The less obvious, and more important, benefit is that building the data mart forces me to define the problem clearly before delegating.
Second, I gave the model access to scripts written by past analysts. Formal documentation is always incomplete, but a codebase is a living map of how the data is really used. That helps the LLM figure out the relevant tables and column semantics much faster.
Third, I connected it to CLI tools that can interact directly with the data infrastructure: running SQL and Spark jobs, deploying ETL tasks, scheduling jobs, and checking task health, without going through a user interface. UIs are designed for humans, not for models. With those pieces in place, asking the AI to break down metric movements by many segmentations started to produce useful leads. In some cases, it could explore more directions than a human analyst would realistically have time to check manually.
However, there are still obstacles to clear before this workflow can be adopted widely across a team. One obstacle is habit. Many analysts are comfortable with UI-driven workflows and may not be trained or ready to use CLI tools or manage a team of AI analysts. Another challenge is that most analysts are not trained to define the problem up front. Scoping well is hard. It takes practice and discipline, and materializing things like key metrics into ETL tasks can feel like too much overhead when you are under pressure to deliver fast. LLMs plus CLI tooling can help, but that loops back to the first issue: not everyone has the technical depth to understand what the model is doing under the hood. In addition, if there are few visible examples inside the team, people default to what is familiar, which feels fast enough simply because that is how it has always been done.
One learning from this journey is that data analysis can be abstracted into a few reusable skills. One such skill is systematic metric decomposition: breaking a key metric into flows or segments to see where the movement comes from. Another is exploring broadly across many dimensions rather than just a few obvious cuts. These skills can be codified and shared across the team as playbooks. With LLMs, they can be executed repeatedly and consistently, giving analysts a structured head start before applying their own judgment and intuition.
Overall, my experience is that using LLMs for data analysis is less about writing clever prompts and more about redesigning how we work. If we simply ask an LLM to analyze a vaguely defined problem on raw data, the result is often unreliable. But if we invest in the right foundations—clear metrics, codebases, CLI access, and shared playbooks—then LLMs become strong partners that extend our analytical reach.
The future here isn’t analysts being replaced, but analysts moving up a level: designing the system, steering the process, and focusing on judgment calls.