The shift we are seeing in analytics is not technical. It is organisational.

For years, organisations have invested heavily in dashboards, semantic layers, and modern data platforms. And yet, the same question keeps coming back from the business:

“Why does it still take days to get a simple answer?”

At first glance, it feels like a tooling problem. It isn’t.

It is a consumption problem.

Data teams have become incredibly good at building pipelines, modelling data, and optimising performance. But business users don’t need pipelines. They don’t need models. They need answers. More importantly, they need decisions.

And dashboards, as powerful as they are, sit awkwardly in the middle. They are designed upfront, based on assumptions of what the business might ask. But the reality is that business questions evolve daily. By the time a dashboard is built, it is already slightly outdated.

This is where AI/BI Genie changes the model as it doesn’t replace dashboards, it removes the dependency on people to interpret data.

Once the business context is clear, the conversation naturally shifts to value.

Why should we invest in this?

And this is where many initiatives struggle, not because the value is not there, but because it is not clearly articulated.

There is always a hard ROI story: reducing manual data requests, cutting down time spent on ad hoc analysis, meeting SLAs, potentially even reducing the cost of legacy BI tooling.

But often, the more important story is the soft ROI.

• What happens when hundreds of people across the organisation can actually access and use data?
• What is the value of making decisions earlier rather than later?
• What is the impact of reducing friction across every function, not just within the data team?

This is where Genie becomes more than a tool. It becomes a business accelerator.
That said, one thing must be made explicit from the beginning: expectations.

Genie is powerful, but it is not magic. You need alignment on accuracy, latency, and consistency. Without that, adoption will struggle and trust will erode before it has a chance to build.

From an implementation perspective, Genie is only as good as the context and instructions you give it.

And this is where organisations consistently underestimate the effort.

There is an implicit assumption that:

• AI will understand the business
• Metrics are obvious
• Data will speak for itself

None of this is true.

Genie needs to be grounded in explicit business logic with clearly defined dimensions, agreed measures, consistent filters, and, critically, well-structured SQL that reflects how the business actually operates.

If that layer is missing, Genie does not fail loudly. It fails quietly by giving answers that look right, but are wrong.

And in enterprise environments, that is far more dangerous.

Even with the right logic in place, there is still a fundamental gap in most implementations.

Business users don’t just want to know that revenue is £2M. They want to know whether that is good or bad.

That requires benchmarking comparisons to last week, last year, targets, or industry baselines. This is what transforms data into insight.

But even benchmarking is not enough.

The real question becomes:

How do you know Genie is giving the right answer?

This is where evaluation frameworks (evals) come in.

Evals are the discipline of validating Genie outputs. They operate at multiple levels. At the most basic, they ensure the SQL generated is correct, that filters, joins, and aggregations align with agreed definitions. At a deeper level, they validate business logic ensuring that calculations such as growth rates or time comparisons are meaningful and consistent.

And at the highest level, they assess whether the response actually supports decision-making.

Because there is a fundamental difference between:

“Revenue increased by 5%”
and
“Revenue increased by 5% versus last week, driven by EMEA, but still 8% below target.”

One is information.
The other is insight.

In the early stages, this requires a human-in-the-loop approach with analysts validating outputs, refining logic, and building trust. Over time, this evolves into a structured evaluation framework that ensures consistency at scale.

All of this sits on top of one critical foundation: governance.

Genie relies on governed data, typically through Unity Catalog, to ensure that:

• Sensitive data is protected
• Definitions are consistent
• Lineage is traceable

If this foundation is weak, Genie will not fix it.
It will simply expose the issues faster.

When Genie implementations fail, it is rarely because of AI.

They fail because:

• Metrics are inconsistent
• Business logic is undocumented
• Data quality is unreliable
• Ownership is unclear

AI doesn’t create these problems. It makes them visible.

It’s tempting to see Genie as the next evolution of BI.

It isn’t.

This is an operating model shift.

From a world of:
Requests → Tickets → Dashboards

To a world of:
Questions → Answers → Decisions

And that shift requires more than technology. It requires clarity in how the business defines, governs, and uses data.

If you are investing in Genie, don’t start with tools or interfaces.

Start with:

• Business definitions
• Metric ownership
• Benchmarking logic
• Governance

Because ultimately: The quality of your AI will always reflect the quality of your business thinking.