DataHub Cloud Context and the work of defining a correct business answer

Public developments and original analysis · Announcement date: 2026-10-28 · Sources

DataHub announced general availability of Cloud Context on October 28, bringing domain-scoped context, recurring answer evaluations and user feedback into one product. For a data leader considering an analytics agent, the most useful idea is the order of work: have business experts define questions and expected answers before deciding whether additional context helps.

That changes what a data-questioning project needs to deliver. Connecting a warehouse and demonstrating a few queries can show that a system works in a demonstration. It cannot show what will happen when the finance team changes a definition next month. The company needs questions it can run again, and someone who can decide whether yesterday’s expected answer still applies.

Who decides the expected answer?

The Context Evals documentation describes questions paired with expected answers and optional rules. An agent responds, and an AI judge compares the result. Checks can run on a schedule or before a context change is published. Their scope is whether the agent uses DataHub context correctly; they do not certify an entire business system. Although Cloud Context has been announced as generally available, the feature documentation still identifies Context Evals as Public Beta at the time of access. Buyers should confirm the applicable release and feature access with the supplier.

Automated judging can reduce the work of inspecting responses. It cannot decide how a company should count revenue. Consider a hypothetical sales question: how much revenue came from new customers last month? Contract value, invoiced value and recognized revenue could each produce valid SQL. Choosing among them depends on whether the requester is studying commercial performance, billing or financial reporting. If the expected answer uses the wrong definition, a passing test makes that error more repeatable.

The business meaning of a question is therefore difficult to delegate entirely to a software supplier. Data teams can explain tables and columns. Finance or sales owners must settle the population, period and exclusions. Leaving those decisions in personal conversations forces the next analyst or model to reconstruct them. Recording the decisions as questions, explanations and queries creates a basis for comparison when tools or models change.

It also explains why a supplier’s accuracy figure is a weak purchasing criterion on its own. DataHub reports customer improvements in its announcement and beta review. The questions, domain, context quality and scoring method determine what those figures mean. They are supplier-reported results, not expected outcomes for another company. A buyer learns more by bringing its own frequent questions and past failures, then asking the supplier to explain each miss.

Maintaining definitions over time

Test questions need maintenance as business practices change. In the hypothetical sales example, suppose the company starts reporting refunds separately. It may preserve the old definition for year-on-year comparisons or use a new definition for current operations. Both can be useful if their applicable periods are clear. Overwriting the old definition can make an earlier report impossible to reproduce; leaving two similarly named definitions unexplained can make selection ambiguous.

DataHub’s feedback guide separates missing, incorrect and conflicting context from user corrections, with owners reviewing whether the feedback is valid. That distinction exposes a real operating cost. A user saying an answer is wrong begins an investigation. The cause may be context, an ambiguous question, delayed data or an overly restrictive expected answer. Collecting feedback automatically helps. Turning every correction into a rule without review can create fresh contradictions.

For the sales example, a better response is to retain the original question and its period, document the refund treatment, explain the relationship between definitions and add questions that distinguish them. Evaluation can expose affected responses. A business owner still needs to decide which interpretation serves the analysis. Otherwise, data teams keep adjusting prompts while business teams continue to distrust the numbers.

For a company considering Mokyun digital employees for this analysis, the preparation should produce specific artifacts: a metric definition approved by its business owner, identifiable sources and questions covering normal and exceptional cases. Mokyun’s knowledge-management direction includes enterprise material and source traceability, providing a basis for designing how definitions and sources are maintained. Query execution, data access and a particular evaluation tool require interface, permission and adaptation checks. Defining the questions and material before choosing data tools or models makes it easier to see where existing tools are sufficient.

Choosing what to evaluate

Evaluations consume operating time and computing resources. A company with a small set of stable metrics already served through controlled reports may be better off keeping those reports. Building an elaborate context platform merely to offer another way to request the same table may add little value. Recurring evaluation becomes more useful when questions change, relevant material spans sources and manual explanation or correction already absorbs substantial effort.

Coverage requires judgment too. A suite containing only common, well-defined questions can achieve an impressive pass rate while missing refunds, cross-period treatment or merged customer accounts. A suite dominated by obscure exceptions can divert effort from everyday work. Separating frequent questions from consequential exceptions, and retaining failure explanations, is more informative than hiding both behind an average. That is an evaluation-design choice, not a business decision DataHub makes for the customer.

The important implication of the release is that enterprise data questioning needs an owner after launch. Definitions change, disputed answers require decisions and historical reports need explanations. If nobody is prepared to approve the expected answers, the sensible response is to narrow the service. Connecting more tables and generating more documentation would enlarge a problem the company has not yet resolved.

• Specialist workflows and information preparation

Mokyun Editorial · neo@mokyun.com