Intent Mismatch
Intent mismatch is when a user asks a question expecting one meaning, but the model, metric, or query returns a different meaning. The result is often "technically correct" while being practically wrong for the decision being made.
Common failure signals
- Users repeatedly export data to "sanity check" a dashboard
- Two users interpret the same metric differently
- Filters or slicers change meaning in unexpected ways
- "Correct" totals that contradict operational reality
Often confused with
- Data quality issues (data can be correct; meaning is misaligned)
- UX problems (UX can contribute, but intent mismatch is semantic)
- User error (often the system encodes ambiguous meaning)
Where it shows up in Analytical Reliability
- Semantic Reliability: measures/relationships do not match the intended question
- Execution Reliability: performance/concurrency constraints change how users query
- Change Reliability: small logic changes shift meaning without user awareness