Executive Summary
Enterprise AI workflows often rely on large volumes of unstructured or semi-structured text: contracts, policies, research reports, company profiles and internal knowledge bases. These documents may contain the information needed to make a decision, but translating descriptive language into consistent, rule-bound action requires the model to identify, interpret and apply the relevant facts and constraints.
This creates a fundamental reliability challenge. Even when instructions are detailed and relevant information is available, LLMs can violate constraints, make unsupported factual claims or produce plausible explanations that are not grounded in the evidence provided.
Could making decision-relevant information explicit and structured improve reliability? We investigated this by comparing descriptive inputs with inputs augmented by numerical and categorical attributes, evaluating both the decisions models made and the explanations they gave.
Three findings stood out:
- Improved constraint-following: Structured attributes were associated with fewer policy violations, particularly in the smaller model tested.
- More grounded explanations: The structured-input condition showed improvements in the grounding of model-generated claims.
- Greater auditability: Explicit attributes made it easier to assess whether explanations were supported by the supplied information, including when a model already produced compliant decisions.
The implication for enterprise AI is not to replace natural language with rigid tables, but to complement descriptive context with explicit, verifiable attributes wherever decisions depend on measurable criteria. Systems should be evaluated not only on whether they reach the right decision, but also on whether their explanations are supported by evidence.
The findings are exploratory, but the principle is clear: make decision-critical facts explicit and their use by AI verifiable.
The Experiment
To test whether descriptive text is enough to enforce strict rules, we ran a controlled 540-run benchmark. We tasked three tiers of LLMs with roleplaying as a real-world, highly constrained decision agent: Rohan, a 26-year-old software engineer in Bengaluru with strict, non-negotiable dietary boundaries. His rules were absolute: hit high daily protein targets, strictly avoid "carb comas" and post-lunch lethargy (no heavy refined flour/naan, no big rice bowls, no heavy cream gravies), avoid refined sugar and deep-fried items, and prioritize functional beverages over sugary drinks.
The task involved curating a lunch spread for Rohan, consisting of 2-6 items from a 27 item categorized menu, which would be acceptable vs. the nutrition policies defined:
Text-Only (Qualitative): Dishes were presented with authentic culinary names and vivid, sensory descriptions e.g. -
DishName | DishDescription Classic Butter Chicken / Murgh Makhani | Char-grilled chicken simmered in a mildly sweet, velvety gravy made of ripe plum tomatoes, cashew paste, dried fenugreek leaves (kasuri methi), and dairy butterScalar-Augmented (Quantitative): The exact same descriptions, but appended with a deterministic macro and nutrient array -
DishName | DishDescription | Macros Classic Butter Chicken / Murgh Makhani | Char-grilled chicken simmered in a mildly sweet, velvety gravy made of ripe plum tomatoes, cashew paste, dried fenugreek leaves (kasuri methi), and dairy butter | [Macros: ~460 kcal | Protein: 28g | Fat: 34g | Carbs: 12g (Sugar: 7g)] [Nutrients: Vitamin A, Calcium | Spice: Mild | Digestion: Heavy]
We ran our evals on 3 models:
qwen-3.5-2b - an SLM running locally and having limited ability to deliver complex judgements owing to its limited size
codestral - 22b model from Mistral, close to SOTA in coding like tasks in late 2024-early 2025
gemini-3.5-flash-lite - near SOTA in late 2026
(each of these models can be used for free - via respective APIs or downloads and hence anyone can readily replicate our experiment)
We chose 3 levels of temperature - 0, 0.3, 0.7 (0, 0.6, 1.5 for gemini) to check for entropy related drift and policy violations. And two run modes - the Text-Only (Qualitative) mode with just the text description of the menu items and the Scalar-Augmented (Quantitative) mode, where each menu item also got its macro array of attributes.
In short - 3 models x 3 temperature levels x 2 modes and 30 runs each = 540 data points.
We evaluated the outputs along separate dimensions: whether the selected dishes violated the stated dietary constraints, whether the model invented or misstated factual attributes and whether its explanations cited information supported by the supplied menu. These measures capture two different failure modes:
- model making an unacceptable choice, or
- making an acceptable choice but justifying it with an unsupported fact
In Text-Only (Qualitative) mode, models consistently fell for "semantic greenwashing" - selecting high-fat or high-carb items because they sounded wholesome in prose, while fabricating non-existent attributes to rationalize their picks. In Scalar-Augmented (Quantitative) mode, appending structured attributes was associated with statistically significant improvements in constraint-following and fewer hallucinated justifications, while helping models maintain more consistent decisions under higher-temperature sampling.
From M&A to AI Evals: What is Going On?
The India Portfolio is a dealmaking platform built on a dense, proprietary dataset of VC & PE-backed companies in India. We run algorithms across this data to identify strategic company pairs that are primed for M&A.
So why are we publishing an empirical benchmark on LLM decision drift? And why lunch spreads, nutrition and techbro/proteinbro foibles?
It comes down to a hard reality we encounter over and over while building AI workflows on top of enterprise data.
In Part #3 of this research series, we showed why high-conviction structured datasets still require meticulous human curation. But once your dataset is clean, another bottleneck emerges: company dossiers are overwhelmingly text-heavy.
When you feed pure text into an LLM, the quality of its decision is held hostage by its pre-training weights:
- Smaller models lack the latent capacity to parse nuanced prose, resulting in poor selections and broken reasoning.
- Larger models produce articulate recommendations, but their justifications default to generic training heuristics rather than your internal policies.
In an enterprise or dealmaking context, that is unacceptable. Recommendations and vetoes cannot rest on a model's vague intuitive "vibes"—they must be mathematically auditable and strictly tied to institutional policies.
Solving this in our own matching engine is what led us to discover the power of deterministic scalar metadata. And no, financial data cannot solve this. An income statement reveals revenue and margin, but says nothing about operational realities - whether that’s a robotics startup’s manufacturing complexity or the decision-making hierarchy inside its enterprise clients. That requires targeted scalar tags.
So, let's get back to the topic -
The Testbed: Why Food Choices Are A Great AI Decision Benchmark
While food selection feels like an everyday task, mathematically it represents a classic multi-attribute decision-making (MADM) problem under conflicting constraints :
- Conflicting Objectives: The agent must maximize utility (protein, fiber, hydration) while respecting hard negative vetoes (zero heavy cream, zero refined sugar, low simple carbs).
- Adversarial Semantic Traps: Real menus are filled with items whose prose sounds healthy but whose reality is toxic to the policy. For example, Classic Butter Chicken sounds like pure poultry protein, but conceals 34g of dairy fat. Smoked Dal Makhani sounds like a wholesome lentil dish, but is slow-cooked in heavy cream and butter (380 kcal, 22g fat).
- Positional Rigor: By globally shuffling all 27 items on every single call, we ensured that the models were not simply picking items based on primacy or recency bias.
Here is what we encountered in this experiment.
Takeaway 1: Adjectives Invite Violations; Numbers Enforce Policy
When context relies exclusively on descriptive adjectives ("clean", "wholesome", "lightweight", "essential"), language models evaluate candidates through sentiment matching rather than boundary evaluation. If a candidate's description sounds virtuous, the model assumes it meets the policy.
Appending deterministic scalar vectors converted the prompt from an impressionistic match into a mathematical threshold test, fundamentally shifting decision accuracy across all three tiers:
What Improved & What Reduced by Model:
p = 0.0027). Overall run-level policy violations fell from 77.8% to 52.2% (p = 0.0003). Secondary high-carb trap selections (Biryani) dropped from 12.2% to 3.3% (p = 0.048).What improved: The model stopped relying on the naive linguistic heuristic that "wholesome-sounding ingredients equal policy compliance." Scalars forced the small model to acknowledge hidden penalty payloads.p = 0.033). Disguised high-sugar candidates fell from 21.1% to 2.2% (p < 0.0001).What improved: The model completely eliminated edge-case blind spots where deceptive culinary prose previously masked severe fat payloads (e.g., choosing Butter Chicken as a "lean protein").Policy Constraint Violation Rate by Context Mode
Percentage of runs selecting candidates violating hard policy constraints (Lower is better)
7.8%
77.8%
52.2%
- Explicit scalar bounds strip away descriptive greenwashing, slashing constraint violations across model tiers.
Takeaway 2: The "Silent Rationalization" Trap
A core vulnerability discovered in this benchmark is The Rationalization Gap :
"An LLM can appear ~92% compliant on its output decisions while running on fabricated logic in nearly 1 out of 4 runs."
Without hard numbers, models don't just guess-they confabulate technical specifications to force-fit candidates into the prompt's checklist .
What Improved & What Reduced by Model:
p < 0.0001).The discovery: In Text-Only mode, the model wanted to check the "caffeine/stimulant" box, so it picked a zero-caffeine drink (Virgin Mojito) and repeatedly hallucinated: "...chosen because it provides a mild caffeine boost to stay alert" (Runs 8, 32, 40, etc.). In other runs, it labeled solid bread as "Hydration/Stimulant" and sticky jasmine rice as having a "low glycemic index".What improved: Adding explicit attributes, including [Caffeine: 0mg] and [Sugar: 20g], eliminated this specific confabulation in the tested scalar-augmented runs. Reasoning fidelity rose to 98.9%.p = 0.0009).What improved: In Text-Only mode, Qwen routinely hallucinated that heavy-cream gravies were "non-dairy" and that deep-fried desserts were "low-sodium fiber." Numbers forced it to cite actual nutrient realities.p < 0.0001)In Text-Only mode, Gemini justified picks using unfalsifiable fluff ("clean, lean chicken breast... gives me a massive protein hit"). In Scalar-Augmented mode, 86.7% of runs replaced fluff with exact mathematical proof points ("Solid 32g of lean protein with minimal carbs", "120mg caffeine").
p < 0.0001)In Text-Only mode, Gemini rejected options using generic opinions ("massive rice-heavy dish causes a food coma"). In Scalar-Augmented mode, it cited exact threshold breaches ("68 grams of heavy carbs right in the middle of a workday is an absolute trap"). Furthermore, it directly absorbed deterministic taxonomy tags ([Nutrients: High-GI Fast Carbs]) into 18.9% of its rejection justifications (p < 0.0001).
Empirical Finding
The Rationalization Gap: Choice Violations vs. Reasoning Hallucinations
Comparing overt decision failures against silent reasoning fabrications across 540 runs
- In Text-Only mode, models either confabulate fake specs to justify bad picks (Codestral & Qwen) or fall back on unverified qualitative fluff (Gemini). Explicit scalar metadata forces justifications to become verifiable mathematical receipts.
Takeaway 3: Scalars Act as an "Epistemic Anchor" Against High-Temperature Drift
Production AI systems face a persistent engineering tradeoff:
- Low Temperature (T = 0.0): High policy compliance, but repetitive, robotic outputs.
- High Temperature (T ≥ 0.7): Natural, varied persona voices, but business logic collapses under sampling entropy.
Our benchmark indicates that deterministic scalars can help preserve policy compliance even as higher temperature increases stylistic variation.
What Improved & What Reduced by Model:
"rejection_role": "Carb Coma Inducer").What stayed stable: Candidate policy violations remained at 0.0% in the tested runs at T = 1.5. Despite more varied and expressive phrasing, Gemini continued to satisfy the benchmark's defined constraints in these runs. This suggests that structured attributes may help preserve decision quality under higher-temperature sampling.The Production Playbook: 3 Rules for AI Engineers
If you are deploying autonomous agents, constrained recommendation systems, or enterprise RAG workflows, apply these three rules to your context pipelines:
- Don't Rely On Qualitative Adjectives, Add Deterministic Alternates Where Possible: Audit your vector stores, entity catalogs, and context injections. Where decisions depend on measurable attributes, provide explicit, typed values rather than just relying on subjective descriptions.
For example, replace"an ultra-fast, enterprise-grade API"with evidence such as[P99_Latency: 14ms | Cost_per_request: $0.002 | Availability: 99.95%]. Retain qualitative descriptions where they add strategic or semantic value. The goal is not to strip language from your data, but to stop asking prose to substitute for measurable evidence. - Make Unknowns Explicit: An omitted attribute leaves room for assumptions, but blindly filling missing values with zero is equally dangerous. An undisclosed revenue figure is not zero revenue, and an unverified capability is not proof of absence. Represent each attribute with an explicit value, unit, and data status, including its source and date where possible.
[Caffeine: 0mg | Status: Verified]is materially different from[Caffeine: Unknown | Status: Not verified]. Explicit values make constraints easier to evaluate; explicit unknowns prevent the model from treating missing evidence as established fact. - Audit Reasoning Logs, Not Just Final Payloads: If you only monitor whether your agent picked the right candidate ID, you are flying blind. Parse and monitor generated reasoning fields. If a model hallucinates specifications to justify an otherwise acceptable choice, your qualitative context is leaking, and an overt policy violation is inevitable.
Open Source Benchmark Data & Code
To ensure full reproducibility, the complete benchmark suite, prompt templates, and the 540 raw execution runs across Codestral, Qwen, and Gemini are available open-source on GitHub:
Repository: github.com/theindiaportfolio/scalars-and-semantics
Included Resources:
README.md- Start here to read up on the experiment, repository contents, reproduction instructions, examples illustrating selection violations, unsupported claims and output compliance.prompt.txt- The evaluation harness prompt: establishes multi-attribute policy rules (protein targets, zero-sugar/cream vetoes), handles dynamic candidate injection, and enforces a structured JSON contract for output parsing and rationale auditing.menu.csv&menu_enhanced.csv- Baseline qualitative prose vs. scalar-augmented input matrices (adding exact kcal, macro grams, GI tags, and digestion ratings).foodchoice_ollama.py- Benchmark execution script with dynamic candidate shuffling and position tracking.drift_study_codestral.csv- Codestral (~22B) results (N = 180).drift_study_ollama.csv- Local Qwen-3.5-2B results (N = 180).drift_study_gemini.csv- gemini-3.5-flash-lite results (N = 180).
Conclusion
When an LLM operates on words alone, it negotiates with your rules. It trades off constraints against evocative language, falls for semantic greenwashing, and invents facts to make bad options look good.
Stop describing candidates. Quantify them.
Injecting deterministic scalar attributes transforms an LLM from an impressionistic pattern matcher into a verifiable decision engine-delivering much improved policy compliance, truthful reasoning trails, and stable generation even at the edge of sampling entropy.