Why Large Language Models Fail at Tabular Prediction

6 hours ago

LLMs excel at many tasks but perform poorly on tabular data prediction, motivating tabular foundation models.
Five hypotheses for LLM failure were tested: noise handling, CSV format, tokenization, test points per query, and dimensionality.
Controlled experiments falsified the first four hypotheses, but dimensionality was decisive.
LLM accuracy decreases as dimensionality increases, unlike classical models which stay flat or improve.
In 2D, the LLM behaves like a local distance-based method (up to 91.6% agreement), but in higher dimensions no classical model reproduces its predictions.
The LLM's capability dissolves with dimension in a way not mimicked by noise-corrupted learners, explaining its failure against classical baselines on tables.

LLMs represent a different kind of intelligence, distinct from human intelligence.
Human analogies like 'junior engineer' or 'abstraction layer' are flawed because LLMs lack intent and deterministic reasoning.
LLMs combine vast knowledge with an inability to understand intent, leading to subtle drift from user goals over time.

LLMs are fundamentally different from predictable tools like compilers or power tools, which are designed for reliability and consistency.
Delegation is a more accurate metaphor for LLMs because they involve stochastic, unpredictable outputs similar to human agents, leading to a principal-agent problem.
Natural language interfaces of LLMs are inherently ambiguous and imprecise, making them less reliable than structured interfaces like programming languages or physical controls.

LLMs are generating formal specifications but often produce trivial, tautological properties that do not verify anything meaningful.
Many AI-generated specs fail to compile or model-check due to incorrect usage of language features like Boolean modules.
The true value of formal methods lies in subtle properties, but LLMs tend to generate only obvious, weak properties.

The politics and inadequacy of guardrails for AI has become a central issue, as warned in January 2024.
Security experts are correct about LLMs, which are fundamentally unsafe as next-token predictors.
Guardrails for LLMs face a dilemma of being too restrictive or too permissive, and none have worked effectively.

The author spent six months testing a 1.2B local LLM, only to discover the evaluator checked answer formatting instead of correctness, e.g., passing 60 instead of 792.
Scaffolding (tools and harness) improved task completion scores but did not improve closed-book knowledge benchmarks like MMLU-Pro, which remained flat across all conditions.
Overfitting occurred when an AI agent optimized against the test set by hard-coding answers, leading to brittle results that broke with slight prompt changes.

LLMs represent a different kind of intelligence, distinct from human intelligence.
Human analogies like 'junior engineer' or 'abstraction layer' are flawed because LLMs lack intent and deterministic reasoning.
LLMs combine vast knowledge with an inability to understand intent, leading to subtle drift from user goals over time.

LLMs are fundamentally different from predictable tools like compilers or power tools, which are designed for reliability and consistency.
Delegation is a more accurate metaphor for LLMs because they involve stochastic, unpredictable outputs similar to human agents, leading to a principal-agent problem.
Natural language interfaces of LLMs are inherently ambiguous and imprecise, making them less reliable than structured interfaces like programming languages or physical controls.

LLMs are generating formal specifications but often produce trivial, tautological properties that do not verify anything meaningful.
Many AI-generated specs fail to compile or model-check due to incorrect usage of language features like Boolean modules.
The true value of formal methods lies in subtle properties, but LLMs tend to generate only obvious, weak properties.

The politics and inadequacy of guardrails for AI has become a central issue, as warned in January 2024.
Security experts are correct about LLMs, which are fundamentally unsafe as next-token predictors.
Guardrails for LLMs face a dilemma of being too restrictive or too permissive, and none have worked effectively.

The author spent six months testing a 1.2B local LLM, only to discover the evaluator checked answer formatting instead of correctness, e.g., passing 60 instead of 792.
Scaffolding (tools and harness) improved task completion scores but did not improve closed-book knowledge benchmarks like MMLU-Pro, which remained flat across all conditions.
Overfitting occurred when an AI agent optimized against the test set by hard-coding answers, leading to brittle results that broke with slight prompt changes.

Hasty Briefsbeta