Damon McMillan’s latest research explores the complexities of context engineering—the specialized art of organizing information so AI can process it efficiently—specifically focusing on how Large Language Models handle massive SQL databases. By conducting nearly 10,000 experiments, the study evaluates how various models interact with complex schemas containing up to 10,000 tables, using SQL code generation as a proxy for sophisticated programmatic agent operations.

The findings reveal a stark performance divide between leading frontier models and their open-weight counterparts. Top-tier systems like Claude Opus 4.5, GPT-5.2, and Gemini 2.5 Pro demonstrated a superior ability to navigate filesystem-based context and structured data. In contrast, open-source models such as Llama 4 and DeepSeek V3.2 showed less convincing results, struggling to maintain reliability when tasked with complex coding agent loops involving large external files.

One of the most notable insights from the paper is the discovery of a 'grep tax' associated with specialized data formats. While Token-Oriented Object Notation (TOON) was designed to represent data using as few tokens as possible, its lack of familiarity to the models backfired. Instead of saving resources, the models spent significantly more tokens over multiple iterations trying to decipher the format’s structure. This indicates that for modern agentic systems, standard formats like Markdown or YAML remain more effective due to the models' extensive pre-training on those structures.

Damon McMillan’s latest research explores the complexities of context engineering—the specialized art of organizing information so AI can process it efficiently—specifically focusing on how Large Language Models handle massive SQL databases. By conducting nearly 10,000 experiments, the study evaluates how various models interact with complex schemas containing up to 10,000 tables, using SQL code generation as a proxy for sophisticated programmatic agent operations.

The findings reveal a stark performance divide between leading frontier models and their open-weight counterparts. Top-tier systems like Claude Opus 4.5, GPT-5.2, and Gemini 2.5 Pro demonstrated a superior ability to navigate filesystem-based context and structured data. In contrast, open-source models such as Llama 4 and DeepSeek V3.2 showed less convincing results, struggling to maintain reliability when tasked with complex coding agent loops involving large external files.

One of the most notable insights from the paper is the discovery of a 'grep tax' associated with specialized data formats. While Token-Oriented Object Notation (TOON) was designed to represent data using as few tokens as possible, its lack of familiarity to the models backfired. Instead of saving resources, the models spent significantly more tokens over multiple iterations trying to decipher the format’s structure. This indicates that for modern agentic systems, standard formats like Markdown or YAML remain more effective due to the models' extensive pre-training on those structures.