Subquadratic Claims to Shatter Decade‑Old LLM Bottleneck with Sparse‑Attention Model

Miami‑based AI startup Subquadratic emerged from stealth with a bold claim: it has cracked the mathematical bottleneck that has hampered large language models for almost ten years. The company says its new model, SubQ, delivers ↑ 12× the context length of typical transformers while slashing compute.

Subquadratic’s Sparse‑Attention Breakthrough

Traditional LLMs rely on dense attention, multiplying every token against every other token – a quadratic explosion that drives up energy use and latency. SubQ replaces that with a dynamic sparse‑attention scheme that selects only the most relevant token pairs on the fly, a move Subquadratic says preserves meaning without the overhead.

“If you want to summarize a novel, you don’t need to compare every word with every other word,” says co‑founder Alex Whedon.

Independent Benchmarks Validate Speed Claims

Third‑party evaluator Appen ran a series of tests. In a raw speed benchmark SubQ outpaced the best FlashAttention models by ↑ 56×. On the LiveCodeBench coding suite the model posted 89.7%, putting it in line with OpenAI and DeepMind offerings. In a needle‑in‑a‑haystack retrieval test SubQ achieved 98% accuracy with a 12‑million‑token context window, far beyond the Reuters‑reported limits of most commercial LLMs.

The firm reports that running a comparable retrieval workload on Anthropic’s Opus cost roughly $2,600, whereas SubQ completed the same task for about $8, a claim that still awaits broader verification.

Market Reception and Skepticism

Initial reactions ranged from excitement to caution. AI engineer Dan McAteer summed it up on X: “SubQ is either the biggest breakthrough since the Transformer … or it’s AI Theranos.” Subquadratic acknowledges the doubt, noting that early self‑published scores lacked third‑party confirmation.

Appen’s director of generative AI research, Jeanine Sinanan‑Singh, praised the results: “Seeing such dramatic speed gains from a sparse‑attention model is rare, and it could reshape how we approach large‑scale inference.” Yet she added that real‑world testing across diverse workloads remains essential.

SubQ is currently offered to a limited cohort of enterprise customers – over 500 firms have signed up, but access is throttled while the startup scales its infrastructure. The company also disclosed that it bootstrapped SubQ using weights from the open‑source Qwen model, a common practice that some critics argue tempers the claim of a wholly novel architecture.

Future Outlook

CEO Justin Dangel envisions a shift away from traditional transformers: “We hope this sparks a new age of efficiency; building on dense attention will become a legacy approach.” If SubQ’s performance holds up under broader scrutiny, it could lower the barrier for enterprises that need to process massive document sets or run extensive code analyses.

For now, the AI community watches closely, awaiting wider access and more transparent cost data. As Bloomberg notes, any technology that can deliver comparable results at a fraction of the price will quickly attract attention from the most cost‑sensitive players.

Dispatch from Julian Reed (Consumer Electronics Expert).