Weka’s Augmented Memory Grid Slashes GPU Demand, Caches All Pre‑Calculated Tokens

Weka’s latest AI storage offering hinges on its Augmented Memory Grid, a flash‑based layer that behaves like GPU memory while costing a fraction of the price. By caching every pre‑calculated token, the system eliminates the repetitive “prefill” step that traditionally drains GPU capacity during multi‑turn chats.

How the Augmented Memory Grid cuts GPU waste

Long context windows force large language models to recompute attention for each turn. Weka’s approach stores those attention results in high‑speed NAND, so subsequent prompts retrieve them instantly, freeing up ↑ 30% of GPU cycles for new queries.

NeuralMesh 6 adds composable clusters and unified storage

The platform delivers virtual multi‑tenancy over an RDMA fabric, supporting up to ↓ 20% latency compared with conventional S3 gateways, and offers a single data view for both file and object workloads—no duplicate copies.

“We now allow our customers to grab some allocation of new GPUs and get up and running within an hour,” Liran Zvibel, co‑founder and CEO, told Reuters.

Weka targets non‑AWS GPU clouds such as Lambda, Nebius, G42 and CoreWeave, promising performance gains that rival on‑prem solutions while charging by capacity rather than per‑API call.

Enterprises deploying internal copilots, customer‑service bots, or code‑generation assistants stand to benefit most, as their workloads often hit the GPU memory ceiling. Smaller teams may see modest gains until their usage scales.

For a broader view of how AI‑driven storage reshapes compute strategy, see the recent analysis of the pandemic‑era surge in remote‑work data demands.

Analysis by Dr. Aris Thorne (Artificial Intelligence Researcher).