WEKA and Oracle Cloud Benchmark 10x AI Inference Gains

The AI data infrastructure company WEKA and Oracle Cloud Infrastructure (OCI) have validated production-scale benchmarks that demonstrate significant performance improvements for long-context AI inference. Utilizing a nine-node OCI bare-metal H100 cluster, the partnership implemented WEKA's NeuralMesh platform with Augmented Memory Grid to overcome GPU memory limitations.

The benchmarks revealed that the system achieved 10x more concurrent users and 10x higher token throughput compared to DRAM-only configurations, scaling past 5,000 concurrent users and reaching approximately two million tokens per second. This architecture decouples the key-value cache from local GPU memory by storing it in a high-performance token warehouse, which reduces the cost per token for agentic workflows.

NeuralMesh with Augmented Memory Grid is now generally available to WEKA customers and through the Oracle Marketplace. OCI serves as the exclusive cloud launch partner for the technology, which aims to improve AI economics by eliminating the memory bottlenecks that typically limit the scale of inference workloads.