InsightEngine with NVIDIA

What InsightEngine is

Unveiled in October 2024 and available on NVIDIA DGX systems since GTC in March 2025, VAST InsightEngine is described as the “industry's first secure full-stack system for real-time data inferencing.” It is pre-integrated with NVIDIA AI Enterprise and ships DGX-validated, expanding to NVIDIA-Certified Systems - turning the VAST platform into a real-time RAG engine rather than a stack of parts you wire together yourself.

See the site's RAG & Vector Search, Inference & Serving, KV Cache, and NVIDIA Infrastructure pages for the building blocks InsightEngine assembles into one product.

“Certified” wording varies across announcements; presented as DGX-validated and expanding to NVIDIA-Certified Systems.

The NVIDIA hardware & software stack

InsightEngine is a layered system spanning NVIDIA silicon and software: BlueField-3 DPUs power the NVMe enclosures and run storage services as offloaded containers, Spectrum-X carries the RDMA fabric, cuVS does GPU-accelerated vector search, and NIM microservices plus the AI-Q and VSS blueprints sit on top. Follow a write and a question through the layers, or tap a layer to see what it does.

GPUDirect Storage: feeding GPUs without the CPU detour

On the legacy path, every read is copied into host system memory before a second copy lands it in GPU memory. GPUDirect Storage DMAs data straight from NVMe into GPU memory over RDMA, bypassing the CPU/system-memory copy entirely via NFS-over-RDMA and NVIDIA Magnum IO (the general mechanism is explained on Networking & the Data Path). Toggle the two paths to see VAST's numbers.

Real-time RAG pipeline: embed-on-write → index → retrieve

Instead of a nightly batch job, InsightEngine turns every write into a retrieval-ready record: NIM embedding agents fire on write to create vectors and graph relationships, the index lives next to the source data, and GPU-accelerated cuVS search serves it back at inference time. For documents that start outside VAST, SyncEngine is the step before embed-on-write: it copies them onto the platform and keeps them in sync, so the index is only as fresh as the last sync.

KV-cache offload to fast storage

For long-context and agentic inference, VAST serves as an external KV-cache tier. Inactive KV cache is evacuated from scarce GPU memory through tiers - GPU device memory → host RAM → local NVMe → external VAST pool, reached over NFS over RDMA, GPUDirect Storage, S3, or S3 over RDMA (in the next VAST AI OS release) - and paged back in across turns instead of being recomputed. With NVIDIA Dynamo using VAST as that tier, time to first token fell from 62 s to 3 s in benchmarks (Dec 2025). The Context Memory page works through when reloading beats recomputing and what a cache hit is worth in $/token; NVIDIA STX & CMX covers the BlueField-4 tier. Step through a busy GPU below to watch the KV state move.

Why this matters for AI factories

The throughput of an AI factory is increasingly set by the data layer, not the GPU. InsightEngine attacks all three failure modes at once: stale retrieval, idle GPUs, and repeated prefill.

Together these keep the most expensive hardware in the building serving tokens instead of waiting - the same throughput logic that runs through the NVIDIA Infrastructure, Inference & Serving, KV Cache, and RAG & Vector Search pages, applied end to end.

Key takeaways