Glossary

Atomicity, Consistency, Isolation, Durability - the guarantees that make a database write reliable. Modern lakehouse table formats bring ACID to data lakes.

Trading compute for memory: discard intermediate activations during the forward pass and recompute them in the backward pass.

Post-training 4-bit weight quantization that protects the most salient weight channels (chosen by activation magnitude), preserving accuracy.

An LLM-driven system that plans, calls tools, and acts over multiple steps toward a goal - not just a single chat response.

An agentic deep-research assistant (NVIDIA Blueprint / VAST Foundation Stack). Plans a question into sub-questions, retrieves repeatedly with reflection, then writes a cited report - RAG wrapped in an agent loop.

A columnar in-memory data format (plus Arrow Flight RPC) that moves tabular data between systems with zero serialization overhead. VAST DB's SDK speaks Arrow so query results stream straight into pandas, Spark, or GPU frames.

An open table format that brings warehouse features - schema evolution, time travel, ACID - to files in a data lake.

A distributed engine for large-scale data processing and ETL across a cluster - a workhorse of data pipelines.

A defined interface for software to call a service. Agents use APIs as tools; models are themselves served behind inference APIs.

Algorithms (HNSW, IVF) that find near-closest vectors quickly by trading a little accuracy for huge speed at billion-scale.

The mechanism that lets each token weigh the relevance of every other token. Produces Query, Key, and Value vectors per token.

A queryable audit store built on VAST Database (SQL, Python SDK, or the GUI), alongside per-CNode JSON logs for SIEM. In multi-tenant clusters, tenant managers can query their own audit records.

An encoder-only transformer that reads text in both directions. The basis for many embedding and reranking models used in retrieval.

NVIDIA's DPU product line. BlueField-3 pairs ~400 Gb/s networking with 16 Arm cores to run networking, storage, and security as an on-NIC infrastructure OS, freeing host CPU cores. BlueField-4 (2026, Vera Rubin) doubles networking to 800 Gb/s via ConnectX-9, adds a Grace CPU and PCIe Gen6, and offloads KV-cache / inference context-memory to a petabyte NVMe tier.

Dashboards, reports, and ad-hoc analytics over business data. The traditional consumer of the data warehouse, now sharing the lakehouse with AI workloads.

The dominant tokenization algorithm. Iteratively merges frequent character pairs into subword tokens, balancing vocabulary size against sequence length.

The general-purpose processor that orchestrates a system - data loading, scheduling, and offload - while GPUs do the parallel math.

The center point of a cluster of vectors. Clustered ANN indexes compare a query against centroids first to pick which cluster to search, pruning the rest of the dataset.

Tracking and streaming row-level inserts, updates, and deletes as they happen, so downstream systems stay in sync without full reloads. VAST DataBase's fine-grained mutable chunks make cross-table CDC simple at scale.

Splitting documents into passage-sized pieces before embedding, so retrieval returns focused, relevant context rather than whole files.

A stateless compute node in DASE that serves protocols and runs logic. Because it holds no persistent state, CNodes scale independently of capacity and any CNode can serve any data.

A class of NVIDIA-Certified GPU servers that run the VAST AI Operating System directly on GPU infrastructure - unifying analytics (Sirius), vector search, RAG, and agent runtimes on one platform.

VAST DataBase's fundamental on-flash unit: a ~32 KB columnar chunk carrying its own footer of metadata (projections, sort keys, and min/max + count statistics) in SCM, with no separate metadata manager. Roughly 1/4000th the size of a Parquet row group, so selective queries read one chunk instead of scanning a whole row group.

The information an agent keeps available - short-term (the prompt/context window) and long-term (retrieved from a store) - to stay coherent.

The first rack-scale implementation of NVIDIA STX. A shared tier of Ethernet-attached flash ('G3.5') tuned to hold KV cache close to the GPUs, lowering time-to-first-token for long-context and agentic inference.

The maximum number of tokens a model can attend to at once (e.g. 8K, 128K, 1M). Longer context costs more memory, largely via the KV cache.

Adding and removing requests from a running batch every step instead of waiting for a fixed batch - keeps the GPU busy and lifts throughput.

The heat exchanger that isolates the clean coolant loop over the chips' cold plates from the facility water loop. Mandatory once a rack passes the ~25 kW air-cooling ceiling.

A snapshot/clone technique that writes only changed data and shares the rest by pointer, so the snapshot copies nothing. VAST's V-Tree adds a new root reusing existing subtrees, making snapshots near-free.

A score from -1 to 1 measuring the angle between two embeddings. Closer to 1 means more semantically similar - the usual relevance metric.

The economic KPIs of inference at scale. Once power - not chip price - is the constraint, what matters is dollars per million tokens and tokens generated per watt, not raw FLOPS.

NVIDIA's GPU BLAS library - the dense matrix-multiply (GEMM) engine underneath training and inference.

NVIDIA's parallel-computing platform and programming model. The foundation almost every GPU AI library is built on.

NVIDIA's collection of 400+ domain-specific accelerated libraries (cuDNN, cuBLAS, RAPIDS, NCCL, cuVS…) layered on CUDA. The breadth every framework optimizes for is the real competitive moat.

A GPU DataFrame library (part of RAPIDS) with a pandas-like API. Accelerates data prep and ETL by running it on the GPU.

CUDA Deep Neural Network library - GPU-optimized primitives (convolutions, attention, normalization) used by PyTorch and TensorFlow.

CUDA Vector Search - GPU-accelerated nearest-neighbor index build and search, powering fast vector databases for RAG.

Disaggregated Shared-Everything - VAST's architecture that separates compute from NVMe storage so every node sees all data with no bottleneck.

The tendency of large datasets to pull compute toward them, because data is costly and slow to move. It pushes AI jobs to run where the data lives - or forces expensive copies.

A store for raw data of any type (structured or not) at massive scale and low cost - flexible but easy to turn into a 'data swamp'.

The automated flow that ingests, cleans, transforms, and serves data for analytics or AI. The unglamorous foundation every model depends on.

A programmable network processor (e.g. NVIDIA BlueField) that offloads networking, storage, and security from the CPU.

A store for cleaned, structured data optimized for fast SQL analytics and BI. Reliable and governed but rigid and pricier per TB.

A fault-tolerant storage enclosure (a JBOF) holding the NVMe SSDs and SCM. Each DBox is fronted by two DNodes in an active-active pair, so losing one DNode never takes the box offline.

The memory-bandwidth-bound phase that generates output one token at a time, reading the growing KV cache each step.

Microsoft's training-optimization library, best known for ZeRO sharding and CPU/NVMe offload that let huge models train on limited GPU memory.

NVIDIA's reference GPU server platforms. HGX is the baseboard OEMs build on; DGX is NVIDIA's own fully-integrated system.

NVIDIA-managed GPU clusters rented through hyperscaler clouds - the full NVIDIA stack as a service, with no hardware to own.

NVIDIA's reference data-center design that clusters many DGX systems with InfiniBand into a turnkey AI supercomputer.

A live, physically accurate virtual replica of a real asset (factory, robot, grid) used to test and optimize before acting in the real world.

A simpler alternative to RLHF that optimizes directly on preference pairs with a classification-style loss - no separate reward model or RL loop.

Running prefill and decode on separate GPU pools so each scales independently - the design behind NVIDIA Dynamo.

A storage enclosure (DBox) of NVMe SSDs plus a small Storage Class Memory tier. DNodes hold all the data; CNodes reach every DNode over NVMe-oF, so compute and capacity scale separately.

NVIDIA's software framework for programming BlueField DPUs. DOCA Memos, introduced with STX, handles KV-cache communication and storage between GPUs and the context-memory tier.

Randomizing simulation parameters (lighting, textures, physics) during training so a model generalizes robustly to the messy real world.

The sum of element-wise products of two vectors. Large when vectors both align and are long; the building block of cosine similarity and the negative-inner-product metric.

Main system (host) memory. Larger but far slower than GPU HBM - used to stage data and to offload optimizer state or KV cache.

VAST's foundational data structure - a flat, metadata-rich store where files, objects, table rows, and streams are all 'elements,' enabling true multi-protocol access to one copy of data.

A vector of numbers that represents the meaning of a token, word, or document. Similar meanings land close together in vector space.

The unit of at-rest encryption (AES-XTS-256, a data key wrapped by a key-encryption key). Scoped per cluster, per tenant, or per group; at least one per tenant gives crypto isolation and tenant-scoped crypto-erase - at the cost of cross-tenant data reduction.

Extract-Transform-Load (transform before storing) vs Extract-Load-Transform (load raw, transform later in the warehouse). The core data-movement patterns.

Straight-line distance between two vectors, the square root of the summed squared differences. Sensitive to magnitude, unlike cosine. VAST exposes it in SQL as array_distance().

A weaker model where replicas only converge to the latest value over time, so reads can briefly return stale data. Common in geo-distributed stores; what DataSpace's strict model avoids.

A system that serves consistent, precomputed model inputs ('features') to both training and inference, preventing train/serve skew.

Continuing training of a pretrained model on a smaller, task-specific dataset to specialize its behavior.

Floating-point operations - the unit of compute. Training cost ≈ 6 × params × tokens FLOPs; a GPU's TFLOPS (trillions/sec) sets how fast it clears.

16-bit floating-point formats. BF16 keeps FP32's exponent range with less precision and is the default for training and inference on modern GPUs.

4-bit formats. FP4 is Blackwell's lowest-precision compute mode; NF4 ('normal float 4') is the data type QLoRA uses to freeze a 4-bit base model.

8-bit floating point, accelerated natively on Hopper/Blackwell GPUs. Roughly doubles throughput vs BF16 with careful scaling.

NVIDIA's consumer/workstation GPU line. Capable for local inference and fine-tuning, but lacks the HBM, NVLink, and scale of data-center cards.

The decoder-only transformer family (GPT-3/4) that popularized large-scale next-token pretraining. Most modern LLMs follow its blueprint.

A single, consistently addressable view of data across many clusters and clouds - the foundation of VAST DataSpace, so one file, object, or table is reachable from every site with no copies to sync.

A one-shot post-training method that compresses LLM weights to 3–4 bits with minimal accuracy loss using approximate second-order information.

NVIDIA's roughly yearly datacenter GPU families - Hopper (H100/H200) → Blackwell (B200/GB200) → Blackwell Ultra (B300) → Vera Rubin → Rubin Ultra → Feynman - each adding memory, lower-precision math, and scale.

A direct data path from NVMe/network storage into GPU memory, skipping a CPU bounce buffer - critical for feeding data-hungry GPUs.

NVIDIA's Arm-based server CPU, linked to Blackwell GPUs over NVLink-C2C in the GB200/GB300 superchips so CPU and GPU share fast, coherent memory.

The signal telling each parameter how to change to reduce error. Computed in the backward pass and synchronized across GPUs via NCCL.

The massively parallel processor that runs AI training and inference. Its memory bandwidth and VRAM size are the usual bottlenecks.

An RL method (popularized by DeepSeek) that ranks a group of sampled answers against each other, dropping PPO's value network. Key to reasoning models.

A middle ground: groups of query heads share K/V heads. Used by Llama 2/3 to balance quality and KV-cache size.

Stacked, ultra-fast memory on data-center GPUs (e.g. HBM3e). Its bandwidth (TB/s) sets the ceiling for decode-phase token speed.

Hierarchical Navigable Small World - a graph-based ANN index prized for high recall and low query latency.

A model that interleaves Transformer attention layers with SSM (Mamba) layers. Only the few attention layers grow a KV cache, so it reaches near-Transformer quality at a fraction of the long-context memory.

An immutable, ransomware-proof snapshot whose deletion - and whose policy - require multi-factor challenge-response tokens from VAST Support before expiry. Designed to defeat rogue admins and insider attacks.

A low-latency, high-throughput network fabric used to connect GPU servers across a cluster, typically with RDMA.

The 400 Gb/s-per-port generation of InfiniBand used to wire GPU clusters together with RDMA and very low latency.

8- and 4-bit integer formats for weight quantization. INT4 can shrink a model ~4× versus BF16 for memory-constrained deployment.

The gap between consecutive streamed tokens - essentially TPOT viewed per-step. Steady, low ITL is what makes streaming output read smoothly.

An ANN index that clusters vectors into cells and only searches the nearest few at query time - trading a little recall for a large speed-up over brute force.

A composable, XLA-compiled array framework popular for research and large-scale training, with automatic differentiation and easy hardware acceleration.

A dense, controller-light enclosure of NVMe flash drives exposed over the network (NVMe-oF) rather than fronted by a traditional storage controller. In VAST, a DBox is a fault-tolerant JBOF that CNodes mount directly.

An algorithm that partitions vectors into K clusters around centroids, iterating until they settle. VAST runs it on a sample to build and continuously recluster its vector index.

Stored Key and Value vectors for tokens already processed, so the model doesn't recompute them each step. The dominant memory cost of long-context inference.

Combines a lake's cheap, open storage with a warehouse's reliability (ACID transactions, schemas) via table formats like Iceberg or Delta.

A neural network trained on vast text to predict the next token. Powers chat, code, and reasoning assistants. Most are decoder-only transformers.

VAST's data-protection scheme (e.g. 146+4) that survives drive failures at ~2.7% overhead while rebuilding from only a fraction of the stripe - far cheaper and faster to recover than triple replication.

Freezes the base model and trains tiny low-rank adapter matrices (~0.1–1% of params), making fine-tuning cheap and portable.

A selective state space model that rivals transformers on long sequences with linear-time inference and a constant-size state - no growing KV cache.

NVIDIA's reference implementation of large-scale model parallelism (tensor/pipeline/expert) - the backbone of many frontier training stacks.

NVIDIA's open modular server reference spec - 100+ configurations across CPU/GPU combos and generations - that OEMs build many SKUs from.

A data-skipping technique where each chunk stores the minimum and maximum value of its column, so a selective query reads only chunks whose range can contain the value and skips the rest before any bytes move. VAST DataBase keeps these statistics per ~32 KB chunk for fine-grained pruning.

A sparse architecture where each token is routed to only a few 'expert' sub-networks. Gives large total capacity while activating few params per token.

Apple's array framework for machine learning on Apple Silicon, using the unified memory of M-series chips.

An open standard for connecting agents to tools and data sources through a uniform interface, so the same tool server works across different agent frameworks.

The fraction of a GPU's peak FLOPS actually spent on useful model math. 30–50% is typical at scale; the rest is lost to memory stalls and communication.

Standard attention with many independent heads, each with its own K/V. Highest quality but largest KV cache.

DeepSeek's technique that compresses K/V into a low-rank latent vector, slashing KV-cache memory while keeping MHA-like quality.

The feed-forward sub-layer in each transformer block - two linear layers with a nonlinearity. Holds the bulk of a model's parameters.

All query heads share a single K/V head, shrinking the KV cache dramatically at a small quality cost.

Running many fully-isolated tenants on one shared-everything cluster. Isolation is enforced in software (namespace, identity, network, keys, QoS) rather than by physically partitioning the array - so no stranded capacity and no noisy neighbors.

Block-scaled 4-bit floating-point formats - a shared scale per small block of values - that make FP4 accurate enough for inference on Blackwell.

NVIDIA Collective Communications Library - the optimized all-reduce/all-gather routines that synchronize gradients across many GPUs in training.

The dot product negated, so 'smaller is closer' like a distance. VAST's array_negative_inner_product() ranks magnitude-aware similarity directly in SQL.

A GPU-first specialist cloud (e.g. CoreWeave, Lambda, Crusoe) built on OEM HGX systems - typically far cheaper than hyperscalers but with fewer managed services.

A protocol for fast SSD storage over PCIe. NVMe-over-Fabrics extends it across the network - the basis of disaggregated flash storage.

A reference AI workflow from NVIDIA - runnable sample code built from NIM microservices that developers fork and customize. The starting point VAST Foundation Stacks turn into production pipelines.

NVIDIA's AI-native storage reference architecture (GTC 2026). Inserts a dedicated context-memory tier between GPUs and storage so KV cache can be persisted and reused across a pod instead of recomputed. Built on BlueField-4, Spectrum-X, and Vera Rubin.

A datacenter-scale inference framework that disaggregates prefill and decode and routes requests to where the KV cache already lives.

Prebuilt, optimized models packaged as containers with standard API endpoints - deploy a tuned model anywhere without assembling the serving stack yourself.

An end-to-end framework for pretraining, fine-tuning, and aligning LLMs and multimodal models at scale, built on Megatron and PyTorch.

A platform for building and running physically based 3D simulations and digital twins, connected via the OpenUSD scene format.

Next-gen rack-scale NVLink domains on the Vera Rubin roadmap - NVL144 (144 GPUs) and Rubin Ultra's NVL576 (576 GPUs in one rack), succeeding today's NVL72.

A GB200/GB300 rack of 72 Blackwell GPUs joined by NVLink into one giant GPU - 18 compute trays + 9 switch trays in one NVLink-5 domain.

NVIDIA's high-speed GPU-to-GPU interconnect, far faster than PCIe. Lets many GPUs share data as if one large accelerator.

The switch chip that wires many GPUs into an all-to-all NVLink fabric - the backbone of an NVL72 rack's single NVLink domain.

Universal Scene Description - an open standard for describing, composing, and exchanging complex 3D worlds across tools.

Extra per-parameter data (e.g. Adam's momentum and variance) kept during training - often the largest chunk of training memory (~16 bytes/param).

DataSpace's consistency model: per global folder, one cluster is the Origin holding the authoritative copy and write lease; the others are Satellites caching locally under read leases. A remote write is proxied to the Origin.

vLLM's technique that stores the KV cache in non-contiguous 'pages' like OS virtual memory, eliminating fragmentation so more requests fit per GPU.

Ways to split a training job across GPUs: Data, Tensor, Pipeline, Expert, and Context parallelism - combined as 3D/4D parallelism for frontier models.

A learned weight in the network. Model size (7B, 70B, 405B) counts parameters; more params generally means more capability and more memory.

A columnar file format that compresses well and is fast to scan - the default storage for analytics and lakehouse tables.

The standard bus linking CPUs, GPUs, NICs, and SSDs. Slower than NVLink, so GPU-to-GPU traffic avoids it where possible.

VAST's free, Kubernetes-native control plane for deploying, monitoring, and managing clusters across regions and clouds from one pane of glass - the management substrate of the AI OS.

Quantizing a trained model's weights/activations with little or no retraining. Fast and cheap (GPTQ, AWQ) versus retraining-based QAT.

Sending a query's filters (predicates) and the set of needed columns (projection) down to the storage layer, so only matching rows and requested columns are read - rather than scanning everything and filtering in compute.

The compute-heavy phase that reads the whole prompt in parallel and populates the KV cache before any token is generated.

Reusing the KV cache of a shared prompt prefix (e.g. a system prompt) across requests so it is computed once, not per request.

The expensive first phase: training a model from scratch on trillions of tokens to learn general language and knowledge.

Compresses vectors by splitting them into sub-vectors and encoding each against a small codebook, shrinking the index so billions of vectors fit in memory.

The input text given to a model. In serving it becomes the 'prefill' the model reads before generating a response.

The dominant deep-learning framework for research and production. Defines and trains models with Python and runs them on GPUs via CUDA.

Per-tenant / view / bucket / user performance policy with max caps (containment) and min guarantees (a protected floor), static or capacity-scaled. Enforced by injecting I/O latency in 0.1-second windows - the noisy-neighbor control on a shared cluster.

Storing weights/activations in fewer bits (e.g. INT8, INT4) to cut memory and speed up inference, trading a little accuracy.

Simulating low-precision arithmetic during training so the model learns to tolerate it - higher accuracy than PTQ at the cost of a training run.

LoRA on top of a 4-bit (NF4) frozen base, letting a 65B model fine-tune on a single 48 GB GPU.

Requests handled per second - the throughput axis serving systems trade against latency. Larger batches raise QPS but can hurt TTFT.

Hard and soft limits on capacity and file/inode count, applied per directory, user, group, or tenant (with a grace period and up to 3-level nesting). Counts physical bytes used, so it reflects data reduction.

SGLang's KV-cache reuse scheme that shares cached prefixes across requests via a radix tree - automatic prefix caching for branching prompts.

NVIDIA's suite of GPU-accelerated data-science libraries (cuDF, cuML, cuGraph) for end-to-end analytics without leaving the GPU.

RDMA carried over an Ethernet fabric instead of InfiniBand - the low-latency transport SuperNICs use to move GPU-to-GPU traffic on Ethernet-based AI clusters.

Reason + Act - the agent loop pattern where the model alternates between reasoning steps and tool calls, observing each result before deciding the next move.

The decentralized permissions that keep DataSpace strictly consistent: a Satellite serves cached data under a read lease; a write takes the Origin's write lease, which then invalidates every read lease so no site serves stale data.

For approximate search, the fraction of the true nearest neighbours an index actually returns (e.g. 99% recall) - the accuracy traded away for ANN's speed at scale.

A pre-validated blueprint spanning compute, network, storage, and software - tested for power, thermals, and signal integrity - so an AI cluster goes live in weeks, not months.

Training an agent by trial and error against a reward signal - often run at massive scale in simulation for robotics and control.

Like RLHF, but preferences are labeled by an AI judge instead of humans - scaling alignment data far more cheaply.

Aligning a model to human preferences: train a reward model from human comparisons, then optimize the policy against it (classically with PPO).

Lets one machine read/write another's memory directly over the network, bypassing the CPU and OS - key to fast distributed training and storage.

A second-stage model that re-scores the top retrieved chunks for relevance, sharpening what the LLM finally sees.

Fetching relevant documents at query time and feeding them to an LLM, so answers are grounded in your data instead of just model memory.

A lightweight layer-normalization variant used in Llama-class models - rescales activations by their RMS, with no mean subtraction.

Encodes token position by rotating query/key vectors by an angle proportional to position. Standard in modern LLMs and key to extending context length.

NVIDIA in-network computing that performs all-reduce inside InfiniBand switches, cutting collective-communication time in large training jobs.

A target a service commits to, e.g. 'p99 TTFT under 500 ms.' Serving systems batch and schedule to maximize throughput without breaking the SLO.

A fast LLM serving engine featuring RadixAttention prefix caching and a structured generation language for complex, branching prompts.

A 0-to-1 grade of cluster quality - how tight a cluster is versus how separated from its neighbours. A low score flags a cluster that has drifted and should be reclustered.

Transferring a policy trained in simulation to real hardware. The 'reality gap' is bridged with domain randomization and accurate physics.

Global reduction that finds and encodes byte-level similarity across all data - not just exact-match dedup - shrinking footprints (including embeddings and model data) beyond classic compression.

A roadmap CNode-X compute tier bringing GPU-accelerated SQL (RAPIDS cuDF/cuVS) to VAST DB; VAST reports ~44% query-time reduction, moving analytics from CPU-bound to GPU-bound on the same data.

Restricts each token to attend only to a fixed window of recent tokens, capping KV-cache growth for long sequences (used by Mistral).

Flash-based persistent storage. NVMe SSDs feed datasets and store checkpoints; their aggregate bandwidth gates how fast GPUs can be fed.

A non-attention sequence architecture (e.g. Mamba) that carries a fixed-size recurrent state. Scales linearly with sequence length instead of quadratically.

The base optimization algorithm: nudge weights down the loss gradient on mini-batches. Adam adds momentum and per-parameter scaling on top.

A fast, persistent memory tier that acts as VAST's write buffer: incoming data lands in SCM, is reduced and shaped, then flushed to cheap QLC flash - giving low write latency on bulk-economical media.

A guarantee that once a write commits, every reader anywhere sees the new value immediately - no stale reads. VAST DataSpace provides it across sites via read/write leases.

The standard language for querying and manipulating relational data. Increasingly the interface for analytics over lakehouse and vector data too.

A package joining Grace CPU(s) and Blackwell GPU(s) over NVLink-C2C as one coherent unit. The GB200/GB300 superchip is the building block of the NVL72 rack.

A NIC purpose-built for GPU-to-GPU (east-west) AI traffic. NVIDIA's ConnectX-8 SuperNIC delivers up to 800 Gb/s of RDMA/RoCE with GPUDirect - distinct from a DPU's infrastructure-offload role. ConnectX-9 is the Vera Rubin generation, integrated into the BlueField-4 DPU.

Fine-tuning a base model on curated prompt→response demonstrations so it follows instructions - the first post-training stage before preference alignment.

Machine-generated training data from simulation. Cheap, perfectly labeled, and able to cover rare 'edge cases' real data misses.

An isolated administrative and data domain on a VAST cluster (vastdata_tenant). Each tenant has its own Element Store root, identity provider, encryption keys, quotas, QoS, and network rules; a cluster scales to 10,240 tenants.

A delegated administrator scoped to a single tenant - manages its views, quotas, local identity provider, S3 keys, and tenant roles, but cannot create tenants, make VIP pools, add identity providers, or see cluster hardware. Contrast with Cluster Admin.

Specialized GPU units that do matrix-multiply-accumulate at low precision (FP16/FP8/FP4) - the engine behind AI throughput.

Google's production deep-learning framework with broad deployment tooling. Largely supplanted by PyTorch for research but still widely deployed.

NVIDIA's inference optimizer and runtime that fuses layers, selects fast kernels, and quantizes a trained model into an efficient engine. TensorRT-LLM and NIM build on it.

NVIDIA's compiler/runtime that builds optimized inference engines for LLMs - fused kernels, quantization, in-flight batching.

The sustained heat a chip is designed to dissipate, in watts. Data-center GPU TDP has climbed from ~700 W (H100) toward ~1,400 W (B300), forcing the shift to liquid cooling.

Throughput = tokens/sec across all users; latency = time-to-first-token and per-token speed for one user. Serving tunes the trade-off.

Average time to generate each token after the first, set by the decode phase. TTFT + TPOT × tokens predicts total response time.

Latency from request to the first generated token - dominated by the prefill phase. The key metric for how responsive a stream feels.

The atomic unit a model reads and writes - a word piece, character, or symbol. Text is split into tokens before the model sees it; output is generated one token at a time.

Converting raw text into a sequence of token IDs the model can process. The vocabulary and rules determine how many tokens a given string costs.

Letting a model call external functions, APIs, or code to fetch data or take actions beyond text generation.

The architecture behind nearly all modern LLMs. Stacks of self-attention + feed-forward layers that let every token look at every other token.

NVIDIA's production server for deploying models from any framework with dynamic batching, concurrency, and multi-model hosting.

The wide, shallow metadata tree VAST uses to index the Element Store. Its low depth keeps lookups fast even at exabyte scale, underpinning the platform's flat global namespace.

A runtime for hosting and governing AI agents next to the data, exposing platform tools over MCP so agents act on live data with low latency and central control.

VAST's framing of its stack as one operating system for AI - storage, database, streaming, serverless compute, and agents converging onto a single platform rather than separate products.

A unified data platform combining storage, database, and compute on disaggregated shared-everything (DASE) architecture for AI at scale.

A columnar table format built into the VAST platform - an Iceberg/Delta-style alternative that stores analytical tables natively on VAST storage, with no separate Parquet files or metastore. Mutable and ACID, not an OLTP database.

An event-driven serverless compute fabric that runs functions on data where it lands - triggers, Python functions, and containerized engines - so processing happens in-situ with no ETL copy. GA Nov 2025.

A global namespace spanning edge, on-prem, and every cloud so the same data is addressable everywhere with no copies, using an Origin/Satellite model for strict consistency. Lets training and inference follow GPU capacity.

A Kafka-API-compatible streaming engine built into the platform with stateless brokers - durability comes from VAST storage, and streams land directly as queryable Database tables. GA Mar 2025.

Open-source implementations that turn NVIDIA AI Blueprints into production pipelines on the VAST AI OS - a serverless DataEngine ingest pipeline feeding a unified VastDB store, served by NVIDIA NIMs. Launch stacks cover RAG, AI-Q, and VSS.

VAST's real-time RAG stack co-engineered with NVIDIA (BlueField-3, Spectrum-X, NIM, GPUDirect Storage) that auto-embeds data on ingest for instant, always-fresh retrieval.

Vector search built into the VAST platform, indexing embeddings alongside source data for RAG at massive scale.

A store that indexes embeddings and finds the nearest vectors to a query fast - the retrieval engine behind RAG and semantic search.

Finding items by meaning rather than keywords, by locating the closest embedding vectors to a query vector.

NVIDIA's next-generation platform succeeding Grace Blackwell, pairing the Vera CPU with Rubin GPUs. The compute platform that STX and CMX are designed to feed.

A GPU's on-board memory. Holds the model weights, KV cache, and activations - running out of VRAM is the most common deployment limit.

An NVIDIA Blueprint (and VAST Foundation Stack) that ingests live or archived video, describes each segment with a vision-language model, and makes it searchable and summarizable in natural language.

A group of virtual IPs that CNodes serve protocols through. A pool can be pinned to specific CNodes (cnode_ids) and assigned to a tenant + VLAN - giving that tenant a dedicated serving path while data stays on the shared DNode pool.

A model that takes both images and text as input and reasons over them jointly - e.g. describing a photo or reading a chart. Adds a vision encoder in front of an LLM.

A high-throughput LLM serving engine. Its PagedAttention manages the KV cache like virtual memory to pack many requests onto a GPU.

The RBAC building block in the VAST Management Service: a set of object types plus the allowed actions (create/view/edit/delete) on them. A custom realm can be scoped to one tenant to delegate administration.

A model that learns the dynamics of an environment and predicts future states, letting agents 'imagine' outcomes - e.g. NVIDIA Cosmos.

Techniques that shard model params, gradients, and optimizer state across GPUs so models too big for one card can still train.