Every figure, product name and mechanism the finished picture puts on screen, chased to a primary source. Where a claim is illustrative rather than measured, it is listed under Not checked at the end and is not printed as a number on screen.
NVIDIA’s own local AI post reports over 200 tokens per second for Muse Glimmer, a 30 billion parameter model aimed at always on local agent work, running on a GeForce RTX
This is the figure the video is built around. It is a vendor published number for a named model on a named GPU, not a general claim about all local inference.
NVIDIA’s local AI landing page for developers names the hardware as GeForce RTX, RTX PRO, RTX Spark, DGX Spark and DGX Station, and the software as Ollama, llama.cpp, vLLM, SGLang, TensorRT, PyTorch, ONNX, Windows ML and ComfyUI. It also covers building agents with agentic harnesses and MCP tool connections against a local backend.
NVIDIA states that it worked with vLLM, Ollama, llama.cpp and LM Studio on local deployment of its Nemotron 3.5 Lightning models, in both NVFP4 and GGUF formats.
RTX Spark is a real, announced product: a class of Windows PC built on the GB10 Grace Blackwell superchip, announced at COMPUTEX and covered by NVIDIA on 31 May 2026, with 1 petaflop of AI compute and 128 GB of unified memory.
NVIDIA’s DGX Spark product page states:
Up to 1 PFLOP FP4 (NVIDIA notes this is theoretical FP4 TOPS using sparsity)
The video uses the 128 GB and 200 billion parameter figures together, exactly as NVIDIA pairs them, and immediately separates capacity from speed using the bandwidth figure from the same page.
NVIDIA’s product page states 32 GB GDDR7 on a 512 bit memory interface.
The bandwidth figure of about 1,792 GB/s is the arithmetic of that 512 bit bus with 28 Gbps GDDR7, and is the figure carried consistently across independent coverage of the card. It is listed under Not checked below because NVIDIA’s own page states the bus width and memory type rather than a single bandwidth number.
The contrast the video draws is between two published numbers: about four times the memory on the deskside machine, and roughly a sixth of the bandwidth.
Continuous batching replaces a finished request in a batch with a new one immediately and schedules per iteration rather than waiting for a whole batch to drain, which is what keeps the GPU busy.
Automatic prefix caching reuses the KV cache blocks of a shared prefix across requests. vLLM’s design notes give the motivating case directly: a system prompt that is constant across requests, and multi turn dialogue where each new message builds on all the preceding context.
Speculative decoding has a small draft model propose several tokens which the larger target model then verifies, and vLLM describes it as a way to reduce inter token latency on memory bound workloads, noting that real gains depend on model family, traffic pattern, hardware and sampling settings.
The acceptance rate is not uniform. Structured output, code and instruction following accept a high share of drafted tokens; creative, adversarial and high entropy generation accept fewer. Code editing in particular benefits because much of the original code is reused across successive edits.
The video draws this as an ordering of workloads rather than as a set of measured percentages, because the ordering is what the source supports.