The Reality of DGX Spark: 128GB, So Why Is It Slower Than an RTX 4090?

The DGX Spark is a revolutionary device that can run 100B+ models locally with its 128GB of unified memory, and yet its inference speed is 3x slower than an RTX 4090. The key is "bandwidth, not capacity."

1. What Is the DGX Spark

Hardware Specs

The Magic of Unified Memory

The DGX Spark shares the same 128GB memory pool between CPU and GPU via NVLink-C2C. Conventional GPUs have VRAM separately and exchange data over the PCIe bus, but the DGX Spark removes that bottleneck.

Thanks to this, it can run a 70B model at FP16 (140GB needed) on a single system without quantization. An RTX 4090 cannot run a 70B model on its own.

The Lineup

All products use the same GB10 chip, and connecting two units gives 256GB, enough for a 405B model.

2. Why 128GB Is Slower: The Bandwidth Bottleneck

The Core Formula

Token generation speed (TPS) ≈ memory bandwidth (GB/s) ÷ model size (GB)

Every token generation requires reading the entire model weights from memory. So the read speed (bandwidth) is the speed.

Real Calculation: 8B Model (Q4_K_M, ~4.9GB)

The RTX 4090 is 3.3x faster than the DGX Spark. A 24GB gaming graphics card outpaces a 128GB AI supercomputer.

Why This Happens

Memory bandwidth comparison:

The LPDDR5X the DGX Spark uses is memory specialized for power efficiency. It sacrificed bandwidth to reduce power draw. GDDR6X and HBM, by contrast, focus on performance.

By analogy:

• RTX 4090 = a 4-lane highway (narrow but fast)
• DGX Spark = an 8-lane city avenue (wide but slow because of waiting at lights)

3. Real-World Benchmarks by Model

Small Models (8B and under): RTX 4090 Wins Decisively

Mid-Size Models (13-30B): DGX Spark Catches Up

Large Models (70B+): Only the DGX Spark Can Run Them

NVIDIA Official Benchmark (NVFP4, TRT-LLM)

4. Cases Where the DGX Spark Is Genuinely Useful

DGX Spark Wins

1. Running 70B+ models

• An RTX 4090 cannot run a 70B model on its own
• The DGX Spark runs a 70B model at FP16 with no quantization
• It is a "can it run at all?" problem, so bandwidth does not matter

2. Fine-tuning

• 128GB of memory is essential when fine-tuning a 70B model with QLoRA
• NVIDIA's official benchmark: 5,079 tok/s when QLoRA fine-tuning a 70B model

3. Large context + large model

• 70B model + 128K context = 16-30GB consumed by the KV cache alone
• Handled comfortably on the 128GB unified memory

4. Fully air-gapped security

• Runs a large model locally with no cloud at 128GB
• Ideal for sensitive data in finance, healthcare, and defense

5. Multimodal workloads

• Because CPU and GPU share the same memory, no copy is needed for image/video processing

RTX 4090/5090 Wins

1. Small models of 8B and under

• The RTX 4090 is 3.3x faster (116 vs 35 tps)
• An 8B model fits comfortably within 24GB of VRAM

2. Work that needs fast token generation

• Chatbots, coding assistants, and anything where response speed matters
• The RTX 5090 at 1,792 GB/s is 6.6x faster than the DGX Spark

3. Cost efficiency

• RTX 4090 $1,599 vs DGX Spark $3,999
• For 8B models: 2.5x cheaper and 3.3x faster

4. Gaming plus AI

• The RTX 4090 can do gaming and AI together
• The DGX Spark is AI-only (ARM Linux)

5. Which Device Should You Choose

Decision Table

The Mathematical Criterion

Model size (GB) > VRAM (GB) -> DGX Spark wins (it is about whether it can run at all)
Model size (GB) < VRAM (GB) -> RTX 4090 wins (bandwidth decides the speed)

6. Conclusion

The DGX Spark's 128GB unified memory is a "wide road," and the RTX 4090's 1,008 GB/s GDDR6X is a "highway."

• That it can run a 70B model at 128GB is revolutionary
• But for an 8B model, the RTX 4090 is 3x faster
• The illusion that "more VRAM is faster" overlooks the bottleneck of memory bandwidth

Key lesson: Inference speed is decided not by VRAM capacity but by memory bandwidth. The DGX Spark is "a device that can run larger models," not "a faster device."