If you have spent any time lurking in /r/LocalLLaMA or browsing Hugging Face over the last year, you have almost certainly felt the itch. You see a shiny new open-weights model—maybe Llama 3 70B, DeepSeek-V3, or Qwen 2.5—and you immediately start calculating hardware costs. "If I buy two used RTX 3090s, can I offload enough layers to hit 30 tokens per second? Or do I need to shell out for a Mac Studio with 128GB of Unified Memory?"
Up until now, answering that question required digging through complex mathematical formulas, cross-referencing memory bandwidth specs, guessing quantization loss, or relying on anecdotal benchmark posts that used totally different inference backends than the one you plan to run. Buying hardware for local AI has felt a bit like buying a car without a test drive.
Enter Llmspeed, a project recently showcased on Hacker News that solves this exact problem in a surprisingly visceral way. It doesn't just calculate predicted numbers; it lets you feel the generation speed directly in your browser or terminal before you spend a single dollar on GPUs or unified memory. Today, we are taking a deep dive into how Llmspeed works, the hardware physics governing local LLM performance, and how you can use these principles to spec your next local AI rig.
What is Llmspeed and Why Does It Matter?
At its core, Llmspeed is a simulator for Large Language Model inference performance. Instead of forcing you to set up vLLM, Ollama, or llama.cpp on theoretical infrastructure, Llmspeed takes the architectural parameters of a target model (parameter count, quantization level, context length) and the hardware specifications of a target system (memory bandwidth, compute TFLOPS, interconnect speeds) to simulate text generation in real time.
When you run the tool, it streams text to your console or browser at the exact token-per-second rate that the specified hardware configuration would produce in reality. Seeing "12.4 tokens/sec" written in a static table is one thing; watching text stream out at 12.4 tokens/sec gives you an immediate, intuitive sense of whether a setup is viable for interactive chat, code completion, or background agentic tasks.
The Physics of LLM Inference: Memory Bandwidth vs. Compute
To understand why a simulator like Llmspeed is accurate—and why local LLM hardware specs confuse so many developers—we need to talk about the mechanics of Transformer inference. LLM execution splits into two distinct phases, each bottlenecked by entirely different hardware components:
1. The Prefill Phase (Prompt Processing)
When you feed a prompt into an LLM, the model processes all input tokens concurrently to build the initial Key-Value (KV) cache. This phase is heavily compute-bound (bound by Tensor TFLOPS). The GPU or accelerator needs to execute massive matrix multiplications across all prompt tokens simultaneously.
2. The Autoregressive Phase (Token Generation)
Once the prompt is processed, the model generates output text one single token at a time. To predict