Local LLM (llama.cpp / Ollama) benchmarks

Tokens a second generating text from an open model run locally with llama.cpp or a tool built on it (Ollama, LM Studio). Generation speed follows memory bandwidth; what fits follows memory size.

5 leaderboards · tok/s, higher is better

Llama 2 7B (MLPerf Client 0.5), Generation

5 products · tok/s · main source Tom's Hardware

Sources and dates

RX 9070 XT: Tom's Hardware, Mar 5, 2025 (Ryzen 7 9800X3D, 32GB DDR5-6000, ASRock X670E Taichi (Tom's Hardware 2025 GPU testbed))
RX 9070: Tom's Hardware, Mar 5, 2025 (Ryzen 7 9800X3D, 32GB DDR5-6000, ASRock X670E Taichi (Tom's Hardware 2025 GPU testbed))
Arc B580: Tom's Hardware, Dec 13, 2024 (Tom's Hardware GPU test PC (the review used a Ryzen 7 9800X3D and a Core i9-13900K system))
Arc B570: Tom's Hardware, Jan 16, 2025 (Ryzen 7 9800X3D, 32GB DDR5-6000, ASRock X670E Taichi (Tom's Hardware 2025 GPU testbed))
RX 9060 XT 16GB: Tom's Hardware, Jun 10, 2025 (Ryzen 7 9800X3D, 32GB DDR5-6000, ASRock X670E Taichi (Tom's Hardware 2025 GPU testbed))

Main source: Tom's Hardware.

tg128 Llama 2 7B F16, Generation

4 products · tok/s · main source llama.cpp community benchmarks (GitHub)

Sources and dates

Mac Studio M5 Ultra: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M5 Ultra, 80-core GPU, 1228GB/s (configure-to-order))
Mac Studio M5 Max: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M5 Max, 40-core GPU, 614GB/s (configure-to-order))
MacBook Pro 14 M5 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5 Pro, 16-core GPU, 307GB/s (run on a MacBook Pro 14, 15-core CPU, 24GB))
iMac M4: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M4, 10-core GPU, 120GB/s (four-port models))

Main source: llama.cpp community benchmarks (GitHub).

pp512 Llama 2 7B Q4_0, Prompt processing

24 products · tok/s · main source llama.cpp community benchmarks (GitHub)

Sources and dates

RTX 5090: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RTX 5080: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RTX 5070 Ti: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RTX 5070: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RX 9070 XT: llama.cpp community benchmarks (GitHub), Oct 19, 2026
Mac Studio M5 Ultra: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M5 Ultra, 80-core GPU, 1228GB/s (configure-to-order))
RTX 5060 Ti 16GB: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RTX 5060: llama.cpp community benchmarks (GitHub), Jun 19, 2026
Mac Studio M5 Max: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M5 Max, 40-core GPU, 614GB/s (configure-to-order))
MacBook Pro 14 M5 Max: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5 Max, 40-core GPU, 614GB/s (the 40-core option; the table has no 32-core figure))
MacBook Pro 16 M5 Max: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5 Max, 40-core GPU, 614GB/s (the 40-core option; the table has no 32-core figure))
RX 9070: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RX 9060 XT 16GB: llama.cpp community benchmarks (GitHub), Oct 19, 2026
MacBook Pro 16 M5 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5 Pro, 20-core GPU, 307GB/s (the table lists by chip))
Mac mini M5 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M5 Pro, 16-core GPU, 307GB/s)
MacBook Pro 14 M5 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5 Pro, 16-core GPU, 307GB/s (run on a MacBook Pro 14, 15-core CPU, 24GB))
Mac mini M6: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Mac mini M6, 12-core GPU, 32GB (170GB/s), macOS 27.0.1)
Arc B570: llama.cpp community benchmarks (GitHub), Oct 19, 2026
MacBook Air 15 M5: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5, 10-core GPU, 153GB/s (run on a MacBook Air, 16GB))
MacBook Pro 14 M5: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5, 10-core GPU, 153GB/s (the table lists by chip; this run was on a MacBook Air))
Arc B580: llama.cpp community benchmarks (GitHub), Oct 19, 2026
Mac mini M4 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M4 Pro, 16-core GPU, 273GB/s)
Mac mini M4: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M4, 10-core GPU, 120GB/s)
iMac M4: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M4, 10-core GPU, 120GB/s (four-port models))

Main source: llama.cpp community benchmarks (GitHub).

tg128 Llama 2 7B Q4_0, Generation

24 products · tok/s · main source llama.cpp community benchmarks (GitHub)

Sources and dates

RTX 5090: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RTX 5080: llama.cpp community benchmarks (GitHub), Oct 19, 2026
Mac Studio M5 Ultra: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M5 Ultra, 80-core GPU, 1228GB/s (configure-to-order))
RTX 5070 Ti: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RX 9070 XT: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RTX 5070: llama.cpp community benchmarks (GitHub), Oct 19, 2026
Mac Studio M5 Max: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M5 Max, 40-core GPU, 614GB/s (configure-to-order))
MacBook Pro 14 M5 Max: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5 Max, 40-core GPU, 614GB/s (the 40-core option; the table has no 32-core figure))
MacBook Pro 16 M5 Max: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5 Max, 40-core GPU, 614GB/s (the 40-core option; the table has no 32-core figure))
RX 9070: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RTX 5060: llama.cpp community benchmarks (GitHub), Jun 19, 2026
RTX 5060 Ti 16GB: llama.cpp community benchmarks (GitHub), Oct 19, 2026
RX 9060 XT 16GB: llama.cpp community benchmarks (GitHub), Oct 19, 2026
Arc B580: llama.cpp community benchmarks (GitHub), Oct 19, 2026
Mac mini M5 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M5 Pro, 16-core GPU, 307GB/s)
MacBook Pro 14 M5 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5 Pro, 16-core GPU, 307GB/s (run on a MacBook Pro 14, 15-core CPU, 24GB))
MacBook Pro 16 M5 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5 Pro, 20-core GPU, 307GB/s (the table lists by chip))
Arc B570: llama.cpp community benchmarks (GitHub), Oct 19, 2026
Mac mini M4 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M4 Pro, 16-core GPU, 273GB/s)
Mac mini M6: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Mac mini M6, 12-core GPU, 32GB (170GB/s), macOS 27.0.1)
MacBook Air 15 M5: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5, 10-core GPU, 153GB/s (run on a MacBook Air, 16GB))
MacBook Pro 14 M5: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5, 10-core GPU, 153GB/s (the table lists by chip; this run was on a MacBook Air))
Mac mini M4: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M4, 10-core GPU, 120GB/s)
iMac M4: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M4, 10-core GPU, 120GB/s (four-port models))

Main source: llama.cpp community benchmarks (GitHub).

tg128 Llama 2 7B Q8_0, Generation

8 products · tok/s · main source llama.cpp community benchmarks (GitHub)

Sources and dates

Mac Studio M5 Max: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M5 Max, 40-core GPU, 614GB/s (configure-to-order))
Mac mini M5 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M5 Pro, 16-core GPU, 307GB/s)
MacBook Pro 14 M5 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5 Pro, 16-core GPU, 307GB/s (run on a MacBook Pro 14, 15-core CPU, 24GB))
Mac mini M4 Pro: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M4 Pro, 16-core GPU, 273GB/s)
Mac mini M6: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Mac mini M6, 12-core GPU, 32GB (170GB/s), macOS 27.0.1)
MacBook Air 15 M5: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (Apple M5, 10-core GPU, 153GB/s (run on a MacBook Air, 16GB))
Mac mini M4: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M4, 10-core GPU, 120GB/s)
iMac M4: llama.cpp community benchmarks (GitHub), Oct 19, 2026 (M4, 10-core GPU, 120GB/s (four-port models))

Main source: llama.cpp community benchmarks (GitHub).