GreenBench: Benchmarking Energy Efficiency and Carbon Footprint of Open-Source LLM Inference on Apple Silicon
GreenBench is a new benchmarking framework evaluating energy efficiency, throughput, and carbon footprint of open-source LLM inference on Apple Silicon, a previously unstudied area in Green AI research. The Apple M4 Pro (48 GB unified memory) draws only 0.47 W of CPU+GPU package power during sustained inference, achieving 30-40x better energy efficiency per token than datacenter GPUs in single-user deployment. Smaller models (3-3.8B parameters) deliver 2.6-4.2x higher throughput and up to 62% le
Analysis
TL;DR
- GreenBench is a new benchmarking framework evaluating energy efficiency, throughput, and carbon footprint of open-source LLM inference on Apple Silicon, a previously unstudied area in Green AI research.
- The Apple M4 Pro (48 GB unified memory) draws only 0.47 W of CPU+GPU package power during sustained inference, achieving 30-40x better energy efficiency per token than datacenter GPUs in single-user deployment.
- Smaller models (3-3.8B parameters) deliver 2.6-4.2x higher throughput and up to 62% less energy per token compared to larger models (7-9B parameters).
- Pareto analysis identifies Qwen 2.5 (7B) as the optimal accuracy-efficiency trade-off (57% MMLU, 59 tokens/s), while Llama 3.2 (3B) is best suited for latency-critical applications at 175 tokens/s.
- The study provides per-token energy measurements at both package and system levels, along with CO2 estimates for India and US power grids.
Why It Matters
This research fills a critical gap in Green AI by quantifying the environmental impact of running LLMs on consumer-grade Apple Silicon, which is increasingly used for local inference. For AI practitioners deploying models on edge devices or personal hardware, these findings provide actionable data to balance performance, energy consumption, and carbon footprint without relying on datacenter infrastructure.
Technical Details
- Benchmark Framework: GreenBench evaluates five open-source LLMs (3-9B parameters) across three NLP tasks on an Apple M4 Pro with 48 GB unified memory, using macOS powermetrics for direct power measurement and Ollama's nanosecond-precision timing.
- Power Characteristics: The M4 Pro consumes only 0.47 W at the CPU+GPU package level during sustained inference, with total system power ranging from 8-12 W, demonstrating exceptional energy efficiency compared to datacenter GPUs.
- Model Scaling Analysis: Smaller models (3-3.8B) achieve 2.6-4.2x higher throughput and up to 62% less energy per token than larger models (7-9B), highlighting the significant efficiency gains from model downsizing.
- Pareto-Optimal Models: Qwen 2.5 (7B) emerges as the best accuracy-efficiency compromise at 57% MMLU and 59 tokens/s, while Llama 3.2 (3B) excels in latency-sensitive scenarios at 175 tokens/s.
- Carbon Footprint Estimation: The study reports per-token energy at both package and system levels, with CO2 estimates calibrated for India and US electrical grids, enabling region-specific sustainability assessments.
Industry Insight
- Organizations deploying LLMs on consumer hardware should prioritize smaller model variants (3-4B parameters) for throughput-heavy workloads, as they offer dramatically better energy efficiency without severe accuracy sacrifices.
- Apple Silicon's unified memory architecture presents a compelling alternative for single-user or edge LLM inference, potentially reducing operational carbon costs by 30-40x compared to datacenter GPU deployments.
- GreenBench's methodology and open framework can serve as a template for evaluating other edge AI platforms, encouraging the industry to adopt standardized energy-efficiency benchmarks alongside accuracy-focused evaluations.
Disclaimer: The above content is generated by AI and is for reference only.