Energy per Token: The Power Footprint of AI

How much electricity does an AI response actually use? Measuring energy per token on accessible hardware, and finding which choices save the most.

Pennsylvania is in the middle of a data-center boom, and I’m often asked what AI means for power and water. Answering well requires a number most people never see: how much energy a single AI response uses, measured per token, and what actually moves it.

This fall I’m launching a set of undergraduate-accessible studies that measure inference energy directly on open models, with software-based GPU power measurement:

• Prompting: does asking for a concise answer, or writing a shorter prompt, measurably cut energy per response? The goal is a practical, energy-aware prompting guideline.
• Where the joules go: how energy splits between reading the prompt and generating the answer, and how batch size and context length shift it. This includes a head-to-head of dense and Mixture-of-Experts models at matched quality.
• Accuracy per joule: how much energy lower numerical precision (16-, 8- and 4-bit) saves, how much accuracy it costs, and where that trade-off sits for different kinds of tasks.

Much of the potential saving is engineering discipline rather than new hardware, which makes it a good problem for students to measure and improve.