How to evaluate the environmental impact of training large AI models

To evaluate the environmental impact of training large AI models, you must calculate both the operational carbon emissions from electricity use and the embodied carbon from hardware manufacturing. Operational emissions are measured by multiplying the total energy consumed by the data centre by the carbon intensity of the local electricity grid. Embodied emissions require amortising the carbon cost of manufacturing the computing hardware over its expected lifespan.

As machine learning scales from lightweight classification networks to massive generative systems with billions of parameters, the environmental cost of training has become a critical area of study. When you sit your exam, your examiners will expect you to move beyond vague statements about high energy use. They want to see a systematic breakdown of where that energy goes and how it translates into greenhouse gas emissions. You need to understand the physical reality of the data centre, the metrics used to quantify power efficiency, and the geographical factors that dictate how dirty or clean that power actually is. This post will walk you through the exact methodologies and calculations you need to demonstrate a practitioner-level grasp of AI sustainability.

The core equation of operational emissions

The foundation of evaluating training impact begins with a straightforward calculation. The total operational carbon footprint of a training run is the product of the energy consumed in kilowatt-hours and the carbon intensity of the energy grid in kilograms of carbon dioxide equivalent per kilowatt-hour. While the equation sounds simple, gathering accurate inputs for both sides of the multiplication requires careful accounting.

The energy consumption side of the equation must account for the entire computational cluster, not just the processors doing the matrix multiplications. A typical large language model training run involves hundreds or thousands of graphics processing units running concurrently for weeks or months. Alongside these GPUs, you have central processing units managing the data pipelines, volatile memory storing the model weights, and massive networking switches coordinating the distributed training. Every single one of these components draws continuous power.

Furthermore, you must factor in the overhead of the facility itself. Data centres run extremely hot. The electricity required to cool the servers often rivals the electricity used to power the computing hardware. If you only measure the power draw at the server plug, you are missing a massive portion of the environmental cost. Practitioners always include data centre efficiency metrics when calculating the true energy footprint of a machine learning workload.

Measuring energy consumption and Power Usage Effectiveness

To accurately measure the total energy consumed, engineers rely on a metric called Power Usage Effectiveness, commonly known as PUE. PUE is the ratio of the total amount of energy used by a computer data centre facility to the energy delivered to computing equipment. A PUE of 1.0 would mean perfect efficiency, where every watt of power goes directly to the servers. In reality, most modern data centres operate with a PUE between 1.1 and 1.5, meaning an extra 10 to 50 percent of power is spent on cooling, lighting, and power distribution inefficiencies.

Let us look at a worked example that you could easily adapt for a short-answer exam question. Imagine you are training a transformer model on a cluster of eight GPUs. Each GPU has a Thermal Design Power of 400 watts, and they run at maximum capacity for 100 hours. The compute energy is eight GPUs multiplied by 400 watts multiplied by 100 hours, yielding 320,000 watt-hours, or 320 kilowatt-hours. However, if the data centre housing this cluster has a PUE of 1.2, you must multiply the 320 kilowatt-hours by 1.2 to find the total facility energy footprint. The actual energy consumption is therefore 384 kilowatt-hours.

When estimating these figures before a training run, practitioners often rely on the Thermal Design Power rating, which represents the maximum heat a component is expected to dissipate. However, for retrospective evaluation, it is much more accurate to use software profilers that read the actual power draw from the hardware sensors at regular intervals. Dynamic power consumption fluctuates based on the specific operations the processors are executing at any given millisecond. Relying solely on peak hardware ratings will usually result in an overestimation of the environmental impact, which is why examiners reward students who distinguish between estimated peak power and measured dynamic consumption.

Accounting for carbon intensity and geographical location

Once you have calculated the total energy consumption in kilowatt-hours, you must convert that figure into carbon emissions. This is where the physical location of the data centre becomes the deciding factor. Carbon intensity measures how many grams of carbon dioxide are released to produce a kilowatt-hour of electricity. A data centre powered entirely by a nearby hydroelectric dam will have a drastically lower carbon intensity than a facility relying on a coal-fired power station.

Grid carbon intensity is not static. It fluctuates throughout the day based on weather conditions and consumer demand. On a windy, sunny afternoon, a regional grid might be flooded with renewable energy, dropping the carbon intensity to a fraction of its nighttime level. When evaluating a long training run that spans several weeks, you should ideally use the average carbon intensity for that specific region over that exact time period. Many major cloud providers now publish these figures to help engineers accurately track their emissions.

In an exam scenario, you might be asked to compare the impact of training a model in two different locations. If you have a choice between training in a region with a carbon intensity of 200 grams of CO2 per kilowatt-hour and another with 400 grams per kilowatt-hour, the first location will halve your operational emissions for the exact same compute workload. Highlighting this geographical dependence demonstrates to the examiner that you understand the real-world variables that influence AI sustainability beyond just writing efficient code.

Embodied carbon and the hidden cost of hardware

Operational emissions only tell half the story. The other crucial component is embodied carbon, which refers to the greenhouse gases emitted during the extraction of raw materials, manufacturing, transportation, and eventual disposal of the hardware. Producing high-performance silicon chips, memory modules, and the surrounding server infrastructure is an incredibly energy-intensive process that relies heavily on global supply chains.

For a complete life-cycle assessment, the embodied carbon of the hardware must be amortised over the training run. You calculate this by taking the total embodied carbon of the server, dividing it by its expected lifespan, and multiplying that by the duration of the training process. If a GPU has an embodied carbon cost of 150 kilograms of CO2 and an expected usable life of three years, using that GPU for a one-month training run incurs roughly 4.1 kilograms of embodied carbon emissions.

Including embodied carbon is particularly vital when evaluating the trade-offs of hardware upgrades. Newer, faster GPUs are generally more energy-efficient per operation, which lowers operational emissions. However, manufacturing entirely new hardware incurs a massive upfront embodied carbon penalty. When writing an essay or long-form exam answer, pointing out this tension between operational efficiency and embodied manufacturing costs will show the examiner you are thinking like a senior practitioner rather than just reciting basic formulas.

Tools and frameworks for practical measurement

You are not expected to manually calculate power draw from hardware sensors in the real world. The machine learning community has developed several open-source tools to automate the evaluation of environmental impact. Libraries such as CodeCarbon and the ML CO2 Impact calculator integrate directly into Python training scripts. These tools poll the hardware for power usage, estimate the facility overhead based on the cloud provider being used, and query real-time data feeds to fetch the current carbon intensity of the local energy grid.

Familiarising yourself with how these tools function under the hood is an excellent way to prepare for practical exam questions. They generally operate as a background thread during the training loop, periodically logging the energy metrics without significantly slowing down the primary compute tasks. When the training concludes, they output a comprehensive report detailing the total kilowatt-hours consumed and the estimated kilograms of CO2 equivalent emitted.

Understanding these frameworks also ties into the broader concept of corporate reporting standards, such as the Greenhouse Gas Protocol. Operational emissions from electricity fall under Scope 2 emissions, while the embodied carbon of purchased hardware falls under Scope 3. Using standardised tools ensures that the environmental impact of AI research can be audited and compared fairly across different organisations and academic institutions. The Full Marks Press Deep Learning and Generative AI guide covers this reporting structure with worked exam questions to help you test your understanding of these specific metrics.

How to answer this in an exam

When an exam question asks you to evaluate or assess the environmental impact of an AI model, a weak answer will just say that training takes a lot of electricity and causes pollution. Mark schemes for higher-level modules demand precision. You must explicitly mention the formula connecting compute energy, PUE, and grid carbon intensity. You should define PUE clearly and explain why data centre cooling is a major factor.

Furthermore, top marks are always reserved for students who distinguish between operational carbon and embodied carbon. If you can provide a small mathematical example showing how an energy-efficient GPU might still have a high environmental cost due to the local grid relying on fossil fuels, you will easily hit the highest grade boundaries. Always ground your answers in the physical reality of data centres rather than treating the cloud as an abstract concept.

Frequently asked questions

Does reducing training time always reduce environmental impact? Usually, yes. Fewer hours spent computing means less energy consumed. However, if you reduce training time by distributing the workload across far more GPUs in a region with a much dirtier energy grid, your total carbon emissions might actually increase.

Why is PUE important for machine learning engineers? PUE accounts for the facility overhead, primarily cooling. Because large models push GPUs to their thermal limits for extended periods, the cooling demand is immense. Ignoring PUE means underestimating your actual energy footprint by up to 50 percent.

What is the difference between Scope 2 and Scope 3 emissions in AI? Scope 2 emissions cover the indirect emissions from the generation of purchased electricity used to power the servers. Scope 3 emissions include all other indirect emissions in the value chain, specifically the embodied carbon from manufacturing and shipping the hardware.

How does inference impact compare to training impact? Training is a massive, concentrated energy expenditure. Inference requires very little energy per request, but a successful model might be queried billions of times over several years. Over a model’s lifetime, the aggregate environmental impact of inference can easily surpass the initial cost of training.

You can download a free copy of our complete revision guide at fullmarkspress.com/free.