eGRID

This repository includes all necessary scripts and documentation to create the Emissions & Generation Resource Integrated Database (eGRID) and its supplemental features and datasets.

Origin

This repository is a fork of USEPA/egrid, the code base EPA's Clean Air and Power Division (CAPD) used to produce eGRID. It is maintained independently at cornerstone-data/egrid and is not maintained, reviewed, or endorsed by USEPA. This fork started with USEPA/egrid's development branch at upstream commit 868b42b. The original authors' commit history is preserved, and the original MIT license and copyright notice are retained in LICENSE.md. Changes made since the fork are recorded in FORK.md.

The USEPA eGRID datasets are published by EPA at https://www.epa.gov/egrid. Output produced by this fork may differ from the official releases.

Background

eGRID is a comprehensive source of data on the environmental characteristics of almost all electric power generated in the United States. eGRID is based on available plant-specific data for all U.S. electricity generating plants that provide power to the electric grid and report emissions and electricity data to the U.S. government. Data reported include, but are not limited to, net electric generation; resource mix (the share of generation by resource or fuel type); mass emissions of carbon dioxide (CO₂), nitrogen oxides (NO_x), sulfur dioxide (SO₂), methane (CH₄), and nitrous oxide (N₂O); emission rates for CO₂, NO_x, SO₂, CH₄, and N₂O; heat input; and nameplate capacity. eGRID reports this information on an annual basis (as well as by ozone season for heat input and NO_x) at different levels of geographic aggregation.

eGRID Repository Components

The primary eGRID dataset is referred to in this repository as the eGRID production model. Additional supplementary datasets and tools utilize the eGRID production model to provide more information about certain aspects of the U.S. electric grid. The eGRID repository cumulatively consists of the following components:

• Production model

• PM_2.5, NH₃, and VOC emissions

• Web updates

• Power profiler

Production model

The final eGRID production model dataset includes eight levels of data aggregation:

• Generator: A set of equipment that produces electricity and is connected to the U.S. electricity grid.

• Unit: A set of equipment that either produces electricity and is connected to the U.S electricity grid or a set of equipment that is connected to a generator which produces electricity and is connected to the U.S. electricity grid.

• Plant: A facility with one or more units and/or generators that provide power to the electric grid.

• State: U.S. states, Puerto Rico (PR), and the District of Columbia (DC).

• Balancing authority: Regional power system operators that ensure a balance of supply and demand.

• eGRID subregion: EPA defined subregions designed to limit the impacts of the import and export of electricity (shown in Figure 1).

• NERC (North American Electric Reliability Corporation) regions: Each NERC region listed in eGRID represents one of nine regional portions of the North American electricity transmission grid: six in the contiguous United States, plus Alaska, Hawaii, and Puerto Rico (which are not part of the formal NERC regions but are considered so in eGRID).

• National U.S.: Contains all 50 states, Puerto Rico (PR), and the District of Columbia (DC).

Further information on the eGRID methodology can be found in the eGRID Technical Guide.

Figure 1: eGRID subregions.

PM_2.5, NH₃, and VOC emissions

The eGRID production model does not include fine particulate matter 2.5 microns in diameter or smaller (PM_2.5), ammonia (NH₃), and volatile organic compounds (VOC) emissions as the data used for the production model emissions calculations are not available for those pollutants. Instead, PM_2.5, NH₃, and VOC emissions and emission rates for power plants are estimated using data from the EPA's National Emissions Inventory (NEI).

The final dataset for each pollutant (PM_2.5, NH₃, and VOC) type includes five of the data aggregation levels present in the production model (see region descriptions above under Production model):

• Unit

• Plant

• State

• eGRID subregion

• National U.S.

Further information on the eGRID PM_2.5, NH₃, and VOC methodology can be found in the Particulate Matter Emissions for eGRID2021 methodology documentation.

Web updates

There are eGRID websites that use underlying data from the production model. The eGRID Data Explorer file inputs and Frequently Asked Questions Global Warming Potential table are calculated as post-processing steps in this codebase.

Power profiler

The National Renewable Energy Lab (NREL) produces a dataset of utilities servicing zip codes which is used in conjunction with the eGRID production model data to assign utilities and eGRID subregions to zip codes. This data is then used in the eGRID Power Profiler Tool where users can explore their electricity fuel mix and emission rates based on their zip code and utility.

Architecture

Starting in 2024, USEPA released the methodology to develop eGRID as an RStudio project. To increase transparency in the eGRID production process, USEPA made the R scripts available for users to view and use. USEPA used the RStudio project beginning in 2024 to produce eGRID2023.

Production model

A data dictionary is provided in eGRID Production Model Data Dictionary.xlsx. This file provides the row number, name, description, imperial units, metric units, source, and calculation method for each column reported in the final eGRID production model dataset.

Figure 2 displays a summary of the eGRID production model architecture, which specifies data sources, inputs, and outputs for creating the eGRID production model.

PM_2.5, NH₃, and VOC emissions

Figure 3 displays a summary of eGRID PM_2.5, NH₃, and VOC emissions architecture, which specifies data sources, inputs, and outputs for creating the eGRID PM_2.5, NH₃, and VOC emissions and emission rates datasets.

Code base organization

This project is structured as an RStudio project. To ensure that all scripts run correctly, load the eGRID_R.Rproj within RStudio to enable the project environment. Within the project, the codebase uses the renv to allow users to replicate the package environment. This will automatically load upon opening the project or a user can use the command renv::load().

Production model

eGRID_master.qmd is a Quarto document that serves as a master script (i.e., it runs all necessary scripts in the correct order), while also providing documentation for the scripts and steps performed therein.

The production model code base is structured as follows:

• scripts/1_production_model/: all scripts to download and clean data, create each data aggregation level, format final dataset, and convert to metric units.

• scripts/functions/: all helper functions.

• data/1_production_model/raw_data/: raw data obtained from EPA and EIA sites.

• data/1_production_model/clean_data/: data created from EPA and EIA data cleaning steps.

• data/1_production_model/outputs/: outputs generated by this code base.

• data/1_production_model/static_tables/: static tables used within the code base. These include crosswalks that match data between EIA and EPA data or regions and manual corrections that are made.

The data used to create the eGRID production model are EPA and Energy Information Administration (EIA) electricity data. EPA data are loaded via an application programming interface (API) in scripts/1_production_model/data_load_epa.R and cleaned in scripts/1_production_model/data_clean_epa.R. EIA data are downloaded from EIA's website in scripts/1_production_model/data_load_eia.R and scripts/functions/function_download_eia_files.R and cleaned in scripts/1_production_model/data_clean_eia.R.

PM_2.5, NH₃, and VOC emissions

Similar to the production model, pm_nh3_voc_master.qmd is a master script used to organize all necessary scripts and document the steps taken to produce PM_2.5, NH₃, and VOC emissions and emission rates. PM_2.5, NH₃, and VOC emissions and emission rates calculations utilize the eGRID production model code and build off the final eGRID production model dataset, with a similar code base structure:

• scripts/2a_pm_nh3_voc/: PM_2.5, NH₃, and VOC emissions scripts to create each data aggregation level and format final dataset.

• scripts/functions/: all helper functions.

• data/2a_pm_nh3_voc/inputs/: emissions input data provided by EPA.

• data/2a_pm_nh3_voc/outputs/: outputs generated by this code base for PM_2.5, NH₃, and VOC emissions.

• data/2a_pm_nh3_voc/static_tables/: static tables used within the code base. These include emissions factors for PM_2.5, NH₃, and VOCs.

The data used to create the PM_2.5, NH₃, and VOC emissions and emission rates data are EPA and EIA electricity data, NEI emissions data, an NEI-EIA facility and unit identifier crosswalk, and the eGRID production model dataset. EPA and EIA data are acquired in the same way as the production model while NEI emissions data and NEI-EIA crosswalk data are provided directly from the EPA.

Web updates

The web updates are structured as follows:

• scripts/2b_web_updates: scripts to create the eGRID Data Explorer files and

• data/2b_web_updates: outputs from each of the scripts for the eGRID Data Explorer files and the Frequently Asked Questions Global Warming Potential table.

Power profiler

The power profiler code base is structured as follows:

• scripts/2c_power_profiler/: scripts to create and format the power profile dataset.

• scripts/functions/: all helper functions.

• data/2c_power_profiler/inputs/: input data which includes zip code utility data and the previous year's power profiler dataset.

• data/2c_power_profiler/outputs/: outputs generated by this code base constituting the power profiler dataset.

• data/2c_power_profiler/static_tables/: static tables used within the code base.

The data used to create the power profiler dataset are EIA electricity data, the eGRID annual production model dataset, the previous year's power profiler dataset, and a zip code utility assignment dataset. EIA data are acquired in the same way as the production model while the zip code utility assignment dataset is produced by NREL and is publicly available.

Creating eGRID

Production model

To create the eGRID production model dataset:

1. Obtain an API key for EPA data.
   • Request an API key from https://www.epa.gov/power-sector/cam-api-portal.
   • Create folder api_keys/ within the root of the eGRID.
   • Create a text file named epa_api_key.txt within the folder api_keys/ and save the API key here on a single line.
2. Load eGRID_R.Rproj within RStudio to enable the project environment.
3. Render eGRID_master.qmd.
   • Set data year in params$eGRID_year in the YAML as a string in the format "YYYY" (ex: "2023").
   • Set temporal resolution version in params$temporal_res in the YAML as a string (options: "annual" or "monthly") eGRID.
   • Render eGRID_master.qmd. This will run all scripts and build the eGRID dataset.

PM_2.5, NH₃, and VOC emissions

The PM_2.5, NH₃, and VOC emissions and emission rates datasets use eGRID production model outputs and require that the production model dataset is already produced for the desired data year.

Note on data availability: Emissions data and crosswalks used in these calculations are from a version of EPA's NEI that are not publicly available. Information about the data inputs and the code used to produce the final data are included here to document the process and inform any need for replication as well as inform data production for those with access to the appropriate NEI data.

To create the eGRID PM_2.5, NH₃, and VOC emissions datasets:

1. Obtain an API key for EPA data.
   • Request an API key from https://www.epa.gov/power-sector/cam-api-portal.
   • Create folder api_keys/ within the root of the eGRID.
   • Create a text file named epa_api_key.txt within the folder api_keys/ and save the API key here on a single line.
2. Confirm the eGRID production model data has been produced and is stored in data/1_production_model/outputs/.
3. Load eGRID_R.Rproj within RStudio to enable the project environment.
4. Render pm_nh3_voc_master.qmd.
   • Set data year in params (eGRID_year) in the YAML as a string in the format "YYYY" (ex: "2023").
   • If you do not have access to the NEI data inputs, set evaluation method in params (eval) to false.
   • Render pm_nh3_voc_master.qmd. If eval:true, this will run all scripts and build the PM_2.5, NH₃, and VOC emissions datasets. If eval:false, this will produce the documentation around dataset production but will not build the datasets themselves.

Web updates

To create the web updates:

1. Confirm the eGRID production model data has been produced and is stored in data/1_production_model/outputs/.
2. Run the scripts scripts/2b_web_updates/data_explorer_formatting.R and scripts/2b_web_updates/faq_gwp_table_create.R.

Power profiler

To create the power profiler dataset:

1. Confirm the eGRID production model data has been produced and is stored in data/1_production_model/outputs/.
2. Run the scripts scripts/2c_power_profiler/power_profiler_create.R and scripts/2c_power_profiler/power_profiler_formatting.R.

Outputs

Production model

The codebase outputs each data aggregation level in the eGRID dataset as an .RDS file and the final dataset as an Excel sheet in data/outputs/{params$eGRID_year}/{params$temporal_res} for each annual and monthly temporal resolution, depending on what is specified by the user. Rendering eGRID_master.qmd also creates an HTML file that summarizes the data, methods, and output files used and created throughout the code base.

PM_2.5, NH₃, and VOC emissions

For each pollutant (PM_2.5, NH₃, VOC), the codebase outputs each data aggregation level as an .RDS file. Additionally, the final data is added to the previous year’s data in an Excel sheet and is stored in data/2a_pm_nh3_voc/outputs/{params$eGRID_year}.

The resulting final outputs include:

• egrid{params$eGRID_year}_pm_emissions.xlsx

• egrid{params$eGRID_year}_nh3_emissions.xlsx

• egrid{params$eGRID_year}_voc_emissions.xlsx

Rendering pm_nh3_voc_master.qmd also creates an HTML file that summarizes the data, methods, and output files used and created throughout the code base.

Web updates

For the eGRID Data Explorer, the codebase outputs a group of CSV files. These files contain eGRID data from 2018 to the current params$eGRID_year, formatted as needed for the Data Explorer. For the Frequently Asked Questions Global Warming Potential table, the codebase outputs a single CSV file. These outputs are stored in data/2b_web_updates/outputs/{params$eGRID_year}. The resulting final outputs include:

• data_explorer_ba_file.csv

• data_explorer_nerc_file.csv

• data_explorer_plant_file.csv

• data_explorer_state_file.csv

• data_explorer_subregion_file.csv

• data_explorer_us_file.csv

• faq_gwp_table.csv

Power profiler

For the power profiler, the codebase outputs two datasets as .RDS files which include zip code utility and zip code subregion assignments. The two datasets are combined and formatted to produce the final power profiler dataset and are stored in data/2c_power_profiler/outputs/{params$eGRID_year}.

The resulting intermediate outputs include:

• zip_utility_subregion.RDS

• zip_subregion_assignments.RDS

The resulting final output is:

• ZipSubregion{params$eGRID_year}.xlsx

QA

The codebase contains two QA files as Quarto documents:

• qa_all.qmd: Annual checks to confirm results are as expected for each file created in the codebase.

• qa_annual_comparison.qmd: Comparison of data year outputs to previous eGRID years.

Contributing to eGRID

To ask a question about or report an issue in this code, review the CONTRIBUTING policy and submit an issue under the "Issues" tab of cornerstone-data/egrid. Provide a concise summary as the title of the issue and a clear description, including steps to reproduce the issue.

License

This repository has two licensing layers:

• Code from USEPA/egrid at or before upstream commit 868b42b is licensed under the MIT License in LICENSE.md. The original copyright notice must be retained.
• Modifications and additions made in this fork after that commit are licensed under the Cornerstone Data Pipeline Attribution License 1.0. See ATTRIBUTION.md for the attribution requirement on data produced with those modifications.

Raw EPA and EIA input data and R package dependencies are not covered by either license.