Get to the Milestone

Punts to next version:

ILCD-formatted process names #323
Troubleshoot pandas 3 #321
Add triangular uncertainties to NG LCI
Update olca_jsonld_writer module with NetlOlca class #319

The remaining tasks for v2.1 include the following:

Empty Renewable Generation Processes with Low Mix Percentage #331
The generation mix process, ‘Electricity; at grid; consumption mix - City of Homestead - BA’, has no inputs
- City of Homestead (HST) is an active BA in Florida, so why no generation?
  - Affiliated with eGRID facility 665 (MIXED fuel plant).

These issues are effectively completed (will be closed when PR is submitted/accepted):

Fix IMP location (assign to Global 'GLO')
Remove Hawaii FERC region distribution and consumption processes
To run efficiency filter on OTHF primary fuels? #330
- Use 100.1% upper-bound efficiency
Tick up the electricity_lci_target_year YAML value from 2025 to 2026
- For the ELCI_2023 and ELCI_2024 models.
General metadata updates based on internal metadata review #328
Locations of upstream processes #327
- See KeyLogicLCA@96f829b
Correct mis-matches between power plant construction processes linked to US average, where there is regional process #326
- The error is likely caused by naming differences (e.g., “power plant construction - coal_const/ngcc_const - Midcontinent Independent Transmission System Operator, Inc..” has two dots after “Inc” where the power plant process location name only has one: “Midcontinent Independent System Operator, Inc.”)
- Proposed Solution: use the BA map on everything to ensure consistent naming.
High emissions in Electricity - HYDRO - Midcontinent Independent System Operator, Inc. #324
- Solved via addition to manual_edits.yml (KeyLogicLCA@525b39f)
Process Indicators of Quality #322
Metadata for Canadian generation data is lacking #317
Geothermal dataset contains non-compliant elementary flows #314
- remove zero-quantity flows from geothermal LCI CSV
- resolved all but two land transformation flows (KeyLogicLCA@8563cb0); mostly solved.
Fix non-fedefl water flows #313
- This may get solved in a future release of fedelemflow (update eLCI.csv mapper file)
- See https://github.com/FLCAC-admin/fedelemflowlist/pull/221/changes
Clean up trouble matching dictionary for generation mix Construction - FUEL - REGION logging statements #311
Update inventory data in StEWI through 2023 #275
- Updates for ELCI2023 + other minor bug fixes KeyLogicLCA/ElectricityLCI#15
Update inventories | Natural Gas #260
- Natural Gas Upstream update - Addresses issue #260 KeyLogicLCA/ElectricityLCI#16
Remove legacy epa affiliation in setup.py #316
Improve the reference flow name for natural gas through transmission #320
- KeyLogicLCA@6e0740c
Unit groups and flow properties out of date in json-ld #315
- Just need to delete old JSON files from data store and have them repopulate.
DQI scores of 5 for technological correlation in gas plants #304
- KeyLogicLCA@834ff77
Incorporate residual mix accounting #276
- Residual electricity mix KeyLogicLCA/ElectricityLCI#18
- New facility generation residual mix aggregation method KeyLogicLCA/ElectricityLCI#19
- filter out HECO balancing authority
  - KeyLogicLCA@59b5b67
- Sort out why there's a 'Electricity; at grid; generation mix - Hawaiian Electric Co Inc' with no fuel processes in its exchange table
Correct residual mix processes exchange table internal IDs (residual process creation needs called before the post-processing step that does the internal ID correction)
- KeyLogicLCA@b2ae1ed
Add missing pytz to dependencies (removed from fedelemflowlist >1.3)
- KeyLogicLCA@cd9ee11
Fix EIA T&D links
- KeyLogicLCA@8d0e1e0
Fix nuclear power source URL (VUE Librarian to respond about broken DOI/OSTI links)
Fix nuclear power upstream process description (replace “power plant emissions” with “power plant operations”)

update sources; addresses NETL-RIC#325

correct inline URL refs; addresses NETL-RIC#325

correct nuclear upstream description; addresses NETL-RIC#325

update eLCI ref; addresses NETL-RIC#325

remove zero flow amounts from LCI; addresses NETL-RIC#261, NETL-RIC#325

update documentation; add error handling; addresses NETL-RIC#325, NET…

documentation; addresses NETL-RIC#325

documentation updates; addresses NETL-RIC#325

update main; addresses NETL-RIC#325

clean up inline comments; addresses NETL-RIC#325

update inline comment; addresses NETL-RIC#325

New Canada generation process metadata; addresses NETL-RIC#317, NETL-…

tick up inventory years; addreses NETL-RIC#325

add fuel category code names; addresses NETL-RIC#325

remove HECO from generation mix; addresses NETL-RIC#325

remove HI from consumption & distribution; addresses NETL-RIC#325

Renewable Generation Without Inputs

In the 2023 baseline, there are three:

‘Electricity - SOLAR - Associated Electric Cooperative, Inc.’
‘Electricity - BIOMASS - Los Angeles Department of Water and Power’
‘Electricity - WIND - Duke Energy Progress East’

For biomass, the empty inputs for Los Angeles biomass generation is the same as in 2022. It was not found in the 2016 baseline (the facility was considered MIXED), and it was not found in the 2020 or 2021 baselines ((facility is absent). The only data available for this facility is the AP42; hence, no inputs.

Looking at solar:

>>> # BASIC SETUP FOR ELCI_2023
>>> from electricitylci import basic_setup
>>> basic_setup()
>>> # GET NETL SOLAR LCI
>>> from electricitylci.solar_upstream import *
>>> solar_df = generate_upstream_solar(2023)
>>> solar_plants = solar_df['plant_id'].unique().tolist()
>>> solar_plants = sorted(solar_plants)
>>> # GET AMPD SOLAR LCI
>>> from electricitylci.ampd_plant_emissions import *
>>> ampd_df = generate_plant_emissions(2023)
>>>  ampd_solar_plants = ampd_df.query("PrimaryFuel == 'SUN'")['eGRID_ID'].unique().tolist()
>>> ampd_solar_plants = sorted(ampd_solar_plants)
>>> matched_solar_plants = [x for x in solar_plants if x in ampd_solar_plants]
>>> unmatched_solar_plants = [x for x in solar_plants if x not in ampd_solar_plants]
>>> print("Found %d/%d solar plants in AMPD; %d plants in solar data not in AMPD" % (len(matched_solar_plants), len(solar_plants), len(unmatched_solar_plants)))
Found 3832/3922 solar plants in AMPD; 90 plants in solar data not in AMPD
>>> in_solar_plants = [x for x in ampd_solar_plants if x in solar_plants]
>>> not_in_solar_plants = [x for x in ampd_solar_plants if x not in solar_plants]
>>> print("Found %d/%d AMPD plants in solar data; %d plants in AMPD not in solar data" % (len(in_solar_plants), len(ampd_solar_plants), len(not_in_solar_plants)))
Found 3832/5572 AMPD plants in solar data; 1740 plants in AMPD not in solar data
>>> # GET LOCATION FOR SOLAR PLANTS
>>> from electricitylci.generation import get_facilities_w_fuel_region
>>> facilities_w_fuel_region = get_facilities_w_fuel_region()
>>> facilities_w_fuel_region.rename(columns={'FacilityID': 'eGRID_ID'}, inplace=True)
>>> facilities_w_fuel_region.query("`Balancing Authority Name` == 'Associated Electric Cooperative, Inc.'")[['eGRID_ID', 'FuelCategory', 'PrimaryFuel', 'State']]
       eGRID_ID FuelCategory PrimaryFuel State
602        1127          OIL         DFO    IA
630        1166          OIL         DFO    IA
1017       2099        HYDRO         WAT    MO
1024       2113        MIXED         DFO    MO
1031       2126          GAS          NG    MO
1032       2127        MIXED         DFO    MO
1042       2141        MIXED         DFO    MO
1045       2145          OIL         DFO    MO
1046       2146        MIXED         DFO    MO
1047       2150        MIXED         DFO    MO
1049       2164          OIL         DFO    MO
1051       2167         COAL          RC    MO
1052       2168         COAL          RC    MO
1916       6140        HYDRO         WAT    MO
2093       6563          OIL         DFO    MO
2224       7304        MIXED         DFO    MO
2256       7405          OIL         DFO    MO
2257       7406          OIL         DFO    MO
2334       7604          GAS          NG    MO
2353       7749          GAS          NG    MO
2356       7754          GAS          NG    MO
2357       7757          GAS          NG    OK
2417       7848          GAS          NG    MO
2427       7860        MIXED         DFO    MO
2560       8110        MIXED         DFO    MO
4080      55340          GAS          NG    AR
4431      56126        MIXED         DFO    MO
4432      56127          GAS          NG    MO
4433      56128        MIXED         DFO    MO
4738      56536         WIND         WND    MO
4739      56537         WIND         WND    MO
4740      56538         WIND         WND    MO
4757      56555         WIND         WND    MO
5240      57189         WIND         WND    MO
6537      58683         WIND         WND    OK
7047      59310      BIOMASS         LFG    MO
9822      62624         WIND         WND    MO
10151     63025         WIND         WND    MO
10213     63098          OIL         DFO    MO
12377     66068        SOLAR         SUN    IA
>>> ampd_df.query("eGRID_ID == 66068")
       eGRID_ID  plant_name                             operator_name PrimaryFuel  ...  DataCollection  GeographicalCorrelation  TechnologicalCorrelation  DataReliability
58225     66068  Maple City  Greenbacker Renewable Energy Corporation         SUN  ...               5                        1                         1                1
58226     66068  Maple City  Greenbacker Renewable Energy Corporation         SUN  ...               5                        1                         1                1
58227     66068  Maple City  Greenbacker Renewable Energy Corporation         SUN  ...               5                        1                         1                1
58228     66068  Maple City  Greenbacker Renewable Energy Corporation         SUN  ...               5                        1                         1                1
58229     66068  Maple City  Greenbacker Renewable Energy Corporation         SUN  ...               5                        1                         1                1
>>> solar_df.query("plant_id == 66068")
Empty DataFrame

Notably, there is one plant (66068) listed in 'Associated Electric Cooperative, Inc.' that is labeled as a solar power plant, which is found in the AMPD data set, but not in NETL's solar LCI.

Remaining questions:

Why does this plant not have generation data associated with it elsewhere?
What year is this power plant associated with (note that facilities_w_fuel_region looks across generation years)?
Why this balancing authority's solar generation data gets skipped for power plant construction?

For question 2, we look at individual years:

>>> get_facilities_w_fuel_region(2016).query("FacilityID == 66068")
Empty DataFrame
>>> get_facilities_w_fuel_region(2020).query("FacilityID == 66068")
Empty DataFrame
>>> get_facilities_w_fuel_region(2023).query("FacilityID == 66068") 
       FacilityID FuelCategory PrimaryFuel  ...  NERC Balancing Authority Code               Balancing Authority Name
11610       66068        SOLAR         SUN  ...   MRO                     AECI  Associated Electric Cooperative, Inc.

So it's only found in the current generation year (i.e., 2023). Notably, the NETL solar LCI is based on 2020 generation data, which does not include plant 66068.

Let's look at all other sources of facility-level emissions:

>>> from electricitylci.generation import create_generation_process_df
>>> gen_df = create_generation_process_df()
>>> gen_df.query("eGRID_ID == 66068")
                FlowName Compartment Source  Year PrimaryFuel  Electricity State
254043    Carbon dioxide         air   ap42  2023         SUN       2870.0    IA
254047           Methane         air   ap42  2023         SUN       2870.0    IA
254052  Nitrogen dioxide         air   ap42  2023         SUN       2870.0    IA
254049   Nitrogen oxides         air   ap42  2023         SUN       2870.0    IA
254053     Nitrous oxide         air   ap42  2023         SUN       2870.0    IA
254056    Sulfur dioxide         air   ap42  2023         SUN       2870.0    IA

It turns out that ap42 is the only instance of this plant.

Looking at the 2020, 2021 and 2022 baselines, there are input exchanges including construction for SOLAR generation in AECI.

So what changed in 2023?

Let's check WIND.

>>> ba_mask = facilities_w_fuel_region['Balancing Authority Name'] == 'Duke Energy Progress East'
>>> fc_mask = facilities_w_fuel_region['FuelCategory'] == 'WIND'
>>> facilities_w_fuel_region.loc[(ba_mask & fc_mask), ['eGRID_ID', 'FuelCategory', 'PrimaryFuel', 'State']]
      eGRID_ID FuelCategory PrimaryFuel State
7485     59825         WIND         WND    NC

So there is just one facility. Let's see what year it's associated with.

>>> get_facilities_w_fuel_region(2016).query("FacilityID == 59825")
      FacilityID FuelCategory PrimaryFuel  PercentGenerationfromDesignatedFuelCategory State  NERC Balancing Authority Code          Balancing Authority Name
6370       59825        SOLAR         SUN                                          1.0    NC  SERC                     CPLE  Progress Energy Carolinas - EAST
>>> get_facilities_w_fuel_region(2020).query("FacilityID == 59825")
      FacilityID FuelCategory PrimaryFuel  PercentGenerationfromDesignatedFuelCategory State  NERC Balancing Authority Code   Balancing Authority Name
7091       59825        SOLAR         SUN                                          1.0    NC  SERC                     CPLE  Duke Energy Progress East
>>> get_facilities_w_fuel_region(2023).query("FacilityID == 59825") 
      FacilityID FuelCategory PrimaryFuel  PercentGenerationfromDesignatedFuelCategory State  NERC Balancing Authority Code   Balancing Authority Name
6786       59825         WIND         WND                                          1.0    NC  SERC                     CPLE  Duke Energy Progress East

This is strange. The facility was a solar-dominated plant up to 2020, then switched to wind-dominate power (in 2021).

Let's look at NETL's wind LCI:

>>> from electricitylci.wind_upstream import *
>>> wind_df = generate_upstream_wind(2023)
>>> wind_df.query("plant_id == 59825")
Empty DataFrame

It's not there (it's based on 2020 generation, when the plant was still solar).

Let's see if any other emissions exist for this plant:

>>> gen_df.query("eGRID_ID == 59825")
                FlowName Compartment Source  Year PrimaryFuel  Electricity State
180926    Carbon dioxide         air   ap42  2023         WND       9795.0    NC
180930           Methane         air   ap42  2023         WND       9795.0    NC
180935  Nitrogen dioxide         air   ap42  2023         WND       9795.0    NC
180932   Nitrogen oxides         air   ap42  2023         WND       9795.0    NC
180936     Nitrous oxide         air   ap42  2023         WND       9795.0    NC
180939    Sulfur dioxide         air   ap42  2023         WND       9795.0    NC

Nope. Just AMPD. If we look at the 2021 and 2022 baselines, we see also the empty inputs associated with WIND generation for CPLE. There is no WIND generation for CPLE in 2020 (the one facility is labeled SOLAR).

It's clear now the mismatch between regions and fuel generation and it's caused by a single facility.

The remaining question is why construction is not applied to these regions as an upstream flow. It should be handled in olcaschema_genprocess in generation.py, which is wrapped by write_gen_fuel_database_to_dict in __init__.py, which is called during run_generation in main.py

>>> from electricitylci import get_upstream_process_df
>>> from electricitylci import write_upstream_process_database_to_dict
>>> from electricitylci import write_process_dicts_to_jsonld
>>> from electricitylci import get_generation_process_df
>>> upstream_df = get_upstream_process_df(2023)
>>> upstream_dict = write_upstream_process_database_to_dict(upstream_df)
>>> upstream_dict = write_process_dicts_to_jsonld(upstream_dict)
>>> # Notably, there is no 'wind_const - Duke Energy Progress East' created
>>> # BTW: do yourself a favor and turn off uncertainty calculations
>>> generation_process_df = get_generation_process_df(upstream_df=upstream_df, upstream_dict=upstream_dict, to_agg=True)
>>> # DIVE INTO olcaschema_genprocess
>>> import pandas as pd
>>> from electricitylci.generation import turn_data_to_dict
>>> from electricitylci.aggregation_selector import subregion_col
>>> database = generation_process_df
>>> subregion="BA"
>>> region_agg = subregion_col(subregion)
>>> fuel_agg = ["FuelCategory"]
>>> stage_code = ["stage_code"]
>>> base_cols = region_agg + fuel_agg + stage_code
>>> non_agg_cols = ["FlowName", "FlowUUID",  "Compartment", "Unit", "Year", "source_string", "TemporalCorrelation", "TechnologicalCorrelation", "GeographicalCorrelation", "DataCollection", "DataReliability", "uncertaintyMin", "uncertaintyMax", "uncertaintySigma", "Emission_factor", "GeomMean", "GeomSD"]
>>> non_agg_cols = [x for x in non_agg_cols if x in database.columns]
>>> database_groupby = database.groupby(by=base_cols)
>>> process_df = pd.DataFrame(database_groupby[non_agg_cols].apply(turn_data_to_dict, (upstream_dict)))
>>> process_df.columns = ["exchanges"]
>>> provider_filter = [x for x in process_df.index.values if x[2] in upstream_dict.keys()]

Actually, at this stage, we're just moving upstream processes into provider locations. Looking through the generation processes, it's all at the power plant level and there's no wind construction at all for Duke.

So we need to go back to upstream_dict.py, which is wrapped by write_upstream_process_database_to_dict in __init__.py.

>>> from electricitylci.globals import COAL_TYPE_CODES
>>> from electricitylci.globals import COAL_BASIN_CODES
>>> from electricitylci.globals import COAL_MINE_CODES
>>> from electricitylci.eia860_facilities import add_balancing_authorities_to_plants
>>> merged = upstream_df
>>> coal_type_codes_inv = dict(map(reversed, COAL_TYPE_CODES.items()))
>>> mine_type_codes_inv = dict(map(reversed, COAL_MINE_CODES.items()))
>>> basin_codes_inv = dict(map(reversed, COAL_BASIN_CODES.items()))
>>> coal_transport = ["Barge", "Lake Vessel", "Ocean Vessel", "Railroad", "Truck", "Belt"]
>>> merged['FlowAmount'] = merged['FlowAmount'].astype("float")
>>> merged['plant_id'] = merged['eGRID_ID']
>>> merged_summary = merged.groupby(by=["FuelCategory", "stage_code", "FlowName", "FlowUUID", "Compartment", "ElementaryFlowPrimeContext", "plant_id", "Unit", "input"], as_index=False).agg({"FlowAmount": "sum", "quantity": "mean"})
>>> const_filter = merged_summary['FuelCategory'].str.contains("CONSTRUCTION")
>>> merged_summary_regional = merged_summary.loc[const_filter, :].copy()
>>> merged_summary_regional, plant_dict = add_balancing_authorities_to_plants(merged_summary_regional, "plant_id", 2023)

At this point, I'm thinking the issue is when (1) we have a renewable generation process (e.g., WIND) and (2) there is no construction upstream process. This type of check can only be accomplished when both generation data and upstream processes are together.

So back to olcaschema_genprocess!

The missing construction could be added with something like this:

def gapfill_construction(df, upstream_dict):
    from electricitylci.upstream_dict import _exchange_table_creation_ref

    # These are the current renewable processes w/ construction
    renew_c_dict = {
        'SOLAR': 'solar_pv_const',
        'SOLARTHERMAL': 'solar_thermal_const',
        'WIND': 'wind_const'
    }

    # Renewable construction only available if enabled.
    if model_specs.include_renewable_generation:
        renew_c_filt = df['FuelCategory'].isin(renew_c_dict.keys())
        for _, row in df.loc[renew_c_filt, :].iterrows():
            # Determine if the exchange table has construction.
            has_c = any([
                True if x['flow']['name'] == 'power plant construction'
                else False for x in row['exchanges']
            ])
            if not has_c:
                logging.info(
                    "Generation process '%s - %s' missing construction input "
                    "exchange." % (
                        row['Balancing Authority Name'],
                        row['FuelCategory']
                    ))
                # Initialize construction exchange process
                exchange = _exchange_table_creation_ref("CONSTRUCTION")
                exchange['input'] = True
                exchange['quantitativeReference'] = False
                exchange['comment'] = 'construction gapfill'
                # Hard-code data quality:
                # 1 - reliability - measurement
                # 2 - temporal - less than 6 years (e.g., 2023-2020 = 3)
                # 3 - geographical - within 2 levels of resolution
                # 1 - technological - all tech relevant
                # 2 - methodological - >80% market in a short time period
                exchange['dqEntry'] = '(1,2,3,1,2)'
                # Add upstream dictionary values
                u_key2 = renew_c_dict[row['FuelCategory']]
                exchange['flow']['id'] = upstream_dict[u_key2]['q_reference_id']
                exchange['unit'] = unit(
                    upstream_dict[u_key2]['q_reference_unit']
                )
                exchange['provider'] = {
                    "name": upstream_dict[u_key2]["name"],
                    "categoryPath": upstream_dict[u_key2]["category"],
                    "processType": "UNIT_PROCESS",
                    "@id": upstream_dict[u_key2]["uuid"],
                }
                row['exchanges'].append(exchange)
    return df

Testing the implementation above. It turns out there are five renewable generation processes in 2023 with no construction:

In [226]: process_df.loc[14, :]
Out[226]: 
Balancing Authority Name                Associated Electric Cooperative, Inc.
FuelCategory                                                            SOLAR
stage_code                                                        Power plant
exchanges                   [{'internalId': '', '@type': 'Exchange', 'avoi...
@type                                                                 Process
allocationFactors                                                            
defaultAllocationMethod                                                      
parameters                                                                   
processType                                                      UNIT_PROCESS
category                    22: Utilities/2211: Electric Power Generation,...
Name: 14, dtype: object

In [227]: process_df.loc[78, :]
Out[227]: 
Balancing Authority Name                            Duke Energy Progress East
FuelCategory                                                             WIND
stage_code                                                        Power plant
exchanges                   [{'internalId': '', '@type': 'Exchange', 'avoi...
@type                                                                 Process
allocationFactors                                                            
defaultAllocationMethod                                                      
parameters                                                                   
processType                                                      UNIT_PROCESS
category                    22: Utilities/2211: Electric Power Generation,...
Name: 78, dtype: object

In [228]: process_df.loc[235, :]
Out[228]: 
Balancing Authority Name                             Puget Sound Energy, Inc.
FuelCategory                                                            SOLAR
stage_code                                                        Power plant
exchanges                   [{'internalId': '', '@type': 'Exchange', 'avoi...
@type                                                                 Process
allocationFactors                                                            
defaultAllocationMethod                                                      
parameters                                                                   
processType                                                      UNIT_PROCESS
category                    22: Utilities/2211: Electric Power Generation,...
Name: 235, dtype: object

In [229]: process_df.loc[247, :]
Out[229]: 
Balancing Authority Name                        Seminole Electric Cooperative
FuelCategory                                                            SOLAR
stage_code                                                        Power plant
exchanges                   [{'internalId': '', '@type': 'Exchange', 'avoi...
@type                                                                 Process
allocationFactors                                                            
defaultAllocationMethod                                                      
parameters                                                                   
processType                                                      UNIT_PROCESS
category                    22: Utilities/2211: Electric Power Generation,...
Name: 247, dtype: object

In [230]: process_df.loc[294, :]
Out[230]: 
Balancing Authority Name                                Tucson Electric Power
FuelCategory                                                             WIND
stage_code                                                        Power plant
exchanges                   [{'internalId': '', '@type': 'Exchange', 'avoi...
@type                                                                 Process
allocationFactors                                                            
defaultAllocationMethod                                                      
parameters                                                                   
processType                                                      UNIT_PROCESS
category                    22: Utilities/2211: Electric Power Generation,...
Name: 294, dtype: object

Consumption Mix with No Generation

The culprit is "Electricity; at grid; consumption mix - City of Homestead - BA," which has consumption and distribution mixes, but no generation mix.

Let's look at the generation data used to create the generation mix (i.e., get_generation_mix_process_df in __init__.py):

>>> from electricitylci.generation_mix import create_generation_mix_process_df_from_model_generation_data
>>> from electricitylci.eia923_generation import build_generation_data
>>> regions = 'BA'
>>> generation_data = build_generation_data(generation_years=[2023,])
>>> generation_mix_process_df = create_generation_mix_process_df_from_model_generation_data(generation_data, regions)
>>> generation_mix_process_df.query("Subregion == 'City of Homestead'")
Empty DataFrame

No Homestead. This was confirmed also by looking at the unique list of BA areas (the Subregion column).

What facility are we dealing with?

>>> from electricitylci.generation import get_facilities_w_fuel_region
>>> facilities_w_fuel_region = get_facilities_w_fuel_region()
>>> facilities_w_fuel_region.query("`Balancing Authority Code` == 'HST'")
     eGRID_ID FuelCategory PrimaryFuel  PercentGenerationfromDesignatedFuelCategory State  NERC Balancing Authority Code Balancing Authority Name
403       665        MIXED          NG                                     0.009369    FL  SERC                      HST        City of Homestead

In 2016, this facility was a natural gas powered plant. By 2020, it switched to a DFO-based MIXED fuel facility. In 2023, it's a natural gas powered MIXED fuel facility. The percent from designated fuel category (NG) is quite low: 0.94%, and is progressively getting smaller (i.e., 45% in 2021, 0.98% in 2022, 0.94% in 2023). The at-grid consumption mix for 2016 baseline is purely from Homestead generation mix, which only has GAS generation.

In 2020, City of Homestead's at-grid consumption mix was entirely made up of trades (e.g., with Southern Company Services, Duke Energy Carolinas, Florida Power & Light Co., JEA, Seminole Electric Cooperative, and Southeastern Power Admin). There was no at-grid generation for Homestead.

In 2021 and 2022, no generation and the at-grid consumption mix for Homestead is empty.

It looks like trade data kept it alive in 2020.

What happened to facility 665?

>>> from electricitylci.eia923_generation import eia923_download_extract
>>> gen_fuel_data = eia923_download_extract(2023)
>>> gen_fuel_data.query("`Plant Id` == 665")
       Plant Id Plant Name State NAICS Code  ... Reported Fuel Type Code  YEAR Total Fuel Consumption MMBtu Net Generation (Megawatthours)
15279       665   G W Ivey    FL         22  ...                     DFO  2023                          870                      -3927.852
15280       665   G W Ivey    FL         22  ...                      NG  2023                            8                        -37.148
>>> gen_fuel_data.query("`Plant Id` == 665").describe()
       Total Fuel Consumption MMBtu  Net Generation (Megawatthours)
count                      2.000000                        2.000000
mean                     439.000000                    -1982.500000
std                      609.526045                     2751.143182
min                        8.000000                    -3927.852000
25%                      223.500000                    -2955.176000
50%                      439.000000                    -1982.500000
75%                      654.500000                    -1009.824000
max                      870.000000                      -37.148000

It's there, but has negative net generation, so it gets filtered from our facility list.

Let's look at the trading data.

>>> from electricitylci.eia_io_trading import _read_ba
>>> df_BA_NA, ba_cols, ferc_list = _read_ba()

We see HST in the ba_cols list.
Note that _read_ba has a hard-coded fix to drop HECO (Hawaiian balancing authority).

>>> from electricitylci.eia_io_trading import _read_bulk
>>> from electricitylci.eia_io_trading import _make_net_gen
>>> NET_GEN_ROWS, BA_TO_BA_ROWS, DEMAND_ROWS = _read_bulk(ba_cols)
>>> df_net_gen = _make_net_gen(year, ba_cols, NET_GEN_ROWS)
>>> df_net_gen.loc[:, 'HST']
datetime
2023-01-01    0.0
2023-01-02    0.0
2023-01-03    0.0
2023-01-04    3.0
2023-01-05    0.0
             ... 
2023-12-27    2.0
2023-12-28    4.0
2023-12-29    1.0
2023-12-30    0.0
2023-12-31    0.0

There's 410 MWh of net generation for HST for 2023.

>>> from electricitylci.eia_io_trading import _read_eia_gen
>>> eia_gen_ba, eia860_ba_list = _read_eia_gen(2023)
>>> eia_gen_ba.query("`Balancing Authority Code` == 'HST'")
Empty DataFrame

So HST does not show up in EIA, where we saw it had negative generation. This makes sense since this method employs the same facility filters.

>>> from electricitylci.eia_io_trading import _read_ca_imports
>>> from electricitylci.eia_io_trading import _make_net_gen_sum
>>> df_CA_Imports_Gen, df_CA_Imports_Rows = _read_ca_imports(year)
>>> df_net_gen_sum = _make_net_gen_sum(df_net_gen, eia_gen_ba, df_CA_Imports_Gen)
>>> df_net_gen_sum.loc['HST', :]
0    410.0

The 410 is still there for HST.

>>> from electricitylci.bulk_eia_data import ba_exchange_to_df
>>> df_ba_trade = ba_exchange_to_df(BA_TO_BA_ROWS, data_type='ba_to_ba')
>>> df_ba_trade['ba_to_ba'] = df_ba_trade['ba_to_ba'].astype('int')
>>> from_filt = df_ba_trade['from_region'] == 'HST'
>>> to_filt = df_ba_trade['to_region'] == 'HST'
>>> df_ba_trade.loc[(from_filt | to_filt), :]
      from_region to_region   datetime ba_to_ba
34240         FPL       HST 2023-01-01     1433
34241         FPL       HST 2023-01-02     1470
34242         FPL       HST 2023-01-03     1506
34243         FPL       HST 2023-01-04     1554
34244         FPL       HST 2023-01-05     1530
...           ...       ...        ...      ...
40703         HST       FPL 2023-12-27    -1399
40704         HST       FPL 2023-12-28    -1167
40705         HST       FPL 2023-12-29    -1142
40706         HST       FPL 2023-12-30    -1108
40707         HST       FPL 2023-12-31    -1158

[723 rows x 4 columns]

There are definitely trades happening to/from HST.

Who with?

>>> df_ba_trade.loc[from_filt, 'to_region'].unique().tolist()
['FPL']
>>> df_ba_trade.loc[to_filt, 'from_region'].unique().tolist()
['FPL']

Okay, so only trades with Florida Light & Power.

>>> from electricitylci.eia_io_trading import _make_trade_pivot
>>> from electricitylci.eia_io_trading import _make_square_pivot
>>> df_trade_pivot = _make_trade_pivot(year, ba_cols, df_ba_trade)
>>> df_trade_pivot = _make_square_pivot(df_trade_pivot, ba_cols)
>>> df_trade_pivot.loc['HST', :].sum()
0.0
>>> df_trade_pivot.loc[:, 'HST'].sum()
0.0

Hmm. The trades appear to disappear during the pivot.

Let's dig into the trade pivot.

>>> trade_df = df_ba_trade.copy()
>>> ba_trade = trade_df.set_index('datetime')
>>> ba_trade['transacting regions'] = ba_trade['from_region'] + '-' + ba_trade['to_region']

There are 723 rows with HST trading.

>>> filt1 = ba_trade['from_region'].isin(ba_cols)
>>> filt2 = ba_trade['to_region'].isin(ba_cols)
>>> filt = filt1 & filt2
>>> ba_trade = ba_trade[filt]

There are still 723 rows with HST. Removes mainly states.

>>> df_ba_trade_pivot = ba_trade.pivot(columns='transacting regions', values='ba_to_ba')
>>> df_ba_trade_sum = df_ba_trade_pivot.sum(axis=0).to_frame()
>>> df_ba_trade_sum = df_ba_trade_sum.reset_index()
>>> df_ba_trade_sum.columns = ['BAAs','Exchange']
>>> df_ba_trade_sum[['BAA1', 'BAA2']] = df_ba_trade_sum['BAAs'].str.split('-', n=1, expand=True)
>>> df_ba_trade_sum = df_ba_trade_sum.rename(columns={'BAAs': 'Transacting BAAs'})

Creating the to/from perspectives:

>>> df_trade_sum_1_2 = df_ba_trade_sum.groupby(['BAA1', 'BAA2','Transacting BAAs'], as_index=False)[['Exchange']].sum()
>>> df_trade_sum_2_1 = df_ba_trade_sum.groupby(['BAA2', 'BAA1', 'Transacting BAAs'], as_index=False)[['Exchange']].sum()
>>> df_trade_sum_1_2.columns = ['BAA1_1_2', 'BAA2_1_2','Transacting BAAs_1_2', 'Exchange_1_2']
>>> df_trade_sum_2_1.columns = ['BAA2_2_1', 'BAA1_2_1','Transacting BAAs_2_1', 'Exchange_2_1']
>>> df_concat_trade = pd.concat([df_trade_sum_1_2, df_trade_sum_2_1], axis=1)
>>> df_concat_trade['Exchange_1_2_abs'] = df_concat_trade['Exchange_1_2'].abs()
>>> df_concat_trade['Exchange_2_1_abs'] = df_concat_trade['Exchange_2_1'].abs()
>>> my_filt = df_concat_trade['Transacting BAAs_1_2'].str.contains("HST")
>>> df_concat_trade.loc[my_filt, :]
    BAA1_1_2 BAA2_1_2 Transacting BAAs_1_2  Exchange_1_2 BAA2_2_1 BAA1_2_1 Transacting BAAs_2_1  Exchange_2_1  Exchange_2_1_abs  Exchange_1_2_abs
92       FPL      HST              FPL-HST      617951.0      FPL      SEC              SEC-FPL     7254387.0         7254387.0          617951.0
110      HST      FPL              HST-FPL     -632433.0      IID     CISO             CISO-IID    -2747339.0         2747339.0          632433.0

Something doesn't look right here. The "1_2" fields do not align with the "2_1" fields.

Maybe try sorting the BAA names to better align them.

>>> df_trade_sum_1_2.head()
   BAA1  BAA2 Transacting BAAs   Exchange
0  AECI  MISO        AECI-MISO  1025915.0
1  AECI   SPA         AECI-SPA -1459246.0
2  AECI  SWPP        AECI-SWPP -2095465.0
3  AECI   TVA         AECI-TVA  1198716.0
4   AVA  BPAT         AVA-BPAT -4222594.0
>>> df_trade_sum_2_1.head()
  BAA2  BAA1 Transacting BAAs   Exchange
0  AECI  MISO        MISO-AECI -1017804.0
1  AECI   SPA         SPA-AECI  1459511.0
2  AECI  SWPP        SWPP-AECI  2356114.0
3   AVA  BPAT         BPAT-AVA  5997744.0
4   AVA  CHPD         CHPD-AVA     5628.0

Nope. These data frames do not line up (one BA trading with another does not guarantee that the other BA trades with the first).

How to correctly join these two data frames?

Merge on the common keys ('from' and 'to')
Fill NaNs
Re-create the transacting BAA strings

>>> df_concat_trade = pd.merge(left=df_trade_sum_1_2, right=df_trade_sum_2_1, how='outer', left_on=['BAA1_1_2', 'BAA2_1_2'], right_on=['BAA2_2_1', 'BAA1_2_1'])
>>> df_concat_trade['BAA1_1_2'] = df_concat_trade['BAA1_1_2'].fillna(df_concat_trade['BAA2_2_1'])
>>> df_concat_trade['BAA2_1_2'] = df_concat_trade['BAA2_1_2'].fillna(df_concat_trade['BAA1_2_1'])
>>> df_concat_trade['BAA1_2_1'] = df_concat_trade['BAA1_2_1'].fillna(df_concat_trade['BAA2_1_2'])
>>> df_concat_trade['BAA2_2_1'] = df_concat_trade['BAA2_2_1'].fillna(df_concat_trade['BAA1_1_2'])
>>> df_concat_trade['Exchange_1_2'] = df_concat_trade['Exchange_1_2'].fillna(0)
>>> df_concat_trade['Exchange_2_1'] = df_concat_trade['Exchange_2_1'].fillna(0)
>>> df_concat_trade['Transacting BAAs_1_2'] = df_concat_trade['BAA1_1_2'] + "-" + df_concat_trade['BAA2_1_2']
>>> df_concat_trade['Transacting BAAs_2_1'] = df_concat_trade['BAA1_2_1'] + "-" + df_concat_trade['BAA2_2_1']
>>> df_concat_trade.head()
  BAA1_1_2 BAA2_1_2 Transacting BAAs_1_2  Exchange_1_2 BAA2_2_1 BAA1_2_1 Transacting BAAs_2_1  Exchange_2_1  Exchange_1_2_abs  Exchange_2_1_abs
0     AECI     MISO            AECI-MISO     1025915.0     AECI     MISO            MISO-AECI    -1017804.0         1025915.0         1017804.0
1     AECI      SPA             AECI-SPA    -1459246.0     AECI      SPA             SPA-AECI     1459511.0         1459246.0         1459511.0
2     AECI     SWPP            AECI-SWPP    -2095465.0     AECI     SWPP            SWPP-AECI     2356114.0         2095465.0         2356114.0
3     AECI      TVA             AECI-TVA     1198716.0     AECI      TVA             TVA-AECI           0.0         1198716.0               0.0
4      AVA     BPAT             AVA-BPAT    -4222594.0      AVA     BPAT             BPAT-AVA     5997744.0         4222594.0         5997744.0

Now we see the 'AECI-TVA' exchange (Exchange_1_2) is on the same line as the 'TVA-AECI' trade (Exchange_2_1), the latter being filled in with the code above since this exchange did not show up in the BA-to-BA trades data.

We can continue!

Back at df_trade_pivot, we see:

>>> df_trade_pivot = _make_trade_pivot(year, ba_cols, df_ba_trade)
>>> df_trade_pivot.loc['HST', :].sum()
KeyError
>>> df_trade_pivot.loc[:, 'HST'].sum()
625192.0

Skipping ahead, we see that HST now has quite a slew of import location:

>>> BAA_final_trade.query("`import BAA` == 'HST'").query("fraction > 0")
     export BAA import BAA  fraction                              export_name        import_name
2002        DUK        HST  0.000146                    Duke Energy Carolinas  City of Homestead
2005       FMPP        HST  0.000087             Florida Municipal Power Pool  City of Homestead
2006        FPC        HST  0.001111                Duke Energy Florida, Inc.  City of Homestead
2007        FPL        HST  0.479815                Florida Power & Light Co.  City of Homestead
2016        HST        HST  0.479981                        City of Homestead  City of Homestead
2038         SC        HST  0.000011  South Carolina Public Service Authority  City of Homestead
2041        SEC        HST  0.021923            Seminole Electric Cooperative  City of Homestead
2042       SEPA        HST  0.000068        Southeastern Power Administration  City of Homestead
2043       SOCO        HST  0.008599  Southern Company Services, Inc. - Trans  City of Homestead
2048        TAL        HST  0.000020                      City of Tallahassee  City of Homestead
2049        TEC        HST  0.008238                   Tampa Electric Company  City of Homestead

Implemented the fix in trading, and we have new trades for HST!

minor tweaks associated with NETL-RIC#331

README update, new TOML, uptick v3, add author

update archive background data method; addresses NETL-RIC#325

tick up version to 3; addresses NETL-RIC#325

Navigation Menu

Repository navigation

Get to the Milestone #325

Renewable Generation Without Inputs

Consumption Mix with No Generation

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Get to the Milestone #325

Description

Activity

dt-woods commented on Mar 24, 2026

Renewable Generation Without Inputs

dt-woods commented on Mar 25, 2026

dt-woods commented on Mar 25, 2026

Consumption Mix with No Generation

dt-woods commented on Mar 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions