Skip to content

Fix new entrant renaming - #144

Open
EllieKallmier wants to merge 1 commit into
mainfrom
fix-new-entrant-renaming
Open

EllieKallmier wants to merge 1 commit into
mainfrom
fix-new-entrant-renaming

Conversation

@EllieKallmier

Copy link
Copy Markdown
Member

Reworked the new entrants' renaming after regional granularity collapse/aggregation.

Previously: was just setting the new names to a standard '{geo_id} {technology}' formula; this lost some detail for units whose pre-collapse naming convention had different structures for representing technologies. Wasn't causing bugs or issues at this stage - but for mapping against constraints from plexos files or other AEMO-sourced name-keyed values, losing the names' structure had potential to be annoying AND was an easy fix. (+ keeps consistent with non-renamed units e.g. those in REZs).

Now: the actual 'old' geo_id string in a unit name gets re-keyed to the new geo_id (collapsed), keeping the rest of the name as-is. The first 'name' value for each collapsed-group is kept upon aggregation; this is the name that gets used to re-key. This implementation does assume that the naming convention for aggregated units (grouped on technology, geo_id and optionally resource_type) is the same for all units in each group, and that the 'new' geo_id is correctly assigned regardless of the old geo_id-like string prefix in each name. Based on current 7.5 workbook data this holds. Special case 'BOTN - Cethana' doesn't get aggregated with anything else so name gets passed through unchanged.

Example to illustrate the difference:

Original name Technology New geo_id Prev version 'name' New version 'name'
CSA Biomass Biomass SA SA Biomass SA Biomass
SNW OCGT Large OCGT (large GT) NSW NSW OCGT (large GT) NSW OCGT Large
SQ Battery - 2h Battery Storage (2hrs storage) QLD QLD Battery Storage (2hrs storage) QLD Battery - 2h

Built off #143

@EllieKallmier EllieKallmier added the module: templater Covers contents of `templater` module label Sep 3, 2026
@codecov

codecov Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
src/ispypsa/templater/new_entrants.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@EllieKallmier
EllieKallmier force-pushed the fix-new-entrant-renaming branch from 39e2bf0 to 0a9569e Compare September 14, 2026 05:42
@EllieKallmier
EllieKallmier marked this pull request as ready for review September 14, 2026 05:50
@dylanjmcconnell

Copy link
Copy Markdown
Member

Hey Ellie -

As discussed - perhaps could expand this out / make it slightly more readable?

    return new_entrants.groupby(
        group_key_columns + ["geo_id"], dropna=False, as_index=False
    ).agg({"name": "first", **{col: "mean" for col in value_columns}})

Something like:

name_aggregation_rule = {"name": "first"} 
value_column_aggregation_rules = {col: "mean" for col in value_columns}
return new_entrants.groupby(
          group_key_columns + ["geo_id"], dropna=False, as_index=False
      ).agg(name_aggregation_rule | value_aggregation_rules)

Or variation of ..

Also maybe move :

set(old_geo_ids) | _EXTRA_SUBREGION_IN_NAMES

From collapse_geo_id_to_granularity to _rekey_names_to_collapsed_geo_id

Otherwise looks good!

@nick-gorman nick-gorman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look good Ellie :)

value_columns: list[str],
) -> pd.DataFrame:
"""Groups by ``group_key_columns`` + 'geo_id' and averages ``value_columns``."""
# 'dropna=False' set to keep thermal generator rows (w/ NaN 'resource_type')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this comment incorrect?


# Allowed 'extra' subregion (not in sub_regional_geography) present in the names of
# some new entrant gas plant; see Open-ISP/ISPyPSA#131
_EXTRA_SUBREGION_IN_NAMES = {"WOO"}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure that the tests cover this? i.e. that might all pass fine even without it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually if you make Dylan change move set(old_geo_ids) | _EXTRA_SUBREGION_IN_NAMES then I think the test you added would then cover this.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module: templater Covers contents of `templater` module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants