Skip to content

FinOps Hub upgraded from v13 to v14- two months data is missing in dashboard #2206

Description

@Vikesh1981

🐛 Problem

TODO: Finops Hub upgraded from v13 to v14- two months data is missing in dashboard

👣 Repro steps

TODO: Add repro steps below:

  1. Go to
  2. Click on
  3. Scroll down to
  4. See error

🤔 Expected

TODO: Add a clear and concise description of what you expected to happen.

📷 Screenshots

TODO: If applicable, add screenshots to help explain your problem. Remove if not applicable.

🔧 Environment

TODO: Complete the following (remove any that are not applicable):

  • FinOps hub version: (e.g., 0.10, 0.11, 12.0, 13.0)
  • Billing account type: EA (Enterprise Agreement) / MCA (Microsoft Customer Agreement) / CSP (Cloud Solution Provider)
  • Power BI report type: Storage-based / ADX (Azure Data Explorer)
  • Cost Management export: (e.g., FOCUS 1.2-preview, FOCUS 1.0, actual, amortized)

ℹ️ Additional context

TODO: Add any other context about the problem here. Remove if not applicable.

🙋‍♀️ Ask for the community

We could use your help:

  1. Please vote this issue up (👍) to prioritize it.
  2. Leave comments to help us solidify the vision.

Activity

  1. microsoft-github-policy-service commented on Jul 8, 2026

    @microsoft-github-policy-service

    Thank you for submitting this issue! We try to respond within 3 business days. In the meantime:

  2. Vikesh1981 commented on Jul 8, 2026

    @Vikesh1981
    Author

    FinOps hub version: v14
    Billing account type: EA (Enterprise Agreement)
    Power BI report type: Storage-based / ADX (Azure Data Explorer)
    Cost Management export: Dashboard

  3. fajterini commented on Jul 13, 2026

    @fajterini

    Latest 2 months or oldest 2 months of data are missing ?

  4. Vikesh1981 commented on Jul 13, 2026

    @Vikesh1981
    Author
  5. fajterini commented on Jul 13, 2026

    @fajterini

    I think this could be related to data retention, by default it's 13 months. If you had for example configured 15 months with v13, and used default configuration for v14 this would be changed back to 13. Upgrade of FinOps should not delete any data.

    You can see this in main.bicep file >> https://github.com/microsoft/finops-toolkit/blob/dev/src/templates/finops-hub/main.bicep

    @description('Optional. Number of months of data to retain in the ingestion container. Default: 13.')
    param ingestionRetentionInMonths int = 13
    
    @description('Optional. Number of months of data to retain in the Data Explorer *_final_v* tables. Default: 13.')
    param dataExplorerFinalRetentionInMonths int = 13
    
  6. Vikesh1981 commented on Jul 13, 2026

    @Vikesh1981
    Author
  7. changed the title [-]Finops Hub upgraded from v13 to v14- two months data is missing in dashboard[/-] [+]FinOps Hub upgraded from v13 to v14- two months data is missing in dashboard[/+] on Jul 22, 2026
  8. flanakin commented on Aug 26, 2026

    @flanakin
    Collaborator

    🤖 [AI] Thanks for confirming it's the oldest two months that are missing — that detail pinpoints the mechanism.

    ingestionRetentionInMonths / finalRetentionInMonths are Bicep parameters that default to 13 if not explicitly passed on a given deploy. Copy-FileToAzureBlob.ps1 then unconditionally overwrote settings.json's stored retention with whatever value it received — with no check for whether that value was actually lower than what was already configured. The Data Factory purge pipeline reads that value directly and ages out data older than the cutoff, oldest first.

    So if your v13→v14 redeploy call didn't explicitly re-specify a retention value above 13 months (for example, if it had been customized higher previously, or the v14 portal UI reshuffle — retention fields are now only shown when Data Explorer mode is explicitly selected — caused the field to be skipped), the redeploy would have silently reset settings.json back to the 13-month default, and the next pipeline run would have purged your two oldest months.

    We've opened #2288 to fix this: retention can now only grow on redeploy, never shrink, so this class of silent data loss shouldn't be possible going forward regardless of how a redeploy is triggered.

    This doesn't recover the two months you already lost, unfortunately — once purged, that data isn't recoverable from the pipeline. If you still have the original Cost Management export data or raw exports retained elsewhere for that period, re-ingesting from there would be the only way to backfill it.

  9. added a commit that references this issue on Oct 16, 2026
    c3611de
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions