Draft: Adding model data support to fluid flow WAN model - #268
Open
caitlinross wants to merge 62 commits into
Open
caitlinross wants to merge 62 commits into
caitlinross wants to merge 62 commits into
Conversation
This reverts commit 9cd0c04 as the changes for it are in the digital-twin-sbir-develop-flow-control branch.
Add flowlet and switch-training logs, queued-before/drop fields, configurable switch schedulers, two-phase egress allocation, and fragment coalescing for cleaner rollback and ML training targets. Rename receive counters to received_fragments.
Buffer CSV rows in memory for both sequential and optimistic runs, then merge logs after simulation completion to avoid event-path file I/O and unsafe parallel appends. Refactor common terminal and switch logging paths, add CSV file-open checks, and keep optimistic logging rollback-safe through commit handlers. Use MPI_COMM_CODES consistently, remove dead event/config fields, document the large reverse-allocation event payload, clean up comments, and release switch queue storage during finalization.
This reverts commit eb4748f.
This reverts commit 9b29061.
This reverts commit 1127d8f.
Attach ROSS instrumentation-layer callbacks (--model-stats=1..4, --event-trace) to the fluid-flow WAN terminal and switch LPs so consumers of the ROSS binary instrumentation format (e.g. NetMaestro) get per-LP time series without parsing the model's CSV logs. Document both payload layouts byte-exactly in doc/model-stats-binary-format.md, decode them in scripts/parse-ross-model-stats.py (max_ports inferred from record size), add a demo recipe to the trace-traffic example doc, and add sequential + optimistic CI tests that assert parser round-trip, aggregate/per-port consistency, and unchanged model event counts against an uninstrumented baseline.
The stock 64 Gb deep buffers never cross the PAUSE high watermark with the example trace: rate feedback caps switch B's standing backlog near 1.7 Gb, so PAUSE stays idle. Add a demo variant config pair (fluid-flow-wan-trace-traffic-pause.yaml on a 2 Gb-buffer topology) where the same two A->C flows drive switch B across the 1.6 Gb watermark: B PAUSEs the A->B ingress (13 PAUSE / 12 RESUME frames) and the pause activity is visible in the CSV logs and in the ROSS instrumentation payloads. Stage both files into the build tree and document the variant, plus the config-relative path resolution and the pre-existing requirement that the config-relative logs/ directory exist before a run.
The CSV log paths resolve relative to the config file's directory, so the documented run commands failed from a clean build tree unless the logs/ directory was created by hand first. Create missing parent directories (rank 0 only, before the header write, which precedes every other open of the path) and drop the manual mkdir steps from the example docs.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.