Skip to content

Codex/hammadkh/aft typed bindings public 20261120 - #6310

Open
hmod2001 wants to merge 3 commits into
openconfig:mainfrom
b4firex:codex/hammadkh/aft-typed-bindings-public-20261120
Open

hmod2001 wants to merge 3 commits into
openconfig:mainfrom
b4firex:codex/hammadkh/aft-typed-bindings-public-20261120

Conversation

@hmod2001

Copy link
Copy Markdown
Contributor

Title: AFT-6.1–6.4: carry forward Cisco filtering fixes and migrate to typed Ondatra bindings

Description:

This PR brings the AFT filtered-streaming changes from internal PR #2822 into public FeatureProfiles and replaces its handwritten global-filter configuration paths with generated Ondatra OpenConfig bindings.

It includes the AFT test implementations, shared collector changes, public dependency pin, binding verification test and reproduction instructions.

Background and changes inherited from #2822

PR #2822 adapted AFT-6.1 through AFT-6.4 to the Cisco IOS XR behavior observed during SIM testing. These include changes to test expectations and coverage, alongside configuration, collection and diagnostic fixes.

Area | Changes carried forward -- | -- AFT-6.1: policy references | Cisco allows a global filter to reference a policy before its definition exists. The test expects successful configuration, verifies the reference, then creates the policy and checks filtering. AFT-6.1: policy deletion | Cisco permits deletion of a policy that remains referenced by a global filter. The test verifies successful deletion, restores the policy/reference, and continues the atomic-deletion checks. AFT-6.1: policy modification | Replaces the complete single-statement policy where XR rejects individual field updates. The IPv4 policy transition uses another IPv4 prefix set. AFT-6.1: non-prefix matching | Uses an unconditional drop policy on Cisco to exercise an empty filtered view and live transitions. This substitution does not validate tag matching. AFT-6.2: dual-stack withdrawal | Allows Cisco default routes needed for next-hop resolution to remain visible after a policy swap. Test-created routes must still be withdrawn. Default routes require resolved next-hop objects; test-created routes retain exact ATE next-hop checks. AFT-6.3 / AFT-6.4 | Carries forward refreshed connections after reboot, dynamic neighbor resolution, per-network-instance handling and complete prefix-set replacement where XR rejects individual-entry updates. Shared AFT collection | Improves notification-path matching, synchronization and checks that unwanted prefixes disappear. Adds stream-progress and failure diagnostics. Cisco VRF prefixes resolve their next-hop objects through DEFAULT while preserving per-VRF prefix isolation.

Only AFT-6.1 was rerun from the prepared public checkout for this publication. No Jenkins job or hardware test was launched.

The validated local SIM profile used IOS XR 26.4.1.34I, OTG 1.48.0-5, an existing ten-second pre-traffic delay and process-wide doubled context deadlines. Those timing adjustments belong to the external SIM harness and are not included in this PR.

Running through Jenkins

Select Public as the test repository and this upstream PR’s number under Test PR. Clear conflicting branch/commit selections and old internal or fork repository overrides.

Use Go 1.26.3 and provide the normal testbed, image and SIM profile. See docs/aft-typed-global-filter-bindings.md for setup details and validation limits.

The validation above does not establish that all four tests pass under a runner with different timing settings; AFT-6.4 runtime validation is still outstanding.

**Title:** AFT-6.1–6.4: carry forward Cisco filtering fixes and migrate to typed Ondatra bindings

Description:

This PR brings the AFT filtered-streaming changes from internal PR #2822 into public FeatureProfiles and replaces its handwritten global-filter configuration paths with generated Ondatra OpenConfig bindings.

It includes the AFT test implementations, shared collector changes, public dependency pin, binding verification test and reproduction instructions.

Background and changes inherited from #2822

PR #2822 adapted AFT-6.1 through AFT-6.4 to the Cisco IOS XR behavior observed during SIM testing. These include changes to test expectations and coverage, alongside configuration, collection and diagnostic fixes.

Area Changes carried forward
AFT-6.1: policy references Cisco allows a global filter to reference a policy before its definition exists. The test expects successful configuration, verifies the reference, then creates the policy and checks filtering.
AFT-6.1: policy deletion Cisco permits deletion of a policy that remains referenced by a global filter. The test verifies successful deletion, restores the policy/reference, and continues the atomic-deletion checks.
AFT-6.1: policy modification Replaces the complete single-statement policy where XR rejects individual field updates. The IPv4 policy transition uses another IPv4 prefix set.
AFT-6.1: non-prefix matching Uses an unconditional drop policy on Cisco to exercise an empty filtered view and live transitions. This substitution does not validate tag matching.
AFT-6.2: dual-stack withdrawal Allows Cisco default routes needed for next-hop resolution to remain visible after a policy swap. Test-created routes must still be withdrawn. Default routes require resolved next-hop objects; test-created routes retain exact ATE next-hop checks.
AFT-6.3 / AFT-6.4 Carries forward refreshed connections after reboot, dynamic neighbor resolution, per-network-instance handling and complete prefix-set replacement where XR rejects individual-entry updates.
Shared AFT collection Improves notification-path matching, synchronization and checks that unwanted prefixes disappear. Adds stream-progress and failure diagnostics. Cisco VRF prefixes resolve their next-hop objects through DEFAULT while preserving per-VRF prefix isolation.

These changes are inherited from #2822 rather than introduced by the typed-binding migration.

Typed-binding migration added by this PR

Global-filter operations now use generated paths rooted at:

gnmi.OC().
    NetworkInstance(networkInstance).
    Afts().
    GlobalFilter()

The migration covers policy configuration, deletion, negative checks and CONFIG/STATE readback.

  • Removes handwritten global-filter configuration paths and manual policy-value JSON construction.
  • Preserves IPv4/IPv6 policy updates in one Set RPC.
  • Preserves atomic deletion of the filter references and policy definition.
  • Preserves Cisco JSON_IETF configuration encoding.
  • Allows reboot tests to use their refreshed gNMI transport.
  • Replaces two paired resilience readback requests with four typed leaf Get requests.
  • Retains direct subscriptions where tests inspect notification deletion paths and atomic flags; their global-filter paths are derived from generated queries.

Direct protocol access therefore remains for notification assertions. It is no longer used as the missing-binding workaround for global-filter configuration and readback.

Ondatra dependency

The PR pins the public supporting module:

replace github.com/openconfig/ondatra => github.com/b4firex/ondatra v0.0.0-20261120174419-ff486025d4fa

[Supporting Ondatra commit](b4firex/ondatra@ff48602)

This starts from upstream Ondatra v0.15.1/OpenConfig 5.11.0 and adds the missing openconfig-aft-global-filter.yang generator input and regenerated bindings. Updating the model version alone did not provide the global-filter API required by these tests.

Related tracking: [openconfig/ondatra#159](openconfig/ondatra#159).

A fresh checkout resolves the dependency publicly; no private module, sibling checkout or manual binding generation is required.

The dependency changes also update Go to 1.26.3 and additional direct/transitive dependencies, including gosnappi, KNE and gRPC. This is the validated dependency set; it has not been minimized to establish whether every version bump is necessary.

Validation

Check Result
Compilation of all four AFT test binaries PASS
Typed-binding wire-contract checks PASS
Public dependency resolution through the Jenkins Go entrypoint PASS
AFT-6.1 from the prepared public checkout PASS — all seven groups and four prefix-policy variants
AFT-6.2 Earlier SIM PASS with matching AFT and generated-binding source
AFT-6.3 Earlier SIM PASS with matching source, including cold reboot, six scale cases and per-network-instance filtering
AFT-6.4 Compiled; updated-binding runtime validation remains pending

Only AFT-6.1 was rerun from the prepared public checkout for this publication. No Jenkins job or hardware test was launched.

The validated local SIM profile used IOS XR 26.4.1.34I, OTG 1.48.0-5, an existing ten-second pre-traffic delay and process-wide doubled context deadlines. Those timing adjustments belong to the external SIM harness and are not included in this PR.

Running through Jenkins

Select Public as the test repository and this upstream PR’s number under Test PR. Clear conflicting branch/commit selections and old internal or fork repository overrides.

Use Go 1.26.3 and provide the normal testbed, image and SIM profile. See docs/aft-typed-global-filter-bindings.md for setup details and validation limits.

The validation above does not establish that all four tests pass under a runner with different timing settings; AFT-6.4 runtime validation is still outstanding.

Pin the public OpenConfig 5.11 global-filter regeneration and use typed policy configuration, deletion and CONFIG/STATE reads. Preserve batching, Cisco JSON_IETF encoding and fresh reconnect transports.

Include the wire-contract check and Jenkins reproduction instructions. AFT-6.1 passed all seven groups and four prefix variants from the prepared public checkout; all four AFT binaries compiled.
@hmod2001
hmod2001 requested review from a team as code owners November 20, 2026 22:19
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request completes the migration of AFT filtered-streaming tests to use typed Ondatra OpenConfig bindings. It carries forward critical filtering fixes required for Cisco IOS XR, ensuring consistent test behavior across vendors. The changes include updated test implementations, shared collector improvements, and robust diagnostic capabilities to facilitate future debugging and validation.

Highlights

  • Typed Ondatra Bindings: Migrated global-filter configuration paths from handwritten JSON construction to generated Ondatra OpenConfig bindings, improving type safety and maintainability.
  • Cisco IOS XR Compatibility: Integrated fixes for Cisco-specific behaviors, including support for unconditional drop policies, atomic deletion handling, and timing adjustments for simulator environments.
  • Dependency Updates: Pinned a public Ondatra fork to support missing global-filter YANG models and updated Go to 1.26.3 along with related dependencies like KNE and gRPC.
  • Diagnostic Improvements: Added comprehensive AFT diagnostic tools, including CLI command capture and independent gNMI readbacks, to assist in troubleshooting complex filtering scenarios.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@OpenConfigBot

Copy link
Copy Markdown

Pull Request Functional Test Report for #6310 / d9ea0aa

Virtual Devices

Device Test Status Test Documentation Job Raw Log
Arista cEOS status
status
status
status
AFT-6.4: pending authorization
AFT-6.3: pending authorization
AFT-6.1: pending authorization
AFT-6.2: pending authorization
AFT-6.4: AFT Prefix Filtering Dynamic Updates
AFT-6.3: AFT Prefix Filtering Resilience
AFT-6.1: AFT Prefix Filtering
AFT-6.2: AFT Prefix Filtering Dual-Stack
Cisco 8000E status
status
status
status
AFT-6.4: pending authorization
AFT-6.3: pending authorization
AFT-6.1: pending authorization
AFT-6.2: pending authorization
AFT-6.4: AFT Prefix Filtering Dynamic Updates
AFT-6.3: AFT Prefix Filtering Resilience
AFT-6.1: AFT Prefix Filtering
AFT-6.2: AFT Prefix Filtering Dual-Stack
Cisco XRd status
status
status
status
AFT-6.4: pending authorization
AFT-6.3: pending authorization
AFT-6.1: pending authorization
AFT-6.2: pending authorization
AFT-6.4: AFT Prefix Filtering Dynamic Updates
AFT-6.3: AFT Prefix Filtering Resilience
AFT-6.1: AFT Prefix Filtering
AFT-6.2: AFT Prefix Filtering Dual-Stack
Juniper ncPTX status
status
status
status
AFT-6.4: pending authorization
AFT-6.3: pending authorization
AFT-6.1: pending authorization
AFT-6.2: pending authorization
AFT-6.4: AFT Prefix Filtering Dynamic Updates
AFT-6.3: AFT Prefix Filtering Resilience
AFT-6.1: AFT Prefix Filtering
AFT-6.2: AFT Prefix Filtering Dual-Stack
Nokia SR Linux status
status
status
status
AFT-6.4: pending authorization
AFT-6.3: pending authorization
AFT-6.1: pending authorization
AFT-6.2: pending authorization
AFT-6.4: AFT Prefix Filtering Dynamic Updates
AFT-6.3: AFT Prefix Filtering Resilience
AFT-6.1: AFT Prefix Filtering
AFT-6.2: AFT Prefix Filtering Dual-Stack
Openconfig Lemming status
status
status
status
AFT-6.4: pending authorization
AFT-6.3: pending authorization
AFT-6.1: pending authorization
AFT-6.2: pending authorization
AFT-6.4: AFT Prefix Filtering Dynamic Updates
AFT-6.3: AFT Prefix Filtering Resilience
AFT-6.1: AFT Prefix Filtering
AFT-6.2: AFT Prefix Filtering Dual-Stack

Hardware Devices

Device Test Status Test Documentation Raw Log
Arista status
status
status
status
AFT-6.4: pending authorization
AFT-6.3: pending authorization
AFT-6.1: pending authorization
AFT-6.2: pending authorization
AFT-6.4: AFT Prefix Filtering Dynamic Updates
AFT-6.3: AFT Prefix Filtering Resilience
AFT-6.1: AFT Prefix Filtering
AFT-6.2: AFT Prefix Filtering Dual-Stack
Cisco status
status
status
status
AFT-6.4: pending authorization
AFT-6.3: pending authorization
AFT-6.1: pending authorization
AFT-6.2: pending authorization
AFT-6.4: AFT Prefix Filtering Dynamic Updates
AFT-6.3: AFT Prefix Filtering Resilience
AFT-6.1: AFT Prefix Filtering
AFT-6.2: AFT Prefix Filtering Dual-Stack
Juniper status
status
status
status
AFT-6.4: pending authorization
AFT-6.3: pending authorization
AFT-6.1: pending authorization
AFT-6.2: pending authorization
AFT-6.4: AFT Prefix Filtering Dynamic Updates
AFT-6.3: AFT Prefix Filtering Resilience
AFT-6.1: AFT Prefix Filtering
AFT-6.2: AFT Prefix Filtering Dual-Stack
Nokia status
status
status
status
AFT-6.4: pending authorization
AFT-6.3: pending authorization
AFT-6.1: pending authorization
AFT-6.2: pending authorization
AFT-6.4: AFT Prefix Filtering Dynamic Updates
AFT-6.3: AFT Prefix Filtering Resilience
AFT-6.1: AFT Prefix Filtering
AFT-6.2: AFT Prefix Filtering Dual-Stack

Help

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request transitions the AFT prefix filtering tests (AFT-6.1 through AFT-6.4) to use typed Ondatra and ygnmi bindings instead of raw gNMI paths. It introduces platform exceptions and robust diagnostic helpers to accommodate Cisco IOS XR limitations, such as replacing complete prefix-sets and using BGP-origin AFT summaries for VRF convergence. The review feedback highlights critical improvements: replacing t.Logf calls in asynchronous background goroutines within aft_cache.go to prevent flaky test panics, and replacing several usages of context.Background() with t.Context() in gNMI operations to ensure proper timeout and cancellation handling.

Comment on lines +551 to +573
if err != nil {
counters.receiveFailure.Store(&aftSubscriptionResponse{err: err})
if ctx.Err() == nil {
// Record the client queue at the actual receive failure,
// before its error waits behind any buffered notifications.
// This is not the DUT's MDT destination queue.
t.Logf("DUT %s NI=%s AFT stream receive failed UTC=%s: %v; CLIENT buffered notifications=%d/%d received=%d processed=%d (not the DUT queue)",
dut.Name(), networkInstance, time.Now().UTC().Format(time.RFC3339Nano), err,
len(buffer), cap(buffer), counters.received.Load(), counters.processed.Load())
select {
case buffer <- &aftSubscriptionResponse{err: err}:
case <-ctx.Done():
}
}
return
}
counters.received.Add(1)
counters.lastReceived.Store(time.Now().UnixNano())
if n.GetSyncResponse() {
counters.syncReceived.Add(1)
t.Logf("DUT %s gNMI AFT SyncResponse arrived; buffered notifications=%d/%d",
dut.Name(), len(buffer), cap(buffer))
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Calling t.Logf inside an asynchronous background goroutine is a known anti-pattern in Go tests. If the test finishes or is cancelled while the background goroutine is still receiving messages or handling errors, calling t.Logf will cause a panic (testing: t.Logf called after Test... has finished), leading to flaky test failures. Since these events are already logged or tracked via ss.Progress() and ListenUntil, we should safely remove these asynchronous logging calls.

			if err != nil {
				counters.receiveFailure.Store(&aftSubscriptionResponse{err: err})
				if ctx.Err() == nil {
					select {
					case buffer <- &aftSubscriptionResponse{err: err}:
					case <-ctx.Done():
					}
				}
				return
			}
			counters.received.Add(1)
			counters.lastReceived.Store(time.Now().UnixNano())
			if n.GetSyncResponse() {
				counters.syncReceived.Add(1)
			}

t.Logf("Sending typed AFT global-filter Set batch: %s", batch)
started := time.Now()
t.Logf("AFT global-filter Set start: UTC=%s NI=%s client=%p createdUTC=%s", started.UTC().Format(time.RFC3339Nano), cfg.VRFName, client, createdAt.UTC().Format(time.RFC3339Nano))
response, err := batch.Set(context.Background(), typedClient, ygnmi.WithEncoding(ConfigEncoding(dut)))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using context.Background() for gNMI operations in tests can cause the test to hang indefinitely if the gNMI server blocks or fails to respond. Since t.Context() is available, we should use it to ensure proper cancellation and timeout handling when the test finishes or is interrupted.

Suggested change
response, err := batch.Set(context.Background(), typedClient, ygnmi.WithEncoding(ConfigEncoding(dut)))
response, err := batch.Set(t.Context(), typedClient, ygnmi.WithEncoding(ConfigEncoding(dut)))

batch := &ygnmi.SetBatch{}
ygnmi.BatchDelete(batch, filter.Ipv4Policy().Config())
ygnmi.BatchDelete(batch, filter.Ipv6Policy().Config())
_, err = batch.Set(context.Background(), client)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using context.Background() for gNMI operations in tests can cause the test to hang indefinitely if the gNMI server blocks or fails to respond. Since t.Context() is available, we should use it to ensure proper cancellation and timeout handling when the test finishes or is interrupted.

Suggested change
_, err = batch.Set(context.Background(), client)
_, err = batch.Set(t.Context(), client)

req := &gpb.SetRequest{Update: updates}
_, setErr := client.Set(context.Background(), req)
t.Logf("Sending typed AFT global-filter negative-check Set batch: %s", batch)
_, setErr := batch.Set(context.Background(), client, ygnmi.WithEncoding(aftpf.ConfigEncoding(dut)))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using context.Background() for gNMI operations in tests can cause the test to hang indefinitely if the gNMI server blocks or fails to respond. Since t.Context() is available, we should use it to ensure proper cancellation and timeout handling when the test finishes or is interrupted.

Suggested change
_, setErr := batch.Set(context.Background(), client, ygnmi.WithEncoding(aftpf.ConfigEncoding(dut)))
_, setErr := batch.Set(t.Context(), client, ygnmi.WithEncoding(aftpf.ConfigEncoding(dut)))

}
req := &gpb.SetRequest{Delete: []*gpb.Path{policyDefinitionPath(policyName)}}
t.Logf("Sending policy-definition delete gNMI SetRequest: %v", req)
_, setErr := client.Set(context.Background(), req)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using context.Background() for gNMI operations in tests can cause the test to hang indefinitely if the gNMI server blocks or fails to respond. Since t.Context() is available, we should use it to ensure proper cancellation and timeout handling when the test finishes or is interrupted.

Suggested change
_, setErr := client.Set(context.Background(), req)
_, setErr := client.Set(t.Context(), req)

ygnmi.BatchDelete(batch, filter.Ipv4Policy().Config())
ygnmi.BatchDelete(batch, filter.Ipv6Policy().Config())
ygnmi.BatchDelete(batch, gnmi.OC().RoutingPolicy().PolicyDefinition(policyName).Config())
if _, err := batch.Set(context.Background(), client); err != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using context.Background() for gNMI operations in tests can cause the test to hang indefinitely if the gNMI server blocks or fails to respond. Since t.Context() is available, we should use it to ensure proper cancellation and timeout handling when the test finishes or is interrupted.

Suggested change
if _, err := batch.Set(context.Background(), client); err != nil {
if _, err := batch.Set(t.Context(), client); err != nil {

if _, err := client.Set(context.Background(), req); err != nil {
started := time.Now()
t.Logf("Sending prefix-set reference gNMI SetRequest UTC=%s: %v", started.UTC().Format(time.RFC3339Nano), req)
response, setErr := client.Set(context.Background(), req)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using context.Background() for gNMI operations in tests can cause the test to hang indefinitely if the gNMI server blocks or fails to respond. Since t.Context() is available, we should use it to ensure proper cancellation and timeout handling when the test finishes or is interrupted.

Suggested change
response, setErr := client.Set(context.Background(), req)
response, setErr := client.Set(t.Context(), req)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants