rdma/mlx5_dv: stamp RoCE IPv4 TOS from SL on RTR

[jleeh]

Summary

• rdma_set_option(RDMA_OPTION_ID_TOS) only programs SL / the NIC SQ (tx_prio3). mlx5_dv then moves the QP with ibv_modify_qp, and ConnectX builds the RoCEv2 IPv4 header from the RTR AV traffic_class.
• rdma_init_qp_attr often leaves that field 0, so the leaf trust dscp parks writes in Q0 even when host prio 3 is busy.
• On RTR, if the AV is present and traffic_class is unset, set traffic_class = SL << 5 (SL 3 → TOS 96 / DSCP 24). Both initiator complete_connect and nvmf accept use this helper.

Test plan

• Build SPDK with --with-rdma=mlx5_dv and pin into IM.
• Canary-roll one non-9 v2 IM (ma3-11 or ma5-11). Existing QPs will not pick this up until they reconnect.
• Confirm xe-1/1/5 egress-queues 3 on the same-leaf worker-9 moves with write IO (not only host tx_prio3).
• Q0 on that port should drop by about the RoCE write rate; leftover IMs still sending SL 3 will show DSCP 24 (CS3), not 26.

Made with Cursor