Skip to content

[Celestica] Leh800bcls: Upgrade BCM logDropReasons OSS stub to real decoder - #1606

Open
gang-tao wants to merge 1 commit into
facebook:mainfrom
gang-tao:leh800bcls_agent_fix23
Open

[Celestica] Leh800bcls: Upgrade BCM logDropReasons OSS stub to real decoder#1606
gang-tao wants to merge 1 commit into
facebook:mainfrom
gang-tao:leh800bcls_agent_fix23

Conversation

@gang-tao

@gang-tao gang-tao commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Pre-submission checklist

  • [ ✓ ] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running pip install -r requirements-dev.txt && pre-commit install
  • [ ✓ ] pre-commit run
clang-format.............................................................Passed
shellcheck...........................................(no files to check)Skipped
shfmt................................................(no files to check)Skipped
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check json...........................................(no files to check)Skipped
check for merge conflicts................................................Passed
ruff check...........................................(no files to check)Skipped
ruff format..........................................(no files to check)Skipped
Prevent sai_impl in fboss manifest.......................................Passed

Summary

Following commit 3e2ff0c591 which introduced a no-op OSS stub for logDropReasons() to resolve SAI 15.4 linker failures, Broadcom XGS platforms in the open-source branch still lack actual drop reason decoding and logging. As a result, tests in AgentDropReasonTests.cpp consistently fail with timeouts.
The test expects to observe the drop reason L3_DST_DISCARD but instead logs ingress=[] egress=[]
indefinitely, or fails due to unexpected companion generic drop reasons (e.g. 60, 66) being reported alongside it.

I0905 09:15:52.168330 493017 AgentDropReasonTests.cpp:137] Drop reason test [L3 DST discard] observed: ingress=[] egress=[]
/var/FBOSS/fboss/fboss/agent/test/agent_hw_tests/AgentDropReasonTests.cpp:181: Failure
Value of: reasons.ingress.contains(kL3DstDiscard)
  Actual: false
Expected: true

I0905 09:15:52.169175 493017 AgentDropTestBase.cpp:186] Drop test [L3 DST discard] port 1: inUnicastPkts=29 outUnicastPkts=29 inDiscardsRaw=29 inDiscards=0 inDstNullDiscards=29 outDiscards=0 inAclDiscards=0 outForwardingDiscards=0
I0905 09:15:52.169196 493017 AgentDropTestBase.cpp:186] Drop test [L3 DST discard] port 3: inUnicastPkts=0 outUnicastPkts=0 inDiscardsRaw=0 inDiscards=0 inDstNullDiscards=0 outDiscards=0 inAclDiscards=0 outForwardingDiscards=0
I0905 09:15:52.169201 493017 AgentDropReasonTests.cpp:184] Drop reason test [L3 DST discard] final: ingress=[] egress=[]
/var/FBOSS/fboss/fboss/agent/test/agent_hw_tests/AgentDropReasonTests.cpp:155: Failure
Value of: reported.contains(expected)
  Actual: false
Expected: true
L3 DST discard: expected reason L3_DST_DISCARD not reported

/var/FBOSS/fboss/fboss/agent/test/agent_hw_tests/AgentDropTestBase.cpp:222: Failure
Value of: found
  Actual: false
Expected: true
L3 DST discard log: expected log containing 'DROP reasons ingress' not found

Root Cause

  • The current implementation of logDropReasons in npu/bcm/oss/SaiSwitchManager.cpp is an empty no-op stub since the proprietary implementation resides in Meta's closed-source facebook/ directory. No actual logs containing "DROP reasons ingress" are emitted on public builds.

Solution

  • Upgraded the no-op stub in the BCM-OSS-specific file fboss/agent/hw/sai/switch/npu/bcm/oss/SaiSwitchManager.cpp into a real decoder. It utilizes the standard, public experimental SAI headers <experimental/saidropextensions.h> under the BRCM_SAI_SDK_XGS_GTE_15_0 macro to decode and print named drop reasons. This fully avoids symbol conflicts with Meta's internal builds.
  • Mapped specific experimental ingress drop types such as SAI_PACKET_DROP_TYPE_INGRESS_RDISC (59), SAI_PACKET_DROP_TYPE_INGRESS_RUC (60), SAI_PACKET_DROP_TYPE_INGRESS_RIMDR (65), and SAI_PACKET_DROP_TYPE_INGRESS_RDROP (66) to their string names.

Test Plan

  • Successfully built the target fboss_hw_agent-sai_impl and multi_switch_agent_hw_test using SAI 15.4.0.0_odp and 14.2.0.0_odp.
  • Ran the case on both NPU0 and 1 in Leh800bcls and verified that AgentDropReasonTest.ingressL3DstDiscardDrop passes successfully.
    • NPU0
      ########## Warmboot test results (1/1, iter 1/1): Note: Google Test filter = AgentDropReasonTest.ingressL3DstDiscardDrop
      [==========] Running 1 test from 1 test suite.
      [----------] Global test environment set-up.
      [----------] 1 test from AgentDropReasonTest
      [ RUN      ] AgentDropReasonTest.ingressL3DstDiscardDrop
      [       OK ] AgentDropReasonTest.ingressL3DstDiscardDrop (10813 ms)
      [----------] 1 test from AgentDropReasonTest (10813 ms total)
      
      [----------] Global test environment tear-down
      [==========] 1 test from 1 test suite ran. (10813 ms total)
      [   PASSED ] 1 test.
      
      Failed to stop fboss_hw_agent@0.service: Unit fboss_hw_agent@0.service not loaded.
      Failed to stop fboss_hw_agent_for_testing_0.service: Unit fboss_hw_agent_for_testing_0.service not loaded.
      Removed "/etc/systemd/system/multi-user.target.wants/fboss_hw_agent_oss@0.service".
      Removed "/etc/systemd/system/fboss_hw_agent_oss@0.service".
      Failed to stop fboss_hw_agent@1.service: Unit fboss_hw_agent@1.service not loaded.
      Failed to stop fboss_hw_agent_for_testing_1.service: Unit fboss_hw_agent_for_testing_1.service not loaded.
      Removed "/etc/systemd/system/multi-user.target.wants/fboss_hw_agent_oss@1.service".
      Removed "/etc/systemd/system/fboss_hw_agent_oss@1.service".
      Cleaning up FBOSS HW Agent Service for index=0...
      Cleaning up FBOSS HW Agent Service for index=1...
      Running all tests took 0:00:51.709249 between 2026-09-05 13:40:20.871990 and 2026-09-05 13:41:12.581239
      [ PASSED ] cold_boot.AgentDropReasonTest.ingressL3DstDiscardDrop (14678 ms)
      [ PASSED ] warm_boot.AgentDropReasonTest.ingressL3DstDiscardDrop (10813 ms)
      Summary:
         PASSED : 2
         FAILED : 0
         SKIPPED : 0
         TIMEOUT : 0
      
      Test output stored at: hwtest_results_2026_Sep_05-01_41_12_PM.csv
      
    • NPU1
      ########## Warmboot test results (1/1, iter 1/1): Note: Google Test filter = AgentDropReasonTest.ingressL3DstDiscardDrop
      [==========] Running 1 test from 1 test suite.
      [----------] Global test environment set-up.
      [----------] 1 test from AgentDropReasonTest
      [ RUN      ] AgentDropReasonTest.ingressL3DstDiscardDrop
      [       OK ] AgentDropReasonTest.ingressL3DstDiscardDrop (9751 ms)
      [----------] 1 test from AgentDropReasonTest (9751 ms total)
      
      [----------] Global test environment tear-down
      [==========] 1 test from 1 test suite ran. (9751 ms total)
      [   PASSED ] 1 test.
      
      Failed to stop fboss_hw_agent@0.service: Unit fboss_hw_agent@0.service not loaded.
      Failed to stop fboss_hw_agent_for_testing_0.service: Unit fboss_hw_agent_for_testing_0.service not loaded.
      Removed "/etc/systemd/system/multi-user.target.wants/fboss_hw_agent_oss@0.service".
      Removed "/etc/systemd/system/fboss_hw_agent_oss@0.service".
      Failed to stop fboss_hw_agent@1.service: Unit fboss_hw_agent@1.service not loaded.
      Failed to stop fboss_hw_agent_for_testing_1.service: Unit fboss_hw_agent_for_testing_1.service not loaded.
      Removed "/etc/systemd/system/multi-user.target.wants/fboss_hw_agent_oss@1.service".
      Removed "/etc/systemd/system/fboss_hw_agent_oss@1.service".
      Cleaning up FBOSS HW Agent Service for index=0...
      Cleaning up FBOSS HW Agent Service for index=1...
      Running all tests took 0:00:48.917083 between 2026-09-05 13:42:45.095872 and 2026-09-05 13:43:34.012955
      [ PASSED ] cold_boot.AgentDropReasonTest.ingressL3DstDiscardDrop (13634 ms)
      [ PASSED ] warm_boot.AgentDropReasonTest.ingressL3DstDiscardDrop (9751 ms)
      Summary:
         PASSED : 2
         FAILED : 0
         SKIPPED : 0
         TIMEOUT : 0
      
      Test output stored at: hwtest_results_2026_Sep_05-01_43_34_PM.csv
      

@gang-tao
gang-tao requested a review from a team as a code owner September 5, 2026 11:18
@meta-cla meta-cla Bot added the CLA Signed label Sep 5, 2026
@meta-codesync

meta-codesync Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

@jchallag has imported this pull request. If you are a Meta employee, you can view this in D119363197.

case SAI_PACKET_DROP_TYPE_INGRESS_RDROP:
ingressNames.push_back("RDROP");
break;
case SAI_PACKET_DROP_TYPE_INGRESS_RUC:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SAI_PACKET_DROP_TYPE_INGRESS_RUC is explicitly classified as a non-drop reason in the internal decoder . This is inconsistent with the change

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jchallag
Thanks for the catch! You are completely right—SAI_PACKET_DROP_TYPE_INGRESS_RUC represents Routed Unicast (non-drop) packets, and treating it as an active drop reason is indeed incorrect and inconsistent.

@jchallag

jchallag commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

In general, i think there are many more drop reasons that needs to be taken care of. Did we check if this breaks other platforms?

@gang-tao

gang-tao commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

In general, i think there are many more drop reasons that needs to be taken care of. Did we check if this breaks other platforms?

Hi @jchallag
Great questions!
Regarding the drop reasons, we have added new drop reasons in the new commit.

As for other NPUs, I took that into consideration: the current code is placed in fboss/agent/hw/sai/switch/npu/bcm/oss/SaiSwitchManager.cpp, which is specific to the BCM OSS target platform. Other vendors have their own corresponding directories, so this will not introduce any duplicate symbol conflicts or break other platforms.

retested and case passed:

  • NPU0
Running all tests took 0:00:50.350603 between 2026-09-10 08:56:18.047683 and 2026-09-10 08:57:08.398286
[ PASSED ] cold_boot.AgentDropReasonTest.ingressL3DstDiscardDrop (14530 ms)
[ PASSED ] warm_boot.AgentDropReasonTest.ingressL3DstDiscardDrop (9786 ms)
Summary:
   PASSED : 2
   FAILED : 0
   SKIPPED : 0
   TIMEOUT : 0

Test output stored at: hwtest_results_2026_Sep_10-08_57_08_AM.csv

  • NPU1
Running all tests took 0:00:48.225962 between 2026-09-10 08:58:34.818577 and 2026-09-10 08:59:23.044539
[ PASSED ] cold_boot.AgentDropReasonTest.ingressL3DstDiscardDrop (13548 ms)
[ PASSED ] warm_boot.AgentDropReasonTest.ingressL3DstDiscardDrop (9704 ms)
Summary:
   PASSED : 2
   FAILED : 0
   SKIPPED : 0
   TIMEOUT : 0

Test output stored at: hwtest_results_2026_Sep_10-08_59_23_AM.csv

If you have further questions, feel free to leave additional comments. Thanks!

@gang-tao
gang-tao force-pushed the leh800bcls_agent_fix23 branch from 92d2002 to 9398af3 Compare September 10, 2026 05:42
@facebook-github-tools

Copy link
Copy Markdown

@gang-tao has updated the pull request. You must reimport the pull request before landing.

@gang-tao gang-tao changed the title [Celestica] Leh800bcls: Upgrade BCM logDropReasons OSS stub to real decoder and adapt genericReasons whitelist [Celestica] Leh800bcls: Upgrade BCM logDropReasons OSS stub to real decoder Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants