Dendrite currently supports multiple ASICs
- Tofino: The Tofino 2 ASIC
- Tofino Stub: mocks responses, keeps some state, no data plane
- Softnpu: A virtual ASIC that runs in hypervisors and as a standalone software data plane.
- Chaos: similar to tofino stub but probabilistically returns errors, primarily used for testing error handling in complex/compound operations that require many underlying ASIC operations.
Each of these ASICs lives within their own cargo feature, meaning dendrite is always compiled for a particular ASIC. This makes a lot of sense for the ASICs we have today. Only one is for production use and everything else is for testing. However there are two factors motivating a rethink.
- We have new production ASIC coming on the scene with the X2
- There is a desire to package and treat softnpu as a first class ASIC for e2e testing environments (see the discussion in omicron#11133).
It's worth noting that a single dpd will likely never be managing multiple switch ASICs at once (much less multiple different switch ASICs at once), so the compile time choice of ASIC is still a reasonable path forward from that perspective. We could package and deliver multiple dpd binaries, and it would be the responsibility of the control plane to detect what hardware is present and launch the correct dpd daemon. However, that seems like a complexity leak. If Dendrite is responsible for managing and abstracting switching hardware, then we should not be punting on detection responsibility.
One could imagine a meta-daemon of some kind that is responsible for hardware detection and launching the correct type of dpd binary for the ASIC in play. However at that point we're probably at a comparable amount of work to having a single dpd binary that can handle multiple ASICs. Having one less daemon to manage is definitely a benefit. Another benefit of this approach is that fracturing the codebase over cargo features per ASIC is a bit cumbersome to deal with. It would be nice to have cargo check/build/nextest and the LSP just operating over the whole codebase rather than having to do everything multiple times and constantly reconfigure the LSP for work that touches multiple ASICs.
With all the above considerations, I think we should be moving toward a one build multiple ASIC approach. For softnpu in the near term and for X2 in the medium term.
Dendrite currently supports multiple ASICs
Each of these ASICs lives within their own cargo feature, meaning dendrite is always compiled for a particular ASIC. This makes a lot of sense for the ASICs we have today. Only one is for production use and everything else is for testing. However there are two factors motivating a rethink.
It's worth noting that a single
dpdwill likely never be managing multiple switch ASICs at once (much less multiple different switch ASICs at once), so the compile time choice of ASIC is still a reasonable path forward from that perspective. We could package and deliver multipledpdbinaries, and it would be the responsibility of the control plane to detect what hardware is present and launch the correctdpddaemon. However, that seems like a complexity leak. If Dendrite is responsible for managing and abstracting switching hardware, then we should not be punting on detection responsibility.One could imagine a meta-daemon of some kind that is responsible for hardware detection and launching the correct type of
dpdbinary for the ASIC in play. However at that point we're probably at a comparable amount of work to having a singledpdbinary that can handle multiple ASICs. Having one less daemon to manage is definitely a benefit. Another benefit of this approach is that fracturing the codebase over cargo features per ASIC is a bit cumbersome to deal with. It would be nice to have cargo check/build/nextest and the LSP just operating over the whole codebase rather than having to do everything multiple times and constantly reconfigure the LSP for work that touches multiple ASICs.With all the above considerations, I think we should be moving toward a one build multiple ASIC approach. For softnpu in the near term and for X2 in the medium term.