diff --git a/bgp.rst b/bgp.rst index 8b45192c..479db054 100644 --- a/bgp.rst +++ b/bgp.rst @@ -11,6 +11,18 @@ BGP :local: :depth: 4 +BGP Overview +------------ + +Netris uses BGP to connect tenant VPCs and management networks to the world outside the Netris-managed fabric — upstream carriers, non-Netris-managed data center networks, WAN routers, and security appliances such as firewalls. + +There are two main types of BGP-enabled connectivity: + +- **SoftGate-terminated BGP** — the session terminates on a :doc:`SoftGate ` node. Typically used for connecting to ISPs, border routers, and similar upstream peers, since SoftGate also provides :doc:`NAT ` and :doc:`Layer-4 load balancing ` for that traffic. +- **:doc:`VPC Connect ` (switch-terminated BGP)** — the session terminates directly on a switch port or V-Net SVI. Typically used to establish line-rate connections to external gateways that don't require NAT or L4LB as part of the connection. + +Typical use cases are connecting the Netris-managed fabric to external networks such as WAN, non-Netris-managed parts of the data center network, Internet Service Providers, and similar external networks. + Basic BGP --------- diff --git a/images/add-server-cluster-selecting-servers-shared.png b/images/add-server-cluster-selecting-servers-shared.png index e058375d..c258103e 100644 Binary files a/images/add-server-cluster-selecting-servers-shared.png and b/images/add-server-cluster-selecting-servers-shared.png differ diff --git a/images/add-server-cluster-shared.png b/images/add-server-cluster-shared.png index 826292f2..372061ad 100644 Binary files a/images/add-server-cluster-shared.png and b/images/add-server-cluster-shared.png differ diff --git a/netris-ufm-integration.rst b/netris-ufm-integration.rst index 82d0689b..e17c8b0e 100644 --- a/netris-ufm-integration.rst +++ b/netris-ufm-integration.rst @@ -153,7 +153,7 @@ Netris Controller Configuration - netris * - NETRIS_CONTROLLER_PASSWORD - Password for authenticating with Netris Controller - - newNet0ps + - Password! * - NETRIS_VERIFY_SSL - Whether to verify SSL certificates when connecting to Netris Controller - true or false @@ -184,7 +184,7 @@ NVIDIA UFM Configuration - Whether to verify SSL certificates when connecting to UFM - true or false * - UFM_ID - - Unique identifier for this UFM instance + - Unique identifier for this UFM instance. This will be referenced from the :doc:`Server Cluster Template ` - ufm-lab * - UFM_PKEY_RANGE - Range of PKey IDs that can be allocated to clusters, in hexadecimal format @@ -259,7 +259,7 @@ Next, create a Server Cluster Template. .. note:: Netris Controller has no visibility into individual UFM NICs or ports, so the InfiniBand side of the template is just a single generic ``netris-ufm`` fabric entry — you don't (and can't) enumerate individual HCAs the way you would for Ethernet interfaces. Per-host HCA/port mapping happens automatically via the GUID sync described above. -4. Create Server Clusters +3. Create Server Clusters -------------------------- After setting up the template, create server clusters as described in :ref:`creating-server-cluster`. @@ -605,4 +605,4 @@ Additional Resources ===================== - `NVIDIA UFM Documentation `_ -- `Netris NVIDIA Spectrum-X Scenario `_ +- :doc:`Netris NVIDIA Spectrum-X Scenario ` diff --git a/server-cluster.rst b/server-cluster.rst index 9717a62d..f54f541f 100644 --- a/server-cluster.rst +++ b/server-cluster.rst @@ -27,7 +27,7 @@ In a Server Cluster Template you define: - What V-Nets to create and their types (VXLAN, VLAN, UFM, NVLink, and others) - What subnets to assign to each these V-Nets, when applicable -- Which server NICs map to these V-Nets +- Which server NICs map to these V-Nets (for Ethernet-based V-Nets only) - Other applicable settings specific to Ethernet, InfiniBand, and NVLink fabrics You can find more information about these primitives in the :doc:`V-Net ` and :doc:`IP Address Management ` Netris documentation. @@ -139,6 +139,9 @@ This example is common for AI fabrics where the frontend is based on Ethernet an } ] +.. tip:: + If your deployment includes more than one InfiniBand fabric (e.g., East-West GPU-to-GPU fabric and a dedicated InifiBand-based storage fabric) you can include multiple stanzas with "type":"netris-ufm". See :doc:`Netris UFM documentation ` for more details. + You can find more details about NVIDIA UFM (InfiniBand) integration in :doc:`Netris UFM documentation `. .. _nvlink-fabric-example: @@ -264,24 +267,14 @@ Each object in the **Vnets** JSON array may include a combination of the followi - A string specifying the IPv4 gateway for V-Net in CIDR notation - A string `specify` to force the operator to enter the gateway explicitly at cluster creation - - an object (see :ref:`advanced-uses`) with the following properties: - - - **assignType**: A string indicating the type of assignment. Only `auto` is permitted at this time. - - **allocation**: A string specifying the IPv4 address allocation, a supernet from which the child subnets will be derived. - - **childSubnetPrefixLength**: An integer specifying the prefix length for child subnets. - - **hostnum**: An integer specifying the host number for the gateway. + - an object (see :ref:`advanced-uses`) with the following properties: **assignType** (only `auto` is permitted at this time), **allocation** (the IPv4 address allocation, a supernet from which the child subnets will be derived), **childSubnetPrefixLength** (the prefix length for child subnets), and **hostnum** (the host number for the gateway). - **ipv4DhcpEnabled**: A boolean to enable/disable DHCP for IPv4. - **ipv6Gateway**: When `type:l2vpn` one of the following values: - A string specifying the IPv6 gateway for V-Net in CIDR notation - A string `specify` to force the operator to enter the gateway explicitly at cluster creation - - an object (see :ref:`advanced-uses`) with the following properties: - - - **assignType**: A string indicating the type of assignment. Only `auto` is permitted at this time. - - **allocation**: A string specifying the IPv6 address allocation, a supernet from which the child subnets will be derived. - - **childSubnetPrefixLength**: An integer specifying the prefix length for child subnets. - - **hostnum**: An integer specifying the host number for the gateway. + - an object (see :ref:`advanced-uses`) with the same **assignType**, **allocation**, **childSubnetPrefixLength**, and **hostnum** properties as above (IPv6-scoped). - **Ufm**: Nvidia UFM controller identifier (`ufm_id`) for V-Net `type:netris-ufm`. See :doc:`Netris UFM documentation ` for details. - **Pkey**: Pkey settings when V-Net `type:netris-ufm`. Only `auto` is permitted at this time. @@ -497,7 +490,11 @@ When you click the blue ``Add`` button, Netris will create the VPC, V-Nets, and - After creation, the template, the VPC, and the site fields are locked. - The same Netris NIC name must be used consistently across all server objects in a cluster. For example, when eth10 is assigned to a V-Net in the template, Netris will assign every switch port that corresponds to every server's eth10 to the same V-Net throughout the server cluster. -You can also assign one or more V-Nets to a separate VPC while assigning others to another VPC. To do so, select **Per template object** under the VPC mapping option. Doing so shows a drop down menu to select an existing VPC or create a new VPC for each V-Net specified in the template. +In some architectures you may want to assign some V-Nets to one VPC, while assigning others to another VPC when creating a Server Cluster. A typical use case might be assigning East-West and North-South scoped V-Nets to a new dedicated VPC, while assigning the management V-Net to an existing management VPC. Alternatively, you can use Labels to achieve a similar outcome as described :doc:`here `. + +To assign the templated V-Nets to different VPCs at the time of Server Cluster creation, select ``Per template object`` under the VPC mapping option and select an existing VPC or create a new VPC for each V-Net specified in the template. + +To assign multiple V-Nets to the same VPC, select them together as shown on the screenshot below. .. image:: images/server-cluster-add-per-vnet-vpc.png :align: center @@ -509,9 +506,7 @@ You can also assign one or more V-Nets to a separate VPC while assigning others .. note:: - - V-Nets selected together on one line will be placed into the same VPC. - - In the screenshot above the Mgmt V-Net will be added to the existing VPC-102 (Shared-infra), while one new VPC will be created with the EastWest and the NorthSouth V-Nets assigned to it. - - If you require a separate VPC per V-Net, add a dedicated line for each V-Net. + In the screenshot above the Mgmt V-Net will be added to the existing VPC-102 (Shared-infra), while one new VPC will be created with the EastWest and the NorthSouth V-Nets assigned to it. .. _server-cluster-shared-endpoints: @@ -530,7 +525,7 @@ To support this, Netris allows administrators to designate specific endpoints as .. raw:: html -
+

Figure. Selecting servers as shared endpoints

.. image:: images/add-server-cluster-shared.png :align: center @@ -538,11 +533,11 @@ To support this, Netris allows administrators to designate specific endpoints as .. raw:: html -
+

Figure. Server01 and server02 are added to th My-Cluster-01 server cluster as shared endpoints

Designating an endpoint as shared changes how the associated switch port is provisioned. Netris automatically configures the switch port in tagged mode, or the functional equivalent in environments such as InfiniBand or NVLink. -In essence: Shared endpoint = Tagged switch port +In essence: Shared endpoint = Tagged switch port. This is the primary behavioral change triggered by marking an endpoint as shared. diff --git a/supported-platform-matrix.rst b/supported-platform-matrix.rst index ae5681b0..9bf16564 100644 --- a/supported-platform-matrix.rst +++ b/supported-platform-matrix.rst @@ -82,7 +82,7 @@ Switch Fabric Management Functions - ✔ - ✔ * - AI switch fabric - - Optimizations for AI workloads. See AI functions section below. + - Optimizations for AI workloads. See AI Specific Functions section below. - ✔ - TBD - ✔ @@ -95,16 +95,16 @@ Switch Fabric Management Functions - TBD * - Upgrade/Downgrade - Upgrade & Downgrade of Netris agent through the controller + - ✔ - Coming Soon - - TBD - - TBD - Coming Soon + - TBD * - Custom Config Snippets - Custom configuration snippets for unique use cases. + - ✔ - Coming Soon - Coming Soon - - Coming Soon - - Coming Soon + - TBD * - :ref:`SNMPv2 polling ` - Enable SNMPv2 server. - ✔ @@ -135,7 +135,7 @@ Host Networking - ✔ - ✔ * - :doc:`Compute VXLAN/EVPN extension (EOH) ` - - Extend VXLAN/EVPN fabric into compute layer. See Compute integrations section below. + - Extend VXLAN/EVPN fabric into compute layer. See Compute Platform Integrations section below. - ✔ - ✔ - TBD @@ -149,7 +149,7 @@ Monitoring & Telemetry * - Function - Description - NVIDIA Cumulus - - Dell SONiC + - Dell-SONiC BCM-SONiC - Arista EOS @@ -214,7 +214,7 @@ External Routing Functions - Arista EOS - EdgeCore-SONiC * - External BGP (SoftGate) - - Terminate full routing table on SoftGate Gateway-server. + - Terminate full routing table on SoftGate Gateway-server. - ✔ - ✔ - ✔ @@ -527,7 +527,7 @@ Management Interfaces Netris and NOS versions compatibility matrix ============================================ -.. note:: SoftGate PRO has been discontinued and superceded by SoftGate HS. The SoftGate PRO column is retained for reference by existing deployments on older Netris versions. +.. note:: SoftGate PRO has been discontinued and superseded by SoftGate HS. The SoftGate PRO column is retained for reference by existing deployments on earlier Netris versions. .. list-table:: :header-rows: 0 diff --git a/supported-switch-hardware.rst b/supported-switch-hardware.rst index 7dd6167b..17392f28 100644 --- a/supported-switch-hardware.rst +++ b/supported-switch-hardware.rst @@ -5,7 +5,7 @@ Hardware Requirements Netris Controller ================= -We recommend three (for HA) Ubuntu 24.04 servers with the below specs. HA Netris Controller can run both in air-gapped or Internet accessible environments. +The Netris Controller requires at least three (for HA) Ubuntu 24.04 bare-metal servers with the specs below. HA Netris Controller can run in both air-gapped and Internet-accessible environments. .. list-table:: :header-rows: 0 @@ -15,12 +15,7 @@ We recommend three (for HA) Ubuntu 24.04 servers with the below specs. HA Netris - **RAM** - **SSD/NVME** - **Network** - * - Leaf/Spine 1-30 switches - - 8 - - 32 GB - - 1 TB - - 4x 25GbE+ NIC - * - Leaf/Spine 30-100 switches + * - Leaf/Spine 1-100 switches - 8 - 32 GB - 1 TB @@ -250,16 +245,20 @@ Dell - Broadcom Tomahawk IV - 64x QSFP-DD 400GbE - Dell-SONiC - - + - Spine only [#dell-fn1]_ - ✔ * - Dell - PowerSwitch Z Series Z9864F-ON - Tomahawk 5 - 64x OSFP800 - Dell-SONiC - - + - Routing VTEP only [#dell-fn2]_ - ✔ +.. [#dell-fn1] This platform's Broadcom Tomahawk IV ASIC does not support VXLAN tunnel termination under either NOS Dell currently offers for it (Dell Enterprise SONiC or SmartFabric OS10). It cannot function as a VTEP and is only suitable for a spine (underlay-only) role. This is a hardware/NOS limitation. See Dell's `Enterprise SONiC Distribution Compatibility Matrix `_. + +.. [#dell-fn2] This platform supports Layer-3 VXLAN routing but not Layer-2 VXLAN bridging or EVPN in Dell Enterprise SONiC. It can serve as a routed-overlay VTEP for Layer-3-only designs. Contact Netris support to confirm fit if your design requires VXLAN bridging or EVPN multihoming. This is a platform/NOS limitation. See Dell's `Enterprise SONiC Distribution Compatibility Matrix `_. + EdgeCore ======== .. list-table:: @@ -305,7 +304,7 @@ EdgeCore - Broadcom Tomahawk 3 - 32 x 400G QSFP-DD - EC-SONiC - - + - Contact Netris support [#edgecore-fn1]_ - ✔ * - EdgeCore - DCS511 (AS9737-32DB) @@ -322,6 +321,8 @@ EdgeCore - - ✔ +.. [#edgecore-fn1] VXLAN/VTEP support for this platform is not clearly confirmed in EdgeCore's published platform documentation. Contact Netris support before deploying this switch in a leaf/VTEP role. See EdgeCore's `DCS510 (AS9716-32D) datasheet `_. + Arista ======== .. list-table:: @@ -374,14 +375,14 @@ Arista - Tomahawk 5 - 32 x 800G OSFP - EOS - - + - Bridging VTEP only [#arista-fn1]_ - ✔ * - Arista - 7060X6-64PE - Tomahawk 5 - 64 x 800G OSFP + 2 x SFP+ - EOS - - + - Bridging VTEP only [#arista-fn1]_ - ✔ * - Arista - 7280R3 @@ -423,7 +424,7 @@ Arista - Tomahawk 3 - 128 x 100G or 32 x 400G - EOS - - + - Contact Netris support [#arista-fn2]_ - ✔ * - Arista - 7500R3 @@ -439,3 +440,7 @@ Arista - EOS - - ✔ + +.. [#arista-fn1] Current Arista EOS releases on this platform support VXLAN bridging but not VXLAN routing, per Arista's published datasheet. It can serve as a bridging-only VTEP. Contact Netris support if your design requires EVPN/VXLAN routing. This is a platform/NOS limitation. See Arista's `7060X6 Series datasheet `_. + +.. [#arista-fn2] VXLAN/VTEP support for this platform is not clearly confirmed in Arista's published documentation. Contact Netris support before deploying this switch in a leaf/VTEP role. See Arista's `7368X4 datasheet `_.