Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ With `localSubnetL2Reachability` enabled, $[prodname] runs a userspace ARP/NDP r

### Userspace ARP/NDP responder

$[prodname] opens a raw socket on each host physical interface that has at least one pod IP or LoadBalancer VIP within its subnet, and replies to ARP/NDP requests for those IPs. The reply carries the node's MAC address, so the external host sends the workload's traffic to that node. From there $[prodname]'s normal dataplane forwards it the rest of the way — directly to a local pod for a pod IP, or load-balanced to a service backend for a LoadBalancer VIP. The responder only answers ARP/NDP; it does not change how traffic is forwarded once it reaches the node.
$[prodname] opens a raw socket on each host physical interface that has at least one pod IP or LoadBalancer VIP within its subnet, and replies to ARP/NDP requests for those IPs. The reply carries the node's MAC address, so the external host sends the workload's traffic to that node. From there $[prodname]'s normal data plane forwards it the rest of the way — directly to a local pod for a pod IP, or load-balanced to a service backend for a LoadBalancer VIP. The responder only answers ARP/NDP; it does not change how traffic is forwarded once it reaches the node.

### One node answers per LoadBalancer VIP

Expand All @@ -37,6 +37,8 @@ $[prodname] answers only for IPs that belong to an IP pool with no encapsulation
- **One node per LoadBalancer VIP.** There is no active-active high availability. If the owning node goes down, the VIP is unreachable until its `Node` object is removed and a new owner is elected.
- **Cloud provider ARP filtering.** Some cloud networks (for example, AWS VPC) filter ARP at the hypervisor. The responder will not work unless the VIP/pod IPs are assigned to the node's cloud interface. This is an infrastructure constraint outside $[prodname]'s control.
- **Heterogeneous subnets.** Node selection for LoadBalancer VIPs hashes over all nodes without checking whether a node has an interface on the VIP's subnet. This is only a concern when nodes do not share the same subnets.
- **Only `externalTrafficPolicy: Cluster` is supported for LoadBalancer services.** $[prodname] elects the node that answers for a VIP with a consistent hash over all nodes, independent of where the service's backends run. Under `externalTrafficPolicy: Local` the elected node may have no local backend, so traffic that reaches it is dropped. Leave `externalTrafficPolicy` at its default of `Cluster`.
- **kube-proxy in IPVS mode requires `strictARP`.** In IPVS mode, kube-proxy binds every LoadBalancer VIP to the `kube-ipvs0` dummy interface on *all* nodes. Because the Linux kernel answers ARP for any IP configured on the host by default, every node replies for the VIP — not just the one $[prodname] elected — so external hosts receive conflicting replies from several MAC addresses. Either run kube-proxy in `iptables` or `nftables` mode, or keep IPVS mode and set `strictARP: true` in kube-proxy's configuration (the `ipvs.strictARP` field of the `KubeProxyConfiguration`, which is stored as YAML inside the `kube-proxy` `ConfigMap`) so the kernel stops answering ARP for the dummy-interface VIPs and leaves it to $[prodname]'s responder.

:::note

Expand All @@ -62,14 +64,21 @@ kubectl patch felixconfiguration default --type='merge' \
-p '{"spec": {"localSubnetL2Reachability": "PodsAndLoadBalancers"}}'
```

The feature is `Disabled` by default. The setting is evaluated at dataplane startup, so **restart Felix (`calico-node`) for the change to take effect**. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx).
The feature is `Disabled` by default. This setting is read when the data plane starts, so changing it makes Felix (`calico-node`) **restart automatically** to pick up the new value — no manual restart is required. The feature takes effect once Felix finishes restarting. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx).

:::note

Enabling the feature alone does nothing. $[prodname] only answers for IPs in a no-encapsulation IP pool that overlaps a host subnet, so you must also create that pool.

:::

**Optional: tune periodic re-announcement.** While the feature is enabled, Felix periodically re-announces every IP it proxies with a gratuitous ARP / unsolicited Neighbor Advertisement, keeping upstream neighbor caches and switch forwarding tables warm even when the set of proxied IPs has not changed. `localSubnetL2ReachabilityRefreshInterval` controls how often this happens (default `120s`). Lower it if your upstream devices age out ARP/NDP entries quickly, or set it to `0` to disable periodic re-announcement and keep only the one-shot announcement sent when an IP is first proxied:

```bash
kubectl patch felixconfiguration default --type='merge' \
-p '{"spec": {"localSubnetL2ReachabilityRefreshInterval": "30s"}}'
Comment on lines +75 to +79
```

### Create a dedicated IP pool in the host subnet

Create an IP pool whose CIDR lies inside your host interface's subnet, with no encapsulation.
Expand Down
13 changes: 11 additions & 2 deletions calico/networking/configuring/local-subnet-l2-reachability.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ With `localSubnetL2Reachability` enabled, $[prodname] runs a userspace ARP/NDP r

### Userspace ARP/NDP responder

$[prodname] opens a raw socket on each host physical interface that has at least one pod IP or LoadBalancer VIP within its subnet, and replies to ARP/NDP requests for those IPs. The reply carries the node's MAC address, so the external host sends the workload's traffic to that node. From there $[prodname]'s normal dataplane forwards it the rest of the way — directly to a local pod for a pod IP, or load-balanced to a service backend for a LoadBalancer VIP. The responder only answers ARP/NDP; it does not change how traffic is forwarded once it reaches the node.
$[prodname] opens a raw socket on each host physical interface that has at least one pod IP or LoadBalancer VIP within its subnet, and replies to ARP/NDP requests for those IPs. The reply carries the node's MAC address, so the external host sends the workload's traffic to that node. From there $[prodname]'s normal data plane forwards it the rest of the way — directly to a local pod for a pod IP, or load-balanced to a service backend for a LoadBalancer VIP. The responder only answers ARP/NDP; it does not change how traffic is forwarded once it reaches the node.

### One node answers per LoadBalancer VIP

Expand All @@ -37,6 +37,8 @@ $[prodname] answers only for IPs that belong to an IP pool with no encapsulation
- **One node per LoadBalancer VIP.** There is no active-active high availability. If the owning node goes down, the VIP is unreachable until its `Node` object is removed and a new owner is elected.
- **Cloud provider ARP filtering.** Some cloud networks (for example, AWS VPC) filter ARP at the hypervisor. The responder will not work unless the VIP/pod IPs are assigned to the node's cloud interface. This is an infrastructure constraint outside $[prodname]'s control.
- **Heterogeneous subnets.** Node selection for LoadBalancer VIPs hashes over all nodes without checking whether a node has an interface on the VIP's subnet. This is only a concern when nodes do not share the same subnets.
- **Only `externalTrafficPolicy: Cluster` is supported for LoadBalancer services.** $[prodname] elects the node that answers for a VIP with a consistent hash over all nodes, independent of where the service's backends run. Under `externalTrafficPolicy: Local` the elected node may have no local backend, so traffic that reaches it is dropped. Leave `externalTrafficPolicy` at its default of `Cluster`.
- **kube-proxy in IPVS mode requires `strictARP`.** In IPVS mode, kube-proxy binds every LoadBalancer VIP to the `kube-ipvs0` dummy interface on *all* nodes. Because the Linux kernel answers ARP for any IP configured on the host by default, every node replies for the VIP — not just the one $[prodname] elected — so external hosts receive conflicting replies from several MAC addresses. Either run kube-proxy in `iptables` or `nftables` mode, or keep IPVS mode and set `strictARP: true` in kube-proxy's configuration (the `ipvs.strictARP` field of the `KubeProxyConfiguration`, which is stored as YAML inside the `kube-proxy` `ConfigMap`) so the kernel stops answering ARP for the dummy-interface VIPs and leaves it to $[prodname]'s responder.

:::note

Expand All @@ -62,14 +64,21 @@ kubectl patch felixconfiguration default --type='merge' \
-p '{"spec": {"localSubnetL2Reachability": "PodsAndLoadBalancers"}}'
```

The feature is `Disabled` by default. The setting is evaluated at dataplane startup, so **restart Felix (`calico-node`) for the change to take effect**. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx).
The feature is `Disabled` by default. This setting is read when the data plane starts, so changing it makes Felix (`calico-node`) **restart automatically** to pick up the new value — no manual restart is required. The feature takes effect once Felix finishes restarting. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx).

:::note

Enabling the feature alone does nothing. $[prodname] only answers for IPs in a no-encapsulation IP pool that overlaps a host subnet, so you must also create that pool.

:::

**Optional: tune periodic re-announcement.** While the feature is enabled, Felix periodically re-announces every IP it proxies with a gratuitous ARP / unsolicited Neighbor Advertisement, keeping upstream neighbor caches and switch forwarding tables warm even when the set of proxied IPs has not changed. `localSubnetL2ReachabilityRefreshInterval` controls how often this happens (default `120s`). Lower it if your upstream devices age out ARP/NDP entries quickly, or set it to `0` to disable periodic re-announcement and keep only the one-shot announcement sent when an IP is first proxied:

```bash
kubectl patch felixconfiguration default --type='merge' \
-p '{"spec": {"localSubnetL2ReachabilityRefreshInterval": "30s"}}'
Comment on lines +75 to +79
```

### Create a dedicated IP pool in the host subnet

Create an IP pool whose CIDR lies inside your host interface's subnet, with no encapsulation.
Expand Down