[deployment] Fix DSS IP in compose deployment: enforce in USS subnet

build/dev/README.md

@@ -46,6 +46,10 @@ When scaling up the local ecosystem by increasing `NUM_USS` or `NUM_NODES`, ther
 ### Host Port Mapping Limit (Max 99 Total Nodes)
 The script maps container ports to host ports using a two-digit padded index (`PADDED_NODE_IDX` from `01` to `99`).  So, NUM_USS*NUM_NODES may not exceed 99 without adjusting run_locally.sh.
 
+### Static IPs of DSS and DB containers limits (Max 128 Nodes)
+In the `dss_internal_network` network the `dss` and `crdb` or `ybdb` containers get assigned a static IP based on their USS and node IDs. In the last byte of the IP address, the first bit is used as a flag distinguishing the DSS from the DB container. E.g. for USS 3 node 2 the IPs would be `172.27.3.2` for the DSS container and `172.27.3.130` for the DB one. I.e. 7 bits only are available for the IP in the subnet, limiting to 128 the maximum number of nodes. Do note that in any case the previous limitation of 99 nodes would be reached before.
+
+
 ### Dynamic IP Limit (Max ~250 Total Nodes)
 The `dss_internal_network` is created with a dynamic IP pool range of `172.27.0.0/24`.  All DSS containers, bootstrap, and init containers are assigned dynamic IPs from this pool. Since `/24` has **253** usable host IPs (with `172.27.0.1` as gateway), starting more than ~250 containers requiring dynamic IPs will cause Docker to run out of IPs.  In addition to 2 containers per node, there are also bootstrap containers.
 

build/dev/docker-compose.yaml

@@ -15,7 +15,7 @@ services:
     hostname: db${USS_NODE_IDX:?}.uss${USS_IDX:?}.localutm
     networks:
       dss_internal_network:
-        ipv4_address: 172.27.${USS_IDX:?}.${USS_NODE_IDX:?}
+        ipv4_address: ${DB_IP:?}
     ports:
       - "81${PADDED_NODE_IDX:?}:8080"
       - "262${PADDED_NODE_IDX:?}:26257"
@@ -43,7 +43,7 @@ services:
     hostname: db${USS_NODE_IDX:?}.uss${USS_IDX:?}.localutm
     networks:
       dss_internal_network:
-        ipv4_address: 172.27.${USS_IDX:?}.${USS_NODE_IDX:?}
+        ipv4_address: ${DB_IP:?}
     ports:
       - "70${PADDED_NODE_IDX:?}:7000"
       - "90${PADDED_NODE_IDX:?}:9000"
@@ -171,8 +171,9 @@ services:
         required: false
     hostname: dss${USS_NODE_IDX:?}.uss${USS_IDX:?}.localutm
     networks:
-      - dss_internal_network
-      - interop_ecosystem_network
+      dss_internal_network:
+          ipv4_address: ${DSS_IP:?}
+      interop_ecosystem_network:
     ports:
       - "40${PADDED_NODE_IDX:?}:4000"
       - "80${PADDED_NODE_IDX:?}:80"

build/dev/run_locally.sh

@@ -80,6 +80,11 @@ for ((i=1; i<=NUM_USS; i++)); do
       export COMPOSE_PROFILES=${COMPOSE_PROFILES},bootstrap-${DB_TYPE}
     fi
 
+    # keep the DSS and the DB in the same subnet by using the first bit of the last byte of the IP
+    # e.g. for USS 3 node 2 the IPs would be 172.27.3.2 for the DSS container and 172.27.3.130 for the DB container
+    export DSS_IP="172.27.$USS_IDX.$USS_NODE_IDX"
+    export DB_IP="172.27.$USS_IDX.$((2#10000000 | USS_NODE_IDX))"
+
     # shellcheck disable=SC2086
     docker compose -f docker-compose.yaml -p "local_infra_${USS_IDX}-${USS_NODE_IDX}" $DC_COMMAND $DC_OPTIONS &
     sleep 0.1 # reduce probability of race condition in joining network at container start