Skip to content

dockerContainerStats subscription returns frozen NetIO/BlockIO values #2007

@jandrop

Description

@jandrop

Environment

Unraid OS Version: 7.3 (Unraid OS)

Are you using a reverse proxy? No — verified by querying the API directly on the server's IP at http://192.168.1.132/graphql.

Pre-submission Checklist

  • I have verified that my Unraid OS is up to date
  • I have tested this issue by accessing my server directly (not through a reverse proxy)
  • This is not an Unraid Connect related issue

Issue Description

The dockerContainerStats GraphQL subscription is documented as emitting live runtime stats for every running Docker container, but the cumulative NetIO and BlockIO string fields stay frozen at the first sample for the entire lifetime of the subscription. CPU% and memory percentages DO update correctly in each emission, so the subscription itself is alive — only the I/O counters are stale.

This breaks any consumer that derives per-second download/upload speeds by taking the delta of netIO between two consecutive emissions (e.g. a third-party dashboard rendering a "live speed" cell per container): the delta is always zero.

Steps to Reproduce

  1. SSH into an Unraid 7.3 server with at least one container that has measurable network traffic (a torrent client actively downloading is ideal).

  2. Subscribe to dockerContainerStats with any GraphQL client that supports graphql-transport-ws. Minimal Node example:

    const ws = new WebSocket('ws://<host>/graphql', 'graphql-transport-ws');
    ws.on('open', () => ws.send(JSON.stringify({
      type: 'connection_init',
      payload: { 'x-api-key': API_KEY }  // lowercase header is required
    })));
    ws.on('message', (raw) => {
      const m = JSON.parse(raw);
      if (m.type === 'connection_ack') ws.send(JSON.stringify({
        id: '1', type: 'subscribe',
        payload: { query: 'subscription { dockerContainerStats { id cpuPercent netIO } }' }
      }));
      if (m.type === 'next') console.log(new Date().toISOString(),
        m.payload.data.dockerContainerStats);
    });
  3. Watch the output for ~15 seconds while the target container has active traffic.

Expected Behavior

netIO (and blockIO) should reflect fresh cumulative byte counts on every emission, so the receiving client can compute rates via delta:

14:32:01  netIO=40.1GB / 219.7GB
14:32:02  netIO=40.2GB / 219.7GB   ← Δ ≈ 100MB rx in 1s = 100 MB/s
14:32:03  netIO=40.3GB / 219.7GB

This matches what /containers/<id>/stats?stream=true on the Docker UNIX socket returns (verified directly — see below).

Actual Behavior

netIO is identical across every emission for the full lifetime of the subscription, even while the container has active traffic:

14:32:01  netIO=40.1GB / 219.7GB
14:32:02  netIO=40.1GB / 219.7GB   ← Δ = 0 B
14:32:03  netIO=40.1GB / 219.7GB
...
14:32:13  netIO=40.1GB / 219.7GB   ← still 0 B after 12 emissions

Yet during those same 12 seconds, the Docker daemon's socket shows real growth on the same container:

$ curl -s --unix-socket /var/run/docker.sock \
    http://localhost/containers/qbittorrent/stats?stream=false \
    | jq '.networks.eth0 | {rx_bytes, tx_bytes}'
# t=0
{ "rx_bytes": 40358834652, "tx_bytes": 219765369309 }

$ sleep 3 && curl -s ...   # same query
# t=3s
{ "rx_bytes": 40467550325, "tx_bytes": 219775453453 }

# Δrx ≈ 109 MB over 3 s → 36 MB/s real download rate

And docker stats --no-stream reports the same frozen 40.1GB / 220GB regardless of timing — pointing at the docker CLI itself, not the API.

Additional Context

Root cause appears to be in api/src/unraid-api/graph/resolvers/docker/docker-stats.service.ts, which spawns execa('docker', ['stats', '--format', ..., '--no-trunc']) and parses each output line. The docker CLI's "live" mode keeps the cumulative counters from its initial snapshot — they don't refresh across output ticks the way the per-container /stats socket endpoint does.

I've prepared a fix that swaps the CLI spawn for a direct dockerode stats stream per container (plus subscribing to docker events to add/remove streams on start/die/stop/kill/destroy). GraphQL schema is unchanged — only the data source for the resolver. Tests follow the docker-event.service.spec.ts template.

I plan to open a Work Intent for the fix referencing this bug.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions