Add structured, physical read traces

[TomAugspurger]

This updates our logging setup to optionally emit structured, physical traces.

There are two main sets of changes:

1. Log physical reads at a TRACE level. This takes the "logical" reads from the user's program and traces the actual reads done by kvikio, which might be smaller thanks to kvikio's read splitting
2. Optionally output to a structured ndjson format, which is easier for downstream tools to parse.

Here's an example program:

import kvikio
import cupy as cp


remote_file = kvikio.RemoteFile.open_s3_url("s3://rapids-tpch/tpch-rs/scale-1/nation/part.0.parquet")
buf = cp.empty(remote_file.nbytes(), dtype=cp.uint8)

size = 1024

remote_file.read(buf, size, 0)
remote_file.read(buf, remote_file.nbytes() - size, size)

print("done")

When run with KVIKIO_LOG_LEVEL=TRACE KVIKIO_LOG_FORMAT=JSON KVIKIO_LOG_FILE=trace.ndjson python test_remote.py that writes

$ cat trace.ndjson 
{"event":"http","level":"trace","source":"https://rapids-tpch.s3.us-east-2.amazonaws.com/tpch-rs/scale-1/nation/part.0.parquet","start":1785255602404710891,"end":1785255602626790456,"threadId":4621804004366526777,"method":"HEAD","status":"ok","purpose":"metadata"}
{"event":"read","level":"trace","source":"https://rapids-tpch.s3.us-east-2.amazonaws.com/tpch-rs/scale-1/nation/part.0.parquet","start":1785255603191754437,"end":1785255603272868402,"offset":0,"size":1024,"threadId":6135855871742540183,"bytesRead":1024,"backend":"remote","status":"ok","isDeviceBuffer":true,"requestId":1,"method":"GET"}
{"event":"read","level":"trace","source":"https://rapids-tpch.s3.us-east-2.amazonaws.com/tpch-rs/scale-1/nation/part.0.parquet","start":1785255603273011822,"end":1785255603349365176,"offset":1024,"size":1226,"threadId":6135855871742540183,"bytesRead":1226,"backend":"remote","status":"ok","isDeviceBuffer":true,"requestId":2,"method":"GET"}

Closes #967

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[TomAugspurger]

/ok to test 19db970

[TomAugspurger]

/ok to test 908041a

[TomAugspurger]

This is deliberately private. It's been useful for me, and might be useful for others, but I don't think we should commit to maintaining it.

[TomAugspurger]

I wonder if this should just go under event: read with method: HEAD. Why split it out into its own thing?

[TomAugspurger]

This will need to be benchmarked to see how much overhead we've introduced.