diff --git a/.gitignore b/.gitignore index 4d9213bd..436e1c30 100644 --- a/.gitignore +++ b/.gitignore @@ -28,3 +28,7 @@ Thumbs.db references/ vendor/ bssh-server.yaml + +# tools/bench artifacts +tools/bench/interop/sshj/lib/ +*.class diff --git a/tools/bench/README.md b/tools/bench/README.md new file mode 100644 index 00000000..e091813f --- /dev/null +++ b/tools/bench/README.md @@ -0,0 +1,96 @@ +# bssh benchmark and interop harness + +Scripts for measuring `bssh-server` SFTP performance relative to OpenSSH on +the same host, capturing CPU flamegraphs, and validating interoperability +with third-party SFTP client stacks (sshj, paramiko). + +This harness produced the measurements recorded in issues +[#225](https://github.com/lablup/bssh/issues/225) and +[#226](https://github.com/lablup/bssh/issues/226), and found the paramiko +read-path hang tracked in [#227](https://github.com/lablup/bssh/issues/227). +Background: [#187](https://github.com/lablup/bssh/issues/187) and PR +[#224](https://github.com/lablup/bssh/pull/224). + +## Layout + +| Script | Purpose | +|---|---| +| `bench.sh` | Relative SFTP throughput: OpenSSH sshd vs bssh-server (optionally plus a baseline bssh build) | +| `profile.sh` | `perf` CPU profile and flamegraph of bssh-server during uploads | +| `interop/paramiko_test.py` | Round-trip integrity check with paramiko (Python SSH stack, used by Ansible) | +| `interop/sshj/run-sshj.sh` | Round-trip integrity and negotiation check with sshj (Java SSH stack, used by Cyberduck) | +| `lib.sh` | Shared setup: keys, configs, test file, server lifecycle, timed transfers | + +All servers run on loopback only, with keys and configs generated fresh under +`BENCH_DIR`. Nothing touches the user's real SSH configuration. + +## Prerequisites + +- A release build of the server: `cargo build --release --bin bssh-server` + (for `profile.sh`, prefer `CARGO_PROFILE_RELEASE_DEBUG=true cargo build --release --bin bssh-server` so stacks resolve) +- OpenSSH client (`ssh`, `sftp`, `ssh-keygen`); OpenSSH server (`/usr/sbin/sshd`) for `bench.sh` +- `profile.sh`: `perf`, `kernel.perf_event_paranoid <= 2`, and optionally `cargo install inferno` for SVG output +- `interop/paramiko_test.py`: `pip install paramiko` +- `interop/sshj/`: a JDK (`javac`/`java`, set `JAVA_HOME` if not on PATH), then `./fetch-deps.sh` once to download the pinned jars into `lib/` (not committed) + +## Configuration + +Everything is an environment variable with a default (see `lib.sh`): + +| Variable | Default | Meaning | +|---|---|---| +| `BENCH_DIR` | `${TMPDIR:-/dev/shm}/bssh-bench-$USER` | Working directory (keys, configs, test file, results) | +| `BSSH_SERVER_BIN` | `target/release/bssh-server` | Server binary under test | +| `BSSH_BASELINE_BIN` | unset | Optional second binary for before/after comparison (`bench.sh`) | +| `BSSH_PORT` / `SSHD_PORT` | `22200` / `22022` | Loopback ports | +| `FILE_SIZE_MIB` | `2048` | Test file size | +| `RUNS` | `3` | Timed runs per direction | +| `SERVER_CORE` / `CLIENT_CORE` | unset (no pinning) | `taskset` core lists; pin the server to one core to emulate a single-core container | +| `CIPHER` | `chacha20-poly1305@openssh.com` | Cipher forced on OpenSSH-client transfers | +| `PROFILE_UPLOADS`, `PERF_FREQ`, `PERF_CALLGRAPH` | `3`, `499`, `fp` | `profile.sh` knobs | + +## Examples + +```bash +# Single-core relative benchmark (the #187 scenario), cores 8 and 9: +SERVER_CORE=8 CLIENT_CORE=9 tools/bench/bench.sh + +# Before/after against an older build: +BSSH_BASELINE_BIN=/path/to/old/bssh-server tools/bench/bench.sh + +# Flamegraph of the upload path: +CARGO_PROFILE_RELEASE_DEBUG=true cargo build --release --bin bssh-server +SERVER_CORE=8 CLIENT_CORE=9 tools/bench/profile.sh + +# sshj interop (2 GiB round trip, negotiation log included): +tools/bench/interop/sshj/fetch-deps.sh +tools/bench/interop/sshj/run-sshj.sh + +# paramiko interop; --no-prefetch works today, the default reproduces #227: +. tools/bench/lib.sh && bench_setup && start_bssh +python3 tools/bench/interop/paramiko_test.py \ + --user "$USER" --key "$BENCH_DIR/bench_key_rsa" \ + --file "$BENCH_DIR/testfile_2048M" --remote-dir "$BENCH_DIR/up" --no-prefetch +stop_servers +``` + +## Methodology notes + +- Loopback removes the NIC bottleneck, so absolute numbers exceed any real + network; the point is the ratio between servers measured under identical + conditions. Pin both servers to the same single core (`SERVER_CORE`) to + reproduce the CPU-bound single-core container scenario from #187. +- Every transfer is verified byte-for-byte (`cmp`) before it counts. +- Keep `BENCH_DIR` on tmpfs (the default) so disk I/O does not dominate. +- Timing includes the SSH handshake; with multi-GiB files its share is a few + percent and identical across servers. + +## Known issues + +- `interop/paramiko_test.py` with default prefetch currently hangs against + bssh-server (pre-existing server bug, not a #224 regression); tracked in + [#227](https://github.com/lablup/bssh/issues/227). Once fixed, the default + invocation doubles as the regression test. +- FileZilla and WinSCP are GUI-only and cannot run headless; both use + PuTTY-derived transports, so a `psftp` (putty-tools) check is the closest + scriptable proxy if needed. diff --git a/tools/bench/bench.sh b/tools/bench/bench.sh new file mode 100755 index 00000000..9f3186ca --- /dev/null +++ b/tools/bench/bench.sh @@ -0,0 +1,49 @@ +#!/bin/bash +# Relative SFTP throughput: OpenSSH sshd vs bssh-server on the same host. +# +# Runs both servers on loopback (optionally pinned to the same CPU core to +# emulate a single-core container) and measures timed uploads and downloads +# with the OpenSSH sftp client. Set BSSH_BASELINE_BIN to also measure an +# older bssh-server build for a before/after comparison. +# +# Example (single-core scenario, big cores 8 and 9): +# SERVER_CORE=8 CLIENT_CORE=9 tools/bench/bench.sh +# +# See README.md for all configuration variables and methodology notes. + +set -u +cd "$(dirname "$0")" +. ./lib.sh + +bench_setup + +measure() { + local label=$1 port=$2 dir vals i + for dir in up dl; do + vals="" + for i in $(seq 1 "$RUNS"); do + vals="$vals $(sftp_xfer "$port" "$dir")" + done + echo "RESULT $label $dir$vals" + done +} + +echo "# cipher: $CIPHER, file: ${FILE_SIZE_MIB} MiB, runs: $RUNS, server core: ${SERVER_CORE:-unpinned}, client core: ${CLIENT_CORE:-unpinned}" +echo "# results are MiB/s per run; every transfer is integrity-checked with cmp" + +if start_sshd; then + measure openssh "$SSHD_PORT" + stop_servers +fi + +if start_bssh; then + measure bssh "$BSSH_PORT" + stop_servers +fi + +if [ -n "${BSSH_BASELINE_BIN:-}" ]; then + if start_bssh "$BSSH_BASELINE_BIN"; then + measure bssh-baseline "$BSSH_PORT" + stop_servers + fi +fi diff --git a/tools/bench/interop/paramiko_test.py b/tools/bench/interop/paramiko_test.py new file mode 100755 index 00000000..28e4eb17 --- /dev/null +++ b/tools/bench/interop/paramiko_test.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python3 +"""SFTP interop check against bssh-server using paramiko. + +Uploads a file, downloads it back, and verifies byte-for-byte integrity. + +Known issue: with paramiko's default prefetch (a burst of READ requests for +the whole file), the download currently hangs against bssh-server; see +https://github.com/lablup/bssh/issues/227. Until that is fixed, the default +invocation reproduces the bug (bounded by --timeout), and --no-prefetch +exercises the sequential read path, which completes. + +Requires: pip install paramiko +""" + +import argparse +import filecmp +import signal +import sys +import time + +import paramiko + + +def main() -> int: + parser = argparse.ArgumentParser( + description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter + ) + parser.add_argument("--host", default="127.0.0.1") + parser.add_argument("--port", type=int, default=22200) + parser.add_argument("--user", required=True) + parser.add_argument("--key", required=True, help="RSA private key in PEM format") + parser.add_argument("--file", required=True, help="local test file to round-trip") + parser.add_argument("--remote-dir", required=True, help="writable directory on the server") + parser.add_argument( + "--no-prefetch", + action="store_true", + help="download sequentially instead of with paramiko's read-ahead burst", + ) + parser.add_argument( + "--timeout", + type=int, + default=120, + help="hard limit in seconds for the whole round trip (default 120)", + ) + args = parser.parse_args() + + def on_timeout(signum, frame): + print(f"TIMEOUT after {args.timeout}s (see lablup/bssh#227)", flush=True) + sys.exit(2) + + signal.signal(signal.SIGALRM, on_timeout) + signal.alarm(args.timeout) + + transport = paramiko.Transport((args.host, args.port)) + transport.banner_timeout = 15 + transport.connect( + username=args.user, pkey=paramiko.RSAKey.from_private_key_file(args.key) + ) + print("remote version:", transport.remote_version, flush=True) + sftp = paramiko.SFTPClient.from_transport(transport) + channel = sftp.get_channel() + print( + "server max packet:", channel.out_max_packet_size, + "window:", channel.out_window_size, + flush=True, + ) + + remote = f"{args.remote_dir}/paramiko_roundtrip" + local_back = args.file + ".paramiko_back" + + t0 = time.time() + sftp.put(args.file, remote) + t1 = time.time() + sftp.get(remote, local_back, prefetch=not args.no_prefetch) + t2 = time.time() + print( + f"put {t1 - t0:.1f}s, get {t2 - t1:.1f}s (prefetch={not args.no_prefetch})", + flush=True, + ) + + sftp.remove(remote) + sftp.close() + transport.close() + signal.alarm(0) + + if not filecmp.cmp(args.file, local_back, shallow=False): + print("INTEGRITY FAIL", flush=True) + return 1 + print("OK", flush=True) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/bench/interop/sshj/SshjTest.java b/tools/bench/interop/sshj/SshjTest.java new file mode 100644 index 00000000..152993f4 --- /dev/null +++ b/tools/bench/interop/sshj/SshjTest.java @@ -0,0 +1,56 @@ +import net.schmizz.sshj.SSHClient; +import net.schmizz.sshj.sftp.RemoteResourceInfo; +import net.schmizz.sshj.sftp.SFTPClient; +import net.schmizz.sshj.transport.verification.PromiscuousVerifier; + +import java.util.List; + +/** + * SFTP interop check against bssh-server using sshj (the SSH stack used by + * Cyberduck and other Java tools). Uploads a file, downloads it back, and + * prints timing; integrity is verified by the calling script with cmp. + * + * Args: port user keyPath localFile remoteUploadPath localDownloadPath listDir + */ +public class SshjTest { + public static void main(String[] args) throws Exception { + String host = "127.0.0.1"; + int port = Integer.parseInt(args[0]); + String user = args[1]; + String keyPath = args[2]; + String localFile = args[3]; + String remoteUp = args[4]; + String localDown = args[5]; + String listDir = args[6]; + + SSHClient ssh = new SSHClient(); + ssh.addHostKeyVerifier(new PromiscuousVerifier()); + ssh.connect(host, port); + try { + ssh.authPublickey(user, keyPath); + System.out.println("NEGOTIATED " + ssh.getTransport().getClientVersion() + + " <-> " + ssh.getTransport().getServerVersion()); + SFTPClient sftp = ssh.newSFTPClient(); + + List ls = sftp.ls(listDir); + System.out.println("LS_ENTRIES " + ls.size()); + + long t0 = System.nanoTime(); + sftp.put(localFile, remoteUp); + long t1 = System.nanoTime(); + sftp.get(remoteUp, localDown); + long t2 = System.nanoTime(); + + long size = sftp.stat(remoteUp).getSize(); + System.out.println("REMOTE_SIZE " + size); + System.out.printf("UPLOAD_MIBS %.0f%n", size / 1048576.0 / ((t1 - t0) / 1e9)); + System.out.printf("DOWNLOAD_MIBS %.0f%n", size / 1048576.0 / ((t2 - t1) / 1e9)); + + sftp.rm(remoteUp); + sftp.close(); + } finally { + ssh.disconnect(); + } + System.out.println("SSHJ_OK"); + } +} diff --git a/tools/bench/interop/sshj/fetch-deps.sh b/tools/bench/interop/sshj/fetch-deps.sh new file mode 100755 index 00000000..f39db532 --- /dev/null +++ b/tools/bench/interop/sshj/fetch-deps.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# Download the pinned sshj jars into lib/ (not committed to the repository). +set -eu +cd "$(dirname "$0")" +mkdir -p lib +M=https://repo1.maven.org/maven2 +DEPS=" +com/hierynomus/sshj/0.38.0/sshj-0.38.0.jar +com/hierynomus/asn-one/0.6.0/asn-one-0.6.0.jar +org/bouncycastle/bcprov-jdk18on/1.78.1/bcprov-jdk18on-1.78.1.jar +org/bouncycastle/bcpkix-jdk18on/1.78.1/bcpkix-jdk18on-1.78.1.jar +org/bouncycastle/bcutil-jdk18on/1.78.1/bcutil-jdk18on-1.78.1.jar +org/slf4j/slf4j-api/2.0.13/slf4j-api-2.0.13.jar +org/slf4j/slf4j-simple/2.0.13/slf4j-simple-2.0.13.jar +net/i2p/crypto/eddsa/0.3.0/eddsa-0.3.0.jar +" +for dep in $DEPS; do + jar=$(basename "$dep") + [ -f "lib/$jar" ] && continue + echo "fetching $jar" + curl -sfL -o "lib/$jar" "$M/$dep" +done +echo "done: $(ls lib/*.jar | wc -l) jars in $(pwd)/lib" diff --git a/tools/bench/interop/sshj/run-sshj.sh b/tools/bench/interop/sshj/run-sshj.sh new file mode 100755 index 00000000..d2f6d376 --- /dev/null +++ b/tools/bench/interop/sshj/run-sshj.sh @@ -0,0 +1,44 @@ +#!/bin/bash +# Compile and run the sshj interop check against a locally started bssh-server. +# +# Requires a JDK (javac + java). Set JAVA_HOME to use a specific one, and run +# ./fetch-deps.sh once to populate lib/. See ../../README.md for the +# configuration variables shared with the rest of the harness. + +set -u +cd "$(dirname "$0")" +. ../../lib.sh + +JAVA=${JAVA_HOME:+$JAVA_HOME/bin/}java +JAVAC=${JAVA_HOME:+$JAVA_HOME/bin/}javac +command -v "$JAVAC" >/dev/null 2>&1 || { echo "FATAL: javac not found; install a JDK or set JAVA_HOME" >&2; exit 1; } +[ -f lib/sshj-0.38.0.jar ] || { echo "FATAL: jars missing; run ./fetch-deps.sh first" >&2; exit 1; } + +CP=$(ls lib/*.jar | tr '\n' ':') +"$JAVAC" -cp "$CP" SshjTest.java || exit 1 + +bench_setup +start_bssh || exit 1 + +"$JAVA" -cp "$CP." \ + -Dorg.slf4j.simpleLogger.log.net.schmizz.sshj.transport=debug \ + -Dorg.slf4j.simpleLogger.log.net.schmizz.sshj.connection=debug \ + SshjTest "$BSSH_PORT" "$USER_NAME" "$BENCH_DIR/bench_key_rsa" \ + "$TEST_FILE" "$BENCH_DIR/up/sshj_up" "$BENCH_DIR/dl/sshj_down" "$BENCH_DIR" \ + > "$BENCH_DIR/sshj-run.out" 2> "$BENCH_DIR/sshj-debug.log" +RC=$? +stop_servers + +echo "=== exit: $RC ===" +cat "$BENCH_DIR/sshj-run.out" +echo "=== integrity ===" +if cmp -s "$TEST_FILE" "$BENCH_DIR/dl/sshj_down"; then + echo "ROUNDTRIP_INTEGRITY_OK" +else + echo "ROUNDTRIP_INTEGRITY_FAIL" + RC=1 +fi +rm -f "$BENCH_DIR/dl/sshj_down" +echo "=== negotiation (client debug log) ===" +grep -iE "Negotiated algorithms|Sending SSH_MSG_KEXINIT|Received SSH_MSG_KEXINIT" "$BENCH_DIR/sshj-debug.log" | head -5 +exit $RC diff --git a/tools/bench/lib.sh b/tools/bench/lib.sh new file mode 100644 index 00000000..bdee4ea2 --- /dev/null +++ b/tools/bench/lib.sh @@ -0,0 +1,130 @@ +#!/bin/bash +# Shared helpers for the bssh benchmark and interop harness. +# All configuration comes from environment variables; see README.md. + +set -u + +REPO_ROOT=${REPO_ROOT:-$(git -C "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" rev-parse --show-toplevel 2>/dev/null || pwd)} +BENCH_DIR=${BENCH_DIR:-${TMPDIR:-/dev/shm}/bssh-bench-$(id -un)} +BSSH_SERVER_BIN=${BSSH_SERVER_BIN:-$REPO_ROOT/target/release/bssh-server} +SSHD_BIN=${SSHD_BIN:-/usr/sbin/sshd} +BSSH_PORT=${BSSH_PORT:-22200} +SSHD_PORT=${SSHD_PORT:-22022} +FILE_SIZE_MIB=${FILE_SIZE_MIB:-2048} +RUNS=${RUNS:-3} +SERVER_CORE=${SERVER_CORE:-} +CLIENT_CORE=${CLIENT_CORE:-} +CIPHER=${CIPHER:-chacha20-poly1305@openssh.com} +SFTP_BIN=${SFTP_BIN:-/usr/bin/sftp} +SSH_BIN=${SSH_BIN:-/usr/bin/ssh} +TIMEOUT_BIN=${TIMEOUT_BIN:-timeout} +XFER_TIMEOUT=${XFER_TIMEOUT:-600} +USER_NAME=$(id -un) +BSSH_PID= +SSHD_PID= + +# Echo a taskset prefix for the given core list, or nothing when unset. +pin() { + [ -n "$1" ] && echo "taskset -c $1" +} + +ssh_opts() { + echo "-i $BENCH_DIR/bench_key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o Ciphers=$CIPHER -o BatchMode=yes" +} + +# Create keys, server configs, and the random test file (idempotent). +bench_setup() { + mkdir -p "$BENCH_DIR/up" "$BENCH_DIR/dl" + chmod 700 "$BENCH_DIR" + [ -f "$BENCH_DIR/bench_key" ] || ssh-keygen -q -t ed25519 -N '' -f "$BENCH_DIR/bench_key" + # RSA in PEM format: paramiko and sshj both parse it without extra support code. + [ -f "$BENCH_DIR/bench_key_rsa" ] || ssh-keygen -q -t rsa -b 3072 -m PEM -N '' -f "$BENCH_DIR/bench_key_rsa" + [ -f "$BENCH_DIR/host_key" ] || ssh-keygen -q -t ed25519 -N '' -f "$BENCH_DIR/host_key" + cat "$BENCH_DIR/bench_key.pub" "$BENCH_DIR/bench_key_rsa.pub" > "$BENCH_DIR/authorized_keys" + chmod 600 "$BENCH_DIR/authorized_keys" + + TEST_FILE=$BENCH_DIR/testfile_${FILE_SIZE_MIB}M + [ -f "$TEST_FILE" ] || dd if=/dev/urandom of="$TEST_FILE" bs=1M count="$FILE_SIZE_MIB" status=none + + cat > "$BENCH_DIR/bssh-server.yaml" < "$BENCH_DIR/sshd_config" <&2; return 1; } + $(pin "$SERVER_CORE") "$bin" run -c "$BENCH_DIR/bssh-server.yaml" -D 2>>"$BENCH_DIR/bssh.log" & + BSSH_PID=$! + sleep 2 + echo "ls /" | "$TIMEOUT_BIN" 15 "$SFTP_BIN" $(ssh_opts) -q -P "$BSSH_PORT" -b - "$USER_NAME@127.0.0.1" >/dev/null 2>&1 \ + || { echo "FATAL: bssh-server not reachable on port $BSSH_PORT" >&2; tail -5 "$BENCH_DIR/bssh.log" >&2; return 1; } +} + +start_sshd() { + [ -x "$SSHD_BIN" ] || { echo "FATAL: sshd not found: $SSHD_BIN (install openssh-server)" >&2; return 1; } + $(pin "$SERVER_CORE") "$SSHD_BIN" -f "$BENCH_DIR/sshd_config" -D -e 2>>"$BENCH_DIR/sshd.log" & + SSHD_PID=$! + sleep 2 + "$TIMEOUT_BIN" 15 "$SSH_BIN" $(ssh_opts) -p "$SSHD_PORT" "$USER_NAME@127.0.0.1" true 2>/dev/null \ + || { echo "FATAL: sshd not reachable on port $SSHD_PORT" >&2; tail -5 "$BENCH_DIR/sshd.log" >&2; return 1; } +} + +stop_servers() { + [ -n "$BSSH_PID" ] && kill "$BSSH_PID" 2>/dev/null + [ -n "$SSHD_PID" ] && kill "$SSHD_PID" 2>/dev/null + wait 2>/dev/null + BSSH_PID= + SSHD_PID= +} + +# One timed transfer via the OpenSSH sftp client. +# $1 = port, $2 = up|dl. Prints integer MiB/s on success, FAIL(rc=N) otherwise. +# Every transfer is integrity-checked with cmp before it counts. +sftp_xfer() { + local port=$1 dir=$2 dest cmd rc t0 t1 + if [ "$dir" = up ]; then + dest="$BENCH_DIR/up/x$$" + cmd="put $TEST_FILE $dest" + else + dest="$BENCH_DIR/dl/x$$" + cmd="get $TEST_FILE $dest" + fi + rm -f "$dest" + t0=$(date +%s.%N) + echo "$cmd" | $(pin "$CLIENT_CORE") "$TIMEOUT_BIN" "$XFER_TIMEOUT" "$SFTP_BIN" $(ssh_opts) -q -P "$port" -b - "$USER_NAME@127.0.0.1" >/dev/null 2>&1 + rc=$? + t1=$(date +%s.%N) + if [ $rc -ne 0 ] || ! cmp -s "$TEST_FILE" "$dest"; then + echo "FAIL(rc=$rc)" + else + awk -v s="$FILE_SIZE_MIB" -v a="$t0" -v b="$t1" 'BEGIN{printf "%.0f", s/(b-a)}' + fi + rm -f "$dest" +} diff --git a/tools/bench/profile.sh b/tools/bench/profile.sh new file mode 100755 index 00000000..5a14774b --- /dev/null +++ b/tools/bench/profile.sh @@ -0,0 +1,51 @@ +#!/bin/bash +# Capture a CPU profile (and flamegraph, when inferno is installed) of +# bssh-server while it serves SFTP uploads. +# +# Requirements: +# - kernel.perf_event_paranoid <= 2 (sudo sysctl kernel.perf_event_paranoid=1) +# - a bssh-server release build, ideally with debug symbols: +# CARGO_PROFILE_RELEASE_DEBUG=true cargo build --release --bin bssh-server +# - optional: cargo install inferno (for SVG flamegraph generation) +# +# See README.md for configuration variables. + +set -u +cd "$(dirname "$0")" +. ./lib.sh + +PARANOID=$(cat /proc/sys/kernel/perf_event_paranoid 2>/dev/null || echo "?") +if [ "$PARANOID" != "?" ] && [ "$PARANOID" -gt 2 ]; then + echo "kernel.perf_event_paranoid=$PARANOID forbids unprivileged profiling." >&2 + echo "Fix with: sudo sysctl kernel.perf_event_paranoid=1" >&2 + exit 1 +fi + +bench_setup +start_bssh || exit 1 + +perf record -F "${PERF_FREQ:-499}" -g --call-graph "${PERF_CALLGRAPH:-fp}" \ + -o "$BENCH_DIR/perf.data" -p "$BSSH_PID" 2>/dev/null & +PERF_PID=$! +sleep 1 + +for i in $(seq 1 "${PROFILE_UPLOADS:-3}"); do + echo "upload $i: $(sftp_xfer "$BSSH_PORT" up) MiB/s" +done + +kill -INT "$PERF_PID" 2>/dev/null +wait "$PERF_PID" 2>/dev/null +stop_servers + +if command -v inferno-collapse-perf >/dev/null 2>&1 && command -v inferno-flamegraph >/dev/null 2>&1; then + perf script -i "$BENCH_DIR/perf.data" 2>/dev/null | inferno-collapse-perf > "$BENCH_DIR/stacks.folded" + inferno-flamegraph --title "bssh-server SFTP upload (${FILE_SIZE_MIB} MiB x ${PROFILE_UPLOADS:-3}, $CIPHER)" \ + "$BENCH_DIR/stacks.folded" > "$BENCH_DIR/flamegraph.svg" + echo "flamegraph: $BENCH_DIR/flamegraph.svg" +else + echo "inferno not found (cargo install inferno); raw profile kept at $BENCH_DIR/perf.data" +fi + +echo "top symbols (self time):" +perf report -i "$BENCH_DIR/perf.data" --stdio --no-children --sort symbol -g none --percent-limit 1.0 2>/dev/null \ + | grep -vE "^#|^[[:space:]]*$" | head -15