Skip to content

Movement layer benchmarks

Reproducible harness for CowCow sync efficiency vs a naive full re-upload baseline.

Quick run

# Smoke run (~1 min, uses cargo run)
./scripts/bench-movement-layer.sh --smoke

# Local micro run (~4 MiB payload)
./scripts/bench-movement-layer.sh

# JSON for dashboards or release notes
./scripts/bench-movement-layer.sh --smoke --json benches/results/smoke.json

Use a release binary for stable timings:

cargo build --release -p cowcow-cli
CARGO=$PWD/target/release/cowcow ./scripts/bench-movement-layer.sh

Methodology

Each run uses a temporary CowCow project under /tmp:

  1. Setup: init, ingest, chunk, qc, score on synthetic multimodal fixtures.
  2. Cold upload: first sync to an empty filesystem destination (initial wall time and bytes on wire).
  3. Interrupted sync + resume: simulate network-failure uploads N chunks, fails, then resumes with --resume (content-addressed skip of verified chunks).
  4. Naive baseline: delete local sync_state.jsonl and sync to a fresh destination without resume semantics (re-sends every chunk; models tools that do not skip verified bytes).

Metrics are read from the latest sync_completed journal event (uploaded_chunks, skipped_chunks, bytes_uploaded, bytes_skipped, skip_rate_percent).

Install rclone and aws locally to compare against external baselines.

Latest smoke results (linux/arm64 dev laptop, cargo run)

Recorded from ./scripts/bench-movement-layer.sh --smoke on 2026-06-24:

Metric Value
Payload 512 KiB source, 32 KiB chunks → 17 chunks
Cold upload 5.0 s, 528,390 bytes
Resume after fail @ 2 chunks 5.3 s, 100% skip rate (17/17 chunks skipped on resume pass)
Naive re-upload baseline 11.5 s, 528,390 bytes
Bytes saved vs naive re-upload 528,390 (100%) on resume pass

After a mid-sync failure, CowCow's resume pass sends zero duplicate bytes when all chunks were already verified. A naive re-upload would send the full payload again. On the interrupt path itself, only chunks after the failure point upload on resume.