Why CowCow¶
CowCow is the sync pipeline for physical AI and robotics data: moving terabyte-scale, multi-sensor dataset volume from edge to cloud with minimal wasted bytes, time, and retries, especially when connectivity is poor.
Rust keeps this layer fast, predictable, and verifiable at scale.
What we deliver¶
| Pillar | Mechanism | Proof |
|---|---|---|
| Volume efficiency | BLAKE3 content-addressed chunks, ingest dedup, resume skips verified bytes | Skip rate %, bytes avoided on resume |
| Scale | Object-store sync, worker queue, /v1 API, optional gRPC |
Parallel uploads, benchmark vs common sync tools |
| Low connectivity | Priority-ordered sync, offline queue, verify-before-trust | Resume after failure, adaptive bandwidth controls |
Capabilities¶
- Fixed-size chunking with BLAKE3 verification
- Priority sync classes (
urgent,high,normal;holdstays local) - Resume via sync state and remote hash verification
- S3, R2, MinIO, and filesystem destinations
- Chunk compression, remote exists-check, and bandwidth-aware sync
- HTTP
/v1REST API, OpenAPI, and optional gRPC upload service - Live sync flow telemetry (SSE) for dashboards and UI animation
Integration¶
API routes, authentication, and wiring patterns: Integration.
Benchmark methodology and sample numbers: Movement benchmarks.