Edge Runner
Neural network inference benchmarking and edge deployment tool
A neural network inference benchmarking and edge deployment tool. Provides per-stage timing (tokenization, embedding, QKV GEMM, attention, FFN, pooling), telemetry CSV output, accuracy verification, configurable iterations, and memory tracking.
Pricing
Bundle with V Engine / CAT VLM
bundle
Market comparison
MLPerf inference benchmarks are free and industry-standard but require significant setup. Custom edge inference profilers can cost $5k-$25k to build internally. Cloud inference benchmarking tools like OctoML are usage-based.
Features
- Per-stage inference timing
- Tokenization, embedding, attention, FFN profiling
- Telemetry CSV output
- Accuracy verification
- Memory tracking
- Embedding checksum
Fact Check
tools/edge_runner/ is listed in tools/README.md. The tool implements per-stage timing and telemetry CSV output.