Commit bc4521a
refactor(bench): replace benchmark tools with unified comprehensive solution
Replaced multiple benchmark scripts with a single integrated tool that
provides human-readable reports and optional historical performance graphs.
**Key Changes**:
Removed:
- scripts/bench-viz.py (replaced)
- scripts/bench-report.sh (replaced)
- scripts/bench-compare.sh (replaced)
- bench.txt, bench-report.txt, bench_results.json from .gitignore
Added:
- scripts/bench.py - Comprehensive benchmark tool with:
- Human-readable reports grouped by category (API, Parsers, Format)
- Multiple performance metrics per benchmark
- Summary statistics (fastest/slowest, averages)
- Optional historical performance graphs across git commits
**Metrics Reported**:
- **Ops/Sec** - Operations per second (throughput)
- **ns/op** - Nanoseconds per operation (latency)
- **MB/s** - Megabytes per second (data throughput)
- **B/op** - Bytes allocated per operation (memory usage)
- **allocs/op** - Number of allocations per operation
**Usage**:
```bash
make bench # Run benchmarks with comprehensive report
make bench N=50 # Run benchmarks + generate graphs for last 50 commits
```
**Historical Graphs** (when N specified):
- out/ops_per_sec.png - Throughput trends over time
- out/ns_per_op.png - Latency trends over time
- out/throughput.png - Data throughput (MB/s) trends
- out/bytes_per_op.png - Memory allocation trends
- out/allocs_per_op.png - Allocation count trends
**Makefile Updates**:
- Simplified bench target to use new tool
- Removed bench-all, bench-report, bench-viz targets
- Single command for all benchmark needs
**Documentation Updates**:
- README.md: Updated with new benchmark usage and metrics
- CONTRIBUTING.md: Detailed explanation of benchmark tool and graphs
- All documentation now reflects unified approach
The tool automatically skips commits where benchmarks fail to compile,
making it robust for historical analysis across code structure changes.
Requirements: Python 3 + matplotlib (only for historical graphs)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>1 parent 4052c8d commit bc4521a
File tree
8 files changed
+546
-658
lines changed- scripts
8 files changed
+546
-658
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | | - | |
28 | | - | |
29 | | - | |
30 | | - | |
31 | 27 | | |
32 | 28 | | |
33 | 29 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
293 | 293 | | |
294 | 294 | | |
295 | 295 | | |
296 | | - | |
297 | | - | |
298 | | - | |
299 | | - | |
300 | | - | |
301 | | - | |
302 | | - | |
303 | | - | |
| 296 | + | |
| 297 | + | |
304 | 298 | | |
305 | | - | |
306 | | - | |
307 | | - | |
308 | | - | |
309 | | - | |
310 | | - | |
311 | | - | |
312 | | - | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
313 | 302 | | |
314 | 303 | | |
315 | | - | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
316 | 330 | | |
317 | 331 | | |
318 | 332 | | |
319 | 333 | | |
320 | | - | |
321 | | - | |
322 | | - | |
323 | | - | |
324 | | - | |
325 | | - | |
326 | | - | |
327 | | - | |
328 | | - | |
329 | | - | |
330 | | - | |
331 | 334 | | |
332 | 335 | | |
333 | 336 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| |||
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
93 | | - | |
| 93 | + | |
94 | 94 | | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
113 | | - | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
| 95 | + | |
122 | 96 | | |
123 | 97 | | |
124 | 98 | | |
| |||
138 | 112 | | |
139 | 113 | | |
140 | 114 | | |
141 | | - | |
142 | | - | |
143 | | - | |
144 | | - | |
145 | | - | |
| 115 | + | |
146 | 116 | | |
147 | 117 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
196 | 196 | | |
197 | 197 | | |
198 | 198 | | |
199 | | - | |
| 199 | + | |
200 | 200 | | |
201 | | - | |
202 | | - | |
203 | | - | |
204 | | - | |
205 | | - | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
206 | 206 | | |
207 | 207 | | |
208 | | - | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
209 | 219 | | |
210 | | - | |
| 220 | + | |
211 | 221 | | |
212 | 222 | | |
213 | | - | |
214 | | - | |
| 223 | + | |
215 | 224 | | |
216 | 225 | | |
217 | | - | |
218 | | - | |
219 | | - | |
220 | | - | |
221 | | - | |
222 | | - | |
223 | | - | |
224 | | - | |
225 | | - | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
226 | 232 | | |
227 | | - | |
| 233 | + | |
228 | 234 | | |
229 | 235 | | |
230 | 236 | | |
231 | | - | |
232 | 237 | | |
233 | 238 | | |
234 | 239 | | |
| |||
This file was deleted.
This file was deleted.
0 commit comments