Skip to content

Commit 4052c8d

Browse files
rpuneetclaude
andcommitted
feat(bench): add performance visualization tool across git history
Added comprehensive Python tool to visualize benchmark performance over time: **New Tool**: scripts/bench-viz.py - Runs benchmarks across git commit history - Parses Go benchmark output and extracts metrics - Generates performance graphs using matplotlib - Supports parallel execution planning (sequential for now) - Handles compilation failures gracefully (skips commits) - Saves results as JSON for reuse **Features**: - Individual graphs per benchmark showing all metrics - Summary graphs grouping benchmarks by category - CLI with options for commit count, output dir, JSON export - Automatic date-based x-axis formatting - Latest value annotations on graphs **Metrics Visualized**: - ns/op: Nanoseconds per operation over time - B/op: Bytes allocated per operation over time - allocs/op: Allocations per operation over time **Usage**: ```bash make bench-viz N=50 # Last 50 commits ./scripts/bench-viz.py --help # Full options ``` **Requirements**: Python 3 + matplotlib **Documentation Updates**: - README.md: Added Performance Visualization section with examples - CONTRIBUTING.md: Added detailed visualization usage guide - Makefile: Added bench-viz target - .gitignore: Added out/ and bench_results.json Graphs are saved to out/ directory and can be embedded in README to show performance trends across project history. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent c47f8e1 commit 4052c8d

File tree

5 files changed

+542
-1
lines changed

5 files changed

+542
-1
lines changed

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,8 @@ debug/
2727
bench.txt
2828
bench-report.txt
2929
bench_output.txt
30+
bench_results.json
3031
*.prof
3132
*.test
3233
trace.out
34+
out/

CONTRIBUTING.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -297,8 +297,37 @@ make bench # Quick benchmark run
297297
make bench-all # Detailed output (3s timing, saved to bench.txt)
298298
make bench-report # Generate formatted report
299299
make bench-compare OLD=commit1 NEW=commit2 # Compare commits
300+
make bench-viz N=50 # Generate performance graphs across last N commits
300301
```
301302

303+
### Performance Visualization
304+
305+
Generate graphs showing how performance evolved over time:
306+
307+
```bash
308+
# Generate graphs for last 50 commits
309+
make bench-viz N=50
310+
311+
# Custom number of commits
312+
make bench-viz N=100
313+
```
314+
315+
**Requirements**: Python 3 with matplotlib
316+
```bash
317+
pip3 install matplotlib
318+
```
319+
320+
The tool will:
321+
1. Checkout each commit in git history
322+
2. Run benchmarks (skip if they fail to compile)
323+
3. Collect performance metrics
324+
4. Generate graphs in `out/` directory showing trends over time
325+
326+
Graphs are generated for each benchmark showing:
327+
- Nanoseconds per operation (ns/op)
328+
- Bytes allocated per operation (B/op)
329+
- Allocations per operation (allocs/op)
330+
302331
### Continuous Benchmarking
303332

304333
Every commit to `main` and every PR automatically runs benchmarks via GitHub Actions:
@@ -355,6 +384,7 @@ make bench # Run benchmarks
355384
make bench-all # Run benchmarks with detailed output
356385
make bench-report # Generate benchmark report
357386
make bench-compare # Compare benchmarks between commits
387+
make bench-viz # Generate performance graphs across git history
358388
make example # Build and run example
359389
```
360390

Makefile

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.PHONY: build test test-lib lint fmt clean install coverage coverage-html check help bench bench-all bench-report bench-compare
1+
.PHONY: build test test-lib lint fmt clean install coverage coverage-html check help bench bench-all bench-report bench-compare bench-viz
22

33
# Go settings
44
GOCMD = go
@@ -114,6 +114,12 @@ bench-compare:
114114
@./scripts/bench-compare.sh $(OLD) $(NEW)
115115
@echo "Usage: make bench-compare OLD=commit1 NEW=commit2"
116116

117+
# Generate benchmark visualizations across git history
118+
bench-viz:
119+
@echo "Running benchmarks across git history and generating graphs..."
120+
@./scripts/bench-viz.py --max-commits $(or $(N),50) --graphs --output out
121+
@echo "✓ Benchmark visualization complete. Graphs saved to out/"
122+
117123
# Show help
118124
help:
119125
@echo "imx - Image Metadata Extraction Library"
@@ -136,5 +142,6 @@ help:
136142
@echo " bench-all - Run benchmarks with detailed output"
137143
@echo " bench-report - Generate detailed benchmark report"
138144
@echo " bench-compare- Compare benchmarks between commits"
145+
@echo " bench-viz - Generate benchmark graphs across git history (N=commits)"
139146
@echo " example - Build and run example"
140147
@echo " help - Show this help"

README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -202,11 +202,34 @@ make bench # Quick benchmark run
202202
make bench-all # Detailed output with 3s timing
203203
make bench-report # Generate formatted report
204204
make bench-compare OLD=commit1 NEW=commit2 # Compare commits
205+
make bench-viz N=50 # Generate performance graphs across last 50 commits
205206
```
206207

208+
**Performance Visualization**:
209+
210+
Generate performance graphs showing how benchmarks evolved over time:
211+
212+
```bash
213+
# Generate graphs for last 50 commits
214+
make bench-viz N=50
215+
```
216+
217+
This creates visualizations in `out/` directory showing:
218+
- **ns/op** - Nanoseconds per operation over time
219+
- **B/op** - Bytes allocated per operation over time
220+
- **allocs/op** - Number of allocations per operation over time
221+
222+
Example graphs (run `make bench-viz` to generate):
223+
224+
![High-Level API Performance](out/summary_HighLevelAPI.png)
225+
![Parser Performance](out/summary_ParserEXIF.png)
226+
227+
**Requirements**: Python 3 with matplotlib (`pip3 install matplotlib`)
228+
207229
**Tools**:
208230
- Results tracked via [github-action-benchmark](https://github.com/benchmark-action/github-action-benchmark)
209231
- Statistical comparison using [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat)
232+
- Historical visualization using Python + matplotlib
210233

211234
## Contributing
212235

0 commit comments

Comments
 (0)