Use of both `--sort command` and `--reference` options leads to confusing relative speed table

[raviqqe]

When --sort command and --reference options are used together, Hyperfine drops faster than and slower than annotations from relative speed tables where everything looks slower than references.

Examples

Only --reference

You see faster than and slower than adjectives in the summary table.

> hyperfine --reference 'sleep 2' 'sleep 1' 'sleep 3'
Benchmark 1: sleep 2
  Time (mean ± σ):      2.014 s ±  0.003 s    [User: 0.004 s, System: 0.004 s]
  Range (min … max):    2.006 s …  2.018 s    10 runs

Benchmark 2: sleep 1
  Time (mean ± σ):      1.020 s ±  0.010 s    [User: 0.004 s, System: 0.004 s]
  Range (min … max):    1.010 s …  1.045 s    10 runs

Benchmark 3: sleep 3
  Time (mean ± σ):      3.018 s ±  0.004 s    [User: 0.005 s, System: 0.005 s]
  Range (min … max):    3.012 s …  3.022 s    10 runs

Summary
  sleep 2 ran
    1.97 ± 0.02 times slower than sleep 1
    1.50 ± 0.00 times faster than sleep 3

--reference and --sort command

You do not see faster than and slower than adjectives in the relative speed comparison table.

> hyperfine --sort command --reference 'sleep 2' 'sleep 1' 'sleep 3'
Benchmark 1: sleep 2
  Time (mean ± σ):      2.014 s ±  0.003 s    [User: 0.003 s, System: 0.003 s]
  Range (min … max):    2.007 s …  2.017 s    10 runs

Benchmark 2: sleep 1
  Time (mean ± σ):      1.015 s ±  0.002 s    [User: 0.004 s, System: 0.003 s]
  Range (min … max):    1.011 s …  1.017 s    10 runs

Benchmark 3: sleep 3
  Time (mean ± σ):      3.016 s ±  0.001 s    [User: 0.004 s, System: 0.004 s]
  Range (min … max):    3.013 s …  3.017 s    10 runs

Relative speed comparison
        1.00          sleep 2
        1.98 ±  0.01  sleep 1
        1.50 ±  0.00  sleep 3

Solution

I think there are two options.

• Add faster than and slower than annotations when both --sort command and --reference options are specified.
• Use relative speeds below 1 for candidates faster than references.