Methodology

I tried as much as possible to get stable and consistent benchmarking results. These are the main things I did to get there.

Benchmark itself

Version 0.35.4 of rawloader was used throughout the test, compiled in release mode
A set of 60 files from a broad set of cameras was chosen manually to try and exercise most types of decompression algorithm. All the sample files are available at raw.pixls.us
Benchmarking is done by first loading the file into memory and only then benchmarking 50 iterations of decoding the file
All the file and rust version combinations were benchmarked five times in random order (to try to eliminate any systematic biases during the run)
Between every two runs a one second sleep was done to prevent thermal throttling
For the compile timing all versions were also compiled five times and cargo fetch was run previously to make sure there were no delays from network traffic

The value for each file and rust version combination is the median of the five runs
The comparisons are simply the ratio between the time of the two runs
The box plots were generated in R with the default boxplot() function