I understand you, but is more fair in the sense of comparing apples to apples if you are going to do one test of a kind. Currently all of this tests as a batch job take several hours, doing more tests, means more time and more job for me. Specially that when I took that decision I didn't gather all the data automatically and I have to collect all of that by hand and put it on the tables. There is a reason for all. Now that I have (almost, I have to add 1 more) all the tests automated and I collect all the data with a python script, I can start to think to do all kind of "alchemy" and improve the comparison.
Thanks for you feedback. I'll add this as an issue into the repository to improve the comparison.
Thanks for you feedback. I'll add this as an issue into the repository to improve the comparison.