Hello everyone. I am the original author of these benchmarks, and first of all let me say that I am both surprised and humbled by the attention that they have gotten. Thank you!
Secondly, let me reiterate (since some people don't seem to be reading the website that's home to these benchmarks) that while I do think benchmarks like these can be interesting, I don't believe that they carry much real-world value. As the website states:
"When it comes to code, the slightest adjustments have the potential to change
things drastically. While I have tried to perform each test as fairly and
accurately as possible, it would be foolish to consider these results as
scientific in any way."
As others have said here, comparing frameworks can be very apples-to-oranges. Still though, I do think that this type of stuff can be interesting and has its place.
As to the "why isn't XYZ listed?" questions: This started as a pet project of mine out of pure curiosity, so I initially only included frameworks that I was familiar with. Thanks to GitHub, I will try to add more frameworks as the pull requests come in as long as I feel like they are popular enough to deserve a place next to Rails, Django, etc. Keep in mind that I have listed the Amazon EC2 AMI that was used, so feel free to run your own tests if for some reason I opt not to include your favorite framework in this list.
Finally, the last refresh of these tests was done in September 2011, which should explain why some of the version numbers are a bit outdated (I didn't know putting this stuff on GitHub would get so much attention or I would have waited). I hope to have another refresh done soon (Django 1.4, Pyramid 1.3, Rails 3.2, CakePHP 2.1, and a few new faces) but life/work is busy at the moment so please be patient and feel free to run your own tests in the meantime.
Secondly, let me reiterate (since some people don't seem to be reading the website that's home to these benchmarks) that while I do think benchmarks like these can be interesting, I don't believe that they carry much real-world value. As the website states:
As others have said here, comparing frameworks can be very apples-to-oranges. Still though, I do think that this type of stuff can be interesting and has its place.As to the "why isn't XYZ listed?" questions: This started as a pet project of mine out of pure curiosity, so I initially only included frameworks that I was familiar with. Thanks to GitHub, I will try to add more frameworks as the pull requests come in as long as I feel like they are popular enough to deserve a place next to Rails, Django, etc. Keep in mind that I have listed the Amazon EC2 AMI that was used, so feel free to run your own tests if for some reason I opt not to include your favorite framework in this list.
Finally, the last refresh of these tests was done in September 2011, which should explain why some of the version numbers are a bit outdated (I didn't know putting this stuff on GitHub would get so much attention or I would have waited). I hope to have another refresh done soon (Django 1.4, Pyramid 1.3, Rails 3.2, CakePHP 2.1, and a few new faces) but life/work is busy at the moment so please be patient and feel free to run your own tests in the meantime.