Recently there have been discussions about the performance of the play framework, after a large collection of web framework test results has been posted on TechEmpower.
Play does well on these tests, but it’s not exceptionally fast. In the article, simple JSON responses are tested on an EC2 large instance, one of Amazon’s faster virtual servers. According to the authors,
- Play serves ~9000 requests/sec
- PHP serves ~4000 requests/sec
- Netty serves ~40000 (!) requests/sec
… just to name a few of the tested frameworks.
Netty was the fastest “framework” in the test, but it’s actually more a web server than a web framework. As a side note, Play uses Netty for serving HTTP requests.
But how how does that compare to a real-world scenario?
Testing what users (and their browsers) really do
Since we’re also using the play framework (2.1.1-scala) in production, we wanted to see how many pages per second we can really serve in our Amazon EC2 environment.
In contrast to the above article, the following results are obtained by load-testing complete pages that we use in production. This means that we download not only the HTML from the server, but also all images, CSS, Javascripts, like normal browsers do.
There were 3 different tests, run on EC2 micro and large instances. (Look how much you can do with a micro instance!)
Test 1 – Small page with database access
A small page with CSS, Javascript, and a picture. It also selects data from a MySQL database, but it’s a simple indexed query. We use Slick for database access.
- Total page size: 30 kb
- Total # of requests per page: 4 (Html, Css, Javascript, Png-Image)
- EC2 Micro Instance result: 30 pages/sec (120 requests/sec)
- EC2 Large Instance result: 600 pages/sec (2400 requests/sec)
- Bottleneck: CPU
Test 2 – Page with lots of content
A larger page with CSS, Javascript and lots of pictures. No database query here. The page is multi-language and uses Play’s Internationalization feature.
- Total page size: 900 kb
- Total # of requests per page: 15 (Html, Css, Javascript, 10 Png-Images)
- EC2 Micro Instance result: 8 pages/sec (120 requests/sec)
- EC2 Large Instance result: 90 pages/sec (1350 requests/sec)
- Bottleneck: Disk/Bandwidth
Test 3 – On-the-fly server-side image resizing
In this test, just one request is made. The server loads a JPEG image from an Amazon S3 bucket, resizes it from 800×600 to a 300×225 thumbnail, and sends it again JPEG-encoded to the client.
- Total response size: 12 kb
- Total # of requests: 1 (JPEG-Image)
- EC2 Micro Instance result: 4 requests /sec
- EC2 Large Instance result: 160 requests /sec
- Bottleneck: CPU (the image rescaling algorithm)
Conclusion: We can serve quite a lot of users with the play framework, using its default settings. Because we did not optimize our code for the benchmarks, we could not serve as many requests as in the article on TechEmpower; but the results are in the same ballpark.
Surprising EC2 micro instance performance
The EC2 micro instances can actually sustain quite a high number of requests per second. If you need to serve 30 full page requests per second (like in Test 1) during peak hours, you’ll probably serve an average of 8 requests per second during the day (4 times less). That’s about 20 GB of data per day, which means you will actually pay an order of magnitude more for the traffic than for the virtual server itself.
(However, running a production server on a micro instance may not be your thing. AWS allows only around 16% of sustained CPU usage on these instances, meaning it’s trivial to DoS-attack your server.)
From an economic perspective, play’s performance means you will be able to focus on other issues, such as saving bandwidth 😉
