Kestrel vs Gin vs Iris vs Express vs Fasthttp on EC2 nano
Since this post got quite a bit of traction, I decided to update it by rerunning all the benchmarks as well as adding GO’s fasthttp and Node’s express to the comparison.
I came across this blog post on ayende.com. Here Oren Eini tries to see how far he could push a simple ipify style of api on an EC2 by running a synthetic benchmark. He hosts the http server on a T2.nano instance and then uses wrk to benchmark it from a T2.small instance. After reading this, I thought to myself - surely .NET cannot be quicker than GO. I decided to try and make a similar effort and get a little bit of competition going between a .NET implementation with, hopefully, a representative version of Oren’s .NET server made with GO. For GO - I went with 3 candidates gin, fasthttp and iris. I also benchmark Node’s Express. All the tests were performed on the same EC2 instance, with production/release configurations, so variance should be low. I also use the same parameters for wrk as Oren. This is as close as I could get to apples to apples type of comparison.
For Kestrel I used the same code that Oren did. I did manage to get to 35000 requests/sec.
The results were as follows:
And here’s a snap of TOP somewhere mid run:
For Iris I went with the code that Gerasimos from the comments below suggested.
For Gin, I used the built in
ClientIP() which does a few more things than just looking at the
X-Forwarded-For header. Here’s the code I ended up with.
To be fair, I knew Express would not perform too well here but I added it nonetheless. Here’s the Express code.
From this, I drew the following conclusions:
- I’m pleasantly surpised by Kestrels performance(I mostly code Go).
- Kestrel(most likely .NET, not Kestrel itself) is way harder on the memory than GO’s equivalents.
- Nano instance can be quite good at handling not very intensive tasks.
- Fasthttp outperforms kKstrel, but not by a large margin.
- Fasthttp uses only 1/10th of Kestrels memory.
- Gin and Iris are not as fast as Kestrel, which surprised me.
- Express is nowhere close, as expected.
I also ran the tests on my Mac Pro - every server comfortably went into the 100 000 requests/second. Go seemed to do a tiny bit better than .NET on this one though. Express only managed 16000 requests/s.
Take note that I’m not an expert in benchmarking by any means. Let me know if I made any mistakes in the comments below!