http-kit blog
2020-09-18T06:57:00+00:00
http://http-kit.org
Feng Shen
shenedu@gmail.com
http-kit is clean and small, less is exponentially more
2013-02-06T00:00:00+00:00
http://http-kit.org/http-kit-clean-small
<ul>
<li><a href="http://http-kit.org/server.html">HTTP server</a>: event-driven, ring adapter, websocket extension, asynchronous extension</li>
<li><a href="http://http-kit.org/client.html">HTTP Client</a>: event-driven, asynchronous with promise, synchronous with @promise, keep-alive</li>
<li><a href="http://http-kit.org/timer.html">Timer facility</a></li>
</ul>
<p>All the above + high concurrency + high performance + nice API + written from scrach = ~3K lines of code.</p>
<p>Clojure is awesome</p>
<p>Clojure + JAVA = Performance + Nice API</p>
<p>Less is exponentially more</p>
<pre>
http://cloc.sourceforge.net v 1.56 T=0.5 s (94.0 files/s, 8216.0 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Java 44 534 396 2844
Clojure 3 52 15 267
-------------------------------------------------------------------------------
SUM: 47 586 411 3111
-------------------------------------------------------------------------------
</pre>
<p><em>cloc runned time: Tue Feb 5 23:23:58 CST 2013, on master branch, from src directory</em></p>
<p><em>Edit: 2013/3/29 with release 2.0.0, the codebase is slightly larger: 2970 lines of Java, 266 lines of Clojure</em></p>
600k concurrent HTTP connections, with Clojure & http-kit
2013-01-27T00:00:00+00:00
http://http-kit.org/600k-concurrent-connection-http-kit
<p>Inspired by <a href="http://blog.caustik.com/2012/04/08/scaling-node-js-to-100k-concurrent-connections/">Scaling node.js to 100k concurrent connections!</a> and <a href="http://blog.caustik.com/2012/04/10/node-js-w250k-concurrent-connections/">Node.js w/250k concurrent connections!</a>. I did some test for http-kit!</p>
<p>http-kit manages to make <strong>600k</strong> concurrent connections, on PC!</p>
<h3 id="servers-logic">Server’s logic</h3>
<p>The server read the length param from the request, generate a string of that length.</p>
<figure class="highlight"><pre><code class="language-clojure" data-lang="clojure"><span class="c1">;; main.clj</span><span class="w">
</span><span class="c1">;; ~20k string</span><span class="w">
</span><span class="p">(</span><span class="k">def</span><span class="w"> </span><span class="n">const-str</span><span class="w"> </span><span class="p">(</span><span class="nb">apply</span><span class="w"> </span><span class="nb">str</span><span class="w"> </span><span class="p">(</span><span class="nb">repeat</span><span class="w"> </span><span class="mi">200</span><span class="w"> </span><span class="s">"http-kit is a http server & client written from scrach for high performance clojure web applications, support async and websocket"</span><span class="p">)))</span><span class="w">
</span><span class="p">(</span><span class="k">defn</span><span class="w"> </span><span class="n">handler</span><span class="w"> </span><span class="p">[</span><span class="n">req</span><span class="p">]</span><span class="w">
</span><span class="p">(</span><span class="k">let</span><span class="w"> </span><span class="p">[</span><span class="n">length</span><span class="w"> </span><span class="p">(</span><span class="nf">to-int</span><span class="w"> </span><span class="p">(</span><span class="nb">or</span><span class="w"> </span><span class="p">(</span><span class="nb">-></span><span class="w"> </span><span class="n">req</span><span class="w"> </span><span class="no">:params</span><span class="w"> </span><span class="no">:length</span><span class="p">)</span><span class="w"> </span><span class="mi">1024</span><span class="p">))]</span><span class="w">
</span><span class="p">{</span><span class="no">:status</span><span class="w"> </span><span class="mi">200</span><span class="w">
</span><span class="no">:headers</span><span class="w"> </span><span class="p">{</span><span class="s">"Content-Type"</span><span class="w"> </span><span class="s">"text/plain"</span><span class="p">}</span><span class="w">
</span><span class="no">:body</span><span class="w"> </span><span class="p">(</span><span class="nb">subs</span><span class="w"> </span><span class="n">const-str</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="p">(</span><span class="nb">max</span><span class="w"> </span><span class="p">(</span><span class="nb">min</span><span class="w"> </span><span class="mi">10240</span><span class="w"> </span><span class="n">length</span><span class="p">)</span><span class="w"> </span><span class="mi">1</span><span class="p">))}))</span><span class="w">
</span><span class="p">(</span><span class="k">defn</span><span class="w"> </span><span class="n">-main</span><span class="w"> </span><span class="p">[</span><span class="o">&</span><span class="w"> </span><span class="n">args</span><span class="p">]</span><span class="w">
</span><span class="p">(</span><span class="nf">run-server</span><span class="w"> </span><span class="p">(</span><span class="nb">-></span><span class="w"> </span><span class="n">handler</span><span class="w"> </span><span class="n">wrap-keyword-params</span><span class="w"> </span><span class="n">wrap-params</span><span class="p">)</span><span class="w">
</span><span class="p">{</span><span class="no">:port</span><span class="w"> </span><span class="mi">8000</span><span class="p">})</span><span class="w">
</span><span class="p">(</span><span class="nb">println</span><span class="w"> </span><span class="p">(</span><span class="nb">str</span><span class="w"> </span><span class="s">"Server started. listen at 0.0.0.0@8000"</span><span class="p">)))</span></code></pre></figure>
<p>Start the server:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">java <span class="nt">-server</span> <span class="nt">-Xms3072m</span> <span class="nt">-Xmx3072m</span> <span class="nt">-cp</span> <span class="sb">`</span>lein classpath<span class="sb">`</span> clojure.main <span class="nt">-m</span> main</code></pre></figure>
<h3 id="linux-config">Linux config</h3>
<p>The server need to set max allowed open file to a much larger value. The default value is ~1024</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="nb">echo </span>9999999 | <span class="nb">sudo tee</span> /proc/sys/fs/nr_open
<span class="nb">echo </span>9999999 | <span class="nb">sudo tee</span> /proc/sys/fs/file-max
<span class="c"># edit /etc/security/limits.conf, add the following line, need logout and login again</span>
<span class="k">*</span> - nofile 4999999
<span class="c"># set before run the server and test code</span>
<span class="nb">ulimit</span> <span class="nt">-n</span> 4999999</code></pre></figure>
<p>More ports for test code to use</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="nb">sudo </span>sysctl <span class="nt">-w</span> net.ipv4.ip_local_port_range<span class="o">=</span><span class="s2">"1025 65535"</span></code></pre></figure>
<h3 id="hardware--software">Hardware & Software</h3>
<p>The server and test code are both run on my desktop:</p>
<ul>
<li><strong>CPU</strong>: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz, 4 core, 8 threads</li>
<li><strong>RAM</strong>: 16G @ 1333MHZ</li>
<li><strong>OS</strong> : Linux 3.2.0-2-amd64 #1 SMP Sun Apr 15 16:47:38 UTC 2012 x86_64 GNU/Linux</li>
<li><strong>http-kit</strong>: “2.0-rc1”</li>
<li><strong>JVM</strong>: 1.7.0_04</li>
</ul>
<h3 id="how-to-make-600k-concurrent-connections-on-a-single-pc">How to make 600k concurrent connections on a single PC</h3>
<p>A IP can only issue most 65536 connections to a server, since socket port is unsigned short. But we can bypass this limit.
On Linux, it’s quite easy to set up virtual network interface:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="k">for </span>i <span class="k">in</span> <span class="sb">`</span><span class="nb">seq </span>200 230<span class="sb">`</span><span class="p">;</span> <span class="k">do </span><span class="nb">sudo </span>ifconfig eth0:<span class="nv">$i</span> 192.168.1.<span class="nv">$i</span> up <span class="p">;</span> <span class="k">done</span></code></pre></figure>
<p>Then your computer have many IPs, from <code class="language-plaintext highlighter-rouge">192.168.1.200</code> to <code class="language-plaintext highlighter-rouge">192.168.1.230</code>. The server bind to <code class="language-plaintext highlighter-rouge">0.0.0.0@port</code>, the client can connect
<code class="language-plaintext highlighter-rouge">192.168.1.200@port</code>, or <code class="language-plaintext highlighter-rouge">192.168.1.201@port</code>, etc. Per IP can have about 60K concurrent connections. Then the client can issue as many concurrent connections as it needed.</p>
<h3 id="concurrency-test-code">Concurrency test code</h3>
<p>The client opens 600k concurrent keep-alived connections to the server, request the server to return string of length randomly between 1 ~ 4096 bytes, read the response, idle 5s ~ 45s (randomly pick a value between), request again.</p>
<p><a href="blog/600k/test_output">output:</a></p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="nb">time </span>0s, concurrency: 100, total requests: 0, thoughput: 0.00M/s, 0.00 requests/seconds
<span class="nb">time </span>40s, concurrency: 164000, total requests: 230142, thoughput: 11.78M/s, 5688.28 requests/seconds
...
<span class="nb">time </span>89s, concurrency: 340100, total requests: 788985, thoughput: 18.23M/s, 8812.23 requests/seconds
...
<span class="nb">time </span>179s, concurrency: 595166, total requests: 2483174, thoughput: 28.61M/s, 13837.77 requests/seconds
<span class="nb">time </span>180s, concurrency: 597853, total requests: 2506378, thoughput: 28.71M/s, 13888.67 requests/seconds
<span class="nb">time </span>183s, concurrency: 600000, total requests: 2529020, thoughput: 28.52M/s, 13788.14 requests/seconds
<span class="nb">time </span>185s, concurrency: 600000, total requests: 2537212, thoughput: 28.20M/s, 13680.42 requests/seconds
...
<span class="nb">time </span>930s, concurrency: 600000, total requests: 17457773, thoughput: 38.64M/s, 18763.53 requests/seconds
<span class="nb">time </span>931s, concurrency: 600000, total requests: 17477678, thoughput: 38.69M/s, 18764.73 requests/seconds</code></pre></figure>
<h3 id="how-about-ab-test-when-600k-connections-are-kept">How about ab test when 600k connections are kept</h3>
<p>Issue this command from command line:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">ab <span class="nt">-n</span> 100000 <span class="nt">-c</span> 10 <span class="nt">-k</span> http://127.0.0.1:8000/</code></pre></figure>
<p>output: <a href="blog/600k/ab_results">ab output</a></p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">Server Software: http-kit
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /
Document Length: 1024 bytes
Concurrency Level: 10
Time taken <span class="k">for </span>tests: 3.184 seconds
Complete requests: 100000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 100000
Total transferred: 117000000 bytes
HTML transferred: 102400000 bytes
Requests per second: 31405.53 <span class="o">[</span><span class="c">#/sec] (mean)</span>
Time per request: 0.318 <span class="o">[</span>ms] <span class="o">(</span>mean<span class="o">)</span>
Time per request: 0.032 <span class="o">[</span>ms] <span class="o">(</span>mean, across all concurrent requests<span class="o">)</span>
Transfer rate: 35883.27 <span class="o">[</span>Kbytes/sec] received
Connection Times <span class="o">(</span>ms<span class="o">)</span>
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 0 0 9.3 0 913
Waiting: 0 0 9.3 0 913
Total: 0 0 9.3 0 913
Percentage of the requests served within a certain <span class="nb">time</span> <span class="o">(</span>ms<span class="o">)</span>
50% 0
66% 0
75% 0
80% 0
90% 0
95% 0
98% 0
99% 0
100% 913 <span class="o">(</span>longest request<span class="o">)</span></code></pre></figure>
<h3 id="the-clojure-servers-cpu-usage">The Clojure Server’s CPU usage</h3>
<p>jvisualvm’s <a href="blog/600k/jvisualvm_snapshort.apps">snapshot file</a>.</p>
<p><a href="blog/600k/cpu.png"><img src="blog/600k/cpu.png" alt="cpu usage" /></a></p>
<h3 id="the-clojure-servers-heap-usage">The Clojure Server’s heap usage</h3>
<p><a href="blog/600k/heap_usage.png"><img src="blog/600k/heap_usage.png" alt="heap memory usage" /></a></p>
<h3 id="run-it-yourself">Run it yourself</h3>
<p>The complete test code is available on <a href="https://github.com/http-kit/scale-clojure-web-app">github</a>. Checkout and run it yourself!</p>
<p>To report a bug, or general discussion: <a href="https://github.com/http-kit/scale-clojure-web-app/issues">https://github.com/http-kit/scale-clojure-web-app/issues</a></p>