This post originated from an RSS feed registered with Python Buzz
by Phillip Pearson.
Original Post: High performance XML-RPC
Feed Title: Second p0st
Feed URL: http://www.myelin.co.nz/post/rss.xml
Feed Description: Tech notes and web hackery from the guy that brought you bzero, Python Community Server, the Blogging Ecosystem and the Internet Topic Exchange
I was working on a standalone web server for XML-RPC a while back, writing in C++ for performance, and guessed that it might be capable of handling near 1000 hits/second.
Well, I just tried out the (BSD licensed) XMLRPC-C library, and it turns out that it includes a standalone web server that gets very close to that goal on my current Linux box (an Athlon XP 2000+).
Test setup: a single box, running both ApacheBench and the simple server example. I only have the one fast Linux box, so it had to be this way.
First test: time ab -n 5000 -p post.txt -T text/xml http://localhost:8080/RPC2:
This is a single test process hitting the server as fast as it can. Results: 722 hits/second. The server process was using somewhere in the 5-10% CPU range, the ApacheBench process was using 10-20%, and the rest was system.
[...]
Concurrency Level: 1
Time taken for tests: 6.923 seconds
Complete requests: 5000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 1480000 bytes
Total POSTed: 1635000
HTML transferred: 700000 bytes
Requests per second: 722.23 [#/sec] (mean)
Time per request: 1.38 [ms] (mean)
Time per request: 1.38 [ms] (mean, across all concurrent requests)
Transfer rate: 213.78 [Kbytes/sec] received
236.17 kb/s sent
449.95 kb/s total
[...]
real 0m6.866s
user 0m0.200s
sys 0m1.440s
Those times suggest ApacheBench was sucking up 24% of the CPU at the time, so we might guess that the performance here could be as high as 722.23 / (1 - 0.24) = 950 hits per second.
Second test: time ab -n 5000 -c 100 -p post.txt -T text/xml http://localhost:8080/RPC2
[...]
Concurrency Level: 100
Time taken for tests: 14.792 seconds
Complete requests: 5000
Failed requests: 0
Broken pipe errors: 0
Total transferred: 1480000 bytes
Total POSTed: 1660506
HTML transferred: 700000 bytes
Requests per second: 338.02 [#/sec] (mean)
Time per request: 295.84 [ms] (mean)
Time per request: 2.96 [ms] (mean, across all concurrent requests)
Transfer rate: 100.05 [Kbytes/sec] received
112.26 kb/s sent
212.31 kb/s total
[...]
real 0m14.814s
user 0m0.660s
sys 0m7.820s
Here, the hits per second rate is way down, but those eight seconds spent in the system make it look very much like the hundred ApacheBench threads (processes?) were mostly responsible for this. Repeating the calculation from above, we might possibly get 338.02 / (1 - (0.660 + 7.820) / 14.814) = 791 hits/second with 100 concurrent clients.
---
People with underperforming XML-RPC web services might want to seriously consider coding part of their apps in C++ and using XMLRPC-C and the Abyss web server to run it all.
In particular, I remember this post by Dave Winer on xmlrpc.com, talking about their pub-sub hooks for RSS subscription. The SOAP meets RSS story on thetwowayweb.com specifies the actual protocol, which is a little trickier to implement than what Dave describes in the message (as it requires the cloud to do the calls back rather than just manage a few lists).
We were talking about this on the pycs-devel mailing list a while back, I think, wondering whether it would make sense for pub-sub responses to go to your own community server, which has a good 'net connectin, and for you to pull them back down. For Radio and PyDS, you could return them as part of the xmlStorageSystem.ping() response. In this case, the community server would really want to be quick, as it could potentially be working for tens of thousands of users.
Previous Topic
