WARNING: This text is deprecated and refers to an old version of ØMQ. It remains here for historical interest. DO NOT USE THIS TO LEARN ØMQ.
As ØMQ is primarily intended to power stock trading business, we've created an sample application that simulates internal working of a stock exchange. The primary focus of this example is to show how ØMQ perfroms in real-world-like scenario.
The diagram below shows the architecture of the application:
Gateway component should get orders from traders over the network (using FIX protocol or a specific proprietary protocol) and send replies back to the traders. However, as the example application is intended to show what are the possible throughputs/latencies of such a system, gateway generates random orders instead of receiving them from the traders. When starting the gateway, you can specify number of orders per second to be generated.
Matching unit contains the core of stock exchange business logic. It matches orders one to another and produces trades and quotes. Our implementation is minimalistic, based on time/price matching algorithm (implementing pro rata algorithm is left as an excercise for the reader). Still, the algorithm has complexity of O(1) and it is heavily optimised. We've seen it process some 18 million orders per second.
Statistics component receives performance info generated by both gateway and matching engine and displays it in human-readable form. To make reading of statistics even more convenient, simple graphical tool is included in the example.
Following screenshot shows example's performance on two high-end 8-core (3GHz) boxes each having 2 dedicated 1GbE NICs. Keep in mind that if you run it on the underspecified and/or untuned hardware it can be still pretty fast, however, you'll experience lower message throughputs and more latency peaks. If you set the message rate too high you can even experience gatway failure as the ticker component used to send orders at the stable rate won't be able to keep pace.
The yellow line shows roundtrip latency, i.e. how long it took to pass the order from the gateway to the matching engine, process it and send order confirmation back to the gateway. In our test latency fluctuated around 200 microsecond average.
The lower throughput line (900,000 messages/second) is the rate of orders being passed from the gateway to the matching engine. The upper throughput line (2,300,000 messages/second) is the rate of order confirmations, trades and stock quotes passed from the matching engine to the gateway. In total, we've seen approximately 3,200,000 messages per second passing through the network.
To build the example use --with-exchange option with configure:
$ ./configure --with-exchange $ make
To be able to run the graphing tool you have to have Perl-Tk installed (packaged as perl-tk on Debian) as well as Tk::Graph from CPAN.
For example, we have following network topology to run the example on. Boxes represent individual machines, arrows represent physical cables between individual network interfaces (marked by their respective IP addresses):
There are 3 boxes (test01, test02 and test03) connected to the switched network with respective IP addresses of 192.168.0.1, 192.168.0.2 and 192.168.0.3. Moreover there are two direct connections between test02 and test03. One connection connects network interface 10.0.0.1 on test02 with network interface 10.0.0.2 on test03. Other one connectes network interface 10.1.0.1 on test02 with network interface 10.1.0.2 on test03.
We'll run zmq_server and statistical component on test01, matching engine on test02 and gateway on test03. We'll use one of the direct connections between test02 and test03 to pass orders from gatway to matching engine and the other one to pass confirmations, trades and quotes from matching engine to gateway.
First, start zmq_server on test01:
Afterwards start the statistical component on test01. The parameters are the box where zmq_server is running and network interface to receive statistical info on:
$ ./stat test1 192.168.0.1:5555
Alternatively you can pipe the statistical data to the graphing tool:
$ ./stat test1 192.168.0.1:5555 | ./graph.pl
Now start the matching engine. Supply zmq_server's host name, interface to receive messages on and interface to send messages on as parameters:
$ ./me test1 10.0.0.1:5556 10.1.0.1:5557
Finally, run the gatway. Supply location of zmq_server and the number of orders to send per second as parameters:
$ ./gtw test01 20000
Exchange example allows you to test ØMQ's performance in real-world-like scenario. However, getting stable latency at high throughputs is a tricky matter dependent on overall tuning of your hardware, operating system, execution environment etc. If you are serious about performance testing, contact us to help you with the task.
If you found this page useful, please rate it up so others will find it.
Who's watching this page?Dominic Luciano
Fernando J Quintero
... and more