AdroitLogic Private Ltd. announced today the results of the fourth round of ESB Performance testing carried out by Asankha Perera, since June 2007. Previous rounds have compared a leading proprietary ESB, the proprietary version of an open source ESB, the WSO2/Apache Synapse ESB, Mule CE, and Apache ServiceMix. Whats significant about this test suite is that it has now been accepted as a de-facto ESB performance test suite by vendors such as WSO2 and Mule in the past, who have reported results against their implementations publicly. This round does not name the results of any competitive ESB's, but makes it easy for end-users to try out the test framework on the Amazon EC2 or local hardware and measure the performance of any comparable ESB with ease. The scenarios selected will also allow end-users to experience how easy or difficult it would be to setup these simple scenarios against the ESB of their choice.
The performance test suite includes a high performance Echo service based on Java NIO, and an Apache Bench style load generator to test concurrency levels from 20 to 2560 users at the ESB with message sizes ranging from 512 bytes to 100K. The test can easily be run on an "m1.large" instance on the Amazon EC2 with the provided scripts.
AdroitLogic invites developers/vendors of both proprietary and open source ESB's to try out the test framework against their implementations, and make the optimized configurations publicly available to enable end-users to compare their ESB's against other available options. For an end-user who wishes to compare the performance of an ESB of his choice, AdroitLogic will provide free support on use of the test framework, so that results could be measured by the end-users themselves easily.
This round 4 includes tests that measure performance of a Direct Proxy (used for Virtualization of services), a Content Based Routing (CBR) Proxy used to make routing decisions based on an XML/SOAP payload, and XSLT transformation Proxy used for translation of requests and responses, and a WS-Security proxy that allows an ESB to act as a Security gateway.