eHarmony Engineering logo

In Pursuit of Messaging Brokers

David Gevorkyan

June 27, 2015

During this session I will present the path we have taken to:

Prioritize Features -> Compare Brokers -> Eliminate -> Benchmark

I’ll discuss the findings, together with our recommendations.
David Gevorkyan

Messaging Brokers are intrinsic part of our infrastructure. At eHarmony, we have historically used HornetQ, which failed to meet our needs. We have experienced issues that forced us to restart multiple times a day, and sometimes even that failed to resolve the issue. Data cleanup was required, resulting in loss of important messages. Lack of documentation, lack of proper support and lack of stability of the HornetQ broker are some of the reasons for looking into new Messaging Brokers.

As part of this task we have gone through the steps to Prioritize Features -> Compare Brokers -> Eliminate -> Benchmark and come up with specific recommendations. This task resulted in implementation of generic messaging broker benchmarking tool in Java, called Benchmark.io, which we intend to open source.

We went through multiple rounds of identifying more or less stable messaging brokers, and created an initial list of 10. We then created a comparison table to evaluate different features. Once the feature set was defined, we prioritized the features that we needed most within the engineering team, and narrowed down the list to 4 messaging brokers: RabbitMQ, Apache Kafka, ActiveMQ and NSQ.

Below you can see messaging broker interest over time from Google Trends:

Obviously HornetQ lacked popularity compared to the others, and ActiveMQ has recently lost some interest, because most of their users are switching to Apache Apollo. RabbitMQ and Apache Kafka, on the other hand, are trending similarly, with RabbitMQ clearly increasing in popularity.

We also eliminated NSQ in later rounds, since it didn’t have the official support of Java drivers, and eHarmony is a Java shop.

After careful consideration we also eliminated ActiveMQ, as it risks being replaced by Apache Apollo, and we didn’t want to have switch out our solution.

Once we had a final set of two messaging brokers, RabbitMQ and Apache Kafka, we performed very thorough benchmark testing, implementing different scenarios with a combination of different consumer and producer counts and payload sizes.

Here are the resulting recommendations…

Use RabbitMQ as a straight replacement for our current HornetQ installation, or any use case that requires delivery guarantee. Benefits are:

  • Ease of setup
  • Simplicity of migration
  • Support of routing capabilities and JMS compliancy
  • Meets our performance requirements
  • Great documentation and community support

Use Apache Kafka for log processing cases that require no guarantees on delivery. Benefits are:

  • Good for stream processing
  • Good for metrics loading and processing
  • Provides log delivery and processing
    • Exception tracking
    • Fraud analysis

You can find a comparison table of all the messaging brokers we considered, as well as strengths and weaknesses of RabbitMQ and Apache Kafka, in the presentation slides.

Here is the full video of the talk …

View the slides here …