ZeroMQ is a community of projects focused on decentralized messaging and computing. The features of this loose cloud of projects evolves all the time. Having said that, we can sketch its shape and properties as they are now. This is a wiki page, so feel free to add links and corrections.
The Core Engines
The focus of the ZeroMQ community is building and using distributed messaging engines. These range in size and capability from "oops, almost missed that" to "I'm actually afraid to ask now". The ZeroMQ engines share a common view of the universe of decentralized computing. They agree on protocols (RFCs) for connecting to each other and exchanging messages. Messages are blobs of useful data of any reasonable size.
As architect, you embed these engines into your applications. This gives your own code as much power as the engine provides. You use this power to queue, route, and filter messages according to various "patterns."
This messaging power spreads out across the network. Hence, we call it "distributed messaging."
The first of these engines, written in C++, was once called "ZeroMQ." The name has expanded to cover the whole family, and its protocols.
ZeroMQ has the core technical goals of simplicity and scalability. Its two main technical principles are:
- The work you do not do takes no time, and
- The code you do not run cannot crash.
The software and the protocols are open source, and owned by the community of experts who built them.
The community philosophy is "send us your patches, and we will merge them." The ZeroMQ development process ensures the code remains clean and stable.
Any messaging product aims to solve the same broad set of problems. Most do it by centralizing the intelligence. ZeroMQ does it by simplifying and spreading the intelligence out. There are a few other distributed messaging projects.
The most successful competitor is ignorance, which has a long track record and is always a safe bet. To use ZeroMQ with profit, you must learn decentralized computing. It's easier to avoid learning this. Yet if you want to build scalable systems, you either learn it, or you outsource.
The ZeroMQ protocols run on everything of interest. You can connect at 32KB embedded chips to a z/OS mainframes running IBM dialects of Unix. The protocols are trivial to start with, and deep. It is a matter of a few hours to make a minimal ZeroMQ engine. It may take years to make a Really Good One.
The ZeroMQ community used to document the language binding projects around ZeroMQ. These days you may get a better picture just by searching GitHub. For many languages, like C, Python, Java, .NET, Ruby, PHP, Perl, and Erlang, there are several alternatives to choose from.
libzmq can talk over TCP, IPC (on POSIX), inproc, TIPC (a nice protocol from Ericsson), PGM (reliable multicast), NORM (another reliable multicast protocol), and SOCKS5, so presumably it can also run over Tor. The majority of other ZeroMQ engines focus on TCP, the lingua franca of distributed computing. Since version 4.2, libzmq supports UDP in Unicast and Multicast modes.
The ZeroMQ protocols live as RFCs on http://rfc.zeromq.org. The main one would be RFC 23, the ZeroMQ Message Transport Protocol (ZMTP), which describes how two pieces talk, over TCP or IPC. RFC 23 provides backwards compatibility to all older stable releases of ZeroMQ.
ZMTP, the core protocol, has a flexible and pragmatic attitude towards security. This covers everything from "really, you're going to steal my cat photos?" insouciance to "good luck cracking that, see you in a thousand years" paranoia. The core C/C+++ library implements the NULL and PLAIN mechanisms (those were easy), and also CURVE and GSSAPI (rather harder). CURVE is the CurveZMQ.org protocol for authentication and encryption. GSSAPI lets us plug into Kerberos. The core library has an authentication API called "ZAP" that lets you add, e.g. your own IP address blacklisting at the connection level.
CurveZMQ uses Daniel J. Bernstein's Curve25519 elliptic curve. It borrows from CurveCP's security handshake. CurveZMQ claims to offer something called "perfect forward secrecy", and good performance.
We said there are a fair number of those engines. Depending on your language and needs and constraints, you can choose from:
- The original C/C+++ engine: https://github.com/zeromq/libzmq, which tends to represent the State of the Art.
- A native Java stack: https://github.com/zeromq/jeromq, fully compatible at both API and protocol level, sans encryption or PGM.
- A native .NET stack: https://github.com/zeromq/netmq, same constraints as JeroMQ
- A native Erlang stack: https://github.com/zeromq/ezmq
- A native Python stack: https://github.com/caedesvvv/zmqproto
- A native C stack: https://github.com/zeromq/libzmtp, designed for small places you would not normally expect messaging technology to fit
- A Netty stack: https://github.com/spotify/netty-zmtp
ZeroMQ was famous for taking the old, tired BSD socket API and doing a kind of Hollywood makeover. Some people do not like the way ZeroMQ does its stuff, by which we mean hiding things from your applications. Yet, hiding stuff from the user is the name of the scalability game, and ZeroMQ did it right in most cases.
The ZeroMQ sockets have become models for well-defined packages of behavior. They even have their own RFCs full of SHALLs and MUST NOTs, like pub-sub and request-reply. The socket API and metaphor can only stretch so far before we feel "this was cool in 1980 but is kind of bizarre in 2020." There are better ways to make APIs. ZeroMQ has been repositioning itself towards those.
For C/C++, the CZMQ binding wraps the core C API in something more usable. CZMQ has itself been wrapped in a dozen or so languages. It creates a world of intelligent "actors" that you can stop and start on demand, and talk to as if they were sockets. Actors use threads, and CZMQ is not meant for tiny machines. It's meant to stretch across thousands of cores across hundreds of boxes.
Above CZMQ, another next popular abstraction is Zyre. This is a clustering technology based on (as always) RFCs (RFC 36). Zyre was born as a way to cluster mobile phones or tablets around a WiFi hotspot. Today it has implementations in C, Java, Python, and Erlang. Zyre nodes can discover each other using UDP beaconing (a CZMQ service), or a separate TCP gossip discovery network (another CZMQ service).
While ZeroMQ is distributed, it also has a broker, called Malamute. Malamute is useful for cases where you need a central entity to hold mail, abstract services, and so on. The broker is light enough to run as a thread in another process.
The Malamute protocol offers topic-based PubSub, service calls, and mailboxes. Today's version is a minimal working model,
As the ZeroMQ community built out real services over ZeroMQ (like the CZMQ gossip service), they found that they needed tools. In CZMQ and Zyre there are half-dozen or more protocols that have some non-trivial grammar and encoding. Each protocol needs its own codecs, clients and servers.
This raises two distinct problems. One, how to make codecs in arbitrary languages at the lowest cost. Two, how to make clients and servers at the lowest cost. We borrowed from old buried iMatix projects including OpenAMQ (the first AMQP client/server) and GSL (a code generator toolkit). Stripping down and simplifying these over a few years gave zproto, a "protocol framework", which is heavy on model-oriented code generation, and finite state machines.
The ZeroMQ community uses a collaboration contract, C4.1. This is an RFC (of course), at http://rfc.zeromq.org/spec:22. It defines how the community works together and has been the main factor for the happy growth and stability of the community.
Not all ZeroMQ projects use C4.1, particularly those with significant bite before 2012. Since ZeroMQ adopted it in early 2012, it's become the default rulebook for new projects. C4.1 drives change through rules that were unusual at first, and now seem obvious. Don't work alone. Don't use branches. Make minimal plausible solutions to small and clear problems. Make patches, not argument. Merge first, review second. Don't discriminate. Embrace diversity. Aim for community first and code second.
ZeroMQ's libzmq core is owned by its contributors and is distributed under a LGPLv3 ++ static link exception combo license. These days most new projects prefer the simpler MPLv2, which achieves much the same result: leverage on commercial derivatives to merge back, without creating hurdles to adoption.
The ZeroMQ organization is a loose confederation without a clear power structure. Most projects live on GitHub. The organization wiki page explains how anyone can join the Owners' team. You bring in a relevant project, and accept the rules. Every project owner runs their ship as they see fit, and people jump from ship to ship over time. The happier, and better run ships tend to accumulate more party-goers. The less enjoyable projects lose their human passengers.
The originator of the project, Pieter Hintjens, runs the community from the sidelines. He contributes to some projects and is the main RFC editor.
ZeroMQ has no roadmaps, no feature requests, no lists of Things We Could Do, no regular meetings, no Central Planning Committees. ZeroMQ does make a promise though: it will not break user space. You'll find ZeroMQ v2, v3 and v4 apps talking to each other in the same universe. This is unusual for a technology that evolves as fast as ZeroMQ does. The tactic is, as often, quite simple. We add new APIs, leave old ones, deprecate them, remove them over time, and never reuse them.
You could do worse than get the O'Reilly ZeroMQ book, and start reading at Chapter six, then from the beginning again. Or, if you're a student or poor, read the ZeroMQ Guide, which is exactly the same text, minus the royalty payments.
Written: 08 Aug 2014 13:21
Revised: 29 Apr 2016 16:00
If you found this page useful, please rate it up so others will find it.
- An Introduction to ØMQ
- Coding Style Guidelines
- Guidelines for ØMQ bindings
- Installing on Windows
- libzmq Source Git Repository
- Mailing Lists
- Message API Goals
- ØMQ Contribution Policy
- ØMQ Cookbook
- ØMQ Distributions
- ØMQ Labs
- ØMQ Owners
- ØMQ Release Policies
- ØMQ/2.1 Class Diagram
- ØMQ/3.0 Roadmap
- ØMQ/3.2 Roadmap
- Process Manager
- Projects using ØMQ
- Release notes for ØMQ/4.0.0
- Reporting and Tracking an Issue
- The native ØMQ API (libzmq)
- The ØMQ Guide
- The Ten Minute Talk
- Trademark Policy
- Tuning ØMQ
- Upgrading from 2.0 to 2.1
- Upgrading from libzmq 2.x to 3.2
- Upgrading to 4.1 from 4.0
- Using Valgrind
- Welcome from AMQP
Who's watching this page?Dominic Luciano
Fernando J Quintero