Skip to content

Latest commit

 

History

History
158 lines (115 loc) · 5.84 KB

README.md

File metadata and controls

158 lines (115 loc) · 5.84 KB
Raw

ZeroMQ -- Connecting computers via message queue

{:.no_toc}

* TOC {:toc}

The goal

ZeroMQ is a simple to implement way to transfer data between server & client application. It is available for a vast range of programming languages. Including Python PyZMQ. It has a lot of options, including broadcasting. I will only present a very simple example because most of it we don't need.

Questions to David Rotermund

Commands
zmq.Context Create a zmq Context
zmq.Context.socket Create a Socket associated with this Context.
zmq.Socket The ZMQ socket object
zmq.Socket.bind Bind the socket to an address.
zmq.Socket.connect Connect to a remote 0MQ socket.
zmq.Socket.recv Receive a message
zmq.Socket.recv_json Receive a Python object as a message using json to serialize.
zmq.Socket.recv_pyobj Receive a Python object as a message using pickle to serialize.
zmq.Socket.recv_string Receive a unicode string, as sent by send_string.
zmq.Socket.recv_multipart Receive a multipart message as a list of bytes or Frame objects
zmq.Socket.send Send a single zmq message frame on this socket.
zmq.Socket.send_json Send a Python object as a message using json to serialize.
zmq.Socket.send_pyobj Send a Python object as a message using pickle to serialize.
zmq.Socket.send_string Send a Python unicode string as a message with an encoding.
zmq.Socket.send_multipart Send a sequence of buffers as a multipart message.

Socket Types

There are many socket types. I will only quote the two relevant ones for this example from the API documentation:

ZMQ_REQ

A socket of type ZMQ_REQ is used by a client to send requests to and receive replies from a service. This socket type allows only an alternating sequence of zmq_send(request) and subsequent zmq_recv(reply) calls. Each request sent is round-robined among all services, and each reply received is matched with the last issued request.

If no services are available, then any send operation on the socket shall block until at least one service becomes available. The REQ socket shall not discard messages.

Summary of ZMQ_REQ characteristics
Compatible peer sockets ZMQ_REP, ZMQ_ROUTER
Direction Bidirectional
Send/receive pattern Send, Receive, Send, Receive, …
Outgoing routing strategy Round-robin
Incoming routing strategy Last peer
Action in mute state Block

ZMQ_REP

A socket of type ZMQ_REP is used by a service to receive requests from and send replies to a client. This socket type allows only an alternating sequence of zmq_recv(request) and subsequent zmq_send(reply) calls. Each request received is fair-queued from among all clients, and each reply sent is routed to the client that issued the last request. If the original requester does not exist any more the reply is silently discarded.

Summary of ZMQ_REP characteristics
Compatible peer sockets ZMQ_REQ, ZMQ_DEALER
Direction Bidirectional
Send/receive pattern Receive, Send, Receive, Send, …
Incoming routing strategy Fair-queued
Outgoing routing strategy Last peer

Simple example where we move np.ndarrays around

Client

import zmq
import numpy as np

context = zmq.Context()

socket = context.socket(zmq.REQ)
# We connect to the localhost at port 5555
socket.connect("tcp://localhost:5555")

# Generating a random np matrix
rng = np.random.default_rng()
np_array = rng.random((100, 10))

# Send it over the wire
socket.send_pyobj(np_array)

# We get the squared version back 
np_array_return = socket.recv_pyobj()

# Check if we get a correct return message 
print(type(np_array_return))

np_array = np_array**2

print("Difference:")
print(np.sum(np.abs(np_array_return - np_array)))

Output:

<class 'numpy.ndarray'>
Difference:
0.0

Server

import zmq

context = zmq.Context()
socket = context.socket(zmq.REP)
# Listen at port 5555
socket.bind("tcp://*:5555")

while True:
    # We should receive a numpy array
    input_np = socket.recv_pyobj()

    # Let's do something with this information
    print(type(input_np))
    print(input_np.shape)
    input_np = input_np**2

    # We send the results back
    socket.send_pyobj(input_np)

Output:

<class 'numpy.ndarray'>
(100, 10)

Non-blocking

There is this option to receive packages in a non blocking way:

    confirm = False
    while not confirm:
        try:
            packet_in = socket.recv_pyobj(zmq.NOBLOCK)
            confirm = True
        except zmq.error.Again:
            pass