Containerized client-server benchmarking tools for comparing cloud networking solutions.
I developed these with a focus on comparing Skupper to other options for multi-site deployment, but there's nothing Skupper-specific about them.
Each benchmark has a server container and a client container. All tests run with a single client and a single server. The number of client connections and target throughput can be configured. The default is a set with 10 connections, 100 connections, and 500 connections, with each connection running 100 operations per second.
Sample output:
## Configuration
Host: localhost
Port: 55432
Scenarios: 10:100,100:100,500:100
Duration: 60 seconds
Iterations: 1
## Results
CONNECTIONS THROUGHPUT LATENCY AVG LATENCY P50 LATENCY P99
10 994.0 ops/s 0.260 ms 0.230 ms 0.580 ms
100 10,000.9 ops/s 0.200 ms 0.180 ms 0.370 ms
500 49,938.5 ops/s 0.350 ms 0.160 ms 2.840 ms
Each operation is a SQL select.
Throughput is the number of operations per second.
Latency is the duration of an operation in milliseconds.
High and low results from repeated runs are discarded.
h2load - HTTP/2
qbench - Messaging
pgbench - PostgreSQL
wrk - HTTP/1.1
The benchmark client is configured using these environment variables:
BENCHDOG_HOST - The host to connect to (default localhost).
BENCHDOG_PORT - The port to connect to (the default is benchmark-specific).
BENCHDOG_SCENARIOS - A comma-separated list of
<connections>:<rate> pairs, where <connections> is the number of
concurrent client connections and <rate> is the number of operations
per second, per connection (default 10:100,100:100,500:100).
BENCHDOG_DURATION - The time in seconds to run the test (default 60).
BENCHDOG_ITERATIONS - The number of repeated runs for each
scenario (default 1). If greater than 1, Benchdog reports the high
median result selected by average latency.
The individual benchmarks have instructions for running the client and server in Kubernetes and setting up Skupper.
skupper init --router-cpu 0.5
I find these useful for confirming that my configuration has taken effect.
watch kubectl top pod -n benchdog-client
watch kubectl top pod -n benchdog-server