From 71f7ef2d77aee219072b6af063024f913da14416 Mon Sep 17 00:00:00 2001 From: Jun Kurihara Date: Sat, 17 May 2025 04:17:50 +0900 Subject: [PATCH] bench --- README.md | 2 +- bench/Caddyfile | 4 +++ bench/README.md | 71 +++++++++++++++++++--------------------- bench/docker-compose.yml | 6 ++-- bench/nginx.conf | 5 ++- submodules/s2n-quic | 2 +- 6 files changed, 46 insertions(+), 44 deletions(-) diff --git a/README.md b/README.md index e114859..70b3dba 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ Supported features are summarized as follows: [^sanitization]: By default, `rpxy` provides the *TLS connection sanitization* by correctly binding a certificate used to establish a secure channel with the backend application. Specifically, it always keeps the consistency between the given SNI (server name indication) in `ClientHello` of the underlying TLS and the domain name given by the overlaid HTTP HOST header (or URL in Request line). We should note that NGINX doesn't guarantee such a consistency by default. To this end, you have to add `if` statement in the configuration file in NGINX. - This project is still *work-in-progress*. But it is already working in some production environments and serves a number of domain names. Furthermore it *significantly outperforms* NGINX and Caddy, e.g., *1.5x faster than NGINX*, in the setting of a very simple HTTP reverse-proxy scenario (See [`bench`](./bench/) directory). + This project is still *work-in-progress*. But it is already working in some production environments and serves a number of domain names. Furthermore it *significantly outperforms* NGINX and Caddy, e.g., *30% or more faster than NGINX*, in the setting of a very simple HTTP reverse-proxy scenario (See [`bench`](./bench/) directory). ## Installing/Building an Executable Binary of `rpxy` diff --git a/bench/Caddyfile b/bench/Caddyfile index 3ba1fa2..d12b122 100644 --- a/bench/Caddyfile +++ b/bench/Caddyfile @@ -2,9 +2,13 @@ auto_https off } + :80 { # Proxy everything else to Rocket reverse_proxy backend-nginx + log { + level ERROR + } } diff --git a/bench/README.md b/bench/README.md index 55ca4dd..f75b978 100644 --- a/bench/README.md +++ b/bench/README.md @@ -8,44 +8,41 @@ $ rewrk -c 512 -t 4 -d 15s -h http://localhost:8080 --pct ## Tests on `linux/arm64/v8` -Done at Jul. 15, 2023 +Done at May. 17, 2025 ### Environment -- `rpxy` commit id: `1da7e5bfb77d1ce4ee8d6cfc59b1c725556fc192` -- Docker Desktop 4.21.1 (114176) +- `rpxy` commit id: `e259e0b58897258d98fdb7504a1cbcbd7c5b37db` +- Docker Desktop 4.41.2 (191736) - ReWrk 0.3.2 -- Macbook Pro '14 (2021, M1 Max, 64GB RAM) +- MacBook Pro '14 (2021, M1 Max, 64GB RAM) The docker images of `nginx` and `caddy` for `linux/arm64/v8` are pulled from the official registry. ### Result for `rpxy`, `nginx` and `caddy` -``` ----------------------------- +```bash Benchmark on rpxy Beginning round 1... Benchmarking 512 connections @ http://localhost:8080 for 15 second(s) Latencies: Avg Stdev Min Max - 19.64ms 8.85ms 0.67ms 113.22ms + 10.62ms 4.47ms 0.47ms 73.01ms Requests: - Total: 390078 Req/Sec: 26011.25 + Total: 720148 Req/Sec: 48025.92 Transfer: - Total: 304.85 MB Transfer Rate: 20.33 MB/Sec + Total: 563.85 MB Transfer Rate: 37.60 MB/Sec + --------------- + --------------- + | Percentile | Avg Latency | + --------------- + --------------- + -| 99.9% | 79.24ms | -| 99% | 54.28ms | -| 95% | 42.50ms | -| 90% | 37.82ms | -| 75% | 31.54ms | -| 50% | 26.37ms | +| 99.9% | 54.78ms | +| 99% | 35.86ms | +| 95% | 23.09ms | +| 90% | 19.82ms | +| 75% | 16.14ms | +| 50% | 13.54ms | + --------------- + --------------- + -721 Errors: error shutting down connection: Socket is not connected (os error 57) - sleep 3 secs ---------------------------- Benchmark on nginx @@ -53,23 +50,23 @@ Beginning round 1... Benchmarking 512 connections @ http://localhost:8090 for 15 second(s) Latencies: Avg Stdev Min Max - 33.26ms 15.18ms 1.40ms 118.94ms + 14.55ms 13.05ms 0.57ms 255.24ms Requests: - Total: 230268 Req/Sec: 15356.08 + Total: 525866 Req/Sec: 35073.37 Transfer: - Total: 186.77 MB Transfer Rate: 12.46 MB/Sec + Total: 427.78 MB Transfer Rate: 28.53 MB/Sec + --------------- + --------------- + | Percentile | Avg Latency | + --------------- + --------------- + -| 99.9% | 99.91ms | -| 99% | 83.74ms | -| 95% | 70.67ms | -| 90% | 64.03ms | -| 75% | 54.32ms | -| 50% | 45.19ms | +| 99.9% | 235.17ms | +| 99% | 91.77ms | +| 95% | 48.86ms | +| 90% | 39.08ms | +| 75% | 28.78ms | +| 50% | 21.77ms | + --------------- + --------------- + -677 Errors: error shutting down connection: Socket is not connected (os error 57) +227 Errors: connection closed sleep 3 secs ---------------------------- @@ -78,23 +75,21 @@ Beginning round 1... Benchmarking 512 connections @ http://localhost:8100 for 15 second(s) Latencies: Avg Stdev Min Max - 48.51ms 50.74ms 0.34ms 554.58ms + 70.44ms 220.19ms 0.67ms 4140.08ms Requests: - Total: 157239 Req/Sec: 10485.98 + Total: 79980 Req/Sec: 5334.74 Transfer: - Total: 125.99 MB Transfer Rate: 8.40 MB/Sec + Total: 64.45 MB Transfer Rate: 4.30 MB/Sec + --------------- + --------------- + | Percentile | Avg Latency | + --------------- + --------------- + -| 99.9% | 473.82ms | -| 99% | 307.16ms | -| 95% | 212.28ms | -| 90% | 169.05ms | -| 75% | 115.92ms | -| 50% | 80.24ms | +| 99.9% | 3550.19ms | +| 99% | 1847.80ms | +| 95% | 672.82ms | +| 90% | 440.34ms | +| 75% | 224.81ms | +| 50% | 128.79ms | + --------------- + --------------- + - -708 Errors: error shutting down connection: Socket is not connected (os error 57) ``` ## Results on `linux/amd64` diff --git a/bench/docker-compose.yml b/bench/docker-compose.yml index 7b2b043..aaf800d 100644 --- a/bench/docker-compose.yml +++ b/bench/docker-compose.yml @@ -28,7 +28,7 @@ services: dockerfile: docker/Dockerfile restart: unless-stopped environment: - - LOG_LEVEL=info + - LOG_LEVEL=error # almost nolog - LOG_TO_FILE=false ports: - 127.0.0.1:8080:8080 @@ -47,7 +47,7 @@ services: tty: false privileged: true volumes: - - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro + - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro # set as almost nolog - /var/run/docker.sock:/tmp/docker.sock:ro logging: options: @@ -64,7 +64,7 @@ services: restart: unless-stopped tty: false volumes: - - ./Caddyfile:/etc/caddy/Caddyfile:ro + - ./Caddyfile:/etc/caddy/Caddyfile:ro # set as almost no log networks: bench-nw: diff --git a/bench/nginx.conf b/bench/nginx.conf index 0ef89e3..b05910a 100644 --- a/bench/nginx.conf +++ b/bench/nginx.conf @@ -31,11 +31,14 @@ # '"$request" $status $body_bytes_sent ' # '"$http_referer" "$http_user_agent" ' # '"$upstream_addr"'; -# access_log off; +access_log off; + # ssl_protocols TLSv1.2 TLSv1.3; # ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384'; # ssl_prefer_server_ciphers off; # error_log /dev/stderr; +error_log /dev/null crit; + # resolver 127.0.0.11; # # HTTP 1.1 support # proxy_http_version 1.1; diff --git a/submodules/s2n-quic b/submodules/s2n-quic index 250347d..f48dc68 160000 --- a/submodules/s2n-quic +++ b/submodules/s2n-quic @@ -1 +1 @@ -Subproject commit 250347d34e8841a8958437f7c461513d157e1957 +Subproject commit f48dc6846fc363ab668441ff5a471fc27199bb55