Opened 3 years ago
Last modified 2 years ago
#2248 closed defect
Stalled connection when using `aio_write on` — at Version 1
Reported by: | Owned by: | ||
---|---|---|---|
Priority: | minor | Milestone: | |
Component: | nginx-core | Version: | 1.19.x |
Keywords: | aio aio_write proxy | Cc: | aminvakil@… |
uname -a: | Linux hostname 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64 GNU/Linux | ||
nginx -V: |
nginx version: nginx/1.20.1
built by gcc 8.3.0 (Debian 8.3.0-6) built with OpenSSL 1.1.1d 10 Sep 2019 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -fdebug-prefix-map=/data/builder/debuild/nginx-1.20.1/debian/debuild-base/nginx-1.20.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie' |
Description (last modified by )
Connection gets stalled most times and does not continue downloading files most of times, here is the configuration files:
/etc/nginx/nginx.conf
worker_rlimit_nofile 40000; user nginx; worker_processes auto; error_log /var/log/nginx/error.log error; pid /var/run/nginx.pid; events { worker_connections 1024; use epoll; } http { server_tokens off; include /etc/nginx/mime.types; default_type application/octet-stream; access_log /var/log/nginx/access.log main buffer=2048k flush=10s; open_file_cache max=10000 inactive=5m; open_file_cache_valid 2m; open_file_cache_min_uses 2; open_file_cache_errors off; sendfile on; sendfile_max_chunk 512k; tcp_nopush on; tcp_nodelay on; output_buffers 2 8k; keepalive_timeout 15; http2_idle_timeout 1m; # SSL ssl_session_timeout 8h; ssl_session_cache shared:SSL:100m; # one megabyte can store about 4000 sessions ssl_session_tickets off; ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; ssl_prefer_server_ciphers on; include /etc/nginx/conf.d/*.conf; }
/etc/nginx/conf.d/example.org.conf
upstream cdns { server cdn1.example.org; keepalive 20; } server { listen 80 reuseport; listen 443 ssl http2 reuseport; server_name cache.example.org; ssl_certificate /etc/nginx/ssl/example.org.pem; ssl_certificate_key /etc/nginx/ssl/example.org.key; root /usr/share/nginx/html/; aio threads; aio_write on; location /apk-files/ { proxy_pass http://cdns; } }
Then using http2 to connect to server gets stalled like this about 90% of times.
curl -LO "https://example.org/samplefile" -vvvvv % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 1.1.1.1:443... * Connected to example.org (1.1.1.1) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/certs/ca-certificates.crt * CApath: none } [5 bytes data] * TLSv1.3 (OUT), TLS handshake, Client hello (1): } [512 bytes data] * TLSv1.3 (IN), TLS handshake, Server hello (2): { [122 bytes data] * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): { [19 bytes data] * TLSv1.3 (IN), TLS handshake, Certificate (11): { [4766 bytes data] * TLSv1.3 (IN), TLS handshake, CERT verify (15): { [264 bytes data] * TLSv1.3 (IN), TLS handshake, Finished (20): { [52 bytes data] * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): } [1 bytes data] * TLSv1.3 (OUT), TLS handshake, Finished (20): } [52 bytes data] * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN, server accepted to use h2 * Server certificate: * subject: CN=*.example.org * start date: Dec 14 11:59:52 2020 GMT * expire date: Dec 14 09:32:52 2021 GMT * subjectAltName: host "cache.example.org" matched cert's "*.example.org" * issuer: C=PL; O=Unizeto Technologies S.A.; OU=Certum Certification Authority; CN=Certum Domain Validation CA SHA2 * SSL certificate verify ok. * Using HTTP2, server supports multiplexing * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 } [5 bytes data] * Using Stream ID: 1 (easy handle 0x801cbac360) } [5 bytes data] > GET /samplefile HTTP/2 > Host: example.org > user-agent: curl/7.79.0 > accept: */* > { [5 bytes data] * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): { [57 bytes data] * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): { [57 bytes data] * old SSL session ID is stale, removing { [5 bytes data] * Connection state changed (MAX_CONCURRENT_STREAMS == 128)! } [5 bytes data] < HTTP/2 200 < server: nginx < date: Mon, 20 Sep 2021 08:53:41 GMT < content-type: application/octet-stream < content-length: 40703269 < last-modified: Mon, 08 Feb 2021 08:13:22 GMT < etag: "6020f2a2-26d1525" < accept-ranges: bytes < { [7942 bytes data] 0 38.8M 0 73478 0 0 5165 0 2:11:20 0:00:14 2:11:06 0
I couldn't reproduce the issue with curl --http1.1
or after commenting aio_write on
.
Note:
See TracTickets
for help on using tickets.