ssl_protocol and ssl_cipher are hyphens

[hackerfactor@…]

I've modified my logging to include the TLS information:

log_format timed_combined '$remote_addr "$ssl_protocol:$ssl_cipher" $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $upstream_response_time $request_time $pipe';

Most web requests log the correct SSL information, such as "TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384" or "TLSv1.3:TLS_AES_128_GCM_SHA256".

However, there is one bot out of China (also uses a few cloud services) that generates "-:-". For example:
106.75.236.230 "-:-" - [30/Apr/2022:03:51:04 -0600] "GET / HTTP/1.1" 400 650 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" - 0.000 .

I have a separate process that logs the client's cipher set. Here's the entire cipher set that the client sent during the initial TLS connection:
Apr 30 03:51:01 myserver web: 106.75.236.230[64042/tcp] -> myserver[443/tcp] : TLS1.0 Client-Hello Cipher 4866,4867,4865,4868,49196,49200,49195,49199,52393,52392,159,158,52394,49327,49325,49188,49192,49162,49172,49315,49311,107,57,49326,49324,49187,49191,49161,49171,49314,49310,103,51,157,156,49313,49309,49312,49308,61,60,53,47,255

In english, this translates to:
[4866,4867,4865,4868,49196,49200,49195,49199,52393,52392,159,158,52394,49327,49325,49188,49192,49162,49172,49315,49311,107,57,49326,49324,49187,49191,49161,49171,49314,49310,103,51,157,156,49313,49309,49312,49308,61,60,53,47,255]

Type: SSL Cipher
Cipher: 4866 = 0x00001302 = TLS-AES-256-GCM-SHA384
Cipher: 4867 = 0x00001303 = TLS-CHACHA20-POLY1305-SHA256
Cipher: 4865 = 0x00001301 = TLS-AES-128-GCM-SHA256
Cipher: 4868 = 0x00001304 = TLS-AES-128-CCM-SHA256
Cipher: 49196 = 0x0000c02c = TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384
Cipher: 49200 = 0x0000c030 = TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384 (Forward secrecy vulnerable)
Cipher: 49195 = 0x0000c02b = TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256
Cipher: 49199 = 0x0000c02f = TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256 (Forward secrecy vulnerable)
Cipher: 52393 = 0x0000cca9 = TLS-ECDHE-ECDSA-WITH-CHACHA20-POLY1305-SHA256
Cipher: 52392 = 0x0000cca8 = TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256 (Forward secrecy vulnerable)
Cipher: 159 = 0x0000009f = TLS-DHE-RSA-WITH-AES-256-GCM-SHA384 (Forward secrecy vulnerable)
Cipher: 158 = 0x0000009e = TLS-DHE-RSA-WITH-AES-128-GCM-SHA256 (Forward secrecy vulnerable)
Cipher: 52394 = 0x0000ccaa = TLS-DHE-RSA-WITH-CHACHA20-POLY1305-SHA256 (Forward secrecy vulnerable)
Cipher: 49327 = 0x0000c0af = TLS-ECDHE-ECDSA-WITH-AES-256-CCM-8
Cipher: 49325 = 0x0000c0ad = TLS-ECDHE-ECDSA-WITH-AES-256-CCM
Cipher: 49188 = 0x0000c024 = TLS-ECDHE-ECDSA-WITH-AES-256-CBC-SHA384 (BEAST and Lucky13 vulnerable)
Cipher: 49192 = 0x0000c028 = TLS-ECDHE-RSA-WITH-AES-256-CBC-SHA384 (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 49162 = 0x0000c00a = TLS-ECDHE-ECDSA-WITH-AES-256-CBC-SHA (BEAST and Lucky13 vulnerable)
Cipher: 49172 = 0x0000c014 = TLS-ECDHE-RSA-WITH-AES-256-CBC-SHA (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 49315 = 0x0000c0a3 = TLS-DHE-RSA-WITH-AES-256-CCM-8 (Forward secrecy vulnerable)
Cipher: 49311 = 0x0000c09f = TLS-DHE-RSA-WITH-AES-256-CCM (Forward secrecy vulnerable)
Cipher: 107 = 0x0000006b = TLS-DHE-RSA-WITH-AES-256-CBC-SHA256 (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 57 = 0x00000039 = TLS-DHE-RSA-WITH-AES-256-CBC-SHA (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 49326 = 0x0000c0ae = TLS-ECDHE-ECDSA-WITH-AES-128-CCM-8
Cipher: 49324 = 0x0000c0ac = TLS-ECDHE-ECDSA-WITH-AES-128-CCM
Cipher: 49187 = 0x0000c023 = TLS-ECDHE-ECDSA-WITH-AES-128-CBC-SHA256 (BEAST and Lucky13 vulnerable)
Cipher: 49191 = 0x0000c027 = TLS-ECDHE-RSA-WITH-AES-128-CBC-SHA256 (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 49161 = 0x0000c009 = TLS-ECDHE-ECDSA-WITH-AES-128-CBC-SHA (BEAST and Lucky13 vulnerable)
Cipher: 49171 = 0x0000c013 = TLS-ECDHE-RSA-WITH-AES-128-CBC-SHA (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 49314 = 0x0000c0a2 = TLS-DHE-RSA-WITH-AES-128-CCM-8 (Forward secrecy vulnerable)
Cipher: 49310 = 0x0000c09e = TLS-DHE-RSA-WITH-AES-128-CCM (Forward secrecy vulnerable)
Cipher: 103 = 0x00000067 = TLS-DHE-RSA-WITH-AES-128-CBC-SHA256 (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 51 = 0x00000033 = TLS-DHE-RSA-WITH-AES-128-CBC-SHA (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 157 = 0x0000009d = TLS-RSA-WITH-AES-256-GCM-SHA384 (Forward secrecy vulnerable)
Cipher: 156 = 0x0000009c = TLS-RSA-WITH-AES-128-GCM-SHA256 (Forward secrecy vulnerable)
Cipher: 49313 = 0x0000c0a1 = TLS-RSA-WITH-AES-256-CCM-8 (Forward secrecy vulnerable)
Cipher: 49309 = 0x0000c09d = TLS-RSA-WITH-AES-256-CCM (Forward secrecy vulnerable)
Cipher: 49312 = 0x0000c0a0 = TLS-RSA-WITH-AES-128-CCM-8 (Forward secrecy vulnerable)
Cipher: 49308 = 0x0000c09c = TLS-RSA-WITH-AES-128-CCM (Forward secrecy vulnerable)
Cipher: 61 = 0x0000003d = TLS-RSA-WITH-AES-256-CBC-SHA256 (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 60 = 0x0000003c = TLS-RSA-WITH-AES-128-CBC-SHA256 (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 53 = 0x00000035 = TLS-RSA-WITH-AES-256-CBC-SHA (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 47 = 0x0000002f = TLS-RSA-WITH-AES-128-CBC-SHA (BEAST and Lucky13 vulnerable, Forward secrecy vulnerable)
Cipher: 255 = 0x000000ff = TLS-EMPTY-RENEGOTIATION-INFO-SCSV

In my cipher set log, the initial TLS1.0 is expected. Almost everyone connects with TLS1.0 and sends their cipher set before changing the protocol to TLS1.2 or TLS1.3.

The cipher set is unusual because it contains a ton of known-vulnerable ciphers. (This is clearly a vulnerability scanner.)

Here are some other log entries from the same bot:
106.75.236.230 "TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384" - [27/Apr/2022:18:48:21 -0600] "GET /favicon.ico HTTP/1.1" 200 143 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" 0.004 0.001 .
106.75.236.230 "TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384" - [27/Apr/2022:18:50:20 -0600] "GET / HTTP/1.1" 200 5412 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" 0.008 0.008 .
106.75.236.230 "TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384" - [27/Apr/2022:18:50:20 -0600] "GET / HTTP/1.1" 200 5409 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" 0.008 0.007 .
106.75.236.230 "TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384" - [27/Apr/2022:18:50:47 -0600] "GET / HTTP/1.1" 200 5410 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" 0.008 0.008 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [29/Apr/2022:05:48:21 -0600] "GET / HTTP/1.1" 200 5339 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" 0.008 0.010 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [29/Apr/2022:05:48:58 -0600] "GET / HTTP/1.1" 200 5408 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" 0.008 0.010 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [29/Apr/2022:05:51:26 -0600] "GET /favicon.ico HTTP/1.1" 200 143 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" 0.000 0.002 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [30/Apr/2022:03:47:47 -0600] "GET / HTTP/1.1" 200 5418 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" 0.012 0.009 .
106.75.236.230 "-:-" - [30/Apr/2022:03:47:59 -0600] "GET / HTTP/1.1" 400 248 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" - 0.000 .
106.75.236.230 "-:-" - [30/Apr/2022:03:48:04 -0600] "GET / HTTP/1.1" 400 248 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" - 0.000 .
106.75.236.230 "-:-" - [30/Apr/2022:03:48:42 -0600] "GET /favicon.ico HTTP/1.1" 400 248 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" - 0.000 .
106.75.236.230 "-:-" - [30/Apr/2022:03:48:49 -0600] "GET /favicon.ico HTTP/1.1" 400 248 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" - 0.000 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [30/Apr/2022:03:50:06 -0600] "GET / HTTP/1.1" 200 5410 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" 0.008 0.008 .
106.75.236.230 "-:-" - [30/Apr/2022:03:50:15 -0600] "GET / HTTP/1.1" 400 248 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" - 0.000 .
106.75.236.230 "-:-" - [30/Apr/2022:03:50:22 -0600] "GET / HTTP/1.1" 400 248 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" - 0.000 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [30/Apr/2022:03:50:30 -0600] "GET / HTTP/1.1" 200 5326 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" 0.008 0.010 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [30/Apr/2022:03:50:36 -0600] "GET / HTTP/1.1" 200 5349 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" 0.004 0.005 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [30/Apr/2022:03:50:53 -0600] "GET / HTTP/1.1" 200 5407 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" 0.008 0.008 .
106.75.236.230 "-:-" - [30/Apr/2022:03:50:56 -0600] "GET / HTTP/1.1" 400 650 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" - 0.000 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [30/Apr/2022:03:51:02 -0600] "GET / HTTP/1.1" 200 5414 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" 0.008 0.009 .
106.75.236.230 "-:-" - [30/Apr/2022:03:51:04 -0600] "GET / HTTP/1.1" 400 650 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11) AppleWebKit/601.1.27 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/601.1.27" - 0.000 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [30/Apr/2022:03:51:34 -0600] "GET /favicon.ico HTTP/1.1" 200 143 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" 0.004 0.001 .
106.75.236.230 "TLSv1.3:TLS_AES_256_GCM_SHA384" - [30/Apr/2022:03:51:36 -0600] "GET /favicon.ico HTTP/1.1" 200 143 "-" "Dalvik/2.1.0 (Linux; U; Android 9.0; ZTE BA520 Build/MRA58K)" 0.000 0.000 .

While this bot changes user-agent strings, it always goes for the same URLs. This bot changes ciphers during different initial negotiations.

When nginx logs "-:-", it also returns an HTTP 400. However, I can't find any other details about the results.

I suspect that the bot is changing ciphers after the initial negotiation. (Very unusual for a regular user, but not unusual for an SSL vulnerability scanner.)

Looking over the source code for OpenSSL 1.1.1n, the function SSL_get_version cannot return a hyphen. The hyphen has to be coming from the nginx code, but I can't figure out where.

1. Where is the hyphen for cipher and protocol coming from?
2. Why is it a hyphen when it's a valid TLS connection? (If it were not valid, I wouldn't see the "GET / HTTP/1.1" request.)
3. Is there some way to log each time the client changes the TLS cipher or protocol after the initial negotiation? (I want to spot and filter out bots.)

[Maxim Dounin]

Where is the hyphen for cipher and protocol coming from?

A hyphen is logged when a variable value is not found, see ​log_format.

Why is it a hyphen when it's a valid TLS connection? (If it were not valid, I wouldn't see the "GET / HTTP/1.1" request.)

Status code 400 most likely implies there were no valid SSL/TLS connection, but rather an HTTP request sent to HTTPS port, see ​error 497. Further details can be found in the error log, there should be corresponding "client sent plain HTTP request to HTTPS port" errors at the info level.

Hope this helps. From the information provided it looks like everything works as it should, closing this. If you have further question about nginx, please consider using ​support options available.

[hackerfactor@…]

Hi Maxim Dounin,

I'm not seeing an error 497. I'm seeing an error 400.
I'm also running a packet sniffer over my own connections and this connection definitely used HTTPS with TLS, not HTTP over 443.

The request was TLS-wrapped.
The client set a list of ciphers.
The server selected a cipher.
The web browser logged a "GET / HTTP/1.1" that came out of the TLS tunnel.
And yet, $ssl_protocol and $ssl_cipher were both "-".

Because it comes from a bot in China, and a few cloud providers -- and is not widespread -- I am concerned that this may be a vulnerability that someone is trying to exploit. Without knowing why the log format returned a "-" when it was clearly a TLS tunnel, I can't track what the bot may be trying to exploit.

[hackerfactor@…]

I'm reopening this since the explanation from Maxim Dounin does not match the information I provided.

This is happening from a single bot in China and a few cloud providers. I think the chinese bot is also being run on a few cloud providers. This is not a well-known vulnerability that this bot is scanning for. I think this may be a new vulnerability that someone hasn't exploited yet.

Openssl sees the tunnel, but nginx loses track and cannot associate the request with the tunnel. (Hence the TLS connection but "-:-" for the cipher and protocol.) I think this bot/author has found a way to disconnect the web request from the TLS tunnel. Effectively turning it into a potential side-channel exploit.

[Maxim Dounin]

I'm not seeing an error 497. I'm seeing an error 400.

The 49X non-standard error codes are only used during error redirection with error_page, and converted to 400 when the response is actually sent to the client.

I'm also running a packet sniffer over my own connections and this connection definitely used HTTPS with TLS, not HTTP over 443.

If you think the connection used HTTPS, please provide the error as logged by nginx when returning 400, there should be something at the "info" level. Also, a ​debug log might be helpful (though just the error likely will be enough to confirm there were no HTTPS).

I'm reopening this since the explanation from Maxim Dounin does not match the information I provided.

For now, the only thing which doesn't match my explanation is your claim that "this connection definitely used HTTPS with TLS". You may want to provide some details to support this claim. Note that access logs you've provided certainly show requests from multiple connections, and it might not be trivial to match connections and requests based on the information being logged.

[hackerfactor@…]

I've managed to replicate the issue. (Or at least, something that behaves like this issue.)

wget -U TEST -O - '​http://myserver:443/'
This generates the "HTTP 400" as you describe, but it doesn't trigger any TLS exchange.

I had to write a little code to test this:
(1) Open the TCP connection
(2) Call OpenSSL to initiate the TLS connection, and
(3) Write directly to the TCP socket rather than through the TLS handle.
Then I can generate the initial TLS handshake followed by a plain text GET that generates the HTTP 400 error.

Why do it this way?
I'm just guessing here: A stateful packet inspector will see the TLS connection and ignore the rest of the data. (Because it's encrypted.) This permits scanning a web server to see if it supports plain text HTTP over port 443 without being detected by the stateful packet inspector.

I'm going to modify my own packet scanner to look for this situation.

[Maxim Dounin]

I've managed to replicate the issue. (Or at least, something that behaves like this issue.)

wget -U TEST -O - '​http://myserver:443/'
This generates the "HTTP 400" as you describe, but it doesn't trigger any TLS exchange.

That's exactly what I've described in the initial response: sending an HTTP request to HTTPS port. And this perfectly matches all the information you've provided so far.

I had to write a little code to test this:
(1) Open the TCP connection
(2) Call OpenSSL to initiate the TLS connection, and
(3) Write directly to the TCP socket rather than through the TLS handle.
Then I can generate the initial TLS handshake followed by a plain text GET that generates the HTTP 400 error.

It is not clear what do you mean by "to test this". No additional code is needed to test the HTTP request to HTTPS port case, the full test case using wget is already provided in your comment. On the other hand, attempts to send any plain text data after an SSL handshake won't result in error 400, but will rather result in an SSL error, such as SSL_read() failed (SSL: error:1408F10B:SSL routines:ssl3_get_record:wrong version number) while waiting for request....

If you are able to reproduce error 400 by sending plain text data after an SSL handshake, please provide a code (and/or a packet trace) which demonstrates this.

[Maxim Dounin]

Feedback timeout. As previously suggested in comment:1, from the information provided it looks like everything works as it should.