Could not allocate new session in SSL session shared cache

[Johannes Ziemke]

Hi,

I'm using nginx as reverse proxy in front of haproxy. I'm using this ssl_session_cache config:

ssl_session_cache   shared:SSL:10m;
ssl_session_timeout 512m;

Now I get from time to time such errors in the log:

2014-09-03T09:51:34+00:00 hostname nginx: 2014/09/03 09:51:34 [alert] 27#0: *144835271 could not allocate new session in SSL session shared cache "SSL" while SSL handshaking, client: a.b.c.d, server: 0.0.0.0:443

Unfortunately that error doesn't say much. Looking at the code shows that it probably failed to allocate via ngx_slab_alloc_locked():

​https://github.com/nginx/nginx/blob/master/src/event/ngx_event_openssl.c#L2088

I'll try to raise the session cache and see if it helps, but since it's just a cache I would expect only performance differences.

FWIW: The nginx is running in a docker container, to be specific: ​https://registry.hub.docker.com/u/fish/haproxy/ (Although I've raised the cache setting there already

[Maxim Dounin]

As the error says, nginx wasn't able to allocate new session in SSL shared cache. This is something that may happen if the cache is full, and nginx wasn't able to free enough space by removing a least recently used session. Use smaller session timeouts or a larger shared memory zone to avoid cache overflows.

The message is probably a bit too scary (while the situation itself is mostly harmless). It was set to "alert" level as it wasn't expected to happen in real life. It seems to happen though - especially after 5024d29354f1, with configurations assuming nginx will silently throw away old sessions if there will be not enough memory in the shared memory zone. The severity of the error should be reduced, and/or nginx should try harder (free more old sessions) before giving up.

For now you may use smaller session timeouts and/or a larger shared memory zone. This way sessions will expire and will be removed from the cache before the cache will be overflowed, and no errors will happen.

[rosenqui@…]

+1 on changing this log entry from alert to warning.

We've noticed this alert at busy sites with lots of clients, and found that having a mix of sessions that do/don't use client certificates makes it a lot worse.

Without a client certificate, SSL session data is about 150b. If there's a client certificate, the encoded size of the certificate is added to that, making if more like 1kb depending on the DN sizes, extensions, etc.

If the cache is full and there are no expired sessions, then when a new session using a client certificate is created, the code that tries to make space by getting rid of the oldest unexpired session won't free up anywhere near enough room if the oldest session didn't use a client certificate. Event if the oldest session did use a client certificate, its encoded size could be much smaller than the new session.

If ngx_ssl_expire_sessions was told how many bytes to free up rather than how many entries (with perhaps a cap on max # to purge), the alert messages would almost never appear.

[Maxim Dounin <mdounin@…>]

In 8075:38c71f9b2293/nginx:

SSL: reduced logging of session cache failures (ticket #621).

Session cache allocations might fail as long as the new session is different
in size from the one least recently used (and freed when the first allocation
fails). In particular, it might not be possible to allocate space for
sessions with client certificates, since they are noticeably bigger than
normal sessions.

To ensure such allocation failures won't clutter logs, logging level changed
to "warn", and logging is now limited to at most one warning per second.

[Maxim Dounin]

Fix committed, thanks to all involved for prodding this.