NGINX errors about SSL certificate PEM_read_bio_X509_AUX()

yuluma · July 16, 2024, 10:57am

Since shortly I see these errors in the NGINX error log and I can’t pinpoint for what site this is going wrong.

2024/07/16 12:53:01 [error] 3776693#3776693: *5249775 cannot load certificate “data:”: PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 49.12.36.197, server: 0.0.0.0:443
2024/07/16 12:54:01 [error] 3776694#3776694: *5250373 cannot load certificate “data:”: PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 49.12.36.197, server: 0.0.0.0:443
2024/07/16 12:55:01 [error] 3776693#3776693: *5250868 cannot load certificate “data:”: PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 49.12.36.197, server: 0.0.0.0:443

I tried to debug an solve it but have little clue where to start for the solution.

amiedema · July 16, 2024, 5:19pm

Hello @yuluma - try going to Server > Services and then select the ‘Heartbeat’ option in the overflow menu. The output may clarify which site is having the issue. If it’s not clear, another thing to try is SSH’ing into the server and then run nginx -t which may provide the needed details.

yuluma · September 4, 2024, 2:44pm

Hi Adam,

root@webshops1:~# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

while the log still states:

2024/09/04 16:42:01 [error] 72263#72263: *5676917 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.111.244, server: 0.0.0.0:443

I also checked the heartbeat for the nginx service but this responded with a green ACTIVE with no notice or warning/error.

PS: 188 lines of this for the complete NGINX log

amiedema · September 5, 2024, 4:25am

You might try cross referencing the client IP to the NGINX access log to see if that helps identify a site. It may also just be a client that had some issue loading SSL.

Do any of the sites use custom SSL certs?

yuluma · September 10, 2024, 7:27am

Hi Adam,

All the error log lines with PEM_read_bio_X509_AUX() in it are linked with ip adressess from CloudFlare which are not represented in the access log, maybe logical because when they try to access the request fails?

I tried to listnen a while with tcpdump on the ip but with no succes.

These are the latest loglines, just a few to be complete:

2024/09/10 08:26:02 [error] 723750#723750: *670953 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.95.178, server: 0.0.0.0:443
2024/09/10 08:27:02 [error] 723750#723750: *671037 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.164.205, server: 0.0.0.0:443
2024/09/10 08:28:02 [error] 723750#723750: *671130 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.247.41, server: 0.0.0.0:443
2024/09/10 08:29:02 [error] 723750#723750: *671192 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.172.39, server: 0.0.0.0:443
2024/09/10 08:30:02 [error] 723750#723750: *671292 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.172.32, server: 0.0.0.0:443
2024/09/10 08:31:02 [error] 723750#723750: *671369 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.246.234, server: 0.0.0.0:443
2024/09/10 08:32:01 [error] 723751#723751: *671428 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.68.192.177, server: 0.0.0.0:443
2024/09/10 08:33:01 [error] 723750#723750: *671512 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.69.150.175, server: 0.0.0.0:443
2024/09/10 08:34:01 [error] 723750#723750: *671609 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.247.12, server: 0.0.0.0:443
2024/09/10 08:35:01 [error] 723750#723750: *671661 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.246.160, server: 0.0.0.0:443
2024/09/10 08:36:01 [error] 723750#723750: *671707 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.246.223, server: 0.0.0.0:443
2024/09/10 08:37:01 [error] 723750#723750: *671770 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.68.110.101, server: 0.0.0.0:443
2024/09/10 08:38:01 [error] 723750#723750: *671869 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.250.161, server: 0.0.0.0:443
2024/09/10 08:39:01 [error] 723750#723750: *671955 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.69.151.218, server: 0.0.0.0:443
2024/09/10 08:40:01 [error] 723750#723750: *672036 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.251.138, server: 0.0.0.0:443
2024/09/10 08:41:01 [error] 723750#723750: *672119 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.69.150.249, server: 0.0.0.0:443
2024/09/10 08:42:01 [error] 723750#723750: *672214 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.94.154, server: 0.0.0.0:443
2024/09/10 08:43:01 [error] 723750#723750: *672335 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.172.27, server: 0.0.0.0:443
2024/09/10 08:44:01 [error] 723750#723750: *672431 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.246.33, server: 0.0.0.0:443
2024/09/10 08:45:01 [error] 723750#723750: *672544 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.69.150.208, server: 0.0.0.0:443
2024/09/10 08:46:01 [error] 723750#723750: *672616 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.68.192.158, server: 0.0.0.0:443
2024/09/10 08:47:01 [error] 723750#723750: *672696 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.172.33, server: 0.0.0.0:443
2024/09/10 08:48:01 [error] 723750#723750: *672767 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.164.4, server: 0.0.0.0:443
2024/09/10 08:49:01 [error] 723750#723750: *672871 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.94.135, server: 0.0.0.0:443
2024/09/10 08:50:01 [error] 723750#723750: *672983 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.242.11, server: 0.0.0.0:443
2024/09/10 08:51:01 [error] 723750#723750: *673059 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.68.192.227, server: 0.0.0.0:443
2024/09/10 08:52:01 [error] 723750#723750: *673156 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.246.235, server: 0.0.0.0:443
2024/09/10 08:53:01 [error] 723751#723751: *673245 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.242.83, server: 0.0.0.0:443
2024/09/10 08:54:01 [error] 723750#723750: *673365 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.111.71, server: 0.0.0.0:443
2024/09/10 08:55:01 [error] 723750#723750: *673454 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.110.47, server: 0.0.0.0:443
2024/09/10 08:56:01 [error] 723750#723750: *673581 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.86.110, server: 0.0.0.0:443
2024/09/10 08:57:01 [error] 723750#723750: *673690 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.172.44, server: 0.0.0.0:443
2024/09/10 08:58:01 [error] 723750#723750: *673859 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.68.193.185, server: 0.0.0.0:443
2024/09/10 08:59:01 [error] 723750#723750: *673983 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.251.111, server: 0.0.0.0:443
2024/09/10 09:00:01 [error] 723750#723750: *674095 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.246.214, server: 0.0.0.0:443
2024/09/10 09:01:01 [error] 723750#723750: *674247 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.68.110.122, server: 0.0.0.0:443
2024/09/10 09:02:01 [error] 723750#723750: *674451 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.251.146, server: 0.0.0.0:443
2024/09/10 09:03:01 [error] 723750#723750: *674582 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.86.14, server: 0.0.0.0:443
2024/09/10 09:04:01 [error] 723750#723750: *674753 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.69.150.175, server: 0.0.0.0:443
2024/09/10 09:05:01 [error] 723751#723751: *674897 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.69.150.131, server: 0.0.0.0:443
2024/09/10 09:06:01 [error] 723750#723750: *674988 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.246.84, server: 0.0.0.0:443
2024/09/10 09:07:01 [error] 723750#723750: *675107 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.251.93, server: 0.0.0.0:443
2024/09/10 09:08:01 [error] 723750#723750: *675253 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.172.33, server: 0.0.0.0:443
2024/09/10 09:09:01 [error] 723751#723751: *675361 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.172.27, server: 0.0.0.0:443
2024/09/10 09:10:01 [error] 723751#723751: *675466 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.69.150.209, server: 0.0.0.0:443
2024/09/10 09:11:01 [error] 723750#723750: *675546 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.160.121, server: 0.0.0.0:443
2024/09/10 09:12:01 [error] 723750#723750: *675592 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.86.110, server: 0.0.0.0:443
2024/09/10 09:13:01 [error] 723750#723750: *675685 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.68.192.158, server: 0.0.0.0:443
2024/09/10 09:14:01 [error] 723750#723750: *675738 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.111.200, server: 0.0.0.0:443
2024/09/10 09:15:01 [error] 723750#723750: *675846 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.247.41, server: 0.0.0.0:443
2024/09/10 09:16:01 [error] 723750#723750: *676017 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.68.192.159, server: 0.0.0.0:443
2024/09/10 09:17:01 [error] 723750#723750: *676175 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.250.155, server: 0.0.0.0:443
2024/09/10 09:18:01 [error] 723750#723750: *676349 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.69.151.217, server: 0.0.0.0:443
2024/09/10 09:19:01 [error] 723750#723750: *676491 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.94.155, server: 0.0.0.0:443
2024/09/10 09:20:01 [error] 723750#723750: *676636 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.70.251.173, server: 0.0.0.0:443
2024/09/10 09:21:01 [error] 723750#723750: *676768 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.95.175, server: 0.0.0.0:443
2024/09/10 09:22:01 [error] 723750#723750: *676919 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.68.194.191, server: 0.0.0.0:443
2024/09/10 09:23:01 [error] 723750#723750: *677054 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.111.123, server: 0.0.0.0:443
2024/09/10 09:24:01 [error] 723750#723750: *677157 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.172.38, server: 0.0.0.0:443
2024/09/10 09:25:01 [error] 723750#723750: *677301 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 162.158.111.91, server: 0.0.0.0:443
2024/09/10 09:26:01 [error] 723750#723750: *677492 cannot load certificate "data:": PEM_read_bio_X509_AUX() failed (SSL: error:0480006C:PEM routines::no start line:Expecting: TRUSTED CERTIFICATE) while SSL handshaking, client: 172.71.164.137, server: 0.0.0.0:443```