unable to get local issuer certificate: when nginx sends an incomplete certificate chain
Which error message does an incomplete chain show?
A server running nginx serves a certificate, and the client aborts with an error message. The message depends on the client.
What does curl show?
curl: (60) SSL certificate problem: unable to get local issuer certificate
What does openssl s_client show?
Verify return code: 21 (unable to verify the first certificate)
or, depending on the chain form, with the same cause:
Verify return code: 20 (unable to get local issuer certificate)
What do Firefox and Chrome show?
Firefox:
SEC_ERROR_UNKNOWN_ISSUER
Chrome shows, in essence, NET::ERR_CERT_AUTHORITY_INVALID.
What do Java and Android report?
Java:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Android:
java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
Is it always an incomplete chain?
The same curl message and the same openssl codes 20 and 21 also occur when the client does not trust the root that the chain which was sent leads back to (a private or in-house CA). If only that other case applies, the root CA is unknown to the client helps instead. Both cases can also occur at once.
Whether one of the two applies, both, or neither, the message alone does not tell you; the chain that was sent does, if the check under “How do I check the chain that was sent?” comes to an answer. This is measured for curl and openssl. Whether Firefox, Chrome, Java and Android show the same message in both cases has not been measured.
Why does it work in my own browser and not elsewhere?
This is the point where troubleshooting most often goes the wrong way. You open the site in your own desktop browser, the padlock is green, there is no warning - and you conclude that the certificate is fine.
The error must be somewhere else: in the curl call, in the Java version, in the Android app. With an incomplete chain, that conclusion is wrong, for a concrete technical reason - one that differs between browsers but has the same effect.
In the browser, a missing chain therefore often does not show up: Chrome and Safari fetch the certificate, and Firefox has the intermediate preloaded or has already seen it.
Measurement and edge cases: How browsers replace a missing intermediate certificate
- Chrome and Safari may fetch a missing intermediate certificate automatically: the leaf certificate often contains an AIA extension (Authority Information Access) with a URL from which the issuing intermediate certificate can be downloaded, and the browser fetches it on demand.
- Firefox does not do that - Mozilla never implemented AIA fetching. Instead, since version 68, Firefox ships a stock of known intermediate certificates via Remote Settings and adds intermediates from its own earlier connections to other servers of the same CA.
- If the relevant way does not apply - for example because the certificate carries no AIA URL, or because Firefox has neither preloaded the intermediate nor seen it before - the browser also shows a certificate error.
- If the intermediate certificate is missing from the chain that was sent, curl, openssl, the Java runtime and Android have neither a fetch mechanism nor a bundled stock, and the chain breaks.
Why do curl, Java and Android fail then?
curl, openssl, the Java runtime and Android do neither as a rule. They check the chain that was sent against their own trust store and fetch nothing.
If the intermediate certificate is missing, the server really is misconfigured; the browser usually closes the gap itself.
Why does nginx send the chain incomplete?
nginx sends in the TLS connection exactly the file named in ssl_certificate - no more and no less. There is no built-in logic that adds missing intermediate certificates or fetches them online.
Certificate authorities usually hand out two separate files after issuing: the leaf certificate (for your own domain) and one or more intermediate certificates (the sub-CA that signed the leaf). If you enter only the first file in ssl_certificate, you serve a technically valid but incomplete certificate - a client that does not already know the sub-CA cannot close the chain up to a trusted root.
How should fullchain.pem be built?
The combined file, usually called fullchain.pem, follows a fixed order: first the leaf certificate, then the intermediates in ascending order (the certificate that signed the leaf, then the next one up, and so on).
The root certificate does not belong in this file - the client must already have it as trusted, otherwise it would be worthless anyway, and a root that is sent along adds nothing but extra bytes on every connection.
How do I check the chain that was sent?
The most reliable way is to query the chain that was actually sent, directly, without the detour through a browser:
openssl s_client -connect example.com:443 -servername example.com -showcerts </dev/null
The output contains the block Certificate chain. Each entry has two lines: s: says whose certificate it is, i: says who issued it. A measurement with a throwaway CA (not a real server; “Wegwerf” is German for throwaway) shows, for a leaf and one intermediate certificate:
Certificate chain
0 s:CN=localhost
i:C=DE, O=Wegwerf GmbH, CN=Wegwerf Intermediate
1 s:C=DE, O=Wegwerf GmbH, CN=Wegwerf Intermediate
i:C=DE, O=Wegwerf GmbH, CN=Wegwerf Root A
Measurement and edge cases: How the names are written
- How the names are written depends on the openssl program.
- Measured are OpenSSL 3.6.4 (
C=DE, O=Wegwerf GmbH, CN=...) and LibreSSL 3.3.6 (/C=DE/O=Wegwerf GmbH/CN=...). - How other versions write them has not been measured.
How do I read the chain?
You read the chain in three steps, in this order.
Step 1: Which sent entry comes next?
Follow the chain from the leaf. Start at entry 0. Find exactly one entry you have not visited yet whose s: equals the i: line of the current entry. The entries visited this way are the path.
- You repeat the search until no such entry matches any more or you reach an entry whose
s:andi:are identical (identical names do not yet prove a root, see step 2). - If several entries match, the check gives no answer (then only
Verify return codeis left). - Move on to a matching entry only once it has demonstrably signed the current entry (the proof “First” under “Did the issuer really sign the predecessor?”). The proof runs with the PEM block of the current entry as
vorgaenger.pemand the PEM block of the matching entry asaussteller.pem, both from the-showcertsoutput. - If the proof fails with error 20 or 7, the entry is not on the path even if its name matches (for example a foreign self-signed certificate with the issuer’s name). The path then ends at the current entry, as at an issuer that the sent entries do not contain, and you go on with step 3.
Measurement and edge cases: Step 1 in detail
- If several entries match,
Verify return codeshows whether the client closes the chain, but not which case applies. - Any other message from the proof does not answer the question, and the check gives no answer (see “First”).
- Entries that are not on the path contribute nothing to the chain.
- If an entry that was already visited turns up a second time (the sequence runs in a circle), the check gives no answer either. Such chains have not been measured.
- A broken sequence alone does not tell you whether a level is missing: what stands after the break can be a certificate of another chain, and an issuer that appears further down as an
s:line can itself point to a missing level. - Only steps 2 and 3 decide that, applied to the last member of the path.
Step 2: Does the path end at an entry whose s: and i: are identical?
Then this line checks, for the PEM block of the entry saved as entry.pem, whether the self-signature holds. It runs with the same openssl program as s_client; the block comes from the -showcerts output (from -----BEGIN CERTIFICATE----- to -----END CERTIFICATE-----):
openssl verify -check_ss_sig -CAfile entry.pem entry.pem
- If it reports
entry.pem: OK(exit code 0), the entry is a root or a self-signed leaf. - If the client’s trust store contains a certificate with this name (block further down), a certificate of that name is trusted. Whether the client closes the chain with it is shown by
Verify return code(as long as you have obtained no certificate in step 3; afterwards see the end of step 3). - At
0 (ok)the chain is complete. With any other number it is not the certificate that closes the chain (for example because only a certificate with a different key stands under that name), or the cause lies outside both cases, and the check gives no answer. - If no certificate with this name is in the client’s trust store, the client does not trust it and the server is not missing any level: this is the case of the foreign or own root (a private or in-house CA, a self-signed certificate); see when the root CA is unknown to the client.
- If it reports, at depth 0, an error 7 (
certificate signature failure) or 20 (unable to get local issuer certificate), the self-signature is not confirmed; the message says no more than that. Do not treat the entry as a root. Go on with step 3, taking the name in thei:line of this entry as the issuer.
Measurement and edge cases: Step 2 in detail
- Identical names only mean that the entry was issued under its own name (self-issued, RFC 5280, section 3.2), not that it signed itself: when a CA changes its key, it can issue a new certificate under the same name, signed with its old key.
- Whether, after an error 7 or 20, a certificate with the same name that signed the entry is missing is shown only by the check against the issuer you obtain; if you cannot obtain one, the check gives no answer.
- Any other message, such as an expired certificate (error 10), does not answer the question; the check then gives no answer.
Measurement and edge cases: self-signature check
Measured with a throwaway PKI (root, old CA certificate, new CA certificate under the same name signed with the old key, leaf below it), with OpenSSL 3.6.4 and LibreSSL 3.3.6 as the client:
- If the server sends the leaf and the new CA certificate without the old one,
s_clientreports code 20, the line above reports error 20 for the new CA certificate, andOKfor a root that was sent and for a self-signed leaf. - If the new CA certificate carries no Authority Key Identifier extension,
s_clientreports code 19 (self-signed certificate in certificate chain) although nothing signed itself, andopenssl verifywithout-check_ss_sigreportsOKfor that entry; hence the switch. - For that entry the line above reports error 7, with both programs; the complete message also contains error lines from
rsa routines(with OpenSSL 3.6.4 as with LibreSSL 3.3.6). - An error 7 does not prove a missing issuer: for a self-signature in which one byte is changed, the line above also reports error 7,
s_clientreports code 18 (self-signed certificate), andopenssl verifywithout-check_ss_sigreportsOK. - If the server sends both CA certificates,
s_clientreports code 0; two entries then match thei:line of the leaf, and step 1 gives no answer. - Other versions and clients are not measured.
Step 3: What if the issuer was not sent?
Otherwise the path ends at an issuer that the sent entries do not contain. An entry that carries the issuer’s name but is not confirmed as its signer by the proof (error 20 or 7) counts as not contained.
- If the client’s trust store contains a certificate with the name from the last
i:line of the path, a certificate of that name is trusted. - Whether the client closes the chain with it is shown by
Verify return code(as long as you have obtained nothing yet; afterwards see the end of this step): at0 (ok)the chain is complete, with any other number the check gives no answer (for example because only a certificate with a different key stands under that name). - If no certificate with that name is in the client’s trust store, what matters is what that issuer is. Only its certificate shows that, and you have to obtain it for this (how is explained under “How do I get the issuer’s certificate?”).
- A root whose self-signature is confirmed and which the client does not trust: that is the case of the foreign or own root, see when the root CA is unknown to the client.
- An intermediate certificate: it is missing on the server, which is the case of the missing intermediate certificate. Whether the client trusts the chain after that is not yet settled.
Step 3 (continued): How do I go on once I have the certificate?
- The certificate you obtain is, after the proof (see “Did the issuer really sign the predecessor?”), the next member of the chain. Look for its issuer first in the list of sent entries, among those you have not visited yet, and go on there as in step 1, including the proof: the certificate you obtained is
vorgaenger.pemthere. - An issuer is missing on the server if no entry in the list carries its name or the entry that does is not its signer (error 20 or 7 in the proof); you obtain it the same way, and so on.
- In the end the trust store decides as in steps 2 and 3: if it contains a certificate with the name of the last issuer or the last root, the chain is complete once the added intermediate certificates are in place, provided it is the same certificate. Whether the client then closes it is shown by
Verify return codeafter adding them (“How do I fix it?”, fifth). - Until then the current
Verify return codedecides nothing: once you have obtained a certificate it is not 0 (every “missing intermediate” row of the table shows another number), and what counts is the code after adding. - If it is proven that an intermediate certificate was missing and the chain, once it is added, ends at a root the client does not trust, both cases are present at once: the intermediate certificates have to be added and the trust settled.
Measurement and edge cases: Step 3 in detail
- An intermediate certificate with identical subject and issuer counts as missing only once a further obtained certificate has demonstrably signed it (see “Second” further down).
- The path end also includes an entry that carries its name but is not its signer (error 20 or 7 in the proof of step 1): it counts as not contained. The same holds for an entry whose self-signature step 2 did not confirm; its issuer then carries the same name as the entry itself but is a different certificate.
- That does not yet settle whether the server merely did not send that certificate: an error 7 from step 2 can also point to a damaged certificate for which no issuer exists at all (then the check gives no answer).
- If a certificate turns up a second time on the way, or several unvisited entries match, the check gives no answer here either (see step 1).
- If the certificate of an issuer cannot be obtained, the check gives no answer either; then only
Verify return codeis left, with the same limit.
What if the file is sorted wrongly?
If the path entries do not stand as entries 0, 1, 2 … directly one after the other at the start of the list, the file is sorted wrongly; the fix is “How do I fix it?”, second (assemble the chain in the right order). That holds next to the other cases as well: if an intermediate certificate is also missing or the client does not trust the root, the file has to be sorted in addition.
Is the issuer in the client’s trust store?
Whether a name is in the trust store is shown by this block. The name stands in a here-document rather than in the command, because an apostrophe in the name (such as O=Let's Encrypt) would otherwise end the string in the shell:
openssl crl2pkcs7 -nocrl -certfile ca-bundle.pem | openssl pkcs7 -print_certs -noout | grep -c '^subject='
read -r issuer <<'EOF'
C=DE, O=Wegwerf GmbH, CN=Wegwerf Intermediate
EOF
openssl crl2pkcs7 -nocrl -certfile ca-bundle.pem | openssl pkcs7 -print_certs -noout | sed 's/[[:space:]]*$//' | grep -Fx -- "subject=$issuer"
ca-bundle.pemis the CA bundle file the client checks against. The client decides where it is located; with curl and openssl you can name it with--cacertand-CAfile.- The first line is the control and must print a number greater than 0. If it prints 0, the command read nothing (measured with an empty file and with a wrong file path), and an empty result from the last line then tells you nothing.
- Replace the example line between
<<'EOF'andEOFwith the text afteri:from the lasti:line of the path, character for character; the block can then be pasted in one piece. - A line in the output means the name is in the bundle, except for whitespace at the end. No output means it is not.
- Run both
openssllines with the same openssl program ass_client.
Measurement and edge cases: input, whitespace and special characters in the name comparison
- The name stands in the command itself, so
readwaits for no input and cannot swallow a line of the sequence. - A line of its own,
read -r issuer, that waits for input would, pasted in one piece, read the next line as the name; the check line would not run, and only the control number would be printed (measured). That would be a silent result that looks like “not found”. read -rtakes a backslash literally and removes leading and trailing whitespace from the text, such as a space that came along when you copied; whitespace inside the name stays.- The
sedremoves whitespace at the end of the lines of the bundle, so that a name which itself ends in a space is found as well (measured withCN= Wegwerf Rand CA). - Because of the here-document with
'EOF'the shell does not evaluate the text, so a$or a backslash in the name stays literal.-Fxcompares the whole line as fixed text, not as a pattern. - The price: two names that differ only in whitespace at the end are not told apart (measured: the input with a trailing space finds a bundle that contains only the name without one, and the other way round).
- With
IFS= read -rand nosedit would be the other way round: the name with a trailing space would be found, but a margin space that came along when copying would make the match fail, and the command would print nothing although the name is in the bundle;read -ralone, without thesed, on the other hand, would miss a name that itself ends in whitespace.
Measurement and edge cases: Measured name forms in the name comparison
Measured, that is, in bash 3.2 and zsh 5.9, with OpenSSL 3.6.4 and LibreSSL 3.3.6, against test bundles with throwaway certificates, with the whole block sent in one piece through a pipe to the shell:
- A name with an apostrophe (
Let's Encrypt, and in a second case two apostrophes) is found, and if it is not in the bundle there is no output. - A name that is only the beginning of a longer one (
Wegwerf IntermediateagainstWegwerf Intermediate Backup), and a name with special characters such asWegwerf CA (Test) 1.0next to a look-alike (1x0), produce no false matches. - A name containing
$WEGWERF_Xor a backslash is compared literally and not evaluated. - A name with two spaces inside (
Wegwerf CA) is found, and a space typed before or after the name does not change the result. - A name that itself ends in a space is found, and if it is not in the bundle there is no output.
- Names with whitespace directly after
subject=are not measured.
Measurement and edge cases: openssl programs and name matches
- Measured, the
i:line froms_clientand thesubject=line match character for character when both come from the same program (OpenSSL 3.6.4 or LibreSSL 3.3.6); if you mix the two programs they differ, because they write the names differently. - The lines are measured against test bundles, with both programs; the block is not measured against system bundles, nor is pasting into a terminal window (the block came to the shell through a pipe). They are not measured for the trust stores of browsers, Java and Android.
- The name comparison does not replace the client’s check: whether the client closes the chain is not settled until the
Verify return codeline (see below); which case applies, it does not show. - A name match does not prove the key either: measured, the block (with OpenSSL 3.6.4 and LibreSSL 3.3.6) finds the name of a root even when the trust store holds, under that name, a root with a different key, and
openssl verify(OpenSSL 3.6.4) against that trust store reports error 20 (unable to get local issuer certificate) for the leaf and the intermediate certificate; with the right root it reportsOK.
How do I get the issuer’s certificate?
Whether the issuer is a root or an intermediate certificate s_client does not show, because the server does not send that certificate. It is shown by the issuer’s own certificate, which you have to obtain: for a public CA via the AIA entry CA Issuers in the certificate of the last path entry (command under “How do I fix it?”, first, there for the leaf), or from the CA’s website; for your own CA you have it anyway.
For the proof, save the certificate that is meant to be the issuer as a PEM file aussteller.pem (German for issuer), and the certificate whose issuer it is meant to be as vorgaenger.pem (German for predecessor). Both run with the same openssl program as s_client, except the proof, which differs because LibreSSL 3.3.6 does not know -partial_chain (details under “Other messages of the proof”). The same holds for every further member that step 3 asks for.
Measurement and edge cases: What vorgaenger.pem and aussteller.pem are
- The proof is used in two places: in step 1 for a sent entry that matches by name, and in step 3 for a certificate you obtain. In both,
aussteller.pemis the matching sent entry or the obtained certificate. vorgaenger.pemis always the last member of the path so far, the certificate whose issuer is sought, no matter whether it was sent or obtained: the PEM block of a sent entry from the-showcertsoutput, or the certificate you obtained last.- A certificate counts as the next member of the path only once it has demonstrably signed
vorgaenger.pem; an obtained one is classified after that.
Did the issuer really sign the predecessor?
First, the proof that aussteller.pem signed the predecessor:
openssl verify -partial_chain -CAfile aussteller.pem vorgaenger.pem
- If it reports
vorgaenger.pem: OK(exit code 0),aussteller.pemis a member of the chain. - If it reports, at depth 0, an error 20 (
unable to get local issuer certificate) or 7 (certificate signature failure),aussteller.pemis not the issuer of the predecessor, even if its name matches thei:line of the predecessor. For a sent entry (step 1) that means it is not on the path, and you go on with step 3; for a certificate you obtain (step 3) the check gives no answer. - Any other message does not answer the question, and the check gives no answer.
Measurement and edge cases: Other messages of the proof
- A certificate with the same name need not have signed the predecessor (key change, a certificate of another chain, a foreign self-signed certificate with the same name).
- With any other message the check gives no answer, for a sent entry as for a certificate you obtain.
- Error 79 (
invalid CA certificate, reported at depth 1) says thataussteller.pemdid sign the predecessor but is not a CA certificate. - Error 18 (
self-signed certificate) says that the predecessor is itself self-issued (for example a damaged self-signed certificate). - An expired certificate (error 10) says nothing about who signed.
- Without
-partial_chain,openssl verifyaccepts only a root as trust anchor; with an intermediate certificate asaussteller.pemthe line then fails with error 2 (measured). - LibreSSL 3.3.6 does not know
-partial_chain: the line reads only certificate files and prints no names, so an OpenSSL 3.x can run it even whens_clientcame from LibreSSL.
Is it a root or an intermediate certificate?
Second, the classification, only after the proof. If subject and issuer of aussteller.pem have different names, it is an intermediate certificate and is missing on the server. If they have the same name, that is not enough to call it a root (self-issued, see step 2); then this line checks the self-signature:
openssl verify -check_ss_sig -CAfile aussteller.pem aussteller.pem
- If it reports
aussteller.pem: OK, it is a root (or a self-signed leaf). - If it reports an error 7 or 20 at depth 0, the self-signature is not confirmed, and the classification stays open: the certificate can be an intermediate certificate that another certificate with the same name has signed. Obtain that certificate, save it as
aussteller.pem(the previous one becomesvorgaenger.pem) and begin again at “First”.
Measurement and edge cases: If the self-signature is not confirmed
- If the proof succeeds there, the previous certificate is an intermediate certificate that is missing on the server, and the new one is the next member (go on with step 3).
- If it fails, or no such certificate can be obtained, the check gives no answer.
- Any other message from the self-signature line, such as an expired certificate (error 10), does not answer the question either; the check then gives no answer.
Measurement and edge cases: proof and classification of the issuer, end to end
Measured end to end with the throwaway PKI from step 2 (root, old and new CA certificate under the same name, intermediate certificate, leaves below), with OpenSSL 3.6.4 and LibreSSL 3.3.6 as the program for the classification and OpenSSL 3.6.4 for the proof; both runs give the same result.
The self-signature line reports OK for a root and for a self-signed leaf, error 20 for the new CA certificate under the same name, and error 7 for a self-signed certificate with one byte changed.
- If the server sends a certificate with a damaged self-signature (one byte changed) and its undamaged original is obtained (same name, same key), the proof reports error 18 and exit code 2: no answer.
- If the server sends the new CA certificate and the old one is obtained, the proof reports
OK; the old one is an intermediate certificate and is missing. If its root is obtained next, the proof reportsOKagain, and it is a root. - If a leaf is missing its intermediate certificate and that is obtained, the proof reports
OK; it is missing. - If the issuer you obtain is a root that the client does not trust, the proof reports
OK, and it is a root: the case of the foreign or own root.
Measurement and edge cases: proof and classification of the issuer, cases without an answer
The proof also reports no answer, with exit code 2, in these cases:
- The obtained certificate carries a different name and did not sign the predecessor (error 20).
- It carries the name of the right one but a different key (error 20).
- It did sign the predecessor but is not a CA certificate (error 79,
invalid CA certificate). - On further obtaining, a root with the name but not the key of the right one turns up (error 20).
Measurement and edge cases: proof and classification of the issuer, case 5
Three more forms, built in one throwaway PKI (EC keys; one root with separate branches, plus the foreign self-signed certificate of case 5; OpenSSL 3.6.4 and LibreSSL 3.3.6 as the client, the proof with OpenSSL 3.6.4, same verdicts with both):
Case 5: The server sends the leaf and a foreign self-signed certificate that carries the issuer’s name of the leaf (other key), the real intermediate certificate is missing, and the client trusts the root:
s_clientreports code 21 (with a leaf without an Authority Key Identifier extension: code 7 with OpenSSL 3.6.4, code 19 with LibreSSL 3.3.6).- The proof from step 1 (foreign entry as
aussteller.pem, leaf asvorgaenger.pem) reports error 20 and exit code 2 (without the extension, error 7), so the foreign entry is not on the path. - Step 3 obtains the real intermediate certificate, the proof reports
OK, and this is the case of the missing intermediate certificate;openssl verifyagainst the root reportsOKwith the real intermediate certificate and error 20 with the foreign one. - Without the proof the reading ends at the foreign certificate in the case of the foreign or own root (
-check_ss_sigreportsOK, the name is not in the trust store). - If the real intermediate certificate cannot be obtained, or the only certificate under that name that can be obtained is the foreign one (its proof fails), the check gives no answer.
- If the real intermediate certificate is sent instead,
s_clientreports code 0 and the proofOK.
Measurement and edge cases: proof and classification of the issuer, case 6
Case 6: The chain is root, I3, I2, I1, leaf, the server sends the leaf and I2, and the client trusts the root:
s_clientreports code 21, the proof reportsOKfor I1 against the leaf, for I2 against I1 and for I3 against I2, and I1 and I3 are missing: the case of the missing intermediate certificate.- With the previously obtained I1 as
vorgaenger.pemfor I3 the proof reports error 20 and exit code 2. - With I1, I2 and I3 sent
s_clientreports code 0.
Measurement and edge cases: proof and classification of the issuer, case 7
Case 7: The server sends the leaf and a certificate with the issuer’s name that did sign the leaf but is not a CA certificate (a CA certificate with the same key and name can be obtained), and the client trusts the root:
s_clientreports code 26 with OpenSSL 3.6.4 and code 24 with LibreSSL 3.3.6, and the proof from step 1 reports error 79 at depth 1 and exit code 2, so the check gives no answer.- A reading that took every failed proof as “not on the path” would obtain the CA certificate in step 3 and read “missing intermediate certificate” here.
Measurement and edge cases: proof and classification of the issuer, limits
- The chain from step 2 with both CA certificates sent still gives no answer, because two entries match by name before the proof is reached (
s_clientcode 0). - Not measured are an obtained certificate without an Authority Key Identifier extension, a public CA and, in step 1, a matching entry whose proof reports another message than error 20, 7 or 79 (for example an expired certificate, error 10); by the rule the check gives no answer there.
What does Verify return code say?
At the end of the output is the line Verify return code. 0 (ok) means the client has closed the chain. Any other number means it could not, but the number does not tell you whether one of the two cases applies, both at once, or the cause lies outside both cases:
| Case | Chain form (what the server sends; whom the client trusts) | openssl code | curl (exit code) |
|---|---|---|---|
| complete | leaf + intermediate; root trusted | 0 (ok) | 0 |
| complete | leaf + two intermediates; root trusted | 0 (ok) | 0 |
| complete | leaf + intermediate + root sent along; root trusted | 0 (ok) | 0 |
| complete | leaf only, issued directly by a root; root trusted | 0 (ok) | 0 |
| complete | leaf only, self-signed; stored as trusted | 0 (ok) | 0 |
| missing intermediate | leaf only, intermediate missing; root trusted | 21 (unable to verify the first certificate) | 60 |
| missing intermediate | leaf + lower of two intermediates, upper missing; root trusted | 20 (unable to get local issuer certificate) | 60 |
| missing intermediate | leaf + upper of two intermediates, lower missing; root trusted | 21 (unable to verify the first certificate) | 60 |
| missing intermediate | leaf + root, intermediate in between missing; root trusted | 21 (unable to verify the first certificate) | 60 |
| missing intermediate | leaf + top and bottom of three intermediates, middle one missing; root trusted | 20 (unable to get local issuer certificate) | 60 |
| missing intermediate and foreign/own root | leaf + lower of two intermediates, upper missing; root not trusted | 20 (unable to get local issuer certificate) | 60 |
| wrong order | leaf + both intermediates, in swapped order; root trusted | 0 (ok) | 0 |
| foreign/own root | leaf + intermediate; root not trusted | 20 (unable to get local issuer certificate) | 60 |
| foreign/own root | leaf only, issued directly by a root; root not trusted | 21 (unable to verify the first certificate) | 60 |
| foreign/own root | leaf + intermediate + root sent along; root not trusted | 19 (self-signed certificate in certificate chain) | 60 |
| foreign/own root | leaf only, self-signed; not trusted | 18 (self-signed certificate) | 60 |
| foreign/own root | leaf + intermediate of a different chain; the leaf’s root not trusted | 21 (unable to verify the first certificate) | 60 |
| foreign/own root | leaf + a foreign root sent along that the client trusts; the leaf’s root not trusted | 21 (unable to verify the first certificate) | 60 |
| foreign/own root | leaf + both intermediates, in swapped order; root not trusted | 20 (unable to get local issuer certificate) | 60 |
In the table a row reads “wrong order” only if the chain would otherwise be complete.
Measurement and edge cases: setup and rows of the table
About the setup and the individual rows:
- Measured with OpenSSL 3.6.4 (
s_client) and curl 8.7.1 (SecureTransport, LibreSSL 3.3.6) on macOS (Darwin arm64). - The client did not check against the system trust store but only against the throwaway root (
s_clientwith-CAfile <throwaway root> -no-CApath -no-CAstore, curl with--cacert <throwaway root>); the server wasopenssl s_server, not nginx. - The “Case” column is the reading above, applied to the chain that was sent.
- The proof in step 1 changes no row: all 19 chain forms were run again through the whole reading, with OpenSSL 3.6.4 and with LibreSSL 3.3.6 as the client (the proof with OpenSSL 3.6.4), and each keeps its case and its code; in 12 of the 19 the path enters a sent entry by name in step 1 and so runs the proof there, and in none of the 19 does that proof fail.
- Chains for which the check gives no answer (several matching entries, a circle, an issuer certificate that cannot be obtained or whose proof fails) are not in the table; there only
Verify return codeis left, with the same limit. - The code alone does not separate the cases: 21 appears for “missing intermediate” and for “foreign/own root”, and so does 20, also in the row where both apply at once, and for 20 and 21 the curl message is
unable to get local issuer certificatein every measured row. - As for “wrong order”: the
s:/i:sequence is broken in list order, but the path from the leaf finds all certificates, and OpenSSL 3.6.4 and curl 8.7.1 both accepted the chain. Stricter clients are not measured. In the right order (second row of the table) the code is 0 as well. - The row with three intermediates is the counter-case: the sequence is broken and the issuer of the leaf appears further down as an
s:line (entry 2), yet a level is missing, because after entry 2 comes an issuer that nobody sends. - What is missing is the middle one of the three certificates; its issuer stands as entry 1 in the list. The path there runs over entries 0 and 2 and, after the missing certificate, over entry 1, so the file is also sorted wrongly.
- In the row “missing intermediate and foreign/own root” both apply at once: the sequence breaks at the missing intermediate certificate, and the client does not trust the root above it that the chain leads back to. Measured,
openssl verifyagainst the same trust store also fails there with all intermediate certificates of the throwaway CA; adding the intermediate certificate alone therefore does not remove the error. - In the last row the file is also sorted wrongly (the path runs over entries 0, 2, 1), and the client does not trust the root: that is the case of the foreign or own root.
- In the two rows with a foreign certificate the second entry is not on the path: it does not belong to the leaf’s chain.
- Other versions and other clients are not measured.
Measurement and edge cases: Why curl -vI does not show the chain
- The command
curl -vI https://example.com 2>&1 | grep -i "certificate\|issuer"shows no chain (measured with curl 8.7.1, SecureTransport: with a missing intermediate the error message, with a complete chain oneissuer:line with the issuer of the leaf). - For the reading above you need
openssl s_client.
How do I fix it?
First: How do I get the missing intermediate certificate?
The certificate authority usually provides it for download together with the leaf certificate, often as a separate file or directly as a ready-made fullchain.pem. If that file is missing, the issuing CA can also be read out of the leaf certificate itself, via the AIA extension:
openssl x509 -in leaf.pem -noout -text | grep -A1 "CA Issuers"
The URL given there delivers the missing intermediate certificate for direct download, in DER or PEM format. A DER download has to be converted before assembling - a PEM download can be used directly. How to tell them apart: a PEM file is text and begins with -----BEGIN CERTIFICATE-----, a DER file is binary.
openssl x509 -inform DER -in intermediate.der -out intermediate.pem
Second: How do I assemble the chain?
Leaf first, then the intermediates in ascending order, no root:
cat leaf.pem intermediate.pem > fullchain.pem
If several intermediates are needed, they are appended in the same file in the right order.
Third: How do I point nginx at the complete chain?
In the server configuration, ssl_certificate must refer to fullchain.pem, not to the leaf certificate alone:
ssl_certificate /etc/nginx/ssl/fullchain.pem;
ssl_certificate_key /etc/nginx/ssl/privkey.pem;
Fourth: How do I reload the configuration?
Test the configuration and reload, so that the change takes effect without dropping connections:
nginx -t && nginx -s reload
Fifth: How do I measure again?
Run the same openssl s_client command as above once more and check that the path from the leaf now runs through entries 0, 1, 2 … in that order and ends at an issuer that is in the client’s trust store, and that the last line reads Verify return code: 0 (ok).
Does a tool help with the sorting?
The steps above work completely with the built-in tools of any system on which OpenSSL is installed - an additional tool is not a prerequisite. If you have the certificate files in arbitrary order and do not want to work out the sorting and the detection of missing intermediates by hand, you will find the corresponding function in our PKI Toolkit. It runs entirely in the browser and uploads no certificate to a server.