Certificates and Secure Connections

A special tool is required that can manipulate Java Keystore files. The two most common options are:

Depending on the complexity of TLS provisioning, a non-java-specific key and certificate processing toolkit may be useful in some circumstances:

TLS protocol relies on certificates and keys. During an interaction, a client and server start with an encryption “handshake” that relies on your certificate (Figure 1). This handshake starts with the “Client Hello” message along with the list of supported cipher suites and also a guess of which key agreement protocol the server is likely to select.

Next, the server replies to the “Client Hello” message with the chosen key agreement protocol. The “Server Hello” message also contains the server’s key share, its certificate, and the “Server Finished” message. Finally, the client checks the server certificate, generates keys, and sends the “Client Finished” message.

IQ Server is a Java application. The TLS certificates and keypairs used by it are loaded from files known as keystores. Before setting up secure connections with the IQ Server, it’s a good idea to become familiar with how TLS works in Java applications in general.

The following lists some great resources for examples of Java applications using TLS. We suggest you give these a quick read before continuing with this guide:

TLS connections are established with IQ Server in two directions: inbound and outbound.

If a reverse proxy is used between the HTTP client and IQ Server, HTTPS connections to IQ Server can be set up without a special IQ Server configuration.

The reverse proxy will handle the HTTPS connection and retrieve resources over plain HTTP connections behind the scenes on behalf of the client. Traffic will appear to originate from the reverse proxy https:// address instead of IQ server directly.

Advantages of using reverse proxy terminated TLS include:

Your reverse proxy administrator can help you set up a reverse proxy in front of IQ Server.

Another option is to accept HTTPS connections directly without a reverse proxy.

Embedded with the IQ Server is the web container Eclipse Jetty. Jetty handles inbound HTTPS connections on behalf of IQ Server.

Jetty HTTPS connector configuration is via the IQ Server config.yml file. No adjustment of Java system properties such as javax.net.ssl.keyStore and javax.net.ssl.trustStore are required to make this option work.

The basic ingredients to set direct HTTPS connectors are:

In this guide, we provide a best-practice approach for managing the keys and certificates for direct HTTPS connectors.

The three most common paths to obtain a private key and public key/server certificate are:

How to Self Provision a Key Pair and Self-signed Certificate

A self-signed server certificate allows a server to accept HTTPS connections but is not suitable for production environments. A self-signed server certificate will need to be explicitly trusted by every connecting client, making wide-scale client integration difficult. Self-signed server certificates are only suitable for basic testing.

The following command generates a key pair (a public key and associated private key) and then wraps the public key into an X.509 v3 self-signed certificate, which is stored as a single-element certificate chain. The certificate chain and the private key are stored in a keystore entry identified by an alias inside the specified keystore file:

 keytool -genkeypair -keystore nexus-iq-server-inbound.p12 -storetype PKCS12 -storepass password -keypass password -alias nexus-iq-server -keyalg RSA -validity 356 -dname 'CN=iq.example.com, OU=Example, O=Example, L=Fulton, ST=MD, C=US' -ext 'SAN=DNS:iq.example.com,DNS:lifecycle.example.com' -ext 'EKU=serverAuth'

About this command:

• If nexus-iq-server-inbound.p12 file does not exist, it will be created by this command.

• Override the default JKS format using -storetype PKCS12.

• Jetty requires the -keypass and -storepass values to have identical values.

• Use a -alias name that is identifiable and meaningful to the purpose of the keystore entry.

• The key pair algorithm of RSA overrides the less secure default of DSA and sets the keysize to 2048 bits.

• The validity of the certificate is specified in days. When a certificate expires, HTTPS connections will fail, so overriding the default of 90 days is common.

• The CN=iq.example.com part is an example hostname. The CN value at a minimum must exactly match your IQ server host name.

• More than one hostname or IP address may be used by clients to reach the server. In that case, specify subject alternative names (SAN) values by adjusting the common delimited list of values for -ext 'SAN=<values>'.

The resulting keystore file structure will be:

Figure 5: Example keystore structure for self-provisioned key pair and self-signed certificate.

How to Self Provision a Private Key and Request a Signed Certificate

First, generate a private-public key pair and keystore.

Next, create a Certificate Signing Request (CSR) to send to a certificate authority for signing:

keytool -certreq -alias nexus-iq-server -keystore nexus-iq-server-inbound.p12 -storepass password -ext 'SAN=DNS:iq.example.com,DNS:lifecycle.example.com' -ext 'EKU=serverAuth' -file nexus-iq-server.csr.pem 

About this command:

• Specify the alias name which matches your PrivateKeyEntry.

• If you want the CSR to include extensions like Subject Alternative Name (SAN), you must specify these when generating the CSR. It is not enough to only specify these when creating the original self-signed certificate.

• The output file extensions csr and pem help identify the file as a PEM encoded CSR.

Send the resulting $nexus-iq-server.csr.pem file to your certificate authority.

The certificate authority will send back a signed certificate reply, where the Issuer field references a CA certificate.

Commonly, a chain of CA certificates may be used to sign your server certificate. The CA will provide the complete PEM-encoded certificate chain or may expect you to combine the signed server cert with the CA chain certificates.

For example, a CA may provide you with three files:

• Signed server certificate

• Intermediate CA certificate

• ROOT CA certificate

You should ensure these certificates are combined into one file before importing them into your keystore. A simple way to combine PEM encoded files is to cat them together in order:

cat server.cert.pem intermediate.cert.pem root.cert.pem > nexus-iq-server-csr-reply.cert.pem

The resulting signed server certificate and complete chain of CA certificates must be imported into the same keystore that was used to generate the CSR:

keytool -importcert -keystore nexus-iq-server-inbound.p12 -storepass password -alias nexus-iq-server -file nexus-iq-server-csr-reply.cert.pem -trustcacerts

About this command:

• Specify the same keystore file and alias name that the original CSR was generated from.

• nexus-iq-server-csr-reply.cert.pem is the complete PEM encoded chain of certs.

The resulting keystore entries should match the structure in Figure 6.

Figure 6: Example keystore structure with private key and signed server certificate chain.

openssl pkcs12 -export -passin pass:changeit -passout pass:password -name nexus-iq-server -inkey iq.example.com.key.pem -in iq.example.com-chain.cert.pem -out nexus-iq-server-inbound.p12

About this command:

server
  applicationConnectors:
    - type: https
      port: 8443
      keyStorePath: nexus-iq-server-inbound.p12
      keyStorePassword: password
      certAlias: nexus-iq-server

Assuming you’ve identified a certificate trust issue, you need to gather the untrusted certificates for importing into a truststore file.

Ideally, if your organization is using an internal CA, the team that maintains that CA can provide you with the root CA certificates to trust.

Otherwise, you may obtain these yourself by using a client tool such as OpenSSL client or Java keytool to connect to the remote and capture the certificate.

echo -n | openssl s_client -showcerts -connect hostname:636 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > untrusted-hostname.cert.pem

keytool -printcert -rfc -sslserver hostname:443 > untrusted-hostname.cert.pem

echo -n | openssl s_client -showcerts -proxy proxyhost:8888 -connect hostname:636 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > untrusted-hostname.cert.pem 

keytool -J-Dhttps.proxyHost=proxyhost -J-Dhttps.proxyPort=8888 -printcert -rfc -sslserver hostname:443 > untrusted-hostname.cert.pem

Assuming IQ Server is not yet using a custom truststore, determine the Java install that IQ Server is using and locate the default truststore file included with Java.

Make a copy of the default java truststore file to the same directory where nexus-iq-server-<version>.jar is located. Name the copy nexus-iq-server-outbound.jks to identify the purpose of this file is an outbound TLS connection truststore in the JKS format.

Open the untrusted-hostname.cert.pem file captured earlier. You should see at least two entries demarcated by —–BEGIN CERTIFICATE—– . Select the last certificate in the file , starting with the line —–BEGIN CERTIFICATE—– and ending with —–END CERTIFICATE—– and copy it entirely to your clipboard. This should be the root certificate which is not trusted. Put that certificate into its own file called untrusted-root.cert.pem .

Import the PEM-encoded root certificate into the truststore:

keytool -importcert -keystore nexus-iq-server-outbound.jks -storepass changeit -file untrusted-root.cert.pem -alias custom-root -trustcacerts

About this command:

Editing the IQ Server config.yml file is never needed to trust a remote outbound TLS certificate. You can tell IQ Server to use a custom truststore just like any other Java application by specifying Java system properties.

Change the command used to launch IQ Server so that a custom truststore is used by adding these java arguments:

-Djavax.net.ssl.trustStore=/<replace_with_absolute_full_path>/nexus-iq-server-outbound.jks -Djavax.net.ssl.trustStorePassword=changeit 

An example complete command may look like this:

java -Djavax.net.ssl.trustStore=/nexus-iq-server/nexus-iq-server-outbound.jks -Djavax.net.ssl.trustStorePassword=changeit -jar nexus-iq-server-1.65.0-01.jar server config.yml
 

We have an article describing how to determine which outbound TLS connection truststore is being used by IQ Server.

If you are running IQ Server on Windows as a service, then follow the instructions to specify java program arguments overriding the truststore.

The default keystore file format in Java 8 is proprietary and called “JKS.” JKS keystores can only store private keys and trusted public key certificates. The truststore file java-home/lib/security/cacerts included with Java runtime distributions (JRE) is in JKS format.

Java also supports a keystore format called “PKCS12” which is a widely supported extensible standard. As of Java 8, PKCS12 keystores can store private keys, trusted public key certificates, and secret keys.

Using PKCS12 improves keystore integrity and confidentiality. It also opens opportunities for interoperability with other systems that also support PKCS12. As of Java 9 and newer, PKCS12 is the default keystore format.

Some keytool commands take an explicit -storetype option to specify that PKCS12 format be used instead of the default JKS. The Openssl pkcs12 command can create and parse pkcs12 format keystores.

If you create a keystore with Java 8 keytool in the JKS format, you may see a message such as:

Warning: The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format.

When creating a keystore to use with an HTTPS connector, you should specify -storetype PKCS12 to avoid this message. A PKCS12 format keystore will:

Entries in a keystore are assigned a name or alias. Assigning meaningful names provides an easy way to identify the purpose of the entry.

If the inbound HTTPS connector keystore contains multiple private key entries, then the entry used for the HTTPS connector is indeterminant unless a certAlias is explicitly configured inside config.yml

Commands such as keytool -importcert behave differently depending if an existing alias is found in the keystore. If the alias points to a key entry, then the keytool -importcert command assumes you are importing a certificate reply. Otherwise, the command assumes you are only importing a trusted certificate. Incorrectly importing a certificate reply as a trusted cert can prevent your HTTPS connector from working.

Well-defined keystore entry names help:

HTTP clients accessing HTTPS servers use an architecture that has a strategy of trusting TLS certificates signed by well-known public certificate authorities. It is a deployment advantage to also have a server HTTPS certificate to be signed publicly because the clients will not have to make special adjustments to trust the TLS certificate being sent by the server.

A Java runtime includes a default truststore at java-home/lib/security/cacerts containing public certificate authorities. IQ Server will automatically trust remote certificates when those remotes have a certificate signed by a public authority.

Advantages of using server certificates that are signed by public authorities include:

Inbound and outbound TLS connections require IQ Server to read from file-based keystores.

A keystore file can store keys and trusted certificates. When a keystore file only stores trusted certificates, or the purpose of the file is to hold only trusted certificates, it is referred to as a truststore.

To help keep your configuration organized and avoid confusion about the intended file purpose, we recommend creating two store files kept in the same directory as the nexus-iq-server-<version>.jar file.

Inbound and outbound TLS certificates are managed differently and warrant separate files. The inbound keystore holds a private key and a server certificate chain for the inbound HTTPS connections. The outbound keystore is known as a “truststore” because its only purpose is to store trusted certificates used when IQ Server is acting as the TLS client.

When you organize TLS files using a simple convention:

Contrary to some of the web browser error messages, this most likely has nothing to do with cipher problems.

The likely cause of these errors is that you’ve only imported a signed server cert into the HTTPS connection keystore. The keystore either does not contain the corresponding required private key or the server certificate imported did not include the complete certificate chain of all trusted certificate issuers.

First, check the aliased entry in the keystore file where you imported the certificate. If it shows Entry type: trustedCertEntry instead of Entry type: PrivateKeyEntry, then you are missing the private key.

If you do have Entry type: PrivateKeyEntry, then the associated certificate for that entry was replaced with a server certificate lacking a complete certificate chain. The solution is to create a PEM-encoded file containing the complete certificate chain and re-import that certificate into the private key entry.

A simple way to combine PEM encoded files is to cat them together in order:

cat server.cert.pem intermediate.cert.pem root.cert.pem > server-chain.cert.pem

Then import the server-chain.cert.pem into the keystore, instead of the server.cert.pem.

You may notice generating a self-signed certificate using keytool does not have this problem. This is because the keytool -gencert command creates a private key by default and the certificate is self-signed.

The best solution is to contact the team who gave you the certificate file and ask them to review Jetty documentation on loading keys and certificates. Stipulate that for Sonatype products to terminate SSL connections, Java keystore files require the private key used to generate the Certificate Signing Request and a complete certificate chain including all the intermediate and root CA public certificates.

Examine the IQ Server clm-server.log file looking for any mention of the word “PKIX.” Usually, the related log message will include the host and port information for the TLS connection that was not trusted.

Example log messages that indicate a remote certificate that is not trusted:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
 

To resolve PKIX issues, the IQ server needs to be configured to trust the remote certificate being returned.

A self-signed certificate is one for which the Issuer (signer) is the same as the Owner (subject).

Use the keytool -printcert command to display the certificate information Owner and Issuer fields and compare the values. If they are the same, you have a self-signed certificate.

Try to list the contents of the file using the keytool -list command which will work on both of these formats:

keytool -list -keystore <filename>

If the file is not one of these two formats or another format supported by your version of Java, the command will error.

If the file is in either JKS or PKCS12 format, then you will be prompted for a keystore password. Pressing return at the prompt without typing a password will display the type of keystore format the file is and a summary of what entries it contains.

Use the keytool -printcert command:

keytool -printcert -file <filename>

Each certificate will be listed in the output, demarcated with a line containing “Certificate[n],” where n is a number starting at 1 representing the order of the certificate found in the file.

If the command generates an error, then the file does not contain X.509 data standard certificates, encoded as DER or PEM.

Try to process the file using the OpenSSL pkey command:

openssl pkey -in <filename>

If the file is not a private key-only file, the command will generate an error. Otherwise, you will be prompted for the private key password.

Privacy Enhanced Mail (PEM) encoding, defined by RFC 1421 Certificate Encoding Standard is a printable text base64 encoding of ASN.1/DER binary content. View the text of your file. If it contains lines with dashes and the word BEGIN or END, plus some data type, the file is PEM encoded.

For example, a PEM-encoded certificate file will contain lines such as:

----- BEGIN CERTIFICATE
----- END CERTIFICATE

If the file does not contain these demarcations, it is either DER encoded or a binary keystore format.