Skip to main content

Certificates and Secure Connections

Overview

Using Transport Layer Security (TLS/SSL) based connection is an important step in securing data moving through IQ Server. The configuration process is not automated but can be understood and implemented using best practices. This guide shows you how to set up secure connections to and from the IQ Server by helping you:

  • Understand what TLS/SSL is and how certificates work.

  • Describe how TLS works in Java applications in general and the common tools used during configuration.

  • Understand best practice approaches for configuring keypairs, certificates, and trust stores for inbound and outbound connections to the IQ Server.

  • Define common error messages seen with secure communication and provide steps towards resolution.

Prerequisites

A special tool is required that can manipulate Java Keystore files. The two most common options are:

  • Keytool (standard)- The keytool command-line program comes with the Java Development Kit (JDK). The examples in this guide will use keytool. For more information, see the Oracle documentation for Linux or Microsoft Windows.

  • KeyStore Explorer (3rd party)- If you prefer a graphical user interface (GUI), Keystore Explorer is an option that performs all the operations of keytool and more. Despite using a GUI, you should still understand the basics of Java Keystore files, keys, and certificates before using this tool. If you have questions on how to use KeyStore Explorer, contact the maintainers of that software at https://keystore-explorer.org/index.html.

Depending on the complexity of TLS provisioning, a non-java specific key and certificate processing toolkit may be useful in some circumstances:

TLS/SSL Protocol and the IQ Server

Transport Layer Security (TLS) is a standard security protocol for creating an encrypted communication channel between a client and a server. An example of use is when your web browser connects to a website using an https:// address.

Encrypted channels ensure that all data passed remains private. TLS is a newer iteration of the SSL (Secure Sockets Layer) protocol, with TLS supporting newer and more secure algorithms.

How does TLS work?

TLS protocol relies on certificates and keys. During an interaction, a client and server start with an encryption “handshake” that relies on your certificate (Figure 1). This handshake starts with the “Client Hello” message along with the list of supported cipher suites and also a guess of which key agreement protocol the server is likely to select.

Next, the server replies to the “Client Hello” message with the chosen key agreement protocol. The “Server Hello” message also contains the server’s key share, its certificate, and the “Server Finished” message. Finally, the client checks the server certificate, generates keys, and sends the “Client Finished” message.

137204305.png

Figure 1: Graphic of a TLS “Handshake”. Icons representing a Client and a Server with arrows between them, pointing first to the Server, then to the Client, then to the Server. Each arrow is paired with text that describes the information being transmitted.

Java Applications and TLS

IQ Server is a Java application. The TLS certificates and keypairs used by it are loaded from files known as keystores. Before setting up secure connections with the IQ Server, it’s a good idea to become familiar with how TLS works in Java applications in general.

The following lists some great resources for examples of Java applications using TLS. We suggest you give these a quick read before continuing with this guide:

How does IQ Server use TLS Connections?

TLS connections are established with IQ Server in two directions: inbound and outbound.

137204308.png

Figure 2: Graphic showing IQ server inbound and outbound connections. Inbound connections are categorized as Inbound TLS (HTTPS) Connections and are labeled IQ Server UI, REST API, and Sonatype Plugins. Outbound connections are categorized as Outbound TLS (HTTPS/LDAP/SMTP/SSL Connections, and are labeled Application Scans, Email, Webhooks, and User Authentication.

Inbound IQ Server TLS Connections

Inbound connections to IQ Server use HTTPS, which is built on TLS technology.

Some example clients that can establish secure inbound connections to IQ Server are:

  • A web browser viewing the IQ Server user interface at an https:// address.

  • A custom Python program accessing the REST API.

  • An official Sonatype integration such as an IDE or CI server plugin.

There are two options for setting-up secure inbound connectors: reverse proxy HTTPS and direct HTTPS connections.

Reverse Proxy HTTPS Inbound Connections

If a reverse proxy is used in between the HTTP client and IQ Server, HTTPS connections to IQ Server can be set up without a special IQ Server configuration.

137204309.png

Figure 3: Graphic showing an inbound https connection through a reverse proxy server

The reverse proxy will handle the HTTPS connection and retrieve resources over plain HTTP connections behind the scenes on behalf of the client. Traffic will appear to originate from the reverse proxy https:// address instead of IQ server directly.

Advantages of using reverse proxy terminated TLS include:

  1. Configuration is external to IQ Server and you may be able to leverage already established central management of TLS traffic.

  2. Direct inbound HTTPS connector configuration in IQ server config.yml is not needed.

Your reverse proxy administrator can help you set up a reverse proxy in front of IQ Server.

Direct HTTPS Inbound Connectors

Another option is to accept HTTPS connections directly without a reverse proxy.

137204310.png

Figure 4: Graphic showing a direct inbound HTTPS connection

Embedded with the IQ Server is the web container Eclipse Jetty. Jetty handles inbound HTTPS connections on behalf of IQ Server.

Jetty HTTPS connector configuration is via the IQ Server config.yml file. No adjustment of Java system properties such as javax.net.ssl.keyStore and javax.net.ssl.trustStore are required to make this option work.

The basic ingredients to setup direct HTTPS connectors are:

  1. A private key and a server certificate wrapping the public key.

  2. A keystore file to contain the private key and certificate.

  3. IQ Server config.yml changes.

In this guide, we provide a best-practice approach for managing the keys and certificates for direct HTTPS connectors.

Provisioning the Server Key and Certificates

The three most common paths to obtain a private key and public key/server certificate are:

  1. Self-provision a private key and self-signed certificate.

  2. Self-provision a private key and request a certificate authority (CA) signed certificate.

  3. Externally provision a private key and certificates.

How to Self Provision a Key Pair and Self-signed Certificate

A self-signed server certificate allows a server to accept HTTPS connections but is not suitable for production environments. A self-signed server certificate will need to be explicitly trusted by every connecting client, making wide-scale client integration difficult. Self-signed server certificates are only suitable for basic testing.

The following command generates a key pair (a public key and associated private key) and then wraps the public key into an X.509 v3 self-signed certificate, which is stored as a single-element certificate chain. The certificate chain and the private key are stored in a keystore entry identified by an alias inside the specified keystore file:

 keytool -genkeypair -keystore nexus-iq-server-inbound.p12 -storetype PKCS12 -storepass password -keypass password -alias nexus-iq-server -keyalg RSA -validity 356 -dname 'CN=iq.example.com, OU=Example, O=Example, L=Fulton, ST=MD, C=US' -ext 'SAN=DNS:iq.example.com,DNS:lifecycle.example.com' -ext 'EKU=serverAuth'
About this command:
  • If nexus-iq-server-inbound.p12 file does not exist, it will be created by this command.

  • Override the default JKS format using -storetype PKCS12.

  • Jetty requires the -keypass and -storepass values to have identical values.

  • Use a -alias name that is identifiable and meaningful to the purpose of the keystore entry.

  • Key pair algorithm of RSA overrides the less secure default of DSA and sets the keysize to 2048 bits.

  • Validity of the certificate is specified in days. When a certificate expires, HTTPS connections will fail, so overriding the default of 90 days is common.

  • The CN=iq.example.com part is an example hostname. The CN value at a minimum must exactly match your IQ server host name.

  • More than one hostname or IP address may be used by clients to reach the server. In that case, specify subject alternative names (SAN) values by adjusting the common delimited list of values for -ext 'SAN=<values>'.

The resulting keystore file structure will be:

137204311.png

Figure 5: Example keystore structure for self-provisioned key pair and self-signed certificate.

How to Self Provision a Private Key and Request a Signed Certificate

First, generate a private-public key pair and keystore using the command in How to Self Provision a Key Pair and Self-signed Certificate.

Next, create a Certificate Signing Request (CSR) to send to a certificate authority for signing:

keytool -certreq -alias nexus-iq-server -keystore nexus-iq-server-inbound.p12 -storepass password -ext 'SAN=DNS:iq.example.com,DNS:lifecycle.example.com' -ext 'EKU=serverAuth' -file nexus-iq-server.csr.pem 
About this command:
  • Specify the alias name which matches your PrivateKeyEntry.

  • If you want the CSR to include extensions like Subject Alternative Name (SAN), you must specify these when generating the CSR. It is not enough to only specify these when creating the original self-signed certificate.

  • The output file extensions csr and pem help identify the file as a PEM encoded CSR.

Send the resulting $nexus-iq-server.csr.pem file to your certificate authority.

The certificate authority will send back a signed certificate reply, where the Issuer field references a CA certificate.

It is common that a chain of CA certificates may be used to sign your server certificate. The CA will provide the complete PEM-encoded certificate chain or may expect you to combine the signed server cert with the CA chain certificates.

For example, a CA may provide you three files:

  • Signed server certificate

  • Intermediate CA certificate

  • ROOT CA certificate

You should ensure these certificates are combined into one file before importing them into your keystore. A simple way to combine PEM encoded files is to cat them together in order:

cat server.cert.pem intermediate.cert.pem root.cert.pem > nexus-iq-server-csr-reply.cert.pem

The resulting signed server certificate and complete chain of CA certificates must be imported into the same keystore which was used to generate the CSR:

keytool -importcert -keystore nexus-iq-server-inbound.p12 -storepass password -alias nexus-iq-server -file nexus-iq-server-csr-reply.cert.pem -trustcacerts
About this command:
  • Specify the same keystore file and alias name that the original CSR was generated from.

  • nexus-iq-server-csr-reply.cert.pem is the complete PEM encoded chain of certs.

The resulting keystore entries should match the structure in Figure 6.

137204312.png

Figure 6: Example keystore structure with private key and signed server certificate chain.

How to Install Externally Provisioned Private Key and Certificates

Your organization may have a central unit responsible for issuing cryptographic keys and certificates. You will not be responsible for generating a private key or server certificate, but you still need to install them into IQ Server to configure HTTPS connectors.

As we have seen, Java can use PKCS#12 keystores. However, keytool cannot import a private key-only file into an existing keystore.

The solution is to either:

  1. Request a PKCS#12 keystore from your organizational unit containing both the key and server certificate chain, or

  2. Build a PKCS#12 format keystore yourself using a tool other than keytool such as the openssl pkcs12 command.

For the second option, here is an example command creating a PKCS12 format keystore:

openssl pkcs12 -export -passin pass:changeit -passout pass:password -name nexus-iq-server -inkey iq.example.com.key.pem -in iq.example.com-chain.cert.pem -out nexus-iq-server-inbound.p12
About this command:
  • -passin pass:changeit specifies that changeit is the private key password. The actual key password must be provided to you if you were given the key.

  • -passout pass:password specifies that password will be the new password for both the new keystore and the private key entry.

  • iq.example.com.key.pem is the PEM encoded private key input file.

  • iq.example.com-chain.cert.pem is the PEM encoded certificates file containing the complete certificate chain in order from server cert to root CA cert.

How to Configure Direct HTTPS Connectors in config.yml

To start, check out our help docs where we summarize the basic configuration needed in our help site. Advanced configuration options for HTTPS connectors are in the dropwizard documentation.

The following code block is an example direct HTTPS connector using a config.yml configuration that loads the certificate entry alias named nexus-iq-server from the keystore nexus-iq-server-inbound.p12 :

server
  applicationConnectors:
    - type: https
      port: 8443
      keyStorePath: nexus-iq-server-inbound.p12
      keyStorePassword: password
      certAlias: nexus-iq-server

Outbound IQ Server TLS Connections

Outbound TLS connections from IQ Server can use a variety of protocols like HTTPS, LDAPS, and SMTPS. By default, the IQ Server will use the Java truststore to establish the trust of remote certificates. If all remote TLS certificates are signed by public CAs, then no additional configuration is needed.

In the cases where your organization uses an HTTP proxy server that rewrites TLS certificates, or IQ Server is connecting to internal TLS endpoints such as mail or LDAP servers with custom certificates, you may have to customize what certificates IQ Server trusts.

Some example outbound secure connections IQ Server may make are:

  • Performing scans and receiving report data from Sonatype data services.

  • Emailing policy violation notifications via SMTPS by way of your configured email server.

  • Sending payloads to your configured https:// webhook URL.

  • Authenticating users in your LDAP server using an LDAPS connection.

  • Creating policy violation issues in your configured https:// Jira server address.

  • Using HTTPS CONNECT tunneling and trusting certificates signed by your configured HTTP Proxy Server.

The basic ingredients needed to trust outbound TLS certificates are:

  1. Copies of the remote untrusted certificates.

  2. A single truststore file to contain all trusted certificates.

  3. Edited startup command used to launch the IQ Server.

How to Obtain the Remote Server Untrusted Certificates

Assuming you’ve identified you have a certificate trust issue, you need to gather the untrusted certificates for importing into a truststore file.

Ideally, if your organization is using an internal CA, the team that maintains that CA can provide you with the root CA certificates to trust.

Otherwise, you may obtain these yourself by using a client tool such as Openssl s_client or Java keytool to connect to the remote and capture the certificate.

Capture Remote TLS Certificates by Direct Connection

Using Openssl s_client version 1.0.2 or newer, use the following command when the remote connection is HTTPS, SMTPS, or LDAPS:

echo -n | openssl s_client -showcerts -connect hostname:636 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > untrusted-hostname.cert.pem

About this command:

  • echo -n prevents output of a new line for proper formatting.

  • Replace hostname and 636 with the remote server hostname and port that is untrusted.

  • sed is used to properly format the resulting file into a chain of PEM-encoded certs that other tools will recognize.

Alternatively, keytool can be used if the remote is using HTTPS connection only:

keytool -printcert -rfc -sslserver hostname:443 > untrusted-hostname.cert.pem

About this command:

  • -rfc specifies a PEM encoded version of the remote certificates.

  • Replace hostname and 443 with the remote server hostname and port that is untrusted.

Capture Remote HTTPS Certificates Through a Proxy Server

It is fairly common that organizations use HTTP proxy servers to intercept secure communication to external servers for auditing and inspection. This alters the original TLS certificate, making the connection untrusted. If IQ Server is configured to use HTTP proxy server, and the remote that is not trusted is using an HTTPS connection, then this requires special handling.

Using Openssl s_client version 1.1.0 or newer, use the following command when the remote connection is HTTPS, SMTPS, or LDAPS:

echo -n | openssl s_client -showcerts -proxy proxyhost:8888 -connect hostname:636 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > untrusted-hostname.cert.pem 

About this command:

  • -proxy tells s_client to use HTTP CONNECT tunneling to a proxy server.

  • Replace proxyhost and 8888 with your HTTP proxy server hostname and port.

Alternatively, keytool can be used if the remote is using HTTPS connection only:

keytool -J-Dhttps.proxyHost=proxyhost -J-Dhttps.proxyPort=8888 -printcert -rfc -sslserver hostname:443 > untrusted-hostname.cert.pem

About this command:

  • Replace proxyhost and 8888 with your HTTP proxy server hostname and port.

How to Add a Trusted Remote Certificate to a Custom Truststore

Assuming IQ Server is not yet using a custom truststore, determine the Java install that IQ Server is using and locate the default truststore file included with Java.

Make a copy of the default java truststore file to the same directory where nexus-iq-server-<version>.jar is located. Name the copy nexus-iq-server-outbound.jks to identify the purpose of this file is clearly an outbound TLS connections truststore in the JKS format.

Open the untrusted-hostname.cert.pem file captured earlier. You should see at least two entries demarcated by —–BEGIN CERTIFICATE—– . Select the last certificate in the file , starting with the line —–BEGIN CERTIFICATE—– and ending with —–END CERTIFICATE—– and copy it entirely to your clipboard. This should be the root certificate which is not trusted. Put that certificate into its own file called untrusted-root.cert.pem .

Import the PEM encoded root certificate into the truststore:

keytool -importcert -keystore nexus-iq-server-outbound.jks -storepass changeit -file untrusted-root.cert.pem -alias custom-root -trustcacerts

About this command:

  • Optionally replace custom-root alias with a meaningful name.

  • changeit is the default password for the original cacerts truststore and does not need to be changed.

How to Configure IQ Server to Use a Custom Truststore for Outbound Connections

Editing the IQ Server config.yml file is never needed to trust a remote outbound TLS certificate. You can tell IQ Server to use a custom truststore just like any other Java application by specifying Java system properties.

Change the command used to launch IQ Server so that a custom truststore is used by adding these java arguments:

-Djavax.net.ssl.trustStore=/<replace_with_absolute_full_path>/nexus-iq-server-outbound.jks -Djavax.net.ssl.trustStorePassword=changeit 

An example complete command may look like this:

java -Djavax.net.ssl.trustStore=/nexus-iq-server/nexus-iq-server-outbound.jks -Djavax.net.ssl.trustStorePassword=changeit -jar nexus-iq-server-1.65.0-01.jar server config.yml
 

We have an article describing how to determine which outbound TLS connection truststore is being used by IQ Server.

If you are running IQ Server on Windows as a service, then follow our special instructions to specify java program arguments overriding the truststore.

Best Practices

Now that you know what TLS protocol is, and how to set up inbound and outbound connections to the IQ Server, we’ll give you a few best practice examples to better aid in establishing secure connections.

Use PKCS#12 Keystore Format

The default keystore file format in Java 8 is proprietary and called “JKS.” JKS keystores can only store private keys and trusted public key certificates. The truststore file java-home/lib/security/cacerts included with Java runtime distributions (JRE) is in JKS format.

Java also supports a keystore format called “PKCS12” which is a widely supported extensible standard. As of Java 8, PKCS12 keystores can store private keys, trusted public key certificates, and secret keys.

Using PKCS12 improves keystore integrity and confidentiality. It also opens opportunities for interoperability with other systems that also support PKCS12. As of Java 9 and newer, PKCS12 is the default keystore format.

Some keytool commands take an explicit -storetype option to specify that PKCS12 format be used instead of the default JKS. The Openssl pkcs12 command can create and parse pkcs12 format keystores.

If you create a keystore with Java 8 keytool in the JKS format, you may see a message such as:

Warning:The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format.

When creating a keystore to use with an HTTPS connector, you should specify -storetype PKCS12 to avoid this message. A PKCS12 format keystore will:

  • Provide a more secure keystore when keys are stored.

  • Increase interoperability with other systems.

  • Prepare a keystore for use in future Java versions.

Use Meaningful Keystore Entry Alias Names

Entries in a keystore are assigned a name or alias. Assigning meaningful names provides an easy way to identify the purpose of the entry.

If the inbound HTTPS connector keystore contains multiple private key entries, then the entry used for the HTTPS connector is indeterminant unless a certAlias is explicitly configured inside config.yml .

Commands such as keytool -importcert behave differently depending if an existing alias is found in the keystore. If the alias points to a key entry, then the keytool -importcert command assumes you are importing a certificate reply. Otherwise, the command assumes you are only importing a trusted certificate. Incorrectly importing a certificate reply as a trusted cert can prevent your HTTPS connector from working.

Well defined keystore entry names help:

  • Increase comprehension about the purpose of the keystore entries.

  • Avoid the wrong server cert from being chosen on server startup and breaking the HTTPS connector.

  • Influence tool behavior when modifying and reading keystores.

Use Certificates Signed by Public Certificate Authorities

HTTP clients accessing HTTPS servers use an architecture that has a strategy of trusting TLS certificates signed by well known public certificate authorities. It is a deployment advantage to also have a server HTTPS certificate to be signed publicly because the clients will not have to make special adjustments to trust the TLS certificate being sent by the server.

A Java runtime includes a default truststore at java-home/lib/security/cacerts containing public certificate authorities. IQ Server will automatically trust remote certificates when those remotes have a certificate signed by a public authority.

Advantages of using server certificates that are signed by public authorities include:

  • Avoiding maintenance of custom truststores.

  • Less complicated server and client configuration, setup, and upgrades.

Common Error Messages

If the IQ Server appears to start on the secure port without error, but the Java keytool client commands fail with “No certificate from the SSL server,” Web browsers cannot load the application user interface on the secure port, and you receive one of the following error messages:

Google Chrome:

  • This site can’t be reached. Localhost unexpectedly closed the connection. ERR_CONNECTION_CLOSED

  • This site can’t provide a secure connection. Localhost uses an unsupported protocol. ERR_SSL_VERSION_OR_CIPHER_MISMATCH

Firefox:

  • Secure connection failed. The connection to localhost was interrupted while the page was loading.

  • Secure connection failed. An error occurred during connection to localhost. Cannot communicate securely with peer: no common encryption algorithm(s). Error code: SSL_ERROR_NO_CYPHER_OVERLAP

Safari:

  • Safari can’t open the page because Safari can’t establish a secure connection to the server.

Curl commands error:

  • Server aborted the SSL handshake

Wget commands error:

  • Unable to establish SSL connection

Cause

Contrary to some of the web browser error messages, this most likely has nothing to do with cipher problems.

The likely cause of these errors is that you’ve only imported a signed server cert into the HTTPS connection keystore. The keystore either does not contain the corresponding required private key or the server certificate imported did not include the complete certificate chain of all trusted certificate issuers.

Solution

First, check the aliased entry in the keystore file that you imported the certificate. If it shows Entry type: trustedCertEntry instead of Entry type: PrivateKeyEntry, then you are missing the private key.

If you do have Entry type: PrivateKeyEntry, then the associated certificate for that entry was replaced with a server certificate lacking a complete certificate chain. The solution is to create a PEM encoded file containing the complete certificate chain and re-import that certificate into the private key entry.

A simple way to combine PEM encoded files is to cat them together in order:

cat server.cert.pem intermediate.cert.pem root.cert.pem > server-chain.cert.pem

Then import the server-chain.cert.pem into the keystore, instead of the server.cert.pem.

You may notice generating a self-signed certificate using keytool does not have this problem. This is because the keytool -gencert command creates a private key by default and the certificate is self-signed.

The best solution is to contact the team who gave you the certificate file and ask them to review Jetty documentation on loading keys and certificates. Stipulate that in order for Sonatype products to terminate SSL connections, Java keystore files require the private key used to generate the Certificate Signing Request and a complete certificate chain including all the intermediate and root CA public certificates.

FAQs

The following frequently asked questions provide more troubleshooting information on IQ Server and secure connections.

How can I determine if IQ Server does not trust an outbound TLS certificate?

Examine the IQ Server clm-server.log file looking for any mention of the word “PKIX.” Usually the related log message will include the host and port information for the TLS connection that was not trusted.

Example log messages that indicate a remote certificate that is not trusted:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
 

To resolve PKIX issues, IQ server needs to be configured to trust the remote certificate being returned.

How can I identify a self-signed certificate?

A self-signed certificate is one for which the Issuer (signer) is the same as the Owner (subject).

Use the keytool -printcert command to display the certificate information Owner and Issuer fields and compare the values. If they are the same, you have a self-signed certificate.

How can I identify if I have a PKCS#12 or JKS format keystore file?

Try to list the contents of the file using the keytool -list command which will work on both of these formats:

keytool -list -keystore <filename>

If the file is not one of these two formats or another format supported by your version of Java, the command will error.

If the file is in either JKS or PKCS12 format, then you will be prompted for a keystore password. Pressing ENTER/return at the prompt without typing a password will display the type of keystore format the file is and a summary of what entries it contains.

How can I check if I have a certificate-only file?

Use the keytool -printcert command:

keytool -printcert -file <filename>

Each certificate will be listed in the output, demarcated with a line containing “Certificate[n],” where n is a number starting at 1 representing the order of the certificate found in the file.

If the command generates an error, then the file does not contain X.509 data standard certificates, encoded as DER or PEM.

How to check if I have been given a private key file?

Try to process the file using the OpenSSL pkey command:

openssl pkey -in <filename>

If the file is not a private key-only file, the command will generate an error. Otherwise, you will be prompted for the private key password.

How can I check if I have a PEM encoded file?

Privacy Enhanced Mail (PEM) encoding, defined by RFC 1421 Certificate Encoding Standard is a printable text base64 encoding of ASN.1/DER binary content. View the text of your file. If it contains lines with dashes and the word BEGIN or END, plus some data type, the file is PEM encoded.

For example, a PEM encoded certificate file will contain lines such as:

----- BEGIN CERTIFICATE
----- END CERTIFICATE

If the file does not contain these demarcations, it is either DER encoded or a binary keystore format.