How to use multiple TLS client certificates

Knowledge Base Operating
1

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

How to use multiple TLS client certificates

The tls-authenticate-client directive is used to specify which client TLS certificates are valid when using mutual authentication TLS (mTLS).

Multiple TLS client certificates are used in scenarios for which mutual authentication TLS is required and one or more of the following are also true:

  1. The cluster will support multiple client applications and each client application requries a different TLS certificate.

  2. Cross Datacenter Replciation (XDR) is enabled. In this case the XDR source cluster is another TLS client to the XDR destination cluster.

  3. The certificate that an Aerospike server node presents to clients is required to be a different certificate than the one it presents to an XDR destination cluster.

The following example will illustrate all of the above mutual TLS scenarios.

  • There are two client applications called Client A and Client B
  • There are two Aerospike clusters called Cluster A and Cluster B
  • Both Client A and Client B can establish a secure connection with Cluster A
  • Cluster B can establish a secure connection with Cluster A to ship records via XDR
  • Cluster A can establish a secure connection with Cluster B to ship records via XDR

Certificate Installation

For this example there are 6 TLS certificates; one for each client to authenticate itself to the cluster, one for each cluster to authenticate itself to the client, and one for each cluster to authenticate itself to an XDR destination cluster.

All of the certificates in this example are signed using the same Certificate Authority (CA) and the files are all in pem format.

The Client A and Client B applications use certificates with a Common Name (CN) of aerospike.client.a and aerospike.client.b respectively:

$ openssl x509 -in aerospike.client.a.crt -text -noout | grep -E -- "Issuer:|Subject:"
    Issuer: CN = aerospike.ca, O = "Aerospike, Inc.", C = US
    Subject: CN = aerospike.client.a, O = "Aerospike, Inc.", C = US

$ openssl x509 -in aerospike.client.b.crt -text -noout | grep -E -- "Issuer:|Subject:"
    Issuer: CN = aerospike.ca, O = "Aerospike, Inc.", C = US
    Subject: CN = aerospike.client.b, O = "Aerospike, Inc.", C = US

Each client has the CA certificate, the client certificate, and its private key installed on each node.

Note: The client applications DO NOT have the private key from the cluster nodes nor from the other client applications.

  • Client A
    • CA Certificate: /etc/pki/tls/certs/aerospike.ca.crt
    • Client Certificate: /etc/pki/tls/certs/aerospike.client.a.crt
    • Private Key: /etc/pki/tls/private/aerospike.client.a.key
  • Client B
    • CA Certificate: /etc/pki/tls/certs/aerospike.ca.crt
    • Client Certificate: /etc/pki/tls/certs/aerospike.client.b.crt
    • Private Key: /etc/pki/tls/private/aerospike.client.b.key

To authenticate with clients, Cluster A and Cluster B use certificates with a Common Name (CN) of aerospike.cluster.a and aerospike.cluster.b respectively. To authenticate with the XDR destination cluster they use aerospike.xdr_client.a and aerospike.xdr_client.b respectively:

$ openssl x509 -in aerospike.cluster.a.crt -text -noout | grep -E -- "Issuer:|Subject:"
    Issuer: CN = aerospike.ca, O = "Aerospike, Inc.", C = US
    Subject: CN = aerospike.cluster.a, O = "Aerospike, Inc.", C = US

$ openssl x509 -in aerospike.cluster.b.crt -text -noout | grep -E -- "Issuer:|Subject:"
    Issuer: CN = aerospike.ca, O = "Aerospike, Inc.", C = US
    Subject: CN = aerospike.cluster.b, O = "Aerospike, Inc.", C = US

$ openssl x509 -in aerospike.xdr_client.a.crt -text -noout | grep -E -- "Issuer:|Subject:"
    Issuer: CN = aerospike.ca, O = "Aerospike, Inc.", C = US
    Subject: CN = aerospike.xdr_client.a, O = "Aerospike, Inc.", C = US

$ openssl x509 -in aerospike.xdr_client.b.crt -text -noout | grep -E -- "Issuer:|Subject:"
    Issuer: CN = aerospike.ca, O = "Aerospike, Inc.", C = US
    Subject: CN = aerospike.xdr_client.b, O = "Aerospike, Inc.", C = US

Each cluster has the CA certificate, the cluster certificates, and its private keys installed on each node.

Note: The cluster nodes DO NOT have the private key from the client applications nor from the other cluster.

  • Cluster A
    • CA Certificate: /etc/pki/tls/certs/aerospike.ca.crt
    • Server Certificate: /etc/pki/tls/certs/aerospike.cluster.a.crt
    • Server Private Key: /etc/pki/tls/private/aerospike.cluster.a.key
    • XDR Certificate: /etc/pki/tls/certs/aerospike.xdr_client.a.crt
    • XDR Private Key: /etc/pki/tls/private/aerospike.xdr_client.a.key
  • Cluster B
    • CA Certificate: /etc/pki/tls/certs/aerospike.ca.crt
    • Server Certificate: /etc/pki/tls/certs/aerospike.cluster.b.crt
    • Server Private Key: /etc/pki/tls/private/aerospike.cluster.b.key
    • XDR Certificate: /etc/pki/tls/certs/aerospike.xdr_client.b.crt
    • XDR Private Key: /etc/pki/tls/private/aerospike.xdr_client.b.key

Aerospike Configuration

For simplicity, the example aerospike.conf configuration below shows only the stanzas and directives that are relevant for this TLS configuration.

The configuration example also makes use of <cluster-name> which is a convenience variable substitution syntax. Every occurance of <cluster-name> will be replaced with the value of cluster-name defined in the top-level service stanza.

Cluster A

Remember, the configuration for Cluster A will allow secure connections to be established from Client A, Client B, and Cluster B.

service {
    # cluster name matches CN or SAN in the TLS certificate
    cluster-name aerospike.cluster.a
}

network {
    tls <cluster-name> {
        # cert for this server to authenticate with a client (including XDR client)
        ca-file /etc/pki/tls/certs/aerospike_ca.crt
        cert-file /etc/pki/tls/certs/aerospike.cluster.a.crt
        key-file /etc/pki/tls/private/aerospike.cluster.a.key
    }

    tls aerospike.xdr_client.a {
        # cert for this server, acting as an XDR client to authenticate with the other cluster
        ca-file /etc/pki/tls/certs/aerospike_ca.crt
        cert-file /etc/pki/tls/certs/aerospike.xdr_client.a.crt
        key-file /etc/pki/tls/private/aerospike.xdr_client.a.key
    }

    service {
        # tls-name references a 'tls' stanza above
        tls-name <cluster-name>

        tls-authenticate-client aerospike.client.a
        tls-authenticate-client aerospike.client.b
        tls-authenticate-client aerospike.xdr_client.b
    }

    fabric {
        tls-name <cluster-name>
    }
}

xdr {
    datacenter CLUSTER_B {
        # references CN or SAN in the TLS cert the XDR destination will use to
        # authenticate itself
        tls-node 10.0.0.10 aerospike.cluster.b 4000

        # tls-name references a 'tls' stanza above
        tls-name aerospike.xdr_client.a
    }
}

In the above example there are two network tls stanzas defined for two different certificates. The certificate with CN aerospike.cluster.a will be presented to application clients (including XDR clients) and the certificate with CN aerospike.xdr_client.a will be used when connecting as an XDR client to the other clusters.

In the network service stanza the tls-authenticate-client directive declares which certificates the server expects to be presented from clients. The default value of any would allow any certificate signed by the trusted CA. However, in this case the tls-authenticate-client is defined 3 times explicitly specifying three Common Name (CN) or Subject Alternative Name (SAN) values to expect in the certificates presented by the clients.

In other words, the server will only accept connections from clients presenting certificates with a CN or SAN of aerospike.client.a, aerospike.client.b, or aerospike.xdr_client.b.

The xdr datacenter stanza defines the connection string of the destination cluster with the tls-node directive. The second component of that string is the CN or SAN that is expected to be presented by the XDR destination node. In other words, when Cluster A is acting as an XDR client it expects Cluster B to present a certificate with CN or SAN of aerospike.cluster.b.

The xdr datacenter stanza also defines the certificate to present to Cluster B using the tls-name directive. In other words, Cluster A will present a certificate with CN or SAN aerospike.xdr_client.a to Cluster B.

Cluster B

Remember, the configuration for Cluster B will only allow secure connections to be established from Cluster A via XDR.

service {
    # cluster name matches CN or SAN in the TLS certificate
    cluster-name aerospike.cluster.b
}

network {
    tls <cluster-name> {
        # cert for this server to authenticate itself to a client (XDR client only for this example)
        ca-file /etc/pki/tls/certs/aerospike_ca.crt
        cert-file /etc/pki/tls/certs/aerospike.cluster.a.crt
        key-file /etc/pki/tls/private/aerospike.cluster.a.key
    }

    tls aerospike.xdr_client.b {
        # cert for this server, acting as an XDR client to authenticate with the other cluster
        ca-file /etc/pki/tls/certs/aerospike_ca.crt
        cert-file /etc/pki/tls/certs/aerospike.xdr_client.b.crt
        key-file /etc/pki/tls/private/aerospike.xdr_client.b.key
    }

    service {
        # tls-name references a 'tls' stanza above
        tls-name <cluster-name>

        tls-authenticate-client aerospike.xdr_client.a
    }

    fabric {
        tls-name <cluster-name>
    }
}

xdr {
    datacenter CLUSTER_A {
        # references CN or SAN in the TLS cert the XDR destination will use to
        # authenticate itself
        tls-node 10.0.0.20 aerospike.cluster.a 4000

        # tls-name references a 'tls' stanza above
        tls-name aerospike.xdr_client.b
    }
}

The above example is the inverse of the configuration for Cluster A with one key difference: In the network service stanza the tls-authenticate-client directive is only defined once and explicitly specifies aerospike.xdr_client.a. In other words, this server will only accept connections from clients which present a certificate with a CN or SAN of aerospike.xdr_client.a.