FAQ Aerospike TLS

Knowledge Base General
1

The Aerospike Knowledge Base has moved to https://support.aerospike.com. Content on https://discuss.aerospike.com is being migrated to either https://support.aerospike.com or https://docs.aerospike.com. Maintenance on articles stored in this repository ceased on December 31st 2022 and this article may be stale. If you have any questions, please do not hesitate to raise a case via https://support.aerospike.com.

FAQ Aerospike TLS

What is the default protocol of TLS?

Aerospike, by default, uses TLS 1.2. Not all operating system versions have the correct openSSL version to allow TLS1.2 as standard. Aerospike recommends installing an openSSL version that supports TLS 1.2 as earlier versions are considered less safe.

In addition, Aerospike tools require the latest pyOpenSSL release (greater than 0.15) to connect using TLS.

Can Clients easily transition to other certificates with mutual authentication?

The transition should not be noticable if the old and new certificates are signed by the same CA. In other words, if ca-file does not change or ca-path contains both old and new CA Root certificates.

The server checks that the client certificate passes the following:

  1. The certificate was issued by a trusted CA.
  2. The certificate is not expired.
  3. The certificate matches the expected TLS name (only if configured through the tls-authenticate-client configuration – configured with any, then the TLS name does not have to match.)

Using new certificates in addition to old certificates that have not expired, new certificates will pass all of the above.

The exception to the above rule would be if using self-signed certificates or if the CA root certificates have expired. Refer to the article on How to Replace Expiring CA Root Certificates.

What if the certificates expire when there is live traffic?

If a certificate expires, for either client or server, communication on existing active connections will continue to work. New connections will fail and the a certificate expired warning will be logged.

Server Logs will show the following warning:

	Jul 06 2019 21:56:39 GMT: WARNING (cf:tls): (tls_ee.c:249) SSL_accept failed: error:14094415:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate expired
	Jul 06 2019 21:56:39 GMT: WARNING (cf:tls): (tls_ee.c:249) SSL_accept failed: error:14094415:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate expired

Clients will throw the following error:

	2019-07-06 21:56:39 INFO TLS verify result: certificate has expired
	2019-07-06 21:56:39 WARN SSL_connect failed: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
	2019-07-06 21:56:39 INFO TLS verify result: certificate has expired
	2019-07-06 21:56:39 WARN SSL_connect failed: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

In general, ca-path can be used to load multiple CA Root certificates. For further details refer to the How to Replace Expiring CA Root Certificates.

What is the procedure to renew expired certificates in a live environment?

Versions prior to 3.15.0.1

The server loads the certificate file(s) on startup. In order to use a new certificate the current certificate must be replaced and the server restarted. If certificates expire across the cluster a rolling restart is necessary.

Versions later than 3.15.0.1

The server will dynamically reload the certificates for new tls Service connections up to 60 seconds after the certificate files have been overwritten and the file Modified-Time has changed. No rolling restart is needed in this particular case. All that is needed is to overwrite the file for the signed certificate referenced by the cert-file and key-file parameters in the aerospike.conf configuration file. The above dynamic certificate rotation feature did not apply to all tls configurations. Fabric and Heartbeat tls certificates on versions prior to 4.7.0.5, 4.6.0.8, 4.5.3.10, 4.5.2.10, 4.5.1.15, 4.5.0.19. required a rolling restart to rotate expired certificates. ECDSA private keys and certificates rotations, and password-protected private keys rotations were also not supported until those same versions.

When upgrading XDR to use TLS, is it possible to operate in a mixed mode where some replication targets are TLS protected, while others operate with the non-TLS port (3000)?

This is not possible because for this to happen the source cluster’s XDR section would need to include both TLS and non-TLS methods of sending the data to the destination cluster, which is not possible.

It is recommended to first upgrade all the destination nodes to receive traffic via TLS and non-TLS ports and then upgrade all the source cluster nodes to send traffic via TLS. In so doing, the situation where some of the nodes of the destination receive traffic via TLS and others do not is avoided. Refer to the Upgrade XDR Connections to TLS for further details.

Does the mixed mode configuration process change in any way if XDR is in active-active replication mode?

No, it does not.

Notes

Heartbeat and Fabric level TLS:

  1. TLS has priority. If a cluster node supports both TLS and non-TLS for heartbeat or fabric, then other cluster nodes will to connect to that node via TLS.

  2. If a cluster node does not offer TLS, then it will not try to connect to other cluster nodes via TLS. TLS support is symmetric: the same configuration is used for incoming as well as outgoing connections (mutual authentication is enforced). If a node does not offer support for incoming TLS, then this means that it does not have a TLS configuration, which implies that it cannot connect to other nodes via TLS, either.

  3. It is possible to run a cluster in mixed mode i.e. with partial TLS support on only a subset of cluster nodes to enable rolling upgrades to TLS. If the cluster nodes on both ends of a heartbeat or fabric connection support TLS, then TLS will be used. Otherwise, non-TLS will be used.

  4. In addition to supporting both, TLS and non-TLS fabric or heartbeats (e.g., for rolling upgrades to TLS), nodes may be configured to only support TLS.

  5. Rolling upgrades are supported from non-TLS to TLS. The upgrade happens in two rounds:

    Round 1: Enable TLS for heartbeat and fabric across all nodes in a rolling fashion. Keep non-TLS enabled while doing so. The result is a cluster that supports both non-TLS as well as TLS. Under normal circumstances, this cluster will default to TLS. However, downgrade attacks are possible, where an attacker prevents the cluster from establishing TLS connections. The cluster will then fall back to non-TLS.

    Round 2: Disable non-TLS for heartbeat and fabric across all nodes in a rolling fashion. The result is a cluster that only supports TLS. This prevents downgrade attacks.

For further references:

Keywords

FAQ TLS CERTIFICATES EXPIRED REPLACE

Timestamp

November 2019

1 Like