Cluster over TLS

Last updated 12 Oct 2026

Not on HotKV CloudOn-Premise: no license needed

TLS in a cluster covers three kinds of connection: your clients, the links between nodes (the cluster bus), and the replication links from replicas to their primaries. You turn each on separately. Between nodes HotKV always uses mutual TLS: both ends present a certificate and each checks the other’s against your CA, so a host without a certificate from your CA cannot join the cluster even when it can reach the bus port.

This page builds a TLS cluster across two hosts, from the certificates up. The two hosts are 10.0.0.1 (node1.example.com) and 10.0.0.2 (node2.example.com), each running three nodes on ports 7000 to 7002. The TLS settings for a single server, outside a cluster, are on TLS. Read Deploy on VMs and cloud first for the parts that are the same with and without TLS: sizing, firewalls, passwords.

The commands below write under /etc/hotkv, /var/lib/hotkv and /var/log/hotkv and read private keys, so run them in a root shell (sudo -i) on each host.

How the ports work

With tls-cluster yes a node goes by its TLS port in the cluster, and the bus port follows it:

Settings	Client ports	Bus port
`port 0`, `tls-port 7000` (TLS only)	TLS on 7000	17000
`port 7060`, `tls-port 7160` (both)	plain on 7060, TLS on 7160	17160

cluster-port puts the bus on a port of your choice instead. With both ports, each client is told the port of its own kind. This is what a node with port 7060 and tls-port 7160 printed for CLUSTER NODES, to a plain client and to a TLS client:

88c1a627187760c03ab057b49f2eb0c6b19121d8 10.0.0.1:7060@17160 myself,master - 0 0 0 connected
88c1a627187760c03ab057b49f2eb0c6b19121d8 10.0.0.1:7160@17160 myself,master - 0 0 0 connected

The bus was listening on 17160, the TLS port plus 10000, and nothing listened on 17060. Replicas connect to the primary’s TLS port. Open the TLS port and the bus port between the nodes, as described in Ports and firewalls.

1. Create the certificates

You need one CA that signs only the certificates of this cluster, a certificate for each host, and a certificate for your applications. Run these commands in an empty directory on a trusted machine (here, the first host). The certificates here use elliptic-curve keys and last a year; the CA lasts ten.

mkdir pki && cd pki
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out ca.key
openssl req -x509 -new -key ca.key -sha256 -days 3650 -subj "/CN=HotKV cluster CA" \
  -addext "basicConstraints=critical,CA:TRUE" \
  -addext "keyUsage=critical,keyCertSign,cRLSign" -out ca.crt
for node in node1:10.0.0.1 node2:10.0.0.2; do
  name=${node%%:*}; ip=${node##*:}
  openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out $name.key
  openssl req -new -key $name.key -subj "/CN=$name.example.com" -out $name.csr
  openssl x509 -req -in $name.csr -CA ca.crt -CAkey ca.key -CAcreateserial -sha256 -days 365 \
    -extfile <(printf "subjectAltName=DNS:%s.example.com,IP:%s,IP:127.0.0.1\nextendedKeyUsage=serverAuth,clientAuth\n" $name $ip) \
    -out $name.crt
done
ls
cd ..

Certificate request self-signature ok
subject=CN=node1.example.com
Certificate request self-signature ok
subject=CN=node2.example.com
ca.crt
ca.key
ca.srl
node1.crt
node1.csr
node1.key
node2.crt
node2.csr
node2.key

Two things in the certificates matter:

Names. The subject alternative name lists every address and host name that clients use to reach the node, including the IP addresses, and 127.0.0.1 for tools run on the node itself. Many client libraries check that the name they connected to is in the certificate. Between nodes the names are not checked unless you set tls-expected-peer-name (see Bus protection).
Key usage. Each node uses its certificate both to accept connections and to open them to other nodes, so it needs serverAuth and clientAuth. A certificate with serverAuth only is refused by the other nodes; in that case give the node a second certificate with tls-client-cert-file and tls-client-key-file.

(cd pki && openssl x509 -in node1.crt -noout -subject -issuer -ext subjectAltName,extendedKeyUsage -dates)

subject=CN=node1.example.com
issuer=CN=HotKV cluster CA
X509v3 Subject Alternative Name:
    DNS:node1.example.com, IP Address:10.0.0.1, IP Address:127.0.0.1
X509v3 Extended Key Usage:
    TLS Web Server Authentication, TLS Web Client Authentication
notBefore=Jan  1 00:00:00 2027 GMT
notAfter=Jan  1 00:00:00 2028 GMT

Create a certificate for your applications the same way, with extendedKeyUsage=clientAuth. You need it because tls-auth-clients is yes by default, which makes every TLS client present a certificate:

(cd pki
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out app.key
openssl req -new -key app.key -subj "/CN=app" -out app.csr
openssl x509 -req -in app.csr -CA ca.crt -CAkey ca.key -CAcreateserial -sha256 -days 365 \
  -extfile <(printf "extendedKeyUsage=clientAuth\n") -out app.crt
ls app.*)

Certificate request self-signature ok
subject=CN=app
app.crt
app.csr
app.key

Install the CA certificate and the host’s own certificate and key under /etc/hotkv/tls/ on each host (the directory does not exist yet), with the key readable only by root and the hotkv group, the user the HotKV service runs as. On the first host:

install -d -m 755 /etc/hotkv/tls
(cd pki && install -m 644 ca.crt /etc/hotkv/tls/ca.crt && install -m 644 node1.crt /etc/hotkv/tls/node.crt && install -m 640 -g hotkv node1.key /etc/hotkv/tls/node.key)
stat -c '%A %U:%G %s %n' /etc/hotkv/tls/*

-rw-r--r-- root:root 615 /etc/hotkv/tls/ca.crt
-rw-r--r-- root:root 676 /etc/hotkv/tls/node.crt
-rw-r----- root:hotkv 241 /etc/hotkv/tls/node.key

On the second host, install ca.crt, node2.crt as node.crt and node2.key as node.key the same way, copying them over a channel you trust. Never copy ca.key to the nodes: it can sign certificates for anyone. Keep it offline and use it only to issue and renew certificates. A node that runs as the hotkv user, as the systemd units do, cannot start with a key that only root can read: it logs Failed to initialize SSL context: use_private_key_file: Permission denied and exits.

2. Configure the nodes

The file for the first node on the first host, /etc/hotkv/7000.conf:

bind 10.0.0.1 127.0.0.1
port 0
tls-port 7000
tls-cert-file /etc/hotkv/tls/node.crt
tls-key-file /etc/hotkv/tls/node.key
tls-ca-cert-file /etc/hotkv/tls/ca.crt
tls-auth-clients yes
tls-cluster yes
tls-replication yes
requirepass change-me
cluster-enabled yes
dir /var/lib/hotkv/7000

port 0 turns the plain listener off, so the node serves TLS only. tls-cluster yes puts the bus on TLS and tls-replication yes does the same for replication. A node with either setting and no certificate, key and CA (tls-ca-cert-file or tls-ca-cert-dir) does not start. The password is the same one as without TLS: TLS protects the connection, and the password and ACL users still decide who may do what.

The same node in hotkv.toml:

[network]
bind = "10.0.0.1 127.0.0.1"
port = 0

[tls]
port = 7000
cert_file = "/etc/hotkv/tls/node.crt"
key_file = "/etc/hotkv/tls/node.key"
ca_cert_file = "/etc/hotkv/tls/ca.crt"
auth_clients = "yes"
cluster = true
replication = true

[security]
requirepass = "change-me"

[cluster]
enabled = true

[persistence]
dir = "/var/lib/hotkv/7000"

Make the files for ports 7001 and 7002 from the first one, and the files for the second host the same way with bind 10.0.0.2 127.0.0.1:

cat > /etc/hotkv/7000.conf <<'EOF'
bind 10.0.0.1 127.0.0.1
port 0
tls-port 7000
tls-cert-file /etc/hotkv/tls/node.crt
tls-key-file /etc/hotkv/tls/node.key
tls-ca-cert-file /etc/hotkv/tls/ca.crt
tls-auth-clients yes
tls-cluster yes
tls-replication yes
requirepass change-me
cluster-enabled yes
dir /var/lib/hotkv/7000
EOF
for port in 7001 7002; do sed s/7000/$port/g /etc/hotkv/7000.conf > /etc/hotkv/$port.conf; done
chgrp hotkv /etc/hotkv/700?.conf; chmod 640 /etc/hotkv/700?.conf
install -d -o hotkv -g hotkv /var/lib/hotkv/7000 /var/lib/hotkv/7001 /var/lib/hotkv/7002 /var/log/hotkv
cat /etc/hotkv/7001.conf

bind 10.0.0.1 127.0.0.1
port 0
tls-port 7001
tls-cert-file /etc/hotkv/tls/node.crt
tls-key-file /etc/hotkv/tls/node.key
tls-ca-cert-file /etc/hotkv/tls/ca.crt
tls-auth-clients yes
tls-cluster yes
tls-replication yes
requirepass change-me
cluster-enabled yes
dir /var/lib/hotkv/7001

Start the nodes on both hosts. Under systemd, use the unit from Deploy on VMs and cloud. For a quick test, from a shell (the data directories were created above):

for port in 7000 7001 7002; do
  hotkv /etc/hotkv/$port.conf --daemonize yes --pidfile /var/lib/hotkv/$port/hotkv.pid \
    --logfile /var/log/hotkv/$port.log
done

The log of a node says what it set up:

Plain-text port disabled (port 0), serving TLS only
SSL context initialized (cert=/etc/hotkv/tls/node.crt, key=/etc/hotkv/tls/node.key)
TLS on the cluster bus, other nodes verified against the CA
TLS on replication links, other nodes verified against the CA
TLS enabled on 10.0.0.1 127.0.0.1:7000

(Log lines here are shown without the date, time and level that start each line.)

3. Create the cluster

Give redis-cli the CA certificate to verify the nodes, and a certificate and key to present to them. The node’s own certificate works, since it has clientAuth. The password goes in the environment:

export REDISCLI_AUTH=change-me
redis-cli --tls --cacert /etc/hotkv/tls/ca.crt --cert /etc/hotkv/tls/node.crt --key /etc/hotkv/tls/node.key \
  --cluster create 10.0.0.1:7000 10.0.0.1:7001 10.0.0.1:7002 \
  10.0.0.2:7000 10.0.0.2:7001 10.0.0.2:7002 --cluster-replicas 1 --cluster-yes

>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 10.0.0.2:7002 to 10.0.0.1:7000
Adding replica 10.0.0.1:7002 to 10.0.0.2:7000
Adding replica 10.0.0.2:7001 to 10.0.0.1:7001
M: 4fa60fec899cdebbd189d9f7c623365ce7598a9d 10.0.0.1:7000
   slots:[0-5460] (5461 slots) master
M: ef33b047c890fdd8fa707eb7a7e5e51149723ab1 10.0.0.1:7001
   slots:[10923-16383] (5461 slots) master
S: 3a6ba9f419bdc912c80632d823cc8f77e3f2ffe5 10.0.0.1:7002
   replicates 8a58921d8b2ad80e67568d708d701ea0cf6c0fef
M: 8a58921d8b2ad80e67568d708d701ea0cf6c0fef 10.0.0.2:7000
   slots:[5461-10922] (5462 slots) master
S: c61c540db15f2c19f8ff003a31d25762a18d9338 10.0.0.2:7001
   replicates ef33b047c890fdd8fa707eb7a7e5e51149723ab1
S: 2a8484d5defce55e2bb8ecebc82b69c85897b811 10.0.0.2:7002
   replicates 4fa60fec899cdebbd189d9f7c623365ce7598a9d
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join

>>> Performing Cluster Check (using node 10.0.0.1:7000)
...
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

valkey-cli takes the same options (and VALKEYCLI_AUTH for the password). Starting from fresh nodes, its create ends the same way:

export VALKEYCLI_AUTH=change-me
valkey-cli --tls --cacert /etc/hotkv/tls/ca.crt --cert /etc/hotkv/tls/node.crt --key /etc/hotkv/tls/node.key \
  --cluster create 10.0.0.1:7000 10.0.0.1:7001 10.0.0.1:7002 \
  10.0.0.2:7000 10.0.0.2:7001 10.0.0.2:7002 --cluster-replicas 1 --cluster-yes | tail -n 12

   slots: (0 slots) slave
   replicates fa81bb8a7759a36922313e397f1ee40c7423d2fa
S: 9a0f41a11ce7e331604025cc2d00fcad2a41b9de 10.0.0.1:7002
   slots: (0 slots) slave
   replicates 10dfd88397579c4244058f8ec8526936fc6d5f67
M: fa81bb8a7759a36922313e397f1ee40c7423d2fa 10.0.0.1:7001
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

4. Check that everything is encrypted

A plain connection to a client port or the bus port of a TLS-only node is closed:

redis-cli -h 10.0.0.1 -p 7000 PING
redis-cli -h 10.0.0.1 -p 17000 PING

Error: Server closed the connection
Error: Server closed the connection

The node logs each refused connection, and says what it was:

TLS handshake failed for client 4: wrong version number (SSL routines)
Error accepting cluster node connection: wrong version number (SSL routines) (the TLS handshake from 10.0.0.1: tls-cluster is on here, the other nodes must use it too, with a certificate tls-ca-cert-file trusts)

A TLS client that presents no certificate is refused too, while tls-auth-clients is yes:

TLS handshake failed for client 5: peer did not return a certificate (SSL routines)

The bus port speaks TLS. openssl can connect to it with a node certificate and shows the protocol and the peer’s certificate (the long line of signature algorithms is cut here):

openssl s_client -connect 10.0.0.1:17000 -CAfile /etc/hotkv/tls/ca.crt -cert /etc/hotkv/tls/node.crt -key /etc/hotkv/tls/node.key -verify_return_error -brief </dev/null 2>&1 | head -n 8

Connecting to 10.0.0.1
Can't use SSL_get_servername
CONNECTION ESTABLISHED
Protocol version: TLSv1.3
Ciphersuite: TLS_AES_256_GCM_SHA384
...
Peer certificate: CN=node1.example.com
Hash used: SHA256

Replication is encrypted too. A replica’s log shows it, and INFO replication shows the link is up:

Replication: becoming a replica of 10.0.0.2:7000
Replication: connecting to primary 10.0.0.2:7000
Replication: connected to primary 10.0.0.2:7000 over TLS

redis-cli --tls --cacert /etc/hotkv/tls/ca.crt --cert /etc/hotkv/tls/node.crt --key /etc/hotkv/tls/node.key -p 7002 INFO replication | grep -E "role|master_link_status|master_port"

role:slave
master_port:7000
master_link_status:up

CLUSTER LINKS lists the bus links of a node with their buffers, but does not say whether they are encrypted. Use the log lines above for that.

5. Connect clients

redis-cli with the application certificate follows redirects as usual:

REDISCLI_AUTH=change-me redis-cli -c --tls --cacert pki/ca.crt --cert pki/app.crt --key pki/app.key -h 10.0.0.1 -p 7000 SET greeting hello
REDISCLI_AUTH=change-me redis-cli -c --tls --cacert pki/ca.crt --cert pki/app.crt --key pki/app.key -h 10.0.0.2 -p 7002 GET greeting

OK
hello

In Python, the redis package (pip install redis, in a virtual environment as shown in Cluster clients) takes the certificates as arguments. This script works against the cluster above:

from redis.cluster import RedisCluster

rc = RedisCluster(
    host="10.0.0.1",
    port=7000,
    password="change-me",
    ssl=True,
    ssl_ca_certs="pki/ca.crt",
    ssl_certfile="pki/app.crt",
    ssl_keyfile="pki/app.key",
)
rc.set("greeting", "hello from python")
print(rc.get("greeting"))
print(sorted(n.name for n in rc.get_nodes()))

b'hello from python'
['10.0.0.1:7000', '10.0.0.1:7001', '10.0.0.1:7002', '10.0.0.2:7000', '10.0.0.2:7001', '10.0.0.2:7002']

The client verified each node’s name against its certificate, which is why the certificates list the IP addresses. The replies are bytes unless you pass decode_responses=True. Clients over TLS are told TLS ports in redirects, in CLUSTER SLOTS and in CLUSTER NODES. Other libraries are covered in Clients. If your applications cannot present a certificate, tls-auth-clients optional or no relaxes the check for clients only; between nodes it never applies (see TLS).

6. Reshard under TLS

redis-cli --cluster reshard and the other cluster tools work over TLS with the same options. This example moves 2000 slots from one primary to another, with 200 keys in the cluster. The keys in those slots move with them:

rcli() { redis-cli --tls --cacert /etc/hotkv/tls/ca.crt --cert /etc/hotkv/tls/node.crt --key /etc/hotkv/tls/node.key "$@"; }
rcli --cluster check 10.0.0.1:7000 | head -n 4

10.0.0.1:7000 (4b4522cf...) -> 65 keys | 5361 slots | 1 slaves.
10.0.0.2:7000 (d397b1a9...) -> 61 keys | 5462 slots | 1 slaves.
10.0.0.1:7001 (9a93f72e...) -> 75 keys | 5561 slots | 1 slaves.
[OK] 201 keys in 3 masters.

FROM=$(rcli -h 10.0.0.1 -p 7000 CLUSTER MYID)
TO=$(rcli -h 10.0.0.1 -p 7001 CLUSTER MYID)
rcli --cluster reshard 10.0.0.1:7000 --cluster-from $FROM --cluster-to $TO --cluster-slots 2000 --cluster-yes | tail -n 3

Moving slot 2097 from 10.0.0.1:7000 to 10.0.0.1:7001:
Moving slot 2098 from 10.0.0.1:7000 to 10.0.0.1:7001:
Moving slot 2099 from 10.0.0.1:7000 to 10.0.0.1:7001:

rcli --cluster check 10.0.0.1:7000 | head -n 4

10.0.0.1:7000 (4b4522cf...) -> 46 keys | 3361 slots | 1 slaves.
10.0.0.2:7000 (d397b1a9...) -> 61 keys | 5462 slots | 1 slaves.
10.0.0.1:7001 (9a93f72e...) -> 94 keys | 7561 slots | 1 slaves.
[OK] 201 keys in 3 masters.

(The rcli function stands for the full redis-cli command with the TLS options, and REDISCLI_AUTH was set.) Moving keys between nodes also uses the TLS client ports when the nodes go by their TLS ports. The other reshard methods are in Scaling.

7. Renew certificates

A node loads its certificate when it starts, and again when you tell it to. Replace the file, then reload, one node at a time, with no restart and no downtime.

The commands below use the rcli function from the previous section. Look at the certificate a node presents, issue a new one from the same key, and replace the file:

openssl s_client -connect 10.0.0.1:7000 -CAfile /etc/hotkv/tls/ca.crt </dev/null 2>/dev/null | openssl x509 -noout -serial -enddate

serial=1A2B3C4D5E6F708192A3B4C5D6E7F8091A2B3C4D
notAfter=Jan  1 00:00:00 2028 GMT

(cd pki
openssl req -new -key node1.key -subj "/CN=node1.example.com" -out node1.csr
openssl x509 -req -in node1.csr -CA ca.crt -CAkey ca.key -CAcreateserial -sha256 -days 400 \
  -extfile <(printf "subjectAltName=DNS:node1.example.com,IP:10.0.0.1,IP:127.0.0.1\nextendedKeyUsage=serverAuth,clientAuth\n") \
  -out node1.crt
install -m 644 node1.crt /etc/hotkv/tls/node.crt)

The node keeps presenting the old certificate (the same serial) until it reloads. Setting tls-cert-file to the path it already has reloads it, and CONFIG SET takes effect at once:

for port in 7000 7001 7002; do
  rcli -h 10.0.0.1 -p $port CONFIG SET tls-cert-file /etc/hotkv/tls/node.crt
done

OK
OK
OK

openssl s_client -connect 10.0.0.1:7000 -CAfile /etc/hotkv/tls/ca.crt </dev/null 2>/dev/null | openssl x509 -noout -serial -enddate
rcli -h 10.0.0.1 -p 7000 CLUSTER INFO | head -n 2

serial=1A2B3C4D5E6F708192A3B4C5D6E7F8091A2B3C4E
notAfter=Feb  5 00:00:00 2028 GMT
cluster_enabled:1
cluster_state:ok

New connections use the new certificate, and connections that are already up keep the old one until they close. If the new file cannot be loaded, the command fails and the node keeps the certificate it had. Here the file held text instead of a certificate (a separate file, so that the certificate in use stays intact):

echo "not a certificate" > /tmp/not-a-cert.crt
rcli -h 10.0.0.1 -p 7000 CONFIG SET tls-cert-file /tmp/not-a-cert.crt

ERR Unable to update TLS configuration: use_certificate_chain_file: no start line (PEM routines) [...]

(The message ends with an internal error code, left out here.)

To avoid running the command after every renewal, let the node look for changes itself with tls-auto-reload-interval (seconds, 0 is off, the default). It checks the certificate, key, CA and DH files, and reloads when one changed. A renewal job then only replaces the files:

rcli -h 10.0.0.1 -p 7000 CONFIG SET tls-auto-reload-interval 5
rcli -h 10.0.0.1 -p 7000 CONFIG GET tls-auto-reload-interval

OK
tls-auto-reload-interval
5

A few seconds after a new node.crt is copied in, the node logs this and presents the new serial:

TLS configuration reloaded (cert=/etc/hotkv/tls/node.crt, key=/etc/hotkv/tls/node.key)
TLS materials reloaded successfully

A reload that fails is logged and the certificates in use stay:

TLS reload failed, keeping the current certificate: use_certificate_chain_file: no start line (PEM routines) [...]

(The line ends with the same internal error code, left out here.)

Renew every node’s certificate well before it expires.

Bus protection

Without tls-cluster, any host that can reach the bus port can join the cluster and speak its protocol. Each such node logs this at startup:

WARNING: the cluster bus port is not authenticated: tls-cluster is off and cluster-bus-port-protected-mode is no, so any host that reaches the port can join this cluster and speak its protocol. Let only trusted hosts reach it, or set tls-cluster yes.

cluster-bus-port-protected-mode yes makes a node refuse to run in that state. Without TLS on the bus it does not start:

hotkv --port 7053 --bind 127.0.0.1 --cluster-enabled yes --cluster-bus-port-protected-mode yes --dir /var/lib/hotkv/7052 2>&1 | head -n 4

*** FATAL CONFIG FILE ERROR ***
cluster-bus-port-protected-mode is enabled but tls-cluster is disabled: the cluster bus port of this node would take any host that reaches it as a node of the cluster. Set 'tls-cluster yes', so that the other nodes must show a certificate tls-ca-cert-file trusts, or set 'cluster-bus-port-protected-mode no' if only trusted hosts can reach the port (a firewall, say)

With tls-cluster yes it starts without the warning, and turning TLS off at runtime is refused:

$ redis-cli --tls ... -p 7001 CONFIG GET cluster-bus-port-protected-mode
cluster-bus-port-protected-mode
yes
$ redis-cli --tls ... -p 7001 CONFIG SET tls-cluster no
ERR CONFIG SET failed (possibly related to argument 'tls-cluster') - can't disable tls-cluster while cluster-bus-port-protected-mode is enabled: the bus would take any host; set cluster-bus-port-protected-mode no first, or in the same command

(The two commands carry the same --cacert, --cert and --key options as before, and REDISCLI_AUTH.) tls-expected-peer-name adds a name check: a list of names, one of which the other node’s certificate must carry (as a DNS name, or as its common name when it has none). It applies to the links this node opens and to the bus connections it accepts. A mismatch fails like this (see the next section). The setting takes effect from the next connection and changes at runtime:

rcli -h 10.0.0.1 -p 7000 CONFIG SET tls-expected-peer-name other.example.com

When the handshake fails

A refused link is logged on both ends, with the reason. The node that opens the link logs the TLS bus link to node ... failed, and the node that accepts it logs Error accepting cluster node connection. Both lines end with a reminder of what a bus link needs. Each case below happens when you introduce a new node with CLUSTER MEET.

A certificate from another CA. The node that opened the link:

Cluster: the TLS bus link to node bb6d9054e9709ce7ab565cc44430652ee6d574a3 at 10.0.0.1:17050 failed: certificate verify failed (SSL routines): unable to get local issuer certificate (the TLS handshake: it must have tls-cluster yes, and a certificate tls-ca-cert-file trusts)

The node that was met:

Error accepting cluster node connection: tlsv1 alert unknown ca (SSL routines) (the TLS handshake from 10.0.0.1: tls-cluster is on here, the other nodes must use it too, with a certificate tls-ca-cert-file trusts)

An expired certificate.

Cluster: the TLS bus link to node 083ebe900a8561def80a0f6b09ddc855fafdaf4c at 10.0.0.1:17051 failed: certificate verify failed (SSL routines): certificate has expired (the TLS handshake: it must have tls-cluster yes, and a certificate tls-ca-cert-file trusts)
Error accepting cluster node connection: ssl/tls alert certificate expired (SSL routines) (the TLS handshake from 10.0.0.1: tls-cluster is on here, the other nodes must use it too, with a certificate tls-ca-cert-file trusts)

A name that does not match tls-expected-peer-name.

Cluster: the TLS bus link to node 1792999059702fbffadf2f0ba066ed9246d2ca3a at 10.0.0.1:17052 failed: certificate verify failed (SSL routines): hostname mismatch (the TLS handshake: it must have tls-cluster yes, and a certificate tls-ca-cert-file trusts)

A node without tls-cluster. The TLS nodes log the wrong version number line from section 4 for it. A plain node that is told about TLS nodes does not get an answer it can read, and logs this for each of them:

Cluster link message too large from 964febbb1fe2c12407d497160f6ead5514e840f4: 352518912 bytes

In all cases the new node does not join the cluster. Fix. Issue the node a certificate that your CA signed and that has not expired, set tls-cluster yes (and tls-replication yes) on it, and check the names if you use tls-expected-peer-name. A node that does not start because of its certificate or key files names the file in its log. The other errors in Troubleshooting apply here too.

Next steps

TLS: TLS for a single server, protocols and ciphers, and logging in by certificate.
Production checklist: where TLS fits in the list of things to check.
Cluster configuration: every TLS and cluster setting.

Ask the HotKV docs