Pooling configuration with CockroachDB

CockroachDB certificates define the boundaries of the pool and must be generated before services can be deployed.

Prerequisites

• Install CockroachDB 24.1 to generate CockroachDB certificates.
  • These instructions assume CockroachDB Core.
  • You may need to run sudo chmod +x /usr/local/bin/cockroach after completing the installation instructions.
  • Confirm successful installation with cockroach version

In the instructions below:

• $CLUSTER_CONTEXT is the name of the Kubernetes cluster deployed as infrastructure onto which services will be deployed.
• $NAMESPACE is the namespace for this DSS instance
• Each ADDRESS is the DNS entry for a CockroachDB node that will use the certificates generated by the command. This is usually just the nodes constituting this DSS instance, though if you maintain multiple DSS instances in a single pool, the separate instances may share certificates. Note that --node-address must include all the hostnames and/or IP addresses that other CockroachDB nodes will use to connect to your nodes (the nodes using these certificates). Wildcard notation is supported, so you can use *.<subdomain>.<domain>.com>. If following the recommendations above, use a single ADDRESS similar to *.db.yourdeployment.yourdomain.com. The ADDRESS entries should be separated by spaces.

First or only DSS instance

If this DSS instance will be the first and/or only DSS instance in the pool, follow the instructions in this section.

Use make-certs.py script to create certificates for the CockroachDB nodes in this DSS instance:

    ./make-certs.py --cluster $CLUSTER_CONTEXT --namespace $NAMESPACE
        --node-address <ADDRESS> <ADDRESS> <ADDRESS> ...

1.  Note: If you are creating multiple DSS instances at once and joining
    them together, you may want to copy the nth instance's `ca.crt` into
    the rest of the instances, such that ca.crt is the same across all
    instances.

Proceed to "Apply certs" section below.

Joining an existing pool

If this DSS instance is joining an existing pool, follow the instructions in this section.

Obtain existing deployment information

Before certs for this DSS instance can be generated, the following information must be gathered from participants in the existing pool:

• Public certificate of each DSS instance in the existing pool, concatenated into a single ca.crt file located at CA_CERT_FILE
• The address of each DSS instance in the existing pool

Generate certs

Use make-certs.py script to create certificates for the CockroachDB nodes in this DSS instance:

    ./make-certs.py --cluster $CLUSTER_CONTEXT --namespace $NAMESPACE
        --node-address <ADDRESS> <ADDRESS> <ADDRESS> ...
        --ca-cert-to-join <CA_CERT_FILE>

Adopt certs

Before services are deployed to this DSS instance, all existing participants in the pool must update their DSS instances to accept the certificates of this DSS instance (just generated).

Share ca.crt with each participant in the existing pool and have them apply the new ca.crt, which now contains both your instance's and the original instances' public certs, to enable secure bi-directional communication. The operator of each existing DSS instance, upon receipt of the combined ca.crt from this joining instance, should perform the "Updating ca.crt" actions below and provide positive confirmation that they have completed those actions.

Danger

Do not proceed until every operator of a DSS instance in the existing pool has positively confirmed that their DSS instance has adopted the provided ca.crt.

Updating ca.crt

When a participant seeking to deploy a DSS instance into an existing pool requests that a participant with an existing DSS instance in the pool adopt a provided ca.crt file, the participant with the existing DSS instance should:

1. Overwrite its existing ca.crt with the new ca.crt provided by the DSS instance joining the pool.
2. Upload the new ca.crt to its cluster using ./apply-certs.sh $CLUSTER_CONTEXT $NAMESPACE (see top of page for $CLUSTER_CONTEXT and $NAMESPACE)
3. Restart their CockroachDB pods to recognize the updated ca.crt: kubectl rollout restart statefulset/cockroachdb --namespace $NAMESPACE
4. Inform the prospective pool participant when the CockroachDB pods have finished restarting (typically around 10 minutes)

Apply certs

Regardless of how certs were generated, apply/deploy them to the infrastructure using the apply-certs.sh script to create secrets on the Kubernetes cluster containing the certificates and keys generated in the previous step.

    ./apply-certs.sh $CLUSTER_CONTEXT $NAMESPACE

Next steps

Proceed to services deployment.