ClickHouse Cluster Resizing

Initially, selecting the appropriate size for your ClickHouse cluster based on your current needs may appear straightforward. However, as your business requirements expand, it may become necessary to resize your ClickHouse cluster to effectively accommodate the growing workload and ensure optimal performance. This article describes how you could resize your ClickHouse cluster vertically or horizontally using the Instaclutr Console, Instaclustr API, and Terraform Provider.  

For details on creating a ClickHouse cluster, please refer to this support document.  

Vertical Scaling 

Vertical resize refers to modifying the VM instance of the existing nodes in-place so that the number of nodes in the cluster stays the same, but their underlying VM updates to one with more resources. This can be done in one of the following ways.  

Using the Instaclustr Console 

  1. From Instaclustr Console, begin by navigating to the Scaling tab

  2. Review the Data Centre and the current configurations of the cluster, including current node size, and number of nodes. Click Next to proceed to the next step.

  3. Choose Vertical Scaling in the radio list and proceed to the next step.

  4. Choose the nodes you wish to resize in the Node Purpose dropdown list. A few possible options are available depending on the cluster configuration. 
    If there are Dedicated ClickHouse Keeper nodes in the cluster, you’ll see both ClickHouse Server Nodes and Dedicated ClickHouse Keeper Nodes options, which allow you to resize those nodes only respectively.
    If there’s no Dedicated ClickHouse Keeper nodes in the cluster, you’ll only see All Nodes as the single option
  5. You’ll also need to specify the number of nodes per rack that can be resized concurrently. The larger the number, the faster the operation will be, but the smaller the number, the less impact the resizing will have on the availability of the cluster. 

  6. Once the above options are settled, scroll down on the same page, and you should see a list of node sizes that you can resize the selected nodes to.

  7. Choose a size that fits your workload, and a brief summary will be available on the page showing you the cost of the new nodes. Confirm and submit the resize request by clicking the Resize button. You could also choose to tick the box above it, if you would like to be notified when the scaling operation is finished.

  8. Once the scaling operation is confirmed, you’ll be redirected to a page where you can monitor the progress of the resize. 

  9. While the operation is in progress, the Scaling functionality will be temporarily unavailable. Once it finishes, the Scaling page will be available again for further requests.  

 

Using the Instaclustr API 

Our API Documentation is available at https://instaclustr.redoc.ly/. Search for “Update ClickHouse Cluster Details” and you’ll find details on fields in the payload that can be updated. For changing the node size of the ClickHouse server or Dedicated ClickHouse Keeper nodes, simply update the nodeSize field within the corresponding scope and submit the request.  

Upon a successful request, you should see a comprehensive response with the operation status being “OPERATION_IN_PROGRESS”. You’ll also be able to track the progress of the scaling operation from Instaclustr Console.  

 

Using the Instaclustr Terraform Provider 

Vertical scaling is also supported via Instaclustr’s Terraform Provider. A guide for setting it up can be found in this support document. You can also follow another support document for guidance on creating a ClickHouse cluster with Terraform.  

Once a ClickHouse resource is created (see https://registry.terraform.io/providers/instaclustr/instaclustr/latest/docs for details on Instaclustr Terraform resources), in the main.tf file, you can simply modify the node_size property to resize the server nodes (including colocated ClickHouse Keepers if Dedicated ClickHouse Keepers are not used), or to modify the node_size property within the dedicated_click_house_keeper block to resize the Dedicated ClickHouse Keeper nodes.  

Follow the same process of terraform plan and terraform apply to preview and apply the changes to your cluster.  

 

Horizontal Scaling 

Horizontal scaling refers to increasing the number of nodes in a cluster where the configuration of the new nodes is the same as the existing ones, and the extra nodes will serve either as additional replicas or shards. As mentioned in this support document, the number of nodes in a ClickHouse cluster is the product of Shards and Replicas. When performing horizontal scaling, we follow the same concept of allowing modifications on either of these values for the final cluster size. Increasing the number of Shards improves query performance and scalability of the cluster, while increasing the number of Replicas improves its availability. Similar to vertical scaling, horizontal scaling can also be done in one of the following ways.  

 

Using the Instaclustr Console

  1. Start by going to the Scaling tab on the Instaclustr Console, confirm the current status of your cluster, then proceed with Next.

  2. Choose Horizontal Scaling in the radio button list and proceed to the next step.

  3. You’ll now be able to modify the number of Shards or Replicas in your cluster. Once you’ve made valid changes, a summary will show below, indicating the new cost of the cluster. Confirm and click Resize to proceed with the request. 

  4. Note: To ensure consistent stability and performance, Instaclustr Support will verify the health of your cluster before provisioning additional nodes. We aim to complete this process within one business day and will contact you once the additional nodes have been provisioned. 
  5. Next, you’ll be able to track progress of the scaling operation on the Scaling tab.

  6. Once the operation completes, you’ll also see the new nodes and the configuration change on the Cluster Details tab. 

 

Using the Instaclustr API 

Following our API documentation, similar to vertical scaling, you can send a PUT request to the same API endpoint for horizontal scaling. Simply modify either the replicas or shards to the desired number and submit to proceed.  

 

Using the Instaclustr Terraform Provider 

Following our Terraform provider resource definition, similar to vertical resize, simply change either the number of replicas or shards and follow the same process of terraform plan and terraform apply to preview and apply the changes to your cluster.