In-place Resizing for Kafka
Nodes in a Kafka cluster can be resized in-place via the Console or by issuing a request to our Provisioning API. Moving to a new node size lets you scale the CPU core count, memory quotas as well as increase disk capacity to meet changing demands. You may resize a subset of nodes that match a purpose e.g. resize only the broker nodes in your cluster.
Instaclustr recommends that you first test your applications against in place resizing in a non-production setting to verify that the resize process doesn’t interrupt your applications.
During a cluster resize operation one or more nodes will temporarily become unavailable as they are resized (for resizing involving more than a disk resize). Kafka, as a whole, will handle these transient outages, but client applications may need to be programmed with certain considerations in mind.
Instaclustr will only concurrently resize nodes within the same rack, to ensure that replicas stored in other racks remain available during the resize.
The cluster should have topics with a replication factor greater than one, and this replication factor should not equal to the min.insync.replicas config of that topic.
- Current in place resizing is only supported on AWS Nodes which are EBS backed.
- Disks can be increased but not decreased
- Resizing where the disk is modified can only be done once every 6 hours due to EBS limitations
- Resizing only works with single disk node sizes.
- The nodes selected for resize must all be the same size.
It is recommended to produce with the “acknowledgement” setting to “all” to ensure no data would be lost.
We suggest using all brokers that are provided for connection configuration.
To resize data centre nodes, navigate to the Resize page for your Kafka cluster.
You will be able to select which nodes you want to resize by selecting the Node Purpose from those available to your cluster. You can resize Broker or Dedicated ZooKeeper nodes. Resizing both Broker and Dedicated ZooKeeper nodes needs to be conducted in 2 separate operations.
The Concurrency Factor is a number, from 1 to the count of nodes in the largest rack, that specifies how many nodes may be resized at the same time. To ensure the data’s availability within the cluster, Kafka cluster data centres are only allowed to be resized one node at a time. Any resize requests with a concurrency factor greater than one will not be processed.
Finally, select a desired target node size that you wish to resize to.
Once you have selected a new size, you will be shown a summary of the resize, which will include the new price for managing your cluster. Click on Resize Data Centre to begin the resize process.
You can also select to notify your accounts’ designated support contacts via email once the resize is complete. You may opt-out of this notification by deselecting the Notify this account’s designated support contacts on resize completion checkbox.
If the selected data centre isn’t resizable then an explanation will be displayed on the Resize Data Centre page.
While a resize is in progress, the Cluster Details page will display the progress of the resize. You will see individual nodes switch from Running to Pending → Provisioning/Provisioned and then back to Running once their size has been changed.
In some rare cases, it may be desirable to cancel a resize operation while it is in progress. This can be achieved via the API. This may leave the cluster in a non-uniform state with some nodes already resized. Please contact our support team to assist in this case, to ensure your cluster is correctly sized.