NetApp is introducing customer-initiated automated horizontal scaling for Instaclustr for Apache Kafka. Customers can now add or remove broker nodes through the NetApp Instaclustr Console, application programming interface (API) or Terraform provider, while the platform coordinates the operational work needed to help complete the change safely.

The new workflow brings health checks, broker provisioning or removal, partition reassignment, and progress tracking into a single managed operation. Customers can respond to changing capacity requirements without first opening a support ticket and waiting for the operation to be started manually.

Scaling should not create another operational bottleneck

Kafka workloads change. New applications, seasonal traffic, retention changes, and shifts in producer or consumer behavior can all alter the capacity a cluster needs. Adding brokers can provide more compute, storage, and network capacity. Removing brokers can reduce excess capacity when requirements fall.

The broker count is only part of the change. Kafka must remain healthy, partition replicas must be placed correctly, and the remaining brokers must have enough capacity to carry the resulting load. A scaling request that changes infrastructure without coordinating these steps can leave Operations teams with more work and more risk.

Automated horizontal scaling treats the complete change as one managed workflow. Horizontal scaling changes the number of brokers in a data center. Instaclustr also supports vertical scaling which changes the size of the broker nodes.

One scaling experience, in either direction

Customers start an eligible operation from the cluster’s Scaling page in the NetApp Instaclustr Console, select horizontal scaling, and choose the target broker count. The same capability is also available through the API and Terraform for teams that manage infrastructure programmatically.

The platform then coordinates the operation and reports progress through the stages that matter, including pre-scale checks, node provisioning or selection, partition reassignment, drain verification, and completion. If a specified failure requires intervention, the NetApp Instaclustr Support team is notified.

This is designed to give Operations and platform engineering teams:

  • Fewer handoffs. Start eligible scaling operations directly instead of using a support ticket as the trigger.
  • Built-in operational safeguards. Validate cluster health and capacity before making the change.
  • Automated partition movement. Rebalance replicas as part of the scaling workflow instead of treating reassignment as a separate manual task.
  • Clear progress visibility. Follow each stage from the Console while the operation runs.
  • More responsive capacity management. Adjust broker count as workload requirements change.

Add brokers and rebalance automatically

When customers add brokers, NetApp Instaclustr checks that the cluster is healthy before provisioning the new nodes. After the brokers join the cluster, the workflow reassigns existing partition replicas so the new capacity can carry its share of the workload.

The Console shows progress through the health-check, provisioning, and reassignment stages. Customers do not need to coordinate node creation and partition movement as separate activities.

Remove brokers only when the cluster can support it

Scaling down requires a different set of safeguards. Before removing brokers, NetApp Instaclustr checks cluster and broker health, replication, available disk capacity, and projected resource utilization on the brokers that will remain. If the cluster does not have sufficient headroom, the operation does not proceed.

For an eligible request, the workflow selects the brokers to remove and moves their partition replicas to the remaining brokers. It verifies that the selected brokers are drained before deleting and unregistering them. This sequence helps prevent broker removal from getting ahead of Kafka’s data-placement requirements.

Customers can run sequential operations when they need to make a larger reduction, subject to the eligibility and capacity checks applied to each request.

Current eligibility and limits

The current release supports customer-initiated operations for data centers with no more than 30 Kafka brokers before scaling, with up to three brokers added or removed in one operation. Over the coming months, we will work to do away with these restrictions. Remove-node operations are not currently available for data centers using Amazon Web Services (AWS) PrivateLink, Azure Private Link, or Google Cloud Private Service Connect.[1]

These limits define the customer-initiated workflow, not the full range of changes the NetApp Instaclustr Support team can perform. Customers with larger clusters, larger requested changes, or excluded network configurations should contact Support to plan the operation.

Get started

Open an eligible Kafka cluster in the NetApp Instaclustr Console, select Scaling, and choose Horizontal Scaling. Review the target broker count and operation summary before starting the workflow.

See the Apache Kafka horizontal scaling documentation and additional support page about removing nodes for preparation steps, current eligibility, limitations, API guidance, and detailed progress states. The partition reassignment overview explains how NetApp Instaclustr moves replicas while maintaining rack-aware placement.