Using Kafka Client Telemetry

Note: Kafka Client Telemetry is currently available in Preview. Preview features do not include production Service Level Agreements (SLA) and are intended for development and test use. We’re actively working toward General Availability (GA) and expect to release the GA version by late March. As the feature progresses toward GA, an upgrade or migration to the GA implementation may be needed, though we anticipate it to probably be a non-disruptive migration. In some cases, this may involve enabling Kafka Client Telemetry on a newly created cluster rather than migrating an existing one. If you’d like to test this feature, please reach out to our support team to have it enabled for your account or cluster.  

Kafka Client Telemetry feature on the Instalcustr managed platform is based on KIP-714, KIP-1000 and KIP-1076. This enables the collection of performance and behavioral metrics directly from Kafka clients and exports them to a designated OpenTelemetry (OTLP) endpoint. 

 For Instaclustr’s managed Apache Kafka clusters, Kafka Client Telemetry can be enabled for new clusters via terraform or Instaclustr API. To enable this feature for your account or existing clusters, please contact our support team. 

Prerequisites:

The Kafka Client Telemetry feature has the following prerequisites which come from the Apache Kafka project’s implementation: 

  • Kafka clients must be on version 3.7.0 or later. 
  • Kafka cluster must be running in KRaft mode. 

In addition, Instaclustr’s implementation has the following requirement:  

  • Kafka cluster must be running Apache Kafka 3.9.1 or later. 
  • Prometheus server 3.0 or later as an endpoint running in your environment, accessible by the Kafka cluster, and configured for Kafka Client Telemetry. You can set up Prometheus as your OpenTelemetry backend; see appendix for details.

To use Kafka Client Telemetry, client telemetry configurations must be defined to specify the export settings to your OpenTelemetry supported endpoint.  

Client telemetry subscriptions should exist for collecting metrics. Metrics are only emitted for clients that have an active subscription. 

Additional Requirements for Kafka Client Telemetry (Public Preview only)

The following conditions apply while this feature is in Preview: 

  • The cluster must use a Non-Production SLA tier 
  • PCI compliance mode must not have been enabled. (It is disabled by default.) 
  • Updating or deleting client telemetry configurations from the cluster using Instaclustr API or Terraform is not supported at this time. Please submit a request to our support team to update the client telemetry configurations. 

Step 1. Configuring a Kafka Cluster with Kafka Client Telemetry

Kafka Client Telemetry can currently be configured at cluster creation time using the Instaclustr API or the Terraform provider by including client telemetry details in the request body. Creating a cluster with Kafka Client Telemetry is not yet supported via the console in the Preview.  Console support is planned and will be available before the feature reaches General Availability (GA). 

Sending a request from Instaclustr API 

You can send a POST request to 

The body should include the optional clientTelemetry configuration. The basicAuthentication is also optional, but highly recommended. 

Sending a request from Terraform 

Similar to the Instaclustr API, Terraform also provides an optional configuration block for client_telemetry. 

Step 2. Subscribing to Kafka Client Telemetry

A client telemetry subscription defines which metrics should be collected from matching Kafka metrics, how frequently they are reported, and which clients the configuration applies to. Only clients that match an active subscription will emit telemetry data.  

We support POST, GET, UPDATE, DELETE and GET ALL in cluster endpoints for client telemetry subscriptions. 

POST 

Use the POST endpoint below to create a client telemetry subscription. If the request is successful, it will return the ID of the client telemetry subscription. 

Body of the request with examples; 

  • The metrics property is optional. If not set, all metrics will be subscribed. Wildcard values are supported. 
  • The clients property is optional for Kafka version 4.0.0 or later. If not set, metrics for all clients will be subscribed. For Kafka bundle versions < 4.0.0. The clients property is required. It must contain a valid matcher from the following list:  
    • client_id,  
    • client_instance_id,  
    • client_software_name,  
    • client_software_version,  
    • client_source_address,  
    • client_source_port 
  • The interval property is optional. If not set, it defaults to 20000ms(20 seconds), which is also the minimum supported value. 

GET 

Use the GET endpoint provided below to return information for a specific client telemetry subscription. 

PUT 

Use the PUT endpoint to update an existing client telemetry subscription: 

DELETE 

Use the DELETE endpoint provided below to delete the client telemetry subscription: 

GET ALL 

You can also use the GET endpoint below to query all client telemetry subscriptions for the cluster: 

Appendix: Configuring Prometheus Endpoint

  • Prometheus needs to be started with the option –web.enable-otlp-receiver followed by –otlp-deltatocumulative 
  • Only Basic Authentication is supported for use with Kafka Client Telemetry 

Details about these options and other details for this can be found from Using Prometheus as your OpenTelemetry backend | Prometheus.

Questions

 Please contact [email protected] for any further inquiries.