Kafka Connect Metrics

Menu

Authentication

All requests to the API must use Basic Authentication and contain a valid username and monitoring API key. API keys are created per user account and can be retrieved via the Instaclustr Console from the Account > API Key tab.

Requests

Metrics are requested by constructing a GET request, consisting of:

  • type: Either ‘clusters’, ‘datacentres’ or ‘nodes’.
    • ‘clusters’ returns the metrics for each node in the cluster.
    • ‘datacentres’ returns the metrics for each node belonging to the datacenter.
    • ‘nodes’ returns the metrics for a specific node.
  • UUID or public IP: If the type is set to ‘clusters’ or ‘datacentres’, then the UUID of cluster or datacentre must be specified. However, if the type is set to ‘nodes’, then either the nodes’ UUID or public IP may be specified.
  • metrics: The metrics to return are specified as a comma-delimited query string parameter. Up to 20 metrics may be specified.
  • reportNaN: (true|false) If a metric value is NaN or null, reportNaN determines whether API should report it as NaN. The default behaviour is false and NaN and null will be reported as 0. Setting ‘reportNaN=true’ will return NaN values in the API response.

Metrics

Kafka Connect specific metrics in the monitoring API begin with the kc:: prefix, ie. kc::connectorCount.

The currently available metrics are:

  • kc::taskCount: Number of tasks currently assigned to each worker node
  • kc::connectorCount: Number of connectors currently assigned to each worker node
  • kc::connectorStartupAttemptsTotal: Number of times a connector has been instructed to start on each worker node
  • kc::connectorStartupFailurePercentage: Percentage of connecter start-up attempts that have failed to complete
  • kc::connectorStartupFailureTotal: Number of times a connector has been instructed to start and failed to do so
  • kc::connectorStartupSuccessPercentage: Percentage of connecter start-up attempts that have successfully completed
  • kc::connectorStartupSuccessTotal: Number of times a connector has been instructed to start and has succeeded in doing so
  • kc::taskStartupAttemptsTotal: Number of times a task has been instructed to start on each worker node
  • kc::taskStartupFailurePercentage: Percentage of task start-up attempts that have failed to complete
  • kc::taskStartupFailureTotal: Number of times a task has been instructed to start and failed to do so
  • kc::taskStartupSuccessPercentage: Percentage of task start-up attempts that have successfully completed
  • kc::taskStartupSuccessTotal: Number of times a task has been instructed to start and has succeeded in doing so
  • kc::leaderName: Identity of the current leader worker node. Typically this is the IP address of the leader.
  • kc::completedRebalancesTotal: Number of rebalances that have completed since Kafka Connect has started (per node)
  • kc::epoch: Monotonically increasing number that indicates the current state of assigned tasks. Will increase by one for each completed rebalance
  • kc::timeSinceLastRebalanceMs: Time since the last successful rebalance that each node participated in (per node, in milliseconds)
  • kc::rebalanceAvgTimeMs: The average time each rebalance has taken to complete (per node, in milliseconds)
  • kc::rebalanceMaxTimeMs: The maximum time each rebalance has taken to complete (per node, in milliseconds)
  • kc::rebalancing: Whether or not the worked is currently rebalancing (per node)

Example: Endpoint to return the time since the last rebalance and whether or not the nodes are currently rebalancing for every node in the cluster  with a UUID of 7b58eae9-2b72-420a-a544-32a404b70fd7.

Response:

FREE TRIAL

Spin up a cluster in less
than 5 minutes.
(No credit card required)

Sign Up Now
Close

Site by Swell Design Group