Using non-Instaclustr Kafka Clusters
In this article we’ll show how to set up an Instaclustr Kafka Connect cluster that targets a non-Instaclustr Kafka cluster via the Instaclustr console. For information on how to provision an Apache Kafka Cluster via the Instaclustr Provisioning API, see this page: Instaclustr Provisioning API
- Follow the steps in the creation guide for Kafka Connect clusters, except for the following differences when filling out the Kafka Connect Options section.
- Select the ‘Other Cluster’ option, and put in the connection related properties that you will need to insert into the Kafka Connect worker.properties file. Note that in this interface only bootstrap.servers and connection related security options are allowed. Not every valid entry in worker.properties will be allowed here.
- If security protocol is implemented on your Kafka cluster, you would need to create a Kafka user, information on Kafka ACL permission can be found here. Here is an example of ACL permissions for the Kafka user:
Domain Action Topic ALL Group ALL Transactional ID ALL Cluster CREATE Cluster DESCRIBE_CONFIGS Cluster DESCRIBE Cluster IDEMPOTENT_WRITE
- Additionally, if you are using an SSL/TLS connection to your Kafka cluster, you will need to supply a base64 encoded version of a JKS trust store containing your Kafka server certificate. Also note that we do not support setting up client authentication via SSL/TLS.