Custom Kafka Connect Connectors

Menu

In this document we will look at how to configure and provide custom connectors for your Instaclustr managed Kafka Connect cluster.

Using an AWS S3 bucket as your custom connector storage

In this example we will be utilizing an AWS CloudFormation template to provision the required S3 bucket, Access Policy ,User and User access keys required to configure the S3 custom connector storage.

  1. Create a CloudFormation template file instaclustr_custom_connectors with the below content.
  2. Please log into you AWS console and access https://console.aws.amazon.com/cloudformation
  3. Click on the Create Stack dropdown and select With new resources option
  4. In the create stack view select  Template is Ready for the Prerequisite – Prepare template section
  5. In the create stack view select Upload a template file for the Specify template section
  6. Upload the previously created instaclustr_custom_connectors CloudFormation template file. And enter the resource prefix that would be used for naming the resources created
  7. You can follow the rest of the create stack wizard according to the requirements of your account
  8. When the stack creation is complete you can access the S3 bucket name, the created user access key and secret key required to configure the custom connectors with instaclustr in the CloudFormation stack output view
  9. Once you’ve set up a Kafka Connect cluster and an S3 bucket, you can upload connectors to your bucket, see Updating custom connectors for more details

FREE TRIAL

Spin up a cluster in less
than 5 minutes.
(No credit card required)

Sign Up Now
Close

Site by Swell Design Group