This page describes the steps that need to be taken to connect clients in one VPC to a PrivateLink enabled Kafka cluster in another VPC. In short, clients need to create an endpoint with the endpoint service name of the created PrivateLink Kafka cluster, and a Route 53 record to the endpoint created.
Retrieve Required Cluster Connection Information
Log into the console, click the created cluster and enter the Connection Info page.
The Connection Info page contains the specific connection details for PrivateLink. The endpoint created in the next section uses the endpoint service name shown below.
On the AWS VPC Endpoint Console page, click Create endpoint.
On the creation page under Service category, select Other endpoint services.
In Service Settings, paste the endpoint service name from step 2 to the Service name field. Click Verify service and a green box will appear.
For VPC, select the client’s VPC that will connect to the Kafka PrivateLink cluster. Note, the VPC of the client and Kafka PrivateLink cluster should be in the same region as we do not support inter-region access via PrivateLink.
The configuration would then be as follows.
For Subnets, please select one Subnet ID for each Availability Zone. Please note that we recommend that your clients (producers and consumers) be located in separate availability zones to support high availability should any one availability zone be unreachable.
Only select the Security Group with ic in the Group name. This is the security group managed by Instaclustr. You will get the following configurations. Once done, click Create endpoint.
You will be redirected to the main VPC endpoint page which will specifically show the created endpoint. Once the Status of the endpoint reaches Available, copy the first DNS under the DNS names.