Kafka User Management

Instaclustr provides both a User Management UI and User Management API for Kafka clusters to help you with managing users. The User Management UI and API provide four functions:

• List Users
• Add Users
• Change User Passwords
• Delete Users

All users created or listed in the Users list will have read and write access to all topics in the cluster. There are details below on how to change this in Kafka ACL Management. If you require any other configuration, please submit a support request.

All Kafka clusters will have an ickafka user automatically created as a convenience, which can be deleted. If the ickafka user is deleted or has the password changed the new password will not be stored and the credentials will disappear from the cluster’s Connection Info page. Because changed passwords and new user passwords will not be stored, ensure you make a note of the passwords to user accounts.

User Management UI

These examples show how to use the features of the Instaclustr console’s User Management UI.

List Users

1. To list available Kafka users, first select your Kafka cluster in the Instaclustr console. With your Kafka cluster selected, click Users from the Kafka menu.
   
2. The Users page will list all Kafka users on the cluster.
   

Add User

1. To add a new Kafka user, first select your Kafka cluster in the Instaclustr console. With your Kafka cluster selected, click Users from the Kafka menu.
   
2. From the Users page, click the New User… button. You will be redirected to the New User page.
   
   
3. On the New User page, enter the desired username and password for the new Kafka user, the desired level of initial permissions, and your Instaclustr password to confirm your identity. The initial permissions setting can be one of three things:
   • Standard – The new user will be able to read and write to all topics
   • Read-only – The new user will be able to read from all topics, but not write
   • None – The new user will have no initial permissions
4. Once you’ve entered the required information, click the Create User button. If your Instaclustr password is correct and your username and password are valid, your new user will be created.
   
5. Once the user is created, you will be redirected to the Users page, with a success message displayed at the top of the page. Your new user will be shown in the list of users.
   
   

Change User Password

1. To change the password of an existing Kafka user, first select your Kafka cluster in the Instaclustr console. With your Kafka cluster selected, click Users from the Kafka menu.
   
2. From the Users page, click the Change Password… button of the user whose password you want to change. You will be redirected to the Change User Password page.
   
   
3. On the Change User Password page, enter the new password for the User, and your Instaclustr password to confirm your identity. Once you’ve entered the required information, click the Change Password button.
   
   
4. After clicking the Change Password button you will be redirected to the Users page, with a success message displayed at the top of the page.
   
   

Delete User

1. To delete an existing Kafka user, first select your Kafka cluster in the Instaclustr console. With your Kafka cluster selected, click Users from the Kafka menu.
   
   
2. From the Users page, click the Delete User… button of the user whose password you want to change. You will be redirected to the Delete User page.
   
   
3. On the Delete User page, enter your Instaclustr password to confirm your identity. Once you’ve entered your password, click the Delete User button to delete the user.
   
4. If the password was entered correctly and validated, you will be redirected to the Users page, with a success message displayed at the top of the page. The deleted user will no longer be shown in the list of users.
   

User Management API

These examples show how to use the features of the Instaclustr User Management API.

To provide customers with a high level of security, the Kafka User API endpoint will not store or allow customers to retrieve passwords for Kafka users.
For each endpoint listed below, all requests must include basic authentication details:

List Users

1. To retrieve a list of users currently enabled in the Kafka cluster, make a GET request to:
   https://api.instaclustr.com/provisioning/v1/<cluster_id>/kafka/users
2. The API should respond with a 200 status code and a JSON list of users. E.g.:
   
   XHTML

   [ "ickafka" ]

   1 2 3

   [   "ickafka" ]

Add User

1. To add a new user to the Kafka cluster (with read/write access to all topics), make a POST request to:
   https://api.instaclustr.com/provisioning/v1/<cluster_id>/kafka/users
   A request body should be included, specifying the username and password, and initial permissions to use. E.g.:
   
   { "username": "test1", "password": "Test1Test1", "initial-permissions": "standard" }

   1 2 3 4 5

   {   "username": "test1",   "password": "Test1Test1",   "initial-permissions": "standard" }

   
   Valid values for “initial-permissions” are : “standard”, “read-only”, “none”.
2. The API should respond with a 201 status code and a short message. E.g.
   
   { "message": "User test1 created." }

   1 2 3

   {   "message": "User test1 created." }

   
   Once this message is received, the user should be immediately available for use when connecting to the Kafka cluster.

Change User Password

1. To change an existing user’s password, make a POST request to:
   https://api.instaclustr.com/provisioning/v1/<cluster_id>/kafka/users/reset-password
   A request body should be included, specifying the username and password to apply. E.g.:
   
   { "username": "test1", "password": "Test2Test2" }

   1 2 3 4

   {   "username": "test1",   "password": "Test2Test2" }

2. The API should respond with a 200 status code and a short message. E.g.:
   
   { "message": "Updated password for user test1." }

   1 2 3

   {   "message": "Updated password for user test1." }

Delete User

1. To delete a Kafka user, make a DELETE request to:
   https://api.instaclustr.com/provisioning/v1/<cluster_id>/kafka/users
   A request body should be included, specifying the username and password to use. E.g.:
   
   { "username": "test1" }

   1 2 3

   {   "username": "test1" }

2. The API should respond with a 201 status code and a short message. E.g.
   
   { "message": "User test1 has been deleted." }

   1 2 3

   {   "message": "User test1 has been deleted." }

Kafka ACL Management

Warning: It is not recommended that you use the Kafka ACLs to grant higher privileges than the default given by Instaclustr. This could result in your cluster being unrecoverable and not covered under Instaclustr’s SLA’s. If you require any other configuration, please submit a support request before making any changes beyond those outlined below.

Kafka has built in Access Control Lists so that you can control what users and hosts are allowed to do. Instaclustr has enabled modification of these consoles. Currently this accessed through the Kafka CLI.

The default ickafka user is given the ability to modify ACLs by default. If this user is deleted before you have made the required ACL changes please contact the Technical Operations team and they will be able to recover this for you.

Set Up the Kafka CLI

Instructions for setting up the Kafka CLI can be found on the Connection Info page after you have set up a Kafka cluster 

List the ACLs

1. Once you have set up the kafka.properties file as described above you  will be able to list the ACLs.
2. In the bin directory of your kafka cli run:
   
   ./kafka-acls.sh --bootstrap-server public_ip_of_kafka_node:9092 --command-config kafka.properties --list

   1	./kafka-acls.sh --bootstrap-server public_ip_of_kafka_node:9092 --command-config kafka.properties --list

3. This should return a list of ACLs for all users
   
   Current ACLs for resource 'ResourcePattern(resourceType=TRANSACTIONAL_ID, name=*,patternType=LITERAL)': (principal=User:testuser, host=*, operation=ALL, permissionType=ALLOW) Current ACLs for resource `ResourcePattern(resourceType=CLUSTER, name=kafka-cluster, patternType=LITERAL)`: (principal=User:testuser, host=*, operation=CLUSTER_ACTION, permissionType=DENY) (principal=User:testuser, host=*, operation=DESCRIBE_CONFIGS, permissionType=ALLOW) (principal=User:testuser, host=*, operation=IDEMPOTENT_WRITE, permissionType=ALLOW) (principal=User:testuser, host=*, operation=CREATE, permissionType=ALLOW) (principal=User:testuser, host=*, operation=ALTER_CONFIGS, permissionType=DENY) (principal=User:testuser, host=*, operation=DESCRIBE, permissionType=ALLOW) (principal=User:testuser, host=*, operation=ALTER, permissionType=DENY) Current ACLs for resource `ResourcePattern(resourceType=TOPIC, name=*, patternType=LITERAL)`: (principal=User:testuser, host=*, operation=ALL, permissionType=ALLOW) Current ACLs for resource `ResourcePattern(resourceType=GROUP, name=*, patternType=LITERAL)`: (principal=User:testuser, host=*, operation=ALL, permissionType=ALLOW)

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

   Current ACLs for resource 'ResourcePattern(resourceType=TRANSACTIONAL_ID, name=*,patternType=LITERAL)': (principal=User:testuser, host=*, operation=ALL, permissionType=ALLOW) Current ACLs for resource `ResourcePattern(resourceType=CLUSTER, name=kafka-cluster, patternType=LITERAL)`: (principal=User:testuser, host=*, operation=CLUSTER_ACTION, permissionType=DENY) (principal=User:testuser, host=*, operation=DESCRIBE_CONFIGS, permissionType=ALLOW) (principal=User:testuser, host=*, operation=IDEMPOTENT_WRITE, permissionType=ALLOW) (principal=User:testuser, host=*, operation=CREATE, permissionType=ALLOW) (principal=User:testuser, host=*, operation=ALTER_CONFIGS, permissionType=DENY) (principal=User:testuser, host=*, operation=DESCRIBE, permissionType=ALLOW) (principal=User:testuser, host=*, operation=ALTER, permissionType=DENY)   Current ACLs for resource `ResourcePattern(resourceType=TOPIC, name=*, patternType=LITERAL)`: (principal=User:testuser, host=*, operation=ALL, permissionType=ALLOW)   Current ACLs for resource `ResourcePattern(resourceType=GROUP, name=*, patternType=LITERAL)`: (principal=User:testuser, host=*, operation=ALL, permissionType=ALLOW)

4. Results can be further filtered by user or host, for example:
   
   
   ./kafka-acls.sh --bootstrap-server public_ip_of_kafka_node:9092 --command-config kafka.properties --list --principal User:ickafka

   1	./kafka-acls.sh --bootstrap-server public_ip_of_kafka_node:9092 --command-config kafka.properties --list --principal User:ickafka

5. The ACLs first list the ResourceGroup, then underneath each line lists User:Operation

Alter the ACLs for Users

Instaclustr by default gives all users created through the dashboard or the API the right to produce or consume from any topic.

Kafka ACLs work on the principal that Deny takes precedence over Allow. So if a user has both Deny and Allow to the one operation on the one resource group, that user will be denied.

Therefore, when considering altering the default Instaclustr Kafka ACLs, do not think about allowing users access to resources, think in the mindset of denying them, as by default they are allowed.

Note: Kafka also assumes that unless there is a specific Allow ACL then the default is Deny.

Altering Kafka User ACLs

Examples of adding ACLs for Kafka users below:

Examples of removing ACLs for Kafka users below:

Host ACLs

Kafka ACLs also allow restrictions from hosts with the –allow-host or –deny-host flags. Note: IP addresses only supported. Hostnames are not supported.

Additional ACL Options

The kafka-acls.sh tool provides some convenience flags for the most common requirements.  These are as follows.

Different Resource Groups

The examples given so far have been limited to users and topics. However, Kafka can have ACLs added at the cluster or group level also. You can also permit more broadly by specifying * on a resource group.

You can also add ACLs on prefixed resource patterns, the default resource pattern is LITERAL, which performs exact name matching. However you could do something similar to add an ACL to all topics starting with Instaclustr