Instaclustr
Search Toggle search form
Close
FREE TRIAL: Spin up a cluster in less than 5 minutes - No Credit Card Required
Support Contact us Log In Free trial

Support

Menu
ConsoleStatusOpen Ticket

Kafka User Management

Instaclustr provides both a User Management UI and User Management API for Kafka clusters to help you with managing users. The User Management UI and API provide four functions:

  • List Users
  • Add Users
  • Change User Passwords
  • Delete Users

All Kafka users created or listed in the Users list will have read and write access to all topics in the cluster. There are details below on how to change this in Kafka ACL Management. If you require any other configuration, please submit a support request.

All Kafka clusters will have an ickafka user automatically created as a convenience, which can be deleted. If the ickafka user is deleted or has the password changed the new password will not be stored and the credentials will disappear from the cluster’s Connection Info page. Because changed passwords and new user passwords will not be stored, ensure you make a note of the passwords to user accounts.

For Kafka Schema Registry and Kafka Rest Proxy services the user password can be updated from the dashboard kafka user management UI. We do not support user deletion or creation for these services. Make sure you take into account that the services will be restarted when actioning a password change.

User Management UI

These examples show how to use the features of the Instaclustr console’s User Management UI.

List Users

  1. To list available Kafka users, first select your Kafka cluster in the Instaclustr console. With your Kafka cluster selected, click Users from the Kafka menu.
  2. The Users page will list all Kafka users on the cluster. If you have enabled Kafka Schema Registry and/or Kafka Rest Proxy, you will see the default users for those services listed here as well.

Add User

  1. To add a new Kafka user, first select your Kafka cluster in the Instaclustr console. With your Kafka cluster selected, click Users from the Kafka menu.
  2. From the Users page, click the New User button. You will be redirected to the New User page.

  3. On the New User page, enter the desired username and password for the new Kafka user, the desired level of initial permissions, and your Instaclustr password to confirm your identity. The initial permissions setting can be one of three things:
    • Standard – The new user will be able to read and write to all topics
    • Read-only – The new user will be able to read from all topics, but not write
    • None – The new user will have no initial permissions
  4. Once you’ve entered the required information, click the Create User button. If your Instaclustr password is correct and your username and password are valid, your new user will be created.
  5. Once the user is created, you will be redirected to the Users page, with a success message displayed at the top of the page. Your new user will be shown in the list of users.

Change User Password

  1. To change the password of an existing Kafka user, first select your Kafka cluster in the Instaclustr console. With your Kafka cluster selected, click Users from the Kafka menu.
  2. From the Users page, click the Change Password button of the user whose password you want to change. You will be redirected to the Change User Password page.
  3. On the Change User Password page, enter the new password for the User, and your Instaclustr password to confirm your identity. Once you’ve entered the required information, click the Change Password button.

  4. After clicking the Change Password button you will be redirected to the Users page, with a success message displayed at the top of the page.

Delete User

  1. To delete an existing Kafka user, first select your Kafka cluster in the Instaclustr console. With your Kafka cluster selected, click Users from the Kafka menu.

  2. From the Users page, click the Delete User button of the user whose password you want to change. You will be redirected to the Delete User page.

  3. On the Delete User page, enter your Instaclustr password to confirm your identity. Once you’ve entered your password, click the Delete User button to delete the user.
  4. If the password was entered correctly and validated, you will be redirected to the Users page, with a success message displayed at the top of the page. The deleted user will no longer be shown in the list of users.

User Management API

These examples show how to use the features of the Instaclustr User Management API.

Please note that we currently do not support Kafka schema registry and Kafka rest proxy with add user and delete user functions.

To provide customers with a high level of security, the Kafka User API endpoint will not store or allow customers to retrieve passwords for Kafka users.
For each endpoint listed below, all requests must include basic authentication details:

List Users

  1. To retrieve a list of users currently enabled in the Kafka cluster, make a GET request to:
    https://api.instaclustr.com/provisioning/v1/<cluster_id>/kafka/users
  2. The API should respond with a 200 status code and a JSON list of users. E.g.:
  3. To list the users of Kafka rest proxy or Kafka schema registry, use the bundle name as kafka_rest_proxy or kafka_schema_registry respectively in the GET request. E.g.:

Add User

  1. To add a new user to the Kafka cluster (with read/write access to all topics), make a POST request to:
    https://api.instaclustr.com/provisioning/v1/<cluster_id>/kafka/users
    A request body should be included, specifying the username and password, and initial permissions to use. E.g.:

    Valid values for “initial-permissions” are : “standard”, “read-only”, “none”.
  2. The API should respond with a 201 status code and a short message. E.g.

    Once this message is received, the user should be immediately available for use when connecting to the Kafka cluster.

Change User Password

  1. To change an existing user’s password, make a POST request to:
    https://api.instaclustr.com/provisioning/v1/<cluster_id>/kafka/users/reset-password
    A request body should be included, specifying the username and password to apply. E.g.:
  2. The API should respond with a 200 status code and a short message. E.g.:
  3. To change the user passwords of Kafka rest proxy or Kafka schema registry, use the bundle name as kafka_rest_proxy or kafka_schema_registry respectively in the POST request. E.g.:

Delete User

  1. To delete a Kafka user, make a DELETE request to:
    https://api.instaclustr.com/provisioning/v1/<cluster_id>/kafka/users
    A request body should be included, specifying the username and password to use. E.g.:
  2. The API should respond with a 200 status code and a short message. E.g.

Kafka ACL Management

Warning: It is not recommended that you use the Kafka ACLs to grant higher privileges than the default given by Instaclustr. This could result in your cluster being unrecoverable and not covered under Instaclustr’s SLA’s. If you require any other configuration, please submit a support request before making any changes beyond those outlined below.

Kafka has built in Access Control Lists so that you can control what users and hosts are allowed to do. Instaclustr has enabled modification of these consoles. Currently this accessed through the Kafka CLI.

The default ickafka user is given the ability to modify ACLs by default. If this user is deleted before you have made the required ACL changes please contact the Technical Operations team and they will be able to recover this for you.

Set Up the Kafka CLI

Instructions for setting up the Kafka CLI can be found on the Connection Info page after you have set up a Kafka cluster 

List the ACLs

  1. Once you have set up the kafka.properties file as described above you  will be able to list the ACLs.
  2. In the bin directory of your kafka cli run:
  3. This should return a list of ACLs for all users
  4. Results can be further filtered by user or host, for example:

  5. The ACLs first list the ResourceGroup, then underneath each line lists User:Operation

Alter the ACLs for Users

Instaclustr by default gives all users created through the dashboard or the API the right to produce or consume from any topic.

Kafka ACLs work on the principal that Deny takes precedence over Allow. So if a user has both Deny and Allow to the one operation on the one resource group, that user will be denied.

Therefore, when considering altering the default Instaclustr Kafka ACLs, do not think about allowing users access to resources, think in the mindset of denying them, as by default they are allowed.

Note: Kafka also assumes that unless there is a specific Allow ACL then the default is Deny.

Altering Kafka User ACLs

Examples of adding ACLs for Kafka users below:

Examples of removing ACLs for Kafka users below:

Host ACLs

Kafka ACLs also allow restrictions from hosts with the –allow-host or –deny-host flags. Note: IP addresses only supported. Hostnames are not supported.

Additional ACL Options

The kafka-acls.sh tool provides some convenience flags for the most common requirements.  These are as follows.

–producerConvenience option to add/remove ACLs for producer role. This will generate ACLs that allows WRITE, DESCRIBE and CREATE on topic.
–consumerConvenience option to add/remove ACLs for consumer role. This will generate ACLs that allows READ, DESCRIBE on topic and READ on consumer-group.

Different Resource Groups

The examples given so far have been limited to users and topics. However, Kafka can have ACLs added at the cluster or group level also. You can also permit more broadly by specifying * on a resource group.

You can also add ACLs on prefixed resource patterns, the default resource pattern is LITERAL, which performs exact name matching. However you could do something similar to add an ACL to all topics starting with Instaclustr

By Subhani Munasinghe Monday 23rd July 2018

Submit Comment

Your email address will not be published. Required fields are marked *

×

Need Support
Learn More

Already have an account?
Login to the Console

Experiencing difficulties on the website or console?
Status page for known incidents

Don’t have an account yet?
Sign up for a free trial

Why sign up?
To experience the ease of creating and managing clusters via the Instaclustr Console.