Using external cluster resources

Getting started

A Cadence cluster requires an Apache Cassandra® cluster to connect to for its persistence layer.

If you would like to take advantage of Advanced Visibility we additionally need an Apache Kafka® and OpenSearch or Elasticsearch cluster.

The following guide will explain how to configure your Cadence cluster to use these various external clusters by providing connection information when creating your Cadence cluster. 

In the future, Instaclustr plans to support linking your existing Instaclustr Managed Apache Kafka® and OpenSearch clusters to create a Cadence cluster.

If you just want to use Cadence with Apache Cassandra®, see the Creating a Cadence Cluster guide, which simplifies the process by taking full advantage of the Instaclustr Managed Service.

Note : Free trial customers are restricted to a specific number of developer nodes and are not be able to create all prerequisite clusters needed for Advanced Visibility.

Step 1 – Configure your external clusters

For advanced visibility you will require the following clusters:

Note : You can use Instaclustr’s managed service to provision the prerequisites (i.e. Apache Cassandra®, Elasticsearch, Apache Kafka® clusters)

  1. Apache Cassandra
    • You have an option to specify a pre-existing Cassandra cluster within Instaclustr’s managed service or selecting an external Cassandra cluster
    • If you want to use an external Cassandra cluster, you need:
      • Version 3.11.x
      • Password authentication: Enabled or Disabled
        • If Password authentication is enabled, you will need to provide a role that Cadence will use to connect and can create the keyspaces it requires.
      • Client Encryption (TLS) : Disabled, not currently supported for external clusters
      • RPC Broadcast: Public only
  2. Apache Kafka
    • Version 2.x is supported
    • Password authentication: Enabled or Disabled
      • If Password authentication is enabled, you will need to provide the user that Cadence will use to connect
    • Client Encryption (TLS) : Disabled, not currently supported for external clusters
  3. Elasticsearch or OpenSearch 
    • Elasticsearch
      • Elasticsearch version 7.x.x is supported or OpenDistro version 1.11 or 1.8 
      • iPassword authentication: Enabled or Disabled
        • If Password authentication is enabled, you will need to provide the user that Cadence will use to connect
      • Security Pack: Disabled, not currently supported for external clusters
    • OpenSearch
      • Version 1.0.x is supported
      • Password authentication: Enabled or Disabled
        • If Password authentication is enabled, you will need to provide the user that Cadence will use to connect
      • Security Pack: Disabled, not currently supported for external clusters
  4. In addition to creating these clusters, we will need to have some information about these clusters in the following section.

Step 2: Create a Cadence cluster

Now that we have prepared our required clusters, we can create a Cadence cluster and configure it to use them.

  1. Log into the Instaclustr Console and click the Create Cluster button. When prompted, select the Cadence application.
  2. On the Cadence Setup page, we can select the version of Cadence. We are currently only offering Cadence 0.22.4.
    Make sure to enable the Advanced Visibility option and continue.

  3. On the Cadence Dependency Setup page we can select a running Instaclustr Apache Cassandra cluster to link to. For this exercise, we will select Other Cassandra Cluster from the drop down, which will open up the following fields:
    Cassandra Configuration

    • Cassandra Seed Nodes
      A comma separated list IP addresses to use as contact points, it doesn’t have to be every node in the cluster as the client will discover the rest of them.
    • Cassandra Port
      The port to use when contacting your Apache Cassandra cluster.
    • Cassandra DC Name
      The name of the Data Center we want Cadence to connect to. 
    • Cassandra Username
      If password authentication is enabled, enter the username and password. Otherwise leave it blank.
    • Cassandra Password
      If password authentication is enabled, enter the username and password. Otherwise leave it blank.
    • Cassandra Default Keyspace
      The name of the default keyspace that will be created for this Cadence cluster, leave it as the default if it doesn’t matter.
    • Cassandra Visibility Keyspace
      The name of the visibility keyspace that will be created for this Cadence cluster, leave it as the default if it doesn’t matter.
  • Elasticsearch / OpenSearch Configuration
    • Elasticsearch Hosts
      A comma separated list IP addresses to use as contact points, it doesn’t have to be every node in the cluster as the client will discover the rest of them.
    • Elasticsearch Port
      The port to use when contacting your Elasticsearch cluster.
    • Elasticsearch Username
      If password authentication is enabled, enter the username and password. Otherwise leave it blank.
    • Elasticsearch Password
      If password authentication is enabled, enter the username and password. Otherwise leave it blank.
    • Elasticsearch Visibility Index
      The name of the visibility index that will be created for this Cadence cluster, leave it as the default if it doesn’t matter.
  1. Kafka Configuration
    • Kafka Brokers
      A comma separated list IP addresses to use as contact points, it doesn’t have to be every node in the cluster as the client will discover the rest of them.
    • Kafka Port
      The port to use when contacting your Apache Kafka cluster.
    • Kafka Username
      If password authentication is enabled, enter the username and password. Otherwise leave it blank.
    • Kafka Password
      If password authentication is enabled, enter the username and password. Otherwise leave it blank.
    • Kafka SASL Algorithm
      If password authentication is enabled, enter the SASL algorithm that is used. Otherwise leave it blank.
    • Kafka Visibility Topic
      The name of the visibility index that will be created for this Cadence cluster, leave it as the default if it doesn’t matter.
    • Kafka Visibility DQL Topic
      The name of the visibility topic that will be created for this Cadence cluster, leave it as the default if it doesn’t matter.
  2. Under the Data Centre section
    • Select a Data Centre Region, 
    • A Cluster Network address block, 
    • The number of nodes for your application, 
    • Your desired node size.

  1. The Confirmation section displays a brief summary of your cluster configurations and pricing details.
    If you are happy with the cluster configuration and have accepted the terms and conditions, click the Create Cluster button to start provisioning the cluster.

  2. Instaclustr’s provisioning system will begin to create the Cadence cluster instances. In the Cluster Details page we can monitor the progress of the provisioning steps.
    Once all the nodes enter the Provisioned status, there are a few more things we need to do.

    • Note : If your Apache Cassandra®, Apache Kafka® or OpenSearch/Elasticsearch clusters are behind firewalls, you will need to grant access for each of the Cadence node IP addresses.
      Until they have access, the Cadence cluster will not reach the Running state.

  3. You have now finished creating your Cadence cluster! That was a lot of information we needed to create a Cadence cluster with Advanced Visibility wasn’t it?
    As mentioned earlier, Instaclustr is working on streamlining the process when you combine Instaclustr managed Apache Kafka® and Opensearch clusters to make this a quicker experience.

Please contact [email protected] if you have any questions.

 

 

By Instaclustr Support
Need Support?
Experiencing difficulties on the website or console?
Already have an account?
Need help with your cluster?
Contact Support
Why sign up?
To experience the ease of creating and managing clusters via the Instaclustr Console
Spin up a cluster in minutes