Using the Instaclustr Monitoring API with Data Dog
Instaclustr’s monitoring API is designed to allow you to integrate the monitoring information from your Instaclustr managed cluster with the monitoring tool used for the entire application. DataDog (datadoghq.com) is a popular platform for monitoring a range of applications. This help article walks you through how to use the Instaclustr Monitoring API with DataDog.
At a high-level, the approach we will take in this article is to install a script on a server you manage that has the DataDog agent installed. This script calls the Instaclustr Monitoring API at regular intervals and passes the information returned to the DataDog agent which reports it to the central DataDog system.
Install the DataDog DogStatsD API package (pip install datadog).
Set Up The Script
We have created a sample script that calls the Instaclustr API and forwards the data to DataDog. The script is available on GitHub here: https://github.com/instaclustr/ICAPI-DataDog. You can download the ZIP file or clone the repository.
One of our customers has extended our open source script above with some enhancements. Depending on your requirements, you may use that script instead. The script is available on GitHub here: https://github.com/lendi-au/ICAPI-DataDog.
The script (ic2datadog.py) is fairly straightforward. It retrieves a specified list of metrics for all nodes in the cluster and requires a configuration file (called configuration.json) in the format shown below:
dd_options: Your DataDog API key and Application key. Available from Integrations/APIs in the DataDog console (you may need to create a new app key).
ic_options: Your Instaclustr user name and API key. These can be created under the Account/API Keys tab of the Instaclustr console. Make sure you copy them as they will only be displayed once.
Run The Script and View The Results
Running the script is a simple matter of ‘python ic2datadog.py’. The script will then run until interrupted.After a minute or so of running, the metrics will be visible in DataDog.
You can see the results by logging into the DataDog console:
To view the gauge metrics (e.g. CPU):
Go to Metrics/Explorer
In the Graph text box, start typing ‘instaclustr’. You should see a list of available metrics appear in the format “Instaclustr.[Node IP].[metric name]. Choose the metrics you want and DataDog will draw you a graph.
To view the node status information:
Go to Monitors/Check Summary
You should see the Instaclustr node status checks in the list (filter for “Instaclustr” if necessary).
The Instaclustr metrics are now available to use wherever else you would use them in DataDog (dashboards, monitors, etc).