By Alex Lourie Tuesday 26th June 2018

Query Idempotence in GoCql driver

Technical

We have recently helped to implement a new feature in the gocql driver for Apache Cassandra that allows a user to mark queries as idempotent. In this post we’ll cover what idempotent queries are and how they can be used.

Query Idempotence

The query is defined as idempotent if the repeated execution would render the same result each time. That is, it can be applied multiple times without changing the result of the initial operation. For example, the following query would be idempotent:

UPDATE my_table SET col=2 WHERE pk=1

On the other hand, the following query would not be idempotent:

UPDATE my_table SET col=col+1 WHERE pk=1 (as col value changes with every query)

Idempotence is important for retries and speculative queries. If the query is idempotent, it is safe and in some cases could be useful to retry it, knowing that when it succeeds, the result is the expected one.

Speculative query execution

Knowing that idempotent queries can be safely retried, we can run the same query on a multiple nodes in parallel, only waiting for the first response of success. This potentially can speed up query execution, especially if some node is busy, has network issues or is unresponsive. We can send the query to another node if the current is taking too long to respond. This way of query execution is called speculative query execution, and clearly it heavily relies on query idempotence. Unfortunately, it is not yet available in gocql driver.

As a result, we decided to implement query idempotence in the gocql driver, and then implement speculative query execution as a follow up.

Marking query as idempotent in gocql driver

In order to mark the query as idempotent with the gocql driver, pull the latest master if you haven’t for a while, and then use the following example:

For a specific query:

For all the queries for this cluster:

These examples provide the basis for the future work to implement speculative query execution in gocql driver. By starting to use the idempotency settings now you will be able to take advantage of the speculative execution functions when they are available.

Further reading:

Site by Swell Design Group