An In-Memory Database or Data Structure Server?
Redis is a fast in-memory database and cache, open source under a BSD license, written in C and optimized for speed. Redis’ name comes from “REmote DIctionary Server”. It powers live streaming use cases and can also be used to store metadata regarding user profiles, authentication information, viewing history, manifest files for CDNs to stream videos to millions of users, and more.
Redis is often called a data structure server because its core data types are similar to those found in programming languages like strings, lists, dictionaries (or hashes), sets, and sorted sets. It also provides many other data structures and features for approximate counting, geolocation, and stream processing.
Of the NoSQL databases Redis’ various data structures take it closest to the native data structures programmers most commonly use inside applications and algorithms. This ease of use makes it ideal for rapid development and fast applications, as core data structures are easily shared between processes and services.
By default Redis stores data in memory, with periodic disk persistence as a default. As Redis persists data to disk it can serve as a classical database for many use cases as well as a cache. When full, Redis will return an error to a client, but it can be configured as a cache to eject older and less important data as new data comes in. In both cases the size of available memory is the main constraint on its use.
What Is Redis Primarily Used For?
Redis is commonly used as a cache to store frequently accessed data in memory so that applications can be responsive to users. With the capacity to designate how long you want to keep data, and which data to evict first, Redis enables a series of intelligent caching patterns.
There are many reasons for intelligent caching, and it has a large impact on user experiences, user productivity, bounce rates, and revenue for retail. The research on these effects is compiled in our white paper on the advantages of in-memory databases.
Data Expiration and Eviction Policies
Data structures in Redis can be marked with a Time To Live (TTL) set in seconds, after which they will be removed. A series of configurable intelligent “eviction policies” are available. Under these impermanent data, marked with a TTL, can be considered before other data which does not have a TTL allowing the creation of a tiered hierarchy of memory objects.
In some use cases a least recently used (LRU) or least frequently used (LFU) metric makes more sense for eviction. Redis provides tunable probabilistic implementations of these cache policy options as well.
What Are the Other Uses and Features of Redis?
The in-memory architecture of Redis is a keyspace with arbitrary in-memory objects owned by their keys. The versatile architecture of Redis has enabled it to evolve to add extra features that map to this model.
Streams and Stream Processing
Redis makes an excellent task queue and message broker with its lists and pub/sub messaging (below). In version 5.0 of Redis streams and stream processing, inspired by the Apache Kafka project, were added to its feature set. In a similar way to Kafka topics streams of work can have processing “consumer” groups which check out work and acknowledge when it has been completed. If no acknowledgement is received after a period of time has expired then other consumers can pick up this work to ensure it is done.
This enables Kafka-like patterns in-memory and is particularly useful for responsive non-blocking user interface experiences.
Publication and Subscription Messaging (Pub/Sub)
Pub/Sub messaging allows for messages to be passed to channels and for all subscribers to that channel to receive that message. This feature enables information to flow quickly through your infrastructure without using up space in the database as messages are not stored. So you can make services aware of load on other pieces of infrastructure or applications, or to update gaming scores and pass notifications.
Redis has a scripting facility which enables custom scripts to be written and executed in the Lua language. This allows users to add features to Redis themselves in the form of fast executing scripts. Lua has extremely fast initialization, enabling scripts to perform various tasks on the data without significantly slowing Redis down. As the core Redis process is single threaded this ensures atomic operations.
Redis provides a series of geospatial index data structures and commands. Latitude and longitude coordinates are stored and users can query distances between objects or query for objects within a given radius of a point. These commands are able to return their values in a variety of formats (feet, kilometers, etc.).
The speed of Redis allows these data points to be updated quickly. In a ridesharing application these features could be used to connect a user with the nearby drivers, and then provide real-time updates as they approach or during the ride. Redis is in use with major transport and delivery companies for exactly this use case.
The hyperloglog data structure enables approximate set counting in a much smaller space than keeping a full unique set of items. A simple counter can double count, and a set of user IDs or IPs would take up a large amount of space. A hyperloglog allows a very small amount of memory to hold a good approximation of the unique objects.
Bitmaps allow for the highly efficient storage of True and False values as 1 or 0 inside Redis strings. By allowing this type of Boolean data to be stored efficiently many use cases are possible.
Bitmaps can be used to efficiently store a user’s progress through some content, like an online course or a large download. Another use is to represent the online/offline status of someone’s contacts in an application.
Redis’ creator Salvatore Sanfilippo, or Antirez, originally drafted an in-memory data store in a few hundred lines of TCL for a startup. In 2009 he released a version written in C to the open source community.
In the decade since, Redis has gone through years of development and testing and real world use in enterprise, processing trillions of transactions for the largest companies and services in the world. Redis 6.0 includes SSL/TLS encrypted communication between nodes and granular Access Control Lists (ACLs) for secure deployments.
You can try out some of these features by starting our free trial of Redis and taking it for a spin.
Who’s Using Redis?
Redis is used by many websites as a cache but also by some of the largest ridesharing companies and social networks. In the 2020 StackOverflow developer survey 20.5% of developers said they were currently using Redis. In the last four years of the survey Redis was also rated as the most loved database technology by developers.
It’s worth drilling down into that last point. The way the survey defines “love” is if a developer is currently using the technology and plans to use it again. Redis consistently has the highest ratio of any database technology of people who use it, know it, and like it so much they plan to use it again in the future.
A Great Interface
Part of Redis’ attraction for developers is its simple and elegant command interface. Eric Redmond and Jim Wilson summed it up in their book Seven Databases in Seven Weeks:
It’s not simply easy to use; it’s a joy. If an API is UX for programmers, then Redis should be in the Museum of Modern Art alongside the Mac Cube.
Redis simplifies the use of common fundamental data structures between services and processes at high speed. With Lua scripts and modules Redis becomes an extensible domain-specific language (DSL) for your data.