Skip to main content

Indexing and Querying with Redis

Profile picture for Simon Prickett
Author:
Simon Prickett, Principal Developer Advocate at Redis

We chose to store our user and location data in Redis Hashes. Hashes are a great fit for storing domain objects. Recall that we've chosen to store each user in a Hash whose key contains the user ID. For example, here's user 852 as seen in RedisInsight:

User 852 viewed with RedisInsight

If you're using redis-cli, you can look at user 852 with the HGETALL command:

127.0.0.1:6379> hgetall ncc:users:852
 1) "id"
 2) "852"
 3) "firstName"
 4) "Dominik"
 5) "lastName"
 6) "Schiffmann"
 7) "email"
 8) "dominik.schiffmann@example.com"
 9) "password"
10) "$2b$05$xbkSwODz1tWqdE7xWb393eiYIQcdiEdbbvhK88.Xr9sW7WxdI26qi"
11) "numCheckins"
12) "9353"
13) "lastCheckin"
14) "1488517098363"
15) "lastSeenAt"
16) "124"

Storing data in Hashes means that we can easily and efficiently retrieve the contents of the Hash, provided that we know the key. So it's trivial to look up user 852, but how can we perform any of the following operations?

  • Get the user whose email address is dominik.schiffmann@example.com.
  • Find all users that were last seen at location 124.
  • Find all the users who have between 1000 and 3000 checkins.
  • Find all locations within a 10 mile radius of a given latitude / longitude coordinate and which have at least a 3 star rating.

Redis is a key/value database. This means that its data model is optimized for retrieval by key. The queries above can't be resolved by knowing just the Hash key - we need some other mechanism to index our data.

Traditionally in a key/value database, this has meant adding code to create and manually update indexes. For example to resolve the query "which user has the email address dominik.schiffmann@example.com", we might create a new String key containing that email address, with the value being the user's ID:

127.0.0.1:6379> set ncc:users:byemail:dominik.schiffmann@example.com 852
OK

Now, if we want to get Dominik's user details given only his email address, we have a two step process to follow:

  1. Look up the user ID for the user associated with the email address we have.
  2. Use that user ID to retrieve the values from the user's Hash.
127.0.0.1:6379> get ncc:users:byemail:dominik.schiffmann@example.com
"852"
127.0.0.1:6379> hgetall ncc:users:852
 1) "id"
 2) "852"
 3) "firstName"
 4) "Dominik"
 5) "lastName"
 6) "Schiffmann"
 7) "email"
 8) "dominik.schiffmann@example.com"
 9) "password"
10) "$2b$05$xbkSwODz1tWqdE7xWb393eiYIQcdiEdbbvhK88.Xr9sW7WxdI26qi"
11) "numCheckins"
12) "9353"
13) "lastCheckin"
14) "1488517098363"
15) "lastSeenAt"
16) "124"

We'd also need to keep this information up to date and in sync with changes to the Hash at ncc:users:852 ourselves in our application code.

Other sorts of secondary indexes can be created using other Redis data types. For example, we might use a Redis Sorted Set as a secondary index, allowing us to perform range queries such as "Find all the users who have between 1000 and 3000 checkins". Again, we'd have to populate and maintain this extra data structure ourselves in the application code.

Redis Stack solves all of these problems for us and more. It adds an indexing, querying and full-text search engine to Redis that automatically keeps track of changes to data in indexed Hashes. Redis Stack provides a flexible query language to answer questions such as "Find me all the gyms with at least a 3 star rating and more than 200 checkins within 10 miles of Oakland, California" without adding code to build or maintain secondary data structures in our application.

Watch the video to see how Redis Stack is used in our example Node.js application.

Coding Exercise​

In this exercise, you'll finish implementing a route that uses Redis to return all users whose last checkin was at a given location.

Open the node-js-crash-course folder with your IDE, and find the file src/routes/user_routes.js.

In this file, you'll see a partly implemented route /users/at/:locationId. To complete this exercise, you'll need to replace this line:

const searchResults = await redis.performSearch(
  redis.getKeyName('usersidx'),
  'TODO... YOUR QUERY HERE',
);

with one containing the correct query to return users whose "lastSeenAt" field is set to the value of locationId. You'll need to use the "numeric range" syntax for this, as the "lastSeenAt" field was indexed as a number. Be sure to check out the Query Syntax documentation for Redis to get help with this.

To try your code, ensure that the API Server component is running:

$ npm run dev

(remember, this will use nodemon to restart the server any time you save a code change).

Then, point your browser at http://localhost:8081/api/users/at/33. If your query is correct, you should see output similar to the following (actual users may differ, just ensure that the value of lastSeenAt for each matches the location ID you provided - 33 in this case):

[
  {
    "id": "238",
    "firstName": "Jonas",
    "lastName": "Nielsen",
    "numCheckins": "7149",
    "lastCheckin": "1515248028256",
    "lastSeenAt": "33"
  },
  {
    "id": "324",
    "firstName": "Frans",
    "lastName": "Potze",
    "numCheckins": "8623",
    "lastCheckin": "1515976232073",
    "lastSeenAt": "33"
  },
  ...
]

To help you develop your query, use one of the guides in RedisInsight workbench, or read more about the FT.SEARCH command.

External Resources​

Querying, Index, and Full-Text Search in Redis:

Finding Bigfoot RESTfuly with Express + Redis Stack:

Other resources: