Skip to main content

How to use Redis for Query Caching


Profile picture for Prasan Kumar
Author:
Prasan Kumar, Technical Solutions Developer at Redis
Profile picture for Will Johnston
Author:
Will Johnston, Developer Growth Manager at Redis

GITHUB CODE

Below is a command to the clone the source code for the application used in this tutorial

git clone --branch v4.2.0 https://github.com/redis-developer/redis-microservices-ecommerce-solutions

What is query caching?

Have you ever been in a situation where your database queries are slowing down? Query caching is the technique you need to speed database queries by using different caching methods while keeping costs down! Imagine that you built an e-commerce application. It started small but is growing fast. By now, you have an extensive product catalog and millions of customers.

That's good for business, but a hardship for technology. Your queries to primary database (MongoDB/ Postgressql) are beginning to slow down, even though you already attempted to optimize them. Even though you can squeak out a little extra performance, it isn't enough to satisfy your customers.

Why you should use Redis for query caching

Redis is an in-memory datastore, best known for caching. Redis allows you to reduce the load on a primary database while speeding up database reads.

With any e-commerce application, there is one specific type of query that is most often requested. If you guessed that it’s the product search query, you’re correct!

To improve product search in an e-commerce application, you can implement one of following caching patterns:

  • Cache prefetching: An entire product catalog can be pre-cached in Redis, and the application can perform any product query on Redis similar to the primary database.
  • Cache-aside pattern: Redis is filled on demand, based on whatever search parameters are requested by the frontend.
tip

If you use Redis Cloud, cache aside is easier due to its support for JSON and search. You also get additional features such as real-time performance, High scalability, resiliency, and fault tolerance. You can also call upon high-availability features such as Active-Active geo-redundancy.

This tutorial focuses on the cache-aside pattern. The goal of this design pattern is to set up optimal caching (load-as-you-go) for better read operations. With caching, you might be familiar with a "cache miss," where you do not find data in the cache, and a "cache hit," where you can find data in the cache. Let's look at how the cache-aside pattern works with Redis for both a "cache miss" and a "cache hit."

Cache-aside with Redis (cache miss)

Cache miss when using the cache-aside pattern with Redis

This diagram illustrates the steps taken in the cache-aside pattern when there is a "cache miss." To understand how this works, consider the following process:

  1. An application requests data from the backend.
  2. The backend checks to find out if the data is available in Redis.
  3. Data is not found (a cache miss), so the data is fetched from the database.
  4. The data returned from the database is subsequently stored in Redis.
  5. The data is then returned to the application.

Cache-aside with Redis (cache hit)

Now that you have seen what a "cache miss" looks like, let's cover a "cache hit." Here is the same diagram, but with the "cache hit" steps highlighted in green.

Cache hit when using the cache-aside pattern with Redis
  1. An application requests data from the backend.
  2. The backend checks to find out if the data is available in Redis.
  3. The data is then returned to the application.

The cache-aside pattern is useful when you need to:

  1. Query data frequently: When you have a large volume of reads (as is the case in an e-commerce application), the cache-aside pattern gives you an immediate performance gain for subsequent data requests.
  2. Fill the cache on demand: The cache-aside pattern fills the cache as data is requested rather than pre-caching, thus saving on space and cost. This is useful when it isn't clear what data will need to be cached.
  3. Be cost-conscious: Since cache size is directly related to the cost of cache storage in the cloud, the smaller the size, the less you pay.
tip

If you use Redis Cloud and a database that uses a JDBC driver, you can take advantage of Redis Smart Cache, which lets you add caching to an application without changing the code. Click here to learn more!

Microservices architecture for an e-commerce application

The e-commerce microservices application discussed in the rest of this tutorial uses the following architecture:

  1. products service: handles querying products from the database and returning them to the frontend
  2. orders service: handles validating and creating orders
  3. order history service: handles querying a customer's order history
  4. payments service: handles processing orders for payment
  5. digital identity service: handles storing digital identity and calculating identity score
  6. api gateway: unifies services under a single endpoint
  7. mongodb/ postgresql: serves as the primary database, storing orders, order history, products, etc.
  8. redis: serves as the stream processor and caching database
info

You don't need to use MongoDB/ Postgresql as your primary database in the demo application; you can use other prisma supported databases as well. This is just an example.

E-commerce application frontend using Next.js and Tailwind

The e-commerce microservices application consists of a frontend, built using Next.js with TailwindCSS. The application backend uses Node.js. The data is stored in Redis and MongoDB/ Postgressql using Prisma. Below you will find screenshots of the frontend of the e-commerce app:

  • Dashboard: Shows the list of products with search functionality

    redis microservices e-commerce app frontend products page

  • Shopping Cart: Add products to the cart, then check out using the "Buy Now" button redis microservices e-commerce app frontend shopping cart

  • Order history: Once an order is placed, the Orders link in the top navigation bar shows the order status and history

    redis microservices e-commerce app frontend order history page

GITHUB CODE

Below is a command to the clone the source code for the application used in this tutorial

git clone --branch v4.2.0 https://github.com/redis-developer/redis-microservices-ecommerce-solutions

Caching in a microservices application with Redis and primary database (MongoDB/ Postgressql)

In our sample application, the products service publishes an API for filtering products. Here's what a call to the API looks like:

Get products by filter request

docs/api/get-products-by-filter.md
// POST http://localhost:3000/products/getProductsByFilter
{
"productDisplayName": "puma"
}

Get products by filter response (cache miss)

{
"data": [
{
"productId": "11000",
"price": 3995,
"productDisplayName": "Puma Men Slick 3HD Yellow Black Watches",
"variantName": "Slick 3HD Yellow",
"brandName": "Puma",
"ageGroup": "Adults-Men",
"gender": "Men",
"displayCategories": "Accessories",
"masterCategory_typeName": "Accessories",
"subCategory_typeName": "Watches",
"styleImages_default_imageURL": "http://host.docker.internal:8080/images/11000.jpg",
"productDescriptors_description_value": "<p style=\"text-align: justify;\">Stylish and comfortable, ...",
"createdOn": "2023-07-13T14:07:38.020Z",
"createdBy": "ADMIN",
"lastUpdatedOn": "2023-07-13T14:07:38.020Z",
"lastUpdatedBy": null,
"statusCode": 1
}
//...
],
"error": null,
"isFromCache": false
}

Get products by filter response (cache hit)

{
"data": [
//...same data as above
],
"error": null,
"isFromCache": true // now the data comes from the cache rather DB
}

Implementing cache-aside with Redis and primary database (MongoDB/ Postgressql)

The following code shows the function used to search for products in primary database:

server/src/services/products/src/service-impl.ts
async function getProductsByFilter(productFilter: Product) {
const prisma = getPrismaClient();

const whereQuery: Prisma.ProductWhereInput = {
statusCode: DB_ROW_STATUS.ACTIVE,
};

if (productFilter && productFilter.productDisplayName) {
whereQuery.productDisplayName = {
contains: productFilter.productDisplayName,
mode: 'insensitive',
};
}

const products: Product[] = await prisma.product.findMany({
where: whereQuery,
});

return products;
}

You simply make a call to primary database (MongoDB/ Postgressql) to find products based on a filter on the product's displayName property. You can set up multiple columns for better fuzzy searching, but we simplified it for the purposes of this tutorial.

Using primary database directly without Redis works for a while, but eventually it slows down. That's why you might use Redis, to speed things up. The cache-aside pattern helps you balance performance with cost.

The basic decision tree for cache-aside is as follows.

When the frontend requests products:

  1. Form a hash with the contents of the request (i.e., the search parameters).
  2. Check Redis to see if a value exists for the hash.
  3. Is there a cache hit? If data is found for the hash, it is returned; the process stops here.
  4. Is there a cache miss? When data is not found, it is read out of primary database and subsequently stored in Redis prior to being returned.

Here’s the code used to implement the decision tree:

server/src/services/products/src/routes.ts
const getHashKey = (_filter: Document) => {
let retKey = '';
if (_filter) {
const text = JSON.stringify(_filter);
retKey = crypto.createHash('sha256').update(text).digest('hex');
}
return 'CACHE_ASIDE_' + retKey;
};

router.post(API.GET_PRODUCTS_BY_FILTER, async (req: Request, res: Response) => {
const body = req.body;
// using node-redis
const redis = getNodeRedisClient();

//get data from redis
const hashKey = getHashKey(req.body);
const cachedData = await redis.get(hashKey);
const docArr = cachedData ? JSON.parse(cachedData) : [];

if (docArr && docArr.length) {
result.data = docArr;
result.isFromCache = true;
} else {
// get data from primary database
const dbData = await getProductsByFilter(body); //method shown earlier

if (body && body.productDisplayName && dbData.length) {
// set data in redis (no need to wait)
redis.set(hashKey, JSON.stringify(dbData), {
EX: 60, // cache expiration in seconds
});
}

result.data = dbData;
}

res.send(result);
});
tip

You need to decide what expiry or time to live (TTL) works best for your particular use case.

Ready to use Redis for query caching?

You now know how to use Redis for caching with one of the most common caching patterns (cache-aside). It's possible to incrementally adopt Redis wherever needed with different strategies/patterns. For more resources on the topic of microservices, check out the links below:

Additional resources