This tutorial helps you get started with Redis and FastAPI.
FastAPI is a Python web framework based on the Starlette microframework. With deep support for asyncio, FastAPI is indeed very fast. FastAPI also distinguishes itself with features like automatic OpenAPI (OAS) documentation for your API, easy-to-use data validation tools, and more.
Of course, the best way to make your FastAPI service even faster is to use Redis. Unlike most databases, Redis excels at low-latency access because it's an in-memory database.
In this tutorial, we'll walk through the steps necessary to use Redis with FastAPI. We're going to build IsBitcoinLit, an API that stores Bitcoin sentiment and price averages in RedisTimeSeries, then rolls these averages up for the last three hours.
Next, let's look at the learning objectives of this tutorial.
The learning objectives of this tutorial are:
- Learn how to install aioredis-py and connect to Redis
- Learn how to integrate aioredis-py with FastAPI
- Learn how to use RedisTimeSeries to store and query timeseries data
- Learn how to use Redis as a cache with aioredis-py
Let's get started!
Want to check gaps in your knowledge of Redis and FastAPI before you continue? Take our short pre-tutorial quiz!
You can also visit the quiz directly.
You can achieve the learning objectives of this tutorial by reading through the text and code examples that follow.
However, we recommend that you set up the example project yourself, so that you can try out some of the code as you learn. The project has a permissive license that allows you to use it freely.
To get started, fork the example project on GitHub.
Follow the README to the project running.
RedisTimeSeries is a source available Redis Module that adds a timeseries data type to Redis. Timeseries is a great way to model any data that you want to query over time, like in this case, the ever-changing price of Bitcoin.
You can get started by following the setup instructions in the RedisTimeSeries documentation.
However, note that this tutorial's example project configures RedisTimeSeries automatically for you with the redismod Docker image.
The IsBitcoinLit project is completely async. That means we use an asyncio-compatible Redis client called aioredis-py and FastAPI's async features.
If you aren't familiar with asyncio, take a few minutes to watch this primer on asyncio before continuing:
We're going to start this tutorial assuming that you have a FastAPI project to work with. We'll use the IsBitcoinLit project for our examples.
Poetry is the best way to manage Python dependencies today, so we'll use it in this tutorial.
IsBitcoinLit includes a
pyproject.toml file that Poetry uses to manage the project's directories, but if you had not already created one, you could do so like this:
Once you have a
pyproject.toml file, and assuming you already added FastAPI and any other necessary dependencies, you could add aioredis-py to your project like this:
NOTE: This tutorial uses a beta version of aioredis-py 2.0. The 2.0 version of aioredis-py features an API that matches the most popular synchronous Redis client for Python, redis-py.
The aioredis-py client is now installed. Time to write some code!
We're going to use Redis for a few things in this FastAPI app:
- Storing 30-second averages of sentiment and price for the last 24 hours with RedisTimeSeries
- Rolling up these averages into a three-hour snapshot with RedisTimeSeries
- Caching the three-hour snapshot
Let's look at each of these integration points in more detail.
The data for our app consists of 30-second averages of Bitcoin prices and sentiment ratings for the last 24 hours. We pull these from the SentiCrypt API.
NOTE: We have no affiliation with SentiCrypt or any idea how accurate these numbers are. This example is just for fun!
We're going to store price and sentiment averages in a timeseries with RedisTimeSeries, so we want to make sure that when the app starts up, the timeseries exists.
We can use a startup event to accomplish this. Doing so looks like the following:
We'll use the
TS.CREATE RedisTimeSeries command to create the timeseries within our
TIP: An interesting point to note from this code is that when we create a timeseries, we can use the
DUPLICATE_POLICY option to specify how to handle duplicate pairs of timestamp and values.
/refresh endpoint exists in the app to allow a client to trigger a refresh of the 30-second averages. This is the entire function:
As is often the case with Python, a lot happens in a few lines, so let's walk through them.
The first thing we do is get the latest sentiment and price data from SentiCrypt. The response data looks like this:
Then we save the data into two timeseries in Redis with the
persist() function. That ends up calling another helper,
add_many_to_timeseries(), like this:
add_many_to_timeseries() function takes a list of (timeseries key, sample key) pairs and a list of samples from SentiCrypt. For each sample, it reads the value of the sample key in the SentiCrypt sample, like "btc_price," and adds that value to the given timeseries key.
Here's the function:
This code is dense, so let's break it down.
We're using the
TS.MADD RedisTimeSeries command to add many samples to a timeseries. We use
TS.MADD because doing so is faster than
TS.ADD for adding batches of samples to a timeseries.
This results in a single large
TS.MADD call that adds price data to the price timeseries and sentiment data to the sentiment timeseries. Conveniently,
TS.MADD can add samples to multiple timeseries in a single call.
Clients use IsBitcoinLit to get the average price and sentiment for each of the last three hours. But so far, we've only stored 30-second averages in Redis. How do we calculate the average of these averages for the last three hours?
When we run
/refresh, we call
calculate_three_hours_of_data() to do so. The function looks like this:
There is more going on here than we need to know for this tutorial. As a summary, most of this code exists to support calls to
That function is where the core logic exists to calculate averages for the last three hours, so let's see what it contains:
Here, we use the
TS.RANGE command to get the samples in the timeseries from the "top" of the hour three hours ago, until the latest sample in the series. With the
AGGREGATE parameter, we get back the averages of the samples in hourly buckets.
So where does this leave us? With averages of the averages, one for each of the last three hours.
Let's review. We have code that achieves the following:
- Gets the latest sentiment and price data from SentiCrypt.
- Saves the data into two timeseries in Redis.
- Calculates the average of the averages for the last three hours.
The snapshot of averages for the last three hours is the data we want to serve clients when they hit the
/is-bitcoin-lit endpoint. We could run this calculation every time a client requests data, but that would be inefficient. Let's cache it in Redis!
First, we'll look at writing to the cache. Then we'll see how FastAPI reads from the cache.
Take a closer look at the last line of the
In FastAPI, you can run code outside of a web request after returning a response. This feature is called background tasks.
This is not as robust as using a background task library like Celery. Instead, Background Tasks are a simple way to run code outside of a web request, which is a great fit for things like updating a cache.
When you call
add_task(), you pass in a function and a list of arguments. Here, we pass in
set_cache(). This function saves the three-hour averages summary to Redis. Let's look at how it works:
First, we serialize the three-hour summary data to JSON and save it to Redis. We use the
ex parameter to set the expiration time for the data to two minutes.
TIP: You need to provide a default serializer for the
json.dumps() function so that
dumps() knows how to serialize datetime objects.
This means that after every refresh, we've primed the cache. The cache isn't primed for long -- only two minutes -- but it's something!
We haven't even seen the API endpoint that clients will use yet! Here it is:
To use this endpoint, clients make a GET request to
/is-bitcoin-lit. Then we try to get the cached three-hour summary from Redis. If we can't, we calculate the three-hour summary, return it, and then save it outside of the web request.
We've already seen how calculating the summary data works, and we just explored saving the summary data to Redis. So, let's look at the
get_cache() function, where we read the cached data:
Remember that when we serialized the summary data to JSON, we needed to provide a default serializer for
json.dumps() that understood datetime objects. Now that we're deserializing that data, we need to give
json.loads() an "object hook" that understands datetime strings. That's what
Other than parsing dates, this code is relatively straightforward. We get the current hour's cache key, and then we try to get the cached data from Redis. If we can't, we return
Putting all the pieces together, we now have a FastAPI app that can retrieve Bitcoin price and sentiment averages, store the averages in Redis, cache three-hour summary data in Redis, and serve the data to clients. Not too shabby!
Here are a few notes to consider:
- We manually controlled caching in this tutorial, but you can also use a library like aiocache to cache data in Redis.
- We ran RedisTimeSeries commands like
execute_command()method in aioredis-py. If you are instead using redis-py in a synchronous project, you can use the redistimeseries-py library to run RedisTimeSeries commands.