RedisGears is a dynamic server-side data processing engine, where the "server" part is Redis itself. RedisGears is distributed as a Redis module. You can start a Redis instance preconfigured with Gears using the official Docker image:
Or, as I do most of the time, using the "redismod" image which include Gears and all the other Redis, Inc. supported modules:
RedisGears was built with the purpose of providing a data processing engine inside of Redis, with more formal semantics than the simpler Lua server-side scripting. Think of it as a more flexible map-reduce engine for Redis. It supports supports transaction, batch, and event-driven processing of Redis data. Gears allow you to localize computation and data provides a built-in coordinator to facilitate processing distributed data in a clustered environment.
In RedisGears, the main unit of processing is the RedisGears function, which can be (currently) written in Python (more languages are being worked on). These functions run on their own thread, separate from Redis' main thread and can be executed in response to keyspace events or imperatively as a result of external commands. The functions are "registered" (deployed) to the Gears engine, and have an associated name and a registration Id.
During registration we pick a specific reader for our function which defines how the function gets its initial data:
KeysReader: Redis keys and values.
KeysOnlyReader: Redis keys.
StreamReader: Redis Stream messages.
PythonReader: Arbitrary Python generator.
ShardsIDReader: Shard ID.
CommandReader: Command arguments from application client.
Depending on the reader type, Gear Functions can either be run immediately, on demand, as batch jobs or in an event-driven manner by registering it to trigger automatically on various types of events.
The Python function
rate_limit takes 3 parameters:
key: The Redis key backing the counter for a given user.
max_request: The request quota for the user.
expiry: The number of seconds in the future to set the counter TTL.
Place the script under
src/main/resources/scripts. Now, Let's break it down:
Similarly to what we did in the previous implementation, we:
- Retrieve the current number of requests for the passed
- Cast the result to an
intand if not found, default to
- If the quota hasn't been exceeded, perform the
EXPIREcommands in a transactions (
with atomic():) and return
False(no rate limiting - request is allowed)
- Otherwise, return
True(deny the request)
- At the bottom of the script, in the
# Function registrationsection, we instantiate the
GB) using the
GearsBuilder"builds" the context of the function, in parameters, transformations, triggers, etc.
- We use the
mapmethod to performs a one-to-one mapping of records to the params of the
rate_limitfunction via a mapper function callback.
- We can now invoke the
registeraction to register the function as an event handler. The event in our case is the trigger
In order to use our RedisGear function from our SpringBoot application we need to do a few things:
- Deploy the function to the Redis server
- Execute the function to get a yay/nay answer on each request
To use LettuceMod we'll add the dependency to the Maven POM as shown:
To access any of the LettuceMod supported modules we will inject a
FixedWindowRateLimiterApplication class as follows:
Add the matching import statement:
We'll start by writing a function to determine whether the function with the trigger
RateLimiter has been
registered. It takes a
Registrations and digs deep to extract the value of the
using the Java Streams API:
@PostConstruct annotated method
- We retrieve an instance of the
RedisGearsCommandsfrom the previously injected
- We get the currently registered Gears functions via the
- We pass the list of registrations to our
- If we don't find the registration we proceed to register the function:
- Load the function from the classpath into a
- Use the
pyexecutemethod passing the
- Load the function from the classpath into a
Next, we'll modify the filter to include the
StatefulRedisModulesConnection as well as the
quota; the value that we need to pass to the function:
Now we can modify the
filter method to use the function. Gears functions are invoked by
triggering the correct event
RateLimiter and passing the parameters required by the function;
key, the quota and the TTL seconds in the future.
As we've have done previously, if the function returns
false we let the request through, otherwise
we return an
Once again, we use curl loop to test the limiter:
You should see the 21st request being rejected:
If we run Redis in monitor mode, we should see the Lua calls to
RG.TRIGGER and under that you should see the
EXPIRE for allowed requests:
And for rate limited request you should see only the call to
The complete code for this implementation is under the branch