Building Custom Data Structures with Lua and Redis: A Practical Guide

2023-07-17 14:30:00

Intro:

Redis offers a variety of built-in data structures like sets and hash maps. But what if we want to combine them and create something even more powerful?

In this blog post, we’ll explore how to utilize Redis and Lua scripting to create sets of hash maps. By combining these structures, we can store complex data, associate values with keys, and perform efficient operations.

Let’s dive in and discover the possibilities of sets of hash maps in Redis.

Basic:

Redis provides powerful Lua scripting capabilities, allowing us to extend its functionality beyond built-in commands. With Lua scripting, we can write custom scripts to perform complex operations and manipulate data directly within Redis.

When writing Lua scripts in Redis, we have access to both arguments and keys. Arguments refer to the parameters we pass to the script, while keys represent the data stored in Redis. This combination enables us to create dynamic and flexible scripts that can interact with Redis data in a meaningful way.

One useful feature of Redis Lua scripting is the ability to load scripts and obtain their unique hash. This hash serves as a reference to the script, allowing us to call it later without needing to resend the entire script each time. This approach improves efficiency by reducing network overhead and promoting code reusability.

Write out first script:

Let’s dive into some practical examples of using Lua scripting in Redis. We’ll start with a simple script that returns the keys and arguments passed to it. Consider the following Lua code:

1	return {KEYS[1], KEYS[2], ARGV[1], ARGV[2]}

To run this script in Redis, we need to load it and obtain its unique hash. We can do this using the below command. Let’s assume we’ve saved the script in a file called myscript.lua. We can load it into Redis using the following command:

1	redis-cli -x script load < myscript.lua

After loading the script, Redis will return its unique hash, which we can use to call the script later.

To execute the script with specific keys and arguments, we can use the EVALSHA command. The syntax is as follows:

1	EVALSHA <script_hash> <numkeys> <key> <key> ... <arg> <arg> ...

we can execute the script as follows:

1	EVALSHA 88f4758f9feb4c631e8b1f54561962aa670ca6ae 2 key_1 key_2 arg_1 arg_2

Redis will execute the script and return the result, which, in this case, would be a list containing the keys and arguments passed.

Implementing:

In this section, we will implement the functionality to add, get, and delete users . Each user will have the following fields: first_name, last_name, job, and national_code. We will also group the users based on their job, as we want to be able to query the data based on job categories.

The code provided is written in Lua and executed in Redis. It performs the following operations

The HMSET command is used to set the user details in a hash map. The hash map key is constructed by appending “_user” to the national_code. The user details, such as first_name, last_name, job, and national_code, are stored as key-value pairs in the hash map.
The SADD command is used to add the user’s hash map key to a set named after the user’s job. This allows us to group users based on their job category.
Finally, the code returns the string “added” to indicate that the user has been successfully added.

-- keys = first_name, last_name, job, national_code
redis.call(
    "HMSET", KEYS[4]..'_user',
    "first_name", KEYS[1],
    "last_name", KEYS[2],
    "job", KEYS[3],
    "national_code", KEYS[4]
)

redis.call("SADD", KEYS[3].."_users", KEYS[4]..'_user')
return "added"

After adding this script to our Redis instance, we can conveniently add sample data to the database using the script.

Adding sample data

Now, to check if any data has been added to our Redis database, we can use the KEYS * command.

The provided code is a Lua script that retrieves data from Redis based on a given job. Here’s an explanation of how it works:

The SMEMBERS command retrieves all the user keys associated with the given job from the Redis set.
A result table is initialized to store the retrieved field values.
A loop iterates over each user key retrieved in step 1.
For each user key, the HMGET command is used to fetch the field values specified in ARGV (passed as arguments to the script).
The field values obtained from the HMGET command are stored in the result table.
Finally, the result table is returned.

-- keys = job
-- args could be first_name, last_name, job, national_code
local users = redis.call("SMEMBERS", KEYS[1].."_users")
local result = {}
local count = 1
for k, v in pairs(users) do
    local hmget_result = redis.call("HMGET", v, unpack(ARGV))
    for k1, v1 in pairs(hmget_result) do
        result[count] = v1
        count = count + 1
    end
end
return result

Once you add this Lua script to Redis, you can use it to retrieve data based on a provided job and arguments.

The provided code is for the delete operation in the Lua script. Here’s an explanation of how it works:

The script expects the national_code as the key.
It retrieves the associated job field value using HGET.
It then deletes the user’s data stored under the national_code key using DEL.
Additionally, it removes the user from the set of users belonging to the corresponding job using SREM.
Finally, it returns the string deleted to indicate the successful deletion.

By executing this script with the appropriate national_code key, you can remove a user’s data from Redis, ensuring that they are no longer associated with their previous job.

-- keys = national_code
local job = redis.call("HGET",  KEYS[1].."_user", "job")
redis.call("DEL",  KEYS[1].."_user")
redis.call("SREM", job.."_users", KEYS[1].."_user")
return "deleted"

Conclusion:

In a nutshell, Lua scripting in Redis gives developers the freedom to create custom data structures and take Redis to new heights. With the flexibility to handle arguments and keys, we can craft dynamic operations and seamlessly execute scripts in Redis. By merging sets and hash maps, we’ve showcased the prowess of custom structures in adding, retrieving, and deleting data. Lua scripting in Redis unlocks a world of possibilities, enabling tailored solutions and turbocharged data storage and processing. Redis and Lua scripting make an unbeatable team, empowering developers with the flexibility and scalability they crave. Embrace the power of Redis and Lua to fuel innovation and supercharge your applications.

Database Redis

Heydar Dasoomi

heydar