Below is a command to the clone the source code for the application used in this tutorial
git clone --branch v9.2.0 https://github.com/redis-developer/redis-microservices-ecommerce-solutions
You don't need to use MongoDB/ Postgresql as your write-optimized database in the demo application; you can use other prisma supported databases as well. This is just an example.
Sign up for an OpenAI account to get your API key to be used in the demo (add OPEN_AI_API_KEY variable in .env file). You can also refer to the OpenAI API documentation for more information.
Below is a command to the clone the source code for the application used in this tutorial
git clone --branch v9.2.0 https://github.com/redis-developer/redis-microservices-ecommerce-solutions
const products = [
{
productId: '11000',
price: 3995,
productDisplayName: 'Puma Men Slick 3HD Yellow Black Watches',
variantName: 'Slick 3HD Yellow',
brandName: 'Puma',
ageGroup: 'Adults-Men',
gender: 'Men',
displayCategories: 'Accessories',
masterCategory_typeName: 'Accessories',
subCategory_typeName: 'Watches',
styleImages_default_imageURL:
'http://host.docker.internal:8080/images/11000.jpg',
productDescriptors_description_value: 'Stylish and comfortable, ...',
stockQty: 25,
},
//...
];import { Document } from 'langchain/document';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { RedisVectorStore } from 'langchain/vectorstores/redis';
const convertToVectorDocuments = async (
_products: Prisma.ProductCreateInput[],
) => {
const vectorDocs: Document[] = [];
if (_products?.length > 0) {
for (let product of _products) {
let doc = new Document({
metadata: {
productId: product.productId,
},
pageContent: ` Product details are as follows:
productId: ${product.productId}.
productDisplayName: ${product.productDisplayName}.
price: ${product.price}.
variantName: ${product.variantName}.
brandName: ${product.brandName}.
ageGroup: ${product.ageGroup}.
gender: ${product.gender}.
productColors: ${product.productColors}
Category: ${product.displayCategories}, ${product.masterCategory_typeName} - ${product.subCategory_typeName}
productDescription: ${product.productDescriptors_description_value}`,
});
vectorDocs.push(doc);
}
}
return vectorDocs;
};
const seedOpenAIEmbeddings = async (
vectorDocs: Document[],
_redisClient: NodeRedisClientType,
_openAIApiKey: string,
) => {
if (vectorDocs?.length && _redisClient && _openAIApiKey) {
console.log('openAIEmbeddings started !');
const embeddings = new OpenAIEmbeddings({
openAIApiKey: _openAIApiKey,
});
const vectorStore = await RedisVectorStore.fromDocuments(
vectorDocs,
embeddings,
{
redisClient: _redisClient,
indexName: 'openAIProductsIdx',
keyPrefix: 'openAIProducts:',
},
);
console.log('OpenAIEmbeddings completed');
}
};
const addEmbeddingsToRedis = async (
_products: Prisma.ProductCreateInput[],
_redisClient: NodeRedisClientType,
_openAIApiKey: string,
_huggingFaceApiKey?: string,
) => {
if (_products?.length > 0 && _redisClient && _openAIApiKey) {
const vectorDocs = await convertToVectorDocuments(_products);
await seedOpenAIEmbeddings(vectorDocs, _redisClient, _openAIApiKey);
}
};Download RedisInsight to visually explore your Redis data or to engage with raw Redis commands in the workbench.
POST http://localhost:3000/products/getProductsByVSSText
{
"searchText":"pure cotton blue shirts",
//optional
"maxProductCount": 4, // 2 (default)
"similarityScoreLimit":0.2, // 0.2 (default)
}{
"data": [
{
"productId": "11031",
"price": 1099,
"productDisplayName": "Jealous 21 Women Check Blue Tops",
"productDescriptors_description_value": "Composition : Green and navy blue checked round neck blouson tunic top made of 100% cotton, has a full buttoned placket, three fourth sleeves with buttoned cuffs and a belt below the waist<br /><br /><strong>Fitting</strong><br />Regular<br /><br /><strong>Wash care</strong><br />Machine/hand wash separately in mild detergent<br />Do not bleach or wring<br />Dry in shade<br />Medium iron<br /><br />If you're in the mood to have some checked fun, this blouson tunic top from jealous 21 will fulfil your heart's desire with élan. The cotton fabric promises comfort, while the smart checks guarantee unparalleled attention. Pair this top with leggings and ballerinas for a cute, neat look.<br /><br /><em>Model statistics</em><br />The model wears size M in tops<br />Height: 5'7\"; Chest: 33\"; Waist: 25\"</p>",
"stockQty": 25,
"productColors": "Blue,Green",
"similarityScore": 0.168704152107
//...
}
],
"error": null,
"auth": "SES_fd57d7f4-3deb-418f-9a95-6749cd06e348"
}const getSimilarProductsScoreByVSS = async (
_params: IParamsGetProductsByVSS,
) => {
let {
standAloneQuestion,
openAIApiKey,
//optional
KNN,
scoreLimit,
} = _params;
let vectorDocs: Document[] = [];
const client = getNodeRedisClient();
KNN = KNN || 2;
scoreLimit = scoreLimit || 1;
let embeddings = new OpenAIEmbeddings({
openAIApiKey: openAIApiKey,
});
let indexName = 'openAIProductsIdx';
let keyPrefix = 'openAIProducts:';
if (embeddings) {
// create vector store
const vectorStore = new RedisVectorStore(embeddings, {
redisClient: client,
indexName: indexName,
keyPrefix: keyPrefix,
});
// search for similar products
const vectorDocsWithScore = await vectorStore.similaritySearchWithScore(
standAloneQuestion,
KNN,
);
// filter by scoreLimit
for (let [doc, score] of vectorDocsWithScore) {
if (score <= scoreLimit) {
doc['similarityScore'] = score;
vectorDocs.push(doc);
}
}
}
return vectorDocs;
};const getProductsByVSSText = async (
productsVSSFilter: IProductsVSSBodyFilter,
) => {
let { searchText, maxProductCount, similarityScoreLimit } = productsVSSFilter;
let products: IProduct[] = [];
const openAIApiKey = process.env.OPEN_AI_API_KEY || '';
maxProductCount = maxProductCount || 2;
similarityScoreLimit = similarityScoreLimit || 0.2;
if (!openAIApiKey) {
throw new Error('Please provide openAI API key in .env file');
}
if (!searchText) {
throw new Error('Please provide search text');
}
//VSS search
const vectorDocs = await getSimilarProductsScoreByVSS({
standAloneQuestion: searchText,
openAIApiKey: openAIApiKey,
KNN: maxProductCount,
scoreLimit: similarityScoreLimit,
});
if (vectorDocs?.length) {
const productIds = vectorDocs.map((doc) => doc?.metadata?.productId);
//get product with details
products = await getProductByIds(productIds, true);
}
//...
return products;
};This tutorial explores the implementation of semantic text search in product descriptions using LangChain (OpenAI) and Redis. The focus areas include:
Lets take a look at the architecture of the demo application:
products service: handles querying products from the database and returning them to the frontendorders service: handles validating and creating ordersorder history service: handles querying a customer's order historypayments service: handles processing orders for paymentapi gateway: unifies the services under a single endpointmongodb/ postgresql: serves as the write-optimized database for storing orders, order history, products, etc.The e-commerce microservices application consists of a frontend, built using Next.js with TailwindCSS. The application backend uses Node.js. The data is stored in Redis and either MongoDB or PostgreSQL, using Prisma. Below are screenshots showcasing the frontend of the e-commerce app.
Dashboard: Displays a list of products with different search functionalities, configurable in the settings page.
Settings: Accessible by clicking the gear icon at the top right of the dashboard. Control the search bar, chatbot visibility, and other features here.
Dashboard (Semantic Text Search): Configured for semantic text search, the search bar enables natural language queries. Example: "pure cotton blue shirts."
Dashboard (Semantic Image-Based Queries): Configured for semantic image summary search, the search bar allows for image-based queries. Example: "Left chest nike logo."
Chat Bot: Located at the bottom right corner of the page, assisting in product searches and detailed views.
Selecting a product in the chat displays its details on the dashboard.
Shopping Cart: Add products to the cart and check out using the "Buy Now" button.
Order History: Post-purchase, the 'Orders' link in the top navigation bar shows the order status and history.
Admin Panel: Accessible via the 'admin' link in the top navigation. Displays purchase statistics and trending products.
Consider a simplified e-commerce dataset featuring product details for semantic search.
Implement the addEmbeddingsToRedis function to integrate AI-generated product description embeddings with Redis.
This process involves two main steps:
convertToVectorDocuments function, we transform product details into vector documents. This transformation is crucial as it converts product details into a format suitable for Redis storage.seedOpenAIEmbeddings function is then employed to store these vector documents into Redis. This step is essential for enabling efficient retrieval and search capabilities within the Redis database.Examine the structured openAI product details within Redis using RedisInsight.
This section covers the API request and response structure for getProductsByVSSText, which is essential for retrieving products based on semantic text search.
Request Format
The example request format for the API is as follows:
Response Structure
The response from the API is a JSON object containing an array of product details that match the semantic search criteria:
The backend implementation of this API involves following steps:
getProductsByVSSText function handles the API Request.getSimilarProductsScoreByVSS function performs semantic search on product details. It integrates with OpenAI's semantic analysis capabilities to interpret the searchText and identify relevant products from Redis vector store.Semantic text search in the settings pageDiscover the power of semantic text search for enhancing the e-commerce experience. This tutorial guides you through integrating OpenAI's semantic capabilities with Redis for a dynamic product search engine.