Building a Retrieval-Augmented Generation (RAG) Application with MCP, Gemini, and ChromaDB in Node.js

Building a Retrieval-Augmented Generation (RAG) Application with MCP, Gemini, and ChromaDB in Node.js

Introduction

Large Language Models (LLMs) such as Gemini, GPT, and Claude are powerful, but they have one important limitation—they only know what they were trained on and the information provided in the prompt.

Suppose your company has an internal document containing leave policies, HR rules, medical policies, or confidential project documentation. If you ask an LLM a question about those documents, it may not know the answer because that information was never part of its training data.

This is where Retrieval-Augmented Generation (RAG) becomes useful.

Instead of training a new model, RAG retrieves the most relevant information from your own documents and supplies it to the LLM before generating the final answer.

In this project, we build a complete RAG application using:

  • Node.js
  • Google Gemini API
  • ChromaDB (Vector Database)
  • Model Context Protocol (MCP)

By the end of this article, you will understand how these technologies work together to build an AI assistant capable of answering questions from private company documents.

Technologies Used

Node.js
Google Gemini API
   • Gemini 2.5 Flash (LLM)
   • Gemini Embedding Model (Embeddings)
ChromaDB
Docker
Model Context Protocol (MCP)

Install the required packages:

npm install @google/genai chromadb @modelcontextprotocol/sdk dotenv zod

Run ChromaDB using Docker:

docker run -d --name chromadb -p 8000:8000 chromadb/chroma

What is RAG?

RAG stands for Retrieval-Augmented Generation.

Instead of asking the LLM directly, we first search our documents for the most relevant information.

The retrieved information is then supplied to the LLM, allowing it to generate a more accurate answer.

User Question
      │
      ▼
Generate Embedding
      │
      ▼
Search ChromaDB
      │
      ▼
Retrieve Similar Documents
      │
      ▼
Gemini
      │
      ▼
Final Answer

Why Not Simply Ask Gemini?

Imagine asking:

How many annual leaves do employees receive?

Gemini has no knowledge of your company's internal leave policy.

However, if we first retrieve:

Employees are entitled to 20 annual leaves.

Gemini now has the required context and can answer correctly.

This is the core idea behind RAG.

Understanding Embeddings

Computers cannot understand natural language directly.

Every sentence is converted into a high-dimensional vector called an embedding.

Example:

Vacation
↓

[0.0021, -0.0043, 0.0135, ...]

Leave
↓

[0.0018, -0.0039, 0.0128, ...]

Although the numbers appear random, semantically similar sentences produce vectors that are close together.

The Gemini Embedding model generates vectors containing 3072 dimensions.

Generating an Embedding

//server.js
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { z } from "zod";
import fs from "fs/promises";
import dotenv from "dotenv";
dotenv.config();
import { GoogleGenAI } from "@google/genai";

const server = new McpServer({
    name: "time-server",
    version: "1.0.0"
});

const fileContent = await fs.readFile(
    "company-policy.txt",
    "utf8"
);

const chunks = fileContent
    .split("\n")
    .map(line => line.trim())
    .filter(line => line.length > 0);

    console.log(chunks);

const documentEmbeddings = [];

const ai = new GoogleGenAI({
    apiKey: process.env.GEMINI_API_KEY
});

for (const chunk of chunks) {
    const response = await ai.models.embedContent({
        model: "gemini-embedding-2",
        contents: [
            {
                text: chunk
            }
        ]
    });

    documentEmbeddings.push({
        text: chunk,
        embedding: response.embeddings[0].values
    });
}

console.log(response.embeddings[0].values.length);

Output

3072

Measuring Similarity

After converting text into vectors, we compare them using Cosine Similarity.

1     → Identical meaning

0     → Unrelated

-1    → Opposite meaning

This allows us to retrieve documents that are semantically similar instead of relying on exact keywords.

Why Do We Chunk Documents?

Large documents cannot be embedded efficiently.

Instead we split them into smaller chunks.

Example:

Company Leave Policy

Employees are entitled to 20 annual leaves.

Employees may work from home twice a week.

Medical leave requires a doctor's certificate.

Each chunk receives its own embedding.

Introducing ChromaDB

Instead of storing vectors in JavaScript arrays, we use a Vector Database.

We selected ChromaDB because it is:

  • Open source
  • Lightweight
  • Docker friendly
  • Ideal for learning RAG

Each record contains:

  • Document ID
  • Original Text
  • Embedding
  • Metadata

Loading Documents into ChromaDB

The loading process performs these steps:

  1. Read the document.
  2. Split into chunks.
  3. Generate embeddings.
  4. Store inside ChromaDB.

Example:

import dotenv from "dotenv";
import fs from "fs/promises";
import { GoogleGenAI } from "@google/genai";
import { ChromaClient } from "chromadb";

dotenv.config();

const ai = new GoogleGenAI({
    apiKey: process.env.GEMINI_API_KEY
});


const chromaClient = new ChromaClient({
    host: "localhost",
    port: 8000,
    ssl: false
});


// Create collection
const collection = await chromaClient.getOrCreateCollection({
    name: "company_policy",
    embeddingFunction: null
});


// Read document
const fileContent = await fs.readFile(
    "company-policy.txt",
    "utf8"
);


// Create chunks
const chunks = fileContent
    .split("\n")
    .map(line => line.trim())
    .filter(line => line.length > 0);


console.log("Chunks:");
console.log(chunks);


// Generate embeddings and store
for (let i = 0; i < chunks.length; i++) {

    const chunk = chunks[i];

    const response = await ai.models.embedContent({
        model: "gemini-embedding-2",
        contents: [
            {
                text: chunk
            }
        ]
    });


    await collection.add({

        ids: [
            `chunk-${i}`
        ],

        documents: [
            chunk
        ],

        embeddings: [
            response.embeddings[0].values
        ]
    });
    console.log(`Stored chunk ${i}`);
}
console.log("Document loading completed.");

Searching ChromaDB

When the user asks:

How many vacation days do employees get?

The application:

  • Generates a query embedding
  • Searches ChromaDB
  • Retrieves the closest document

Example:

//app.js
import dotenv from "dotenv";
import dotenv from "dotenv";
import { GoogleGenAI } from "@google/genai";
import { ChromaClient } from "chromadb";

dotenv.config();

const ai = new GoogleGenAI({
    apiKey: process.env.GEMINI_API_KEY
});

const chromaClient = new ChromaClient({
    host: "localhost",
    port: 8000,
    ssl: false
});

const collection = await chromaClient.getCollection({
    name: "company_policy",
    embeddingFunction: null
});

/** User Question */
const question = "How many vacation days do employees get?";

/**  Create Question Embedding  */
const embeddingResponse = await ai.models.embedContent({
    model: "gemini-embedding-2",
    contents: [
        {
            text: question
        }
    ]
});

const questionEmbedding =
    embeddingResponse.embeddings[0].values;

/**  Search into ChromaDB  */
const result = await collection.query({
    queryEmbeddings: [questionEmbedding],
    nResults: 1
});

const context = result.documents[0][0];

console.log("\nRetrieved Context:");
console.log(context);

/**  Ask Gemini using Context */
const prompt = `
You are an AI assistant.

Answer ONLY using the context below.

Context:
${context}

Question:
${question}

If the answer is not present in the context, say
"I could not find the answer in the document."
`;

const finalResponse = await ai.models.generateContent({
    model: "gemini-2.5-flash",
    contents: prompt
});

console.log("\nFinal Answer:\n");
console.log(finalResponse.text);

Output

Employees are entitled to 20 annual leaves.

Even with thousands of documents, ChromaDB performs this search in milliseconds.

Introducing Model Context Protocol (MCP)

Model Context Protocol (MCP) provides a standard way for LLMs to communicate with external tools.

Instead of giving Gemini direct database access, we expose functionality as MCP tools.

Examples include:

  • Search Documents
  • Current Time
  • Weather
  • Database Lookup
  • Email

The LLM decides which tool should be called.

MCP Architecture

            User
              │
              ▼
       Gemini 2.5 Flash
              │
      Tool Selection
              │
              ▼
          MCP Client
              │
              ▼
          MCP Server
              │
              ▼
      searchDocuments()
              │
              ▼
          ChromaDB
              │
              ▼
     Retrieved Context
              │
              ▼
       Gemini Response

Our MCP Tools

We implemented two tools.

  • getCurrentTime
  • searchDocuments

Registering the search tool:

function cosineSimilarity(vectorA, vectorB) {

    let dotProduct = 0;
    let magnitudeA = 0;
    let magnitudeB = 0;

    for (let i = 0; i < vectorA.length; i++) {

        dotProduct += vectorA[i] * vectorB[i];
        magnitudeA += vectorA[i] * vectorA[i];
        magnitudeB += vectorB[i] * vectorB[i];
    }

    magnitudeA = Math.sqrt(magnitudeA);
    magnitudeB = Math.sqrt(magnitudeB);

    return dotProduct / (magnitudeA * magnitudeB);
}

server.registerTool(
    "searchDocuments",
    {
        title: "Search Documents",
        description: "Searches company documents and returns relevant information.",
        inputSchema: {
            query: z.string()
        }
    },
    async ({ query }) => {

        // Convert question into tokens
      
        const response = await ai.models.embedContent({
            model: "gemini-embedding-2",
            contents: [
                {
                    text: query
                }
            ]
        });

        const queryEmbedding = response.embeddings[0].values;
        let bestScore = -1;
        let bestMatch = "";

        for (const doc of documentEmbeddings) {

            const score = cosineSimilarity(
                queryEmbedding,
                doc.embedding
            );

            console.log(
                `${score.toFixed(4)} : ${doc.text}`
            );

            if (score > bestScore) {
                bestScore = score;
                bestMatch = doc.text;
            }
        }


        if (bestScore < 0) {
             bestMatch = "No relevant information found.";
           }

        return {
            content: [
                {
                    type: "text",
                    text: bestMatch
                }
            ]
        };
    }
);

Whenever Gemini needs company knowledge, it invokes this tool automatically.

Gemini Tool Calling

Gemini determines whether it should answer directly or call one of the available MCP tools.

const toolDecision = await ai.models.generateContent({
    model: "gemini-2.5-flash",
    contents: toolPrompt
});

If Gemini decides additional information is required, it invokes searchDocuments.

Complete RAG Flow

User Question
      │
      ▼
Gemini
      │
      ▼
searchDocuments Tool
      │
      ▼
Generate Query Embedding
      │
      ▼
ChromaDB Similarity Search
      │
      ▼
Retrieve Context
      │
      ▼
Gemini
      │
      ▼
Final Answer

Console Output

User:
How many vacation days do employees get?

Gemini Selected Tool:
searchDocuments

Retrieved Context:
Employees are entitled to 20 annual leaves.

Final Answer:
Employees are entitled to 20 annual leaves annually.

Why Use ChromaDB Instead of Arrays?

Initially, we stored embeddings inside JavaScript arrays.

Although suitable for learning, arrays have limitations.

They:

  • Lose data when the application stops.
  • Become slow with large datasets.
  • Cannot efficiently search millions of vectors.

ChromaDB provides:

  • Persistent storage
  • Fast similarity search
  • Metadata support
  • Scalable indexing

Project Structure

project/

│

├── app.js               # MCP Client

├── server.js            # MCP Server

├── load-documents.js    # Load company documents

├── search-documents.js  # Query ChromaDB

├── company-policy.txt

├── package.json

└── .env

Source Code

The complete working project is available on GitHub.

git clone --branch feature/RAG --single-branch https://github.com/PersonSimple/ai-mcp-tools.git

Create a .env file and add your own Gemini API key.

GEMINI_API_KEY=your_api_key

Lessons Learned

While building this project, I learned that:

  • Embeddings represent semantic meaning rather than keywords.
  • Cosine similarity measures vector similarity.
  • Vector databases store embeddings efficiently.
  • RAG combines retrieval with generation.
  • MCP separates AI reasoning from tool execution.
  • ChromaDB replaces manual vector arrays.
  • Docker simplifies infrastructure setup.

Conclusion

Building this project helped me understand that modern AI applications are much more than simply calling an LLM API.

A production-quality AI assistant combines several components:

  • LLM
  • Embedding Model
  • Vector Database
  • Retrieval Pipeline
  • MCP Tools
  • Prompt Engineering

Together, these components form a scalable Retrieval-Augmented Generation (RAG) system capable of answering questions using private company knowledge.

Rather than memorizing concepts, implementing each step—from embeddings and cosine similarity to ChromaDB integration and MCP tools—provides a much deeper understanding of how real-world AI systems are built.



Comments

Post a Comment

Popular posts from this blog

Aggregate function with spring data

Image
I was working on a small project. Requirement was different in term of object mapping. There is simple way to map  java object with database table. Aggregate function in query is not mapped with entity class. We don't have direct mapping attribute in JPA annotation to map aggregate function with entity class. Spring data support this using interface. In ORM query we use aggregate function e.g. count , sum etc. the output of these aggregate function cannot directly map with the Entity class. To collect the outcome of aggregate methods. We need to map using interface in java and provide the Interface class with mapping method.      public List<ITeacherReport> getTeacherReport(String teacherName); Our requirement is to count solved question per month by teacher from answer table. We have below environment configuration. Thymeleaf (for view,  this is template engine) JDK 1.8 Maven Springboot framework  MySql  for database operation Open the start....
Read more

Java Persistence API with Spring Data

Image
Let's try to build simple java application with the help of Spring Tool Suites IDE, Spring-Data connecting and MySql's relation database. You are free to choose any IDE (STS, Eclipse, Intellij etc).  We are building simple application which is storing User information into MySQL Database table. Our Requirements are simple IDE (any IDE, STS, Eclipse or Intellij) JDK 1.8 later  Maven 3.2 Create a Maven project using spring initializr   Open the Spring Initializr ( https://start.spring.io/  ).  This is the best way to start the project. Type the https://start.spring.io on url browser  The Initializr provides quick solution to place dependency and create project structure. Our requirement is JPA and MySql dependencies. below image shows the Initializr with required dependency selection: Following  pom.xml  file  will be generated having   spring-boot-starter-data  and   mysql-con...
Read more

Thread , Runnable and ExecutorService

Image
  Previous approach to create a thread.  1.  Directly create an object of Thread Class after overriding the public void run() method of Thread class and call the start method to initiate the thread package com.executor; public class OldThread implements Runnable { private String task ; public OldThread() { super (); } public OldThread(String task ) { super (); this . task = task ; } @Override public void run(){ System. out .println( " Old thread runnable " + this . task ); try { Thread. sleep (2000); } catch (InterruptedException e ) { e .printStackTrace(); } System. out .println( this . task + " completed." ); } } 2. Implement the Runnable interface  and override the public void method and pass the reference of Runnable Object reference  into Thread(Runnable) constructor. call the start method to initiate the thread package com.executor; public cl...
Read more