Vector Databases for Beginners: Chroma, pgvector and OpenSearch

By Ravindra Bagale, trainer · LinkedIn · Last updated: 19 October 2026

Generative AI course · Chapter 39 · Estimated reading time: 14 minutes

1. Suppose the help search gets slow and forgetful

Suppose our friend Sunita's kitchenware shop grows. Her meaning search (from the embeddings chapter) now has 8,000 help articles. Two problems appear:

1. Forgetful: every time the program restarts, it embeds all 8,000 articles again. That takes time, and with a paid API, money.
2. Slow and messy: to answer one question, it compares the question with all 8,000 embeddings. And when she wants "only articles about payments", she has to write extra code.

A vector database fixes both. It saves the embeddings once, builds an index so it can find close ones without checking every row, and lets you filter by labels like topic = payments.

2. What a vector database does

A vector database stores, for each item:

• an ID (a1),
• the text (or a link to it),
• the embedding (the list of numbers),
• metadata: labels such as topic, page number, language or date.

It does 3 jobs:

1. Store: keep everything on disk, safe after a restart.
2. Index: build a structure (often HNSW, a kind of map of nearby points) so search stays fast as data grows. It gives approximate nearest results: very fast, and almost always the same as checking everything.
3. Search: "give me the 3 items closest to this question, where topic is payments".

Words you will see: a collection or table holds one set of items; distance is the opposite of similarity (for cosine, distance = 1 minus similarity, so smaller is closer).

3. Three choices for beginners (October 2026)

Checked on 19 October 2026:

• Amazon RDS for PostgreSQL supports pgvector; PostgreSQL 18.4 on RDS includes pgvector 0.8.2.
• OpenSearch has no free tier. OpenSearch Serverless bills by OCU-hour (a unit of compute) and storage. The newer NextGen vector collections can scale compute to zero after 10 idle minutes; storage is still billed. Classic vector collections have a minimum compute charge even when idle.
• New AWS accounts can choose a Free plan with up to $200 in credits for up to 6 months, and RDS PostgreSQL db.t4g.micro is eligible. Check your own account's plan before you create anything.

4. Worked example 1: a local Chroma database

pip install chromadb fastembed

We use the same free embedding model as in the embeddings chapter, so the scores match what you saw there.

# Local Chroma: store help articles with their embeddings on disk, then search by meaning.
import chromadb
from fastembed import TextEmbedding

model = TextEmbedding("BAAI/bge-small-en-v1.5")
def embed(texts):
    return [v.tolist() for v in model.embed(texts)]

client = chromadb.PersistentClient(path="./shop_db")          # a folder on your laptop
col = client.get_or_create_collection("help", metadata={"hnsw:space": "cosine"})

articles = [
    ("a1", "How to reset your account password", "account"),
    ("a2", "Track your parcel after you place an order", "orders"),
    ("a3", "Change the delivery address of an order", "orders"),
    ("a4", "Cancel an order and get a refund", "payments"),
    ("a5", "Update your mobile number on your account", "account"),
]
col.upsert(ids=[a[0] for a in articles], documents=[a[1] for a in articles],
           metadatas=[{"topic": a[2]} for a in articles], embeddings=embed([a[1] for a in articles]))
print("Stored:", col.count(), "articles")

res = col.query(query_embeddings=embed(["where is my package"]), n_results=2)
for doc, dist in zip(res["documents"][0], res["distances"][0]):
    print(f"  distance {dist:.2f}  {doc}")

print("Only topic = account:")
res = col.query(query_embeddings=embed(["I forgot my login"]), n_results=1, where={"topic": "account"})
print(" ", res["documents"][0][0])

Real output (we ran this on 19 October 2026 with Chroma 1.5.9):

Stored: 5 articles
  distance 0.29  Track your parcel after you place an order
  distance 0.32  Change the delivery address of an order
Only topic = account:
  How to reset your account password

Check it against the embeddings chapter. There, "where is my package" scored 0.71 and 0.68 similarity. Here the distances are 0.29 and 0.32: exactly 1 minus those scores. Same model, same meaning; the database just stores and searches it for us.

Now prove it remembers. We started Python again and opened the same folder:

import chromadb
c = chromadb.PersistentClient(path="./shop_db")
print("After restart:", c.get_collection("help").count(), "articles still there")

Real output:

After restart: 5 articles still there

No re-embedding needed. The whole database is one folder (376 KB for our 5 articles).

5. Worked example 2: pgvector, locally and on Amazon RDS

pgvector is an extension that adds a vector column type and distance operators to PostgreSQL. Your normal tables and your embeddings live together, so you can join them with orders, products or users.

We ran this against a local PostgreSQL 17 with pgvector. On Amazon RDS for PostgreSQL, the only change is the connection string in your .env file.

# pgvector: the same help articles inside PostgreSQL. On Amazon RDS, only the connection details change.
import os
import psycopg
from fastembed import TextEmbedding

model = TextEmbedding("BAAI/bge-small-en-v1.5")
vec = lambda t: "[" + ",".join(f"{x:.6f}" for x in next(iter(model.embed([t])))) + "]"

# Local test database. For RDS, use the endpoint, user and password from your .env file instead.
conn = psycopg.connect(os.getenv("DATABASE_URL", "host=/tmp/pgv port=5499 user=postgres dbname=postgres"), autocommit=True)
cur = conn.cursor()
cur.execute("CREATE EXTENSION IF NOT EXISTS vector")
cur.execute("SELECT extversion FROM pg_extension WHERE extname = 'vector'")
print("pgvector version:", cur.fetchone()[0])
cur.execute("DROP TABLE IF EXISTS help_articles")
cur.execute("""CREATE TABLE help_articles (
    id serial PRIMARY KEY, title text NOT NULL, topic text, embedding vector(384))""")
articles = [("How to reset your account password", "account"),
            ("Track your parcel after you place an order", "orders"),
            ("Change the delivery address of an order", "orders"),
            ("Cancel an order and get a refund", "payments"),
            ("Update your mobile number on your account", "account")]
for title, topic in articles:
    cur.execute("INSERT INTO help_articles (title, topic, embedding) VALUES (%s, %s, %s)", (title, topic, vec(title)))
cur.execute("CREATE INDEX ON help_articles USING hnsw (embedding vector_cosine_ops)")

q = vec("I want my money back")
cur.execute("""SELECT title, topic, round((1 - (embedding <=> %s::vector))::numeric, 2) AS similarity
               FROM help_articles ORDER BY embedding <=> %s::vector LIMIT 2""", (q, q))
for title, topic, sim in cur.fetchall():
    print(f"  {sim}  {title} ({topic})")

Real output (19 October 2026, PostgreSQL 17.11 with pgvector 0.8.0):

pgvector version: 0.8.0
  0.75  Cancel an order and get a refund (payments)
  0.54  Update your mobile number on your account (account)

Again the same 0.75 as in the embeddings chapter. Three things to notice in the SQL:

1. vector(384) must match your model's size (384 numbers for this model).
2. <=> is cosine distance; ORDER BY ... LIMIT 2 gives the 2 closest.
3. The HNSW index keeps search fast when the table has thousands of rows.

On Amazon RDS for PostgreSQL, the steps are:

1. Create an RDS PostgreSQL database (a recent version; check the RDS release notes for its pgvector version).
2. Keep it private: no public access, and a security group that allows only your app server.
3. Connect from your app server and run CREATE EXTENSION vector; once.
4. Put the connection string in .env as DATABASE_URL, never in code.
5. Run the same Python script.

Expected output (yours will be similar, not the same): the same two rows and scores, because the model and the data are the same.

6. Worked example 3: when OpenSearch on AWS makes sense

Amazon OpenSearch is a search engine service. It is strong at keyword search (exact words, spelling mistakes, filters, counts) and also supports vector search. Using both together is called hybrid search, and it helps with things embeddings are weak at, like order IDs and product codes.

Choose it when:

• you have a large amount of data, or need keyword and vector search together;
• your team already runs search or logs on OpenSearch;
• you use a managed AWS RAG service that stores vectors there.

The plan, step by step (no code to copy blindly):

1. In the AWS console, look at OpenSearch Serverless and create a vector search collection (prefer a type that can scale to zero for learning).
2. Create an index with a vector field whose size matches your model (384 here) and a text field for keywords.
3. Load the same 5 articles with their embeddings.
4. Search with a vector query, then with a hybrid query (vector plus keyword).
5. Delete the collection the same day if it was only for practice.

A prompt to plan it with AI:

I have 5 help articles with 384-number embeddings. Explain, step by step, how to create an
Amazon OpenSearch Serverless vector search collection, an index with a 384-dimension vector
field and a text field, load the data, and run a hybrid query in Python. Tell me which steps
cost money, how to keep compute at zero when idle, and how to delete everything afterwards.

Expected output (yours will be similar, not the same): a numbered plan with the index mapping (a knn_vector field with dimension 384), a Python example, and a clean-up section. Check every cost statement against the AWS pricing page yourself before you start.

7. How to choose

Rules that apply to all three:

1. One embedding model per collection. If you change the model, re-embed everything.
2. Store metadata (page, topic, date) with each chunk, for filters and citations.
3. Keep the original text, so you can show it and re-embed it later.
4. Private data stays private: no public databases, keys in .env.

8. Beginner mistakes and quick fixes

9. Try at home

Recap

Move your FAQ finder into Chroma this week and restart it twice. The day it answers instantly without re-embedding, you have built the memory of a real AI app.

Previous chapter: What Is RAG? Build a Chatbot from Your Own PDF (Step by Step)

Next chapter: Fine-Tuning vs RAG vs Prompting: Which One Do You Need?

Ravindra Bagale, trainer: linkedin.com/in/ravindra-bagale. Sunita, her shop and the help articles on this page are made up for learning. Amazon RDS pgvector support, OpenSearch pricing rules and the AWS Free plan were checked on AWS pages on 19 October 2026; plans change. The Python output was produced on 19 October 2026 with Chroma 1.5.9, PostgreSQL 17.11, pgvector 0.8.0, fastembed and the BAAI/bge-small-en-v1.5 model.