If you're building any AI application that needs to work with more information than fits in a context window — and that's most real business applications — you need a vector database. This isn't optional infrastructure. It's the layer that determines whether your AI application can actually use the knowledge you give it.

Understanding how vector databases work, and how to choose between the options, is now a practical requirement for anyone building AI systems. Here's the orientation.

What a Vector Database Actually Does

Language models understand meaning through numbers — specifically, through high-dimensional vectors where similar concepts cluster together in mathematical space. A sentence about "customer refund requests" and a sentence about "returns and reimbursements" will have vectors that are close to each other in this space, even though they use different words.

A vector database stores these numerical representations (embeddings) and makes it fast to find the items closest to any query. When a user asks your AI support agent "how do I return a product," the system converts that question to a vector, finds the closest items in your knowledge base, and retrieves that content to give the AI context.

This is retrieval-augmented generation (RAG), and vector databases are the infrastructure that makes it work at scale.

The Major Options

Pinecone is the managed service that gets the most attention. Fully hosted, horizontally scalable, good SDKs for most languages. The trade-off is cost — at high query volumes, the bill can become meaningful. Best choice when you want to move fast and not manage infrastructure.

Weaviate is open source with a managed cloud option. More complex to set up than Pinecone, but significantly more flexible. Supports hybrid search (combining vector search with keyword search), which is important for many business applications. Good choice if you want control and are comfortable with more setup.

Qdrant is newer but has been gaining ground for its performance and Rust-based efficiency. The filtering capabilities are particularly strong — useful when you need to search within specific subsets of your data (e.g., only documents from a specific customer tier or product category).

pgvector is the PostgreSQL extension that adds vector search to a database you might already be running. Not as fast as purpose-built vector databases at scale, but if your data is already in Postgres and your query volume is moderate, this is often the right answer. Much less new infrastructure to manage.

Chroma is the open-source option that's become popular for local development and smaller deployments. Simple API, easy to get started, not designed for production at scale.

Choosing for Business Applications

For most business AI applications, the decision comes down to two questions:

What's your query volume? Under 100k queries/day, most options work. Over 1M queries/day, performance characteristics start to matter significantly.

Do you need hybrid search? Pure semantic search works well when users ask natural-language questions. But in business contexts, users also search by specific terms, codes, or structured data. If you need to combine "find similar meaning" with "filter by date range and product category," you want a database that handles hybrid search well.

The Embedding Model Question

A vector database is only as good as the embeddings you put into it. The embedding model converts your text into the numerical vectors that get stored and searched.

For business English, OpenAI's text-embedding-3-small is cost-effective and performs well. For multilingual applications, Cohere's multilingual embedding models are worth the cost premium. For maximum control, open-source models like BGE or E5 run on your own infrastructure.

The embedding model you use to store content must be the same one you use to generate query vectors at search time. Changing embedding models means re-embedding your entire dataset.

Production Considerations

Chunking strategy matters more than people expect. How you split documents before embedding them has a significant impact on retrieval quality. A 200-word chunk that captures a complete idea retrieves better than a 1000-word chunk that covers multiple topics. Experiment with chunk size and overlap for your specific content.

Monitor retrieval quality separately from response quality. If your AI is giving bad answers, the failure might be in retrieval (wrong documents surfaced) or in generation (right documents, wrong response). Logging what was retrieved alongside what was generated helps you diagnose this.

Metadata filtering is essential at scale. Once you have more than a few thousand documents, you need to be able to filter by metadata — date, category, customer segment, product line — rather than searching everything every time. Design your metadata schema before you start loading data.

Getting Started

For a first production deployment: start with Pinecone or pgvector (depending on whether you want managed or self-hosted), use OpenAI embeddings, and implement basic metadata filtering from the start. Get something working with real data before optimizing for scale. The right architecture for 10k documents is different from the right architecture for 10M — and most business applications don't reach 10M for a while.

The foundational skill to build is understanding how retrieval quality affects downstream AI quality. Every hour spent improving how your vector search works will pay back in better AI outputs.