Here’s my thought:
Bigger embeddings don’t necessarily mean better search.
In fact, they usually don’t.
If your goal is search... fast, relevant, intent-aligned retrieval, then adding more dimensions often only adds noise.
Noisy search means less useful results, more overhead, and complexity from ranking, and parsing.
Smaller embeddings are tighter, faster, more focused.
They lock in on what matters.
They don’t chase semantic butterflies across thousands of unnecessary dimensions.
Perfect for:
- 🔹 FAQs
- 🔹 Structured content
- 🔹 Fast retrieval
But throw a stack of 90-page court rulings into the mix, and everything changes.
- 🌀 Now you need cross-sectional context
- 🌀 You need nuance that spans paragraphs, not keywords
- 🌀 You need the difference between “dismissed with prejudice” and “dismissed without” to actually register
That’s when the big models, and more importantly, smarter architecture start to shine.
- ⚙️ Chunk by argument, not by page
- ⚙️ Use mid-size, domain-tuned embeddings
- ⚙️ Run a second pass with global context if the first one can’t cut it
Meaning into math
But Lynn... what's an embedding again?
Okay, here goes.
An embedding is how we turn meaning into math.
Sentences, paragraphs, documents, boiled down to vectors, where similar things sit close together.
But... not all vectors are equal.
🧠 Small local models like MiniLM or E5 are good for speed. Great for focus.
Awesome for when you don’t need to read between the lines.
🧠 Larger models like Ada v2 or BGE are slower, heavier. But they see more of the big picture, if you guide them right.
And then there's the fun stuff:
The key thing to remember is that embeddings, at least on their own aren't summaries.
But... there's no rule anywhere that says you can't add summaries and other key elements to your embeddings, if it'll make them easier to search.
Think about Real Estate, where the technical details of the data might not make any sense on their own, without additional context.
Here, you need "rich embeddings." Where you embed not just the raw content, but also the shape of meaning.
Tags, summaries, latent metadata, all baked right into the vector.
Used wisely, they improve your search, make your agents smarter, and provide context.
🎯 My Rule of thumb:
- ➡️ More nuance = more dimensions
- ➡️ More noise = less performance
- ➡️ Know your corpus before you vectorize
And always use the right tools for the job.