Search Relevance Tuning for SaaS: Synonyms & Reranking

Great search feels invisible. Results seem obvious, typos don’t matter, and the top few hits are exactly what you meant. Getting there is rarely plug-and-play. Whether you use a database full-text index, Elasticsearch, Meilisearch, Typesense, or a cloud search API, your relevance is a product decision, not a default setting. In a SaaS context—with multi-tenant data, permissions, and business outcomes on the line—tuning search relevance is both an engineering and product craft.

Define what “good” search means for your product

Before touching a ranking formula, clarify the job of your search. What does “good” look like in your app? For a document management system, relevance might prioritize exact title matches and recently updated files the user can access. For a marketplace, it might favor inventory in-stock, nearby, and highly rated. For a support portal, it might elevate accepted answers, not just keyword matches.

Write this down. Tie it to outcomes: faster task completion, increased conversion, fewer tickets. These outcomes guide trade-offs across recall (how many relevant items you find), precision (how many results are relevant), and speed.

Start with the data and access model

Search relevance sits on top of your data model. If your documents are inconsistent, missing fields, or lack access controls, relevance tuning can’t save you.

Entity inventory: List what can be searched (e.g., projects, files, messages, products) and the fields to index for each (e.g., title, description, tags, owner, status, timestamps).
Normalization: Standardize fields (e.g., states, currencies, categories) and clean text (trim, collapse whitespace, strip control chars, normalize Unicode, remove boilerplate).
Multilingual strategy: Decide per-tenant or per-user language handling (tokenization, stemming, accents, CJK segmentation, transliteration).
Permissions: Bake access rules into the index or filter at query-time so users never see results they can’t access. In multi-tenant SaaS, treat tenant ID as a first-class filter.

Choose a baseline ranking strategy

Most engines start with a keyword relevance score (BM25 or TF-IDF) and then apply additional boosts and tie-breakers. Establish a baseline that behaves predictably before layering complexity.

Exact field matches: Boost strong signals such as an exact title match or ID match.
Phrase and proximity: Favor queries where tokens appear in order and close together.
Field weighting: Title > tags > description > body. Avoid over-indexing on long fields that can drown better signals.
Recency or freshness: For time-sensitive content, add a mild decay boost on updated_at or published_at.
Tie-breakers: Use a deterministic tie-breaker (e.g., popularity or a stable ID) to avoid jitter across identical scores.

Synonyms: helpful, but handle with care

Synonyms bridge language mismatch between user queries and your data. They’re among the highest-ROI improvements if you keep them curated and scoped.

One-way vs. two-way: “NYC → New York City” may be one-way; “laptop ↔ notebook” can be two-way. Avoid exploding recall with overly broad pairs.
Domain-specific: Add industry terms (e.g., “PO ↔ purchase order”, “SLA ↔ service level agreement”).
Phrase synonyms: Support multi-word phrases (e.g., “2FA ↔ two-factor authentication”).
Tenant-scoped: Allow per-tenant synonyms for internal jargon, but keep them from polluting other tenants’ results.

Monitor drift: too many aggressive synonyms can reduce precision. Pair synonyms with evaluation (more on this below).

Typo tolerance and normalization

Everyone mistypes. Your engine should recover without surfacing nonsense.

Edit distance by token length: Short tokens (≤4 chars) allow fewer edits than long tokens. This reduces false positives like “CRM → arm”.
Keyboard proximity: Typos often reflect adjacent keys; some engines leverage this to rank likely misspellings higher.
Unicode normalization and diacritics: Treat “cafe” and “café” the same. Consider transliteration for queries in Latin script searching non-Latin content.
Graceful degradation: Start with exact/phrase matches, then fuzzy/typo-tolerant expansions, and down-rank the latter.
Performance control: Fuzzy matching can be expensive. Cap the candidate set and fuzzy depth, then re-rank.

Field boosts and business signals

Relevance is more than keyword similarity. Product signals encode what “good” means for your use case.

Field weights: As above, titles and structured attributes often deserve stronger boosts than body text.
Exact/phrase boosts: Add explicit boosts when the exact query equals a title or SKU.
Contextual boosts: Upweight entities assigned to the user (e.g., “my tickets”), in-progress status, or owned teams.
Freshness: Use exponential or linear decay on last_activity for time-sensitive content.
Popularity: View count, purchase rate, or internal reference counts can help rank high-utility items. Beware feedback loops—periodically rebaseline.
Cost or risk signals: Down-rank out-of-stock items, deprecated docs, or superseded versions.

Combine signals transparently. Keep a log of how final scores are composed so you can debug unexpected rankings.

Facets, filters, and ranking interplay

Users often narrow by facets before judging relevance. Prioritize filters that materially reduce noise: status, owner, category, tags, date ranges. In multi-entity search, present entity tabs or a blended result list with clear labels. When blending, normalize scores across entity types so one class doesn’t dominate by accident.

Hybrid search: lexical + vector (semantic) retrieval

Keyword search is fast and precise for exact terms. Vector search finds conceptually similar items even if the words don’t match. In practice, hybrid approaches win.

Two-stage retrieval: Use keyword search to pull top N candidates, then apply vector reranking to reorder by semantic similarity. This controls cost and latency.
Blended scoring: Compute final_score = a·keyword_score + b·vector_similarity + c·business_boosts, scaling each to comparable ranges.
Safety & isolation: Store embeddings per tenant or include tenant IDs in the embedding index to prevent cross-tenant leakage. Avoid embedding sensitive free text if you don’t need it.
Query embeddings: Create a query embedding for reranking; for multi-lingual, choose a multilingual model.

Vector reranking shines for ambiguous queries (“access controls”), long-tail synonyms (“purchase order approvals”), and semantic categories (“how to reset 2FA”). Keep lexical as the anchor for precision and permissions filters.

Reranking and learning-to-rank

Beyond hybrid scoring, learning-to-rank (LTR) uses labeled data or implicit feedback to order results. Start simple:

Implicit clicks: Click-through rates can be noisy. Discount position bias; treat early positions carefully.
Success proxies: For documents, time-on-doc or copy events; for products, add-to-cart or conversion; for tasks, successful completion.
Offline judgments: Ask domain experts to label small test sets (“relevant”, “somewhat”, “irrelevant”).
Feature set: Combine textual scores, field matches, recency, popularity, and personalization flags into a model (logistic regression, gradient boosting). Keep features auditable.

Ship LTR behind a feature flag. Run A/B tests, watch tail queries, and monitor for unintended bias (e.g., always favoring older, more clicked docs).

Query understanding: structure beats guesswork

Many queries are structured requests hiding in free text: “status:open assigned:me updated:<7d billing”. Recognize these patterns.

Heuristic parsing: Tokenize colon pairs (field:value), quoted phrases, and ranges (date:2024-01-01..2024-03-31).
Value normalization: Map “me” to user ID; map “last week” to a date range; normalize currencies and units.
Fallback intent: If parsing fails, treat the whole phrase as a textual query.

For natural-language queries, LLMs can assist with intent classification and slot extraction, but keep a deterministic safety net and never grant access beyond the user’s permissions.

Evaluation: measure, don’t guess

Pick metrics aligned to your outcomes and maintain a living test set.

Offline metrics: NDCG@k, MRR, precision@k, recall@k. Use multiple metrics; a single metric can mislead.
Query buckets: Head (frequent), torso (moderate), and tail (rare). Sample from all when testing.
Tenant representation: Include different tenant sizes and languages to avoid overfitting to one cohort.
A/B testing: Randomize users or tenants, not queries. Set minimum runtime to capture seasonality or weekly cycles.
Guardrails: Track zero-result rate, time-to-first-click, and click concentration (too much top-1 dominance can indicate bias).

Write down your hypotheses (“synonyms for ‘PO’ will reduce zero-result rate by 10% on procurement tenants”) and evaluate directly against them.

Performance: fast is part of relevance

Even perfect ranking fails if users wait. Aim for sub-300ms server response on typical queries.

Query shaping: Apply filters early (tenant ID, permissions) to shrink the candidate set.
Caching: Cache frequent queries and facet counts per tenant. Invalidate by index segment or timestamp gates.
Prefix indexes and n-grams: For autocomplete, use prefix fields or edge n-grams for speed; cap result counts tightly.
Bound fuzzy: Configure max edit distance and number of expansions per token.
Asynchronous enrichments: Precompute popularity and embeddings in background jobs and store as doc fields.

Multilingual and transliteration considerations

If your SaaS crosses languages:

Language-aware analyzers: Tokenization and stemming differ by language; pick or auto-detect per field.
Accent folding: Treat accented/unaccented variants equivalently; expose a setting for strict vs. folded matching.
Script bridging: Consider transliteration for names (e.g., Cyrillic ↔ Latin) where users may search in a different script.
Per-tenant defaults: Let tenants choose default language and analyzers; store their preference in index settings.

Admin tools: your synonym and boost workbench

Relevance evolves. Provide internal tools to manage it:

Synonym editor: Add, remove, and scope synonyms (global vs tenant vs language). Track usage and impact.
Boost rules: Configure field weights and business boosts per entity.
Evaluation runner: Upload a labeled test set, run experiments, compare metrics side-by-side.
Query inspector: Visualize how a score is composed for a given result (keyword score, vector similarity, boosts).

Security, privacy, and compliance guardrails

Search can accidentally leak data if not designed carefully:

Tenant isolation: Always filter by tenant before ranking. If using a shared index, include tenant ID as a must-match token.
Permissions at index-time vs query-time: Index-time filters simplify queries but complicate updates; query-time filters are flexible but slower. Many teams do both for critical roles.
Field-level security: Don’t index sensitive fields users shouldn’t see. If needed for ranking, index hashed or coarsened versions.
Embedding safety: Avoid embedding sensitive personal or proprietary text. Consider embeddings from redacted content (e.g., remove names, emails).
Auditing: Log search queries and served results for investigation, with appropriate retention and access controls.

Implementation patterns by backend

The exact API differs by engine, but the patterns translate well.

Relational DB + full-text: Use language-aware full-text indexes. Keep a narrow set of indexed columns, apply tenant and permission filters in WHERE, and compute business boosts as additional score terms or ORDER BY expressions. Add a small synonyms table for simple expansions.
Search engines (Elasticsearch/OpenSearch/Meilisearch/Typesense): Define analyzers (tokenization, stemming, stop words). Map fields with boosts. Use function scoring for recency/popularity. Add a vector field for hybrid reranking.
Cloud APIs: Configure synonyms, typo tolerance, and ranking profiles using provided admin APIs. Manage per-tenant indices or namespaces for clean isolation.

Regardless of backend, codify your ranking profile in versioned configuration and ship it like code. That makes experiments safe and repeatable.

A practical relevance tuning roadmap

Instrument the baseline: Ship a simple BM25 + field boosts profile with strict permission filters. Track zero-result rate, time-to-first-click, CTR@k.
Autocomplete: Add prefix search for titles and IDs. Cap results at 5–10. Show entity badges.
Synonyms (curated): Start with 20–50 high-impact pairs. Measure precision/recall changes.
Typo tolerance: Enable per-token edit distance with conservative thresholds.
Business boosts: Add freshness decay and popularity; validate with A/B tests.
Hybrid reranking: Introduce vector reranking for the top 100 keyword candidates. Tune weight blending and watch latency.
Admin tools: Build a synonym editor, query inspector, and evaluation runner to sustain improvements.
LTR (optional): If you have enough signal, experiment with a simple LTR model behind flags.

Prompts to build components with YeshBe

YeshBe is a platform for building software with AI. You describe the system you want, iterate in natural language, and refine until it matches your product. These prompts can help you quickly assemble search components, then fine-tune them with your data and rules.

Build a multi-entity search API for our SaaS that indexes projects, files, and messages. Fields to index:
Projects: title (boost high), description, tags, owner_id, tenant_id, updated_at.
Files: title (boost medium), content, mime_type, owner_id, tenant_id, updated_at.
Messages: body, author_id, project_id, tenant_id, created_at.
Implement tenant_id and permission filters. Return a blended result list with an entity label.

Create an admin UI to manage synonyms. Support one-way and two-way pairs, phrase synonyms, and scoping per tenant. Include a test form to run queries and preview how synonyms expand them. Changes should be versioned and require publish to go live.

Add typo tolerance that allows 1 edit for tokens 5–7 chars and 2 edits for tokens 8+ chars. Disable fuzzy matching for tokens 1–3 chars. Down-rank fuzzy matches by 20% compared to exact matches.

Implement a hybrid search pipeline: 1) Run a keyword query with field boosts and permission filters. Return top 200 candidates. 2) Compute a query embedding using a multilingual model. 3) Retrieve vector similarities for the 200 candidates (stored per tenant) and produce a blended score: final = 0.6 * keyword + 0.25 * vector + 0.15 * business_boosts 4) Return top 20 with scores and an explanation object.

Build a search evaluation runner. Upload a CSV of queries with expected results and relevance labels (3=high, 2=medium, 1=low, 0=irrelevant). Run experiments against a specific ranking profile and report NDCG@10, MRR, zero-result rate, and average latency. Store historical runs for comparison.

Troubleshooting unexpected rankings

Too many long-body matches: Reduce weight on body fields, increase phrase proximity, or require higher minimum_should_match.
Over-boosted recency: If fresh low-quality items jump to the top, cap freshness boosts or add a minimum textual relevance threshold.
Synonym explosions: Scope synonyms to tenant or language, and convert some two-way pairs to one-way.
Vector dominance: If semantically similar but off-topic results appear, reduce vector weight, improve your candidate cutoff, or re-embed with better domain prompts.
Permission leaks: Confirm filters are applied before ranking, and validate indices don’t carry stale access states.

Make relevance explainable

Expose an internal “Why this result?” view that shows:

Matched fields and highlights
Keyword score and which tokens matched
Vector similarity (if used)
Business boosts applied (freshness, ownership, popularity)
Final score and rank

This helps product, support, and success teams build trust and report actionable feedback instead of “search is bad.”

Governance: treat ranking like code

Version your ranking profiles, synonyms, and boosts. Use feature flags to roll out changes gradually. Keep a changelog that ties experiments to observed metrics. This discipline prevents firefighting when a well-intentioned tweak tanks precision for a key tenant.

A concise checklist

Define success for your search (business outcomes and user behaviors).
Normalize data and enforce tenant and permission boundaries.
Ship a clear baseline: field boosts, phrase proximity, tie-breakers.
Add curated synonyms and conservative typo tolerance.
Layer in business signals: freshness, ownership, popularity.
Consider hybrid reranking for ambiguous or long-tail queries.
Evaluate with offline metrics and A/B tests across tenant types.
Instrument performance; optimize candidate sets and caching.
Build admin tools for synonyms, boosts, and evaluation.
Version everything; deploy changes behind flags.

Closing thoughts

Search relevance isn’t a one-time configuration—it’s a product capability you iterate on. Start with simple, auditable changes that match your users’ intent and your business goals. Add complexity only when you can measure the benefit. With a platform like YeshBe, you can describe the search system you want, generate the scaffolding quickly, and then spend your time where it matters: curating signals, evaluating changes, and making your users feel like the results were obvious all along.

Search Relevance Tuning for SaaS: Synonyms, Boosts, Typos, and Reranking