> For the complete documentation index, see [llms.txt](https://docs.hexabot.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.hexabot.ai/retrieval-augmented-generation/tune-and-troubleshoot.md).
# Tune and troubleshoot
RAG quality depends on four independent layers:
1. source-content quality;
2. retrieval configuration and indexing;
3. ranking quality;
4. generation instructions and model behavior.
Troubleshoot them separately. A fluent model answer does not prove that the correct content was retrieved.
### Start with an evaluation set
Create a small repeatable dataset before tuning:
Column
Example
Query
“I lost my password; how do I get back in?”
Expected Content ID/title
“Reset a password”
Must retrieve in top K
Yes
Must not retrieve
“Change billing address”
Expected answer facts
Reset link is sent by email
Use representative wording from real users, including abbreviations, spelling variants, and multiple supported languages. Run the same set after every helper, model, chunking, or content change.
Measure retrieval independently from generation. At minimum, track whether the expected Content entry appears in the first `K` hits.
### Improve the source content first
#### Use one primary topic per entry
A focused entry gives both lexical and vector helpers a clearer signal. Split a long page that mixes several unrelated procedures.
#### Write descriptive titles
Titles are part of the canonical search text. Prefer **Reset a customer password** over **Account help**.
#### Put searchable facts in string fields
Use Text and Text Area fields for facts that must be retrieved. Numeric, boolean, attachment, and relation values are not expanded into prose automatically.
#### Include important terminology
For lexical retrieval, include common aliases, acronyms, plan names, error codes, and product names in natural text.
#### Keep publication status accurate
Active content is returned by default. An inactive entry may be correctly excluded rather than missing from the index.
#### Do not rely on uploaded files alone
Media Library files are not automatically parsed into the Content corpus. Extract their text into entries or build an ingestion helper.
### Tune retrieval settings
#### Result limit
The default is `3` hits. Increase it only when relevant evidence is consistently ranked just below the cutoff.
A higher limit can:
* improve recall;
* increase model input cost;
* introduce conflicting or weak context;
* make the final answer less focused.
Common starting values are `3` to `5`. Validate larger values with your evaluation set.
#### Content Type filter
A Content Type filter is often the highest-value precision improvement. Use it whenever the workflow knows the domain.
#### Chunk size for vector helpers
The official vector packages, `hexabot-helper-sqlite-vector` and `hexabot-helper-pgvector`, default to `2000` characters with `200` characters of overlap. Chunking is character-based and deterministic; paragraph and line boundaries are preferred when possible.
* Smaller chunks can improve precision for short, independent facts but create more vectors and may lose surrounding context.
* Larger chunks preserve context but can mix topics and dilute the matching signal.
* Overlap helps preserve facts that cross a boundary but increases embedding volume and duplicate context.
Keep overlap smaller than chunk size. Change one parameter at a time and re-run the same evaluation queries.
#### Embedding model and dimensions
Use an embedding model appropriate for your languages and domain. The **Embedding dimensions** value is a request; models or providers that do not support dimension reduction return their native vector size.
Changing provider, model, dimensions, chunk size, or overlap creates a new vector profile and requires re-embedding.
#### Scores
`score` is optional and helper-specific. Do not display it as a universal confidence percentage or apply the same threshold across `fulltext-search`, the `sqlite-vector` and `pgvector` runtime helpers, and custom helpers unless you have calibrated those scores.
### Diagnose common symptoms
| Symptom | Likely causes | Checks and actions |
| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| RAG helper is missing from Settings | Package not installed, API not restarted, wrong database type, compiled helper missing | Verify `package.json`, reinstall dependencies, restart the API, confirm SQLite vs PostgreSQL, inspect startup logs. |
| `warning` says the helper is unavailable | Default helper points to an incompatible or unregistered helper | Select a helper compatible with the active database. |
| `warning` says settings or credential are invalid | Missing credential, empty secret, unsupported provider, invalid base URL/model/chunk values | Reopen the helper settings and credential; verify an embedding model, not a chat model. |
| Exact terms are not found by full-text search | Entry inactive, facts not stored as strings, terminology absent, Content Type filter excludes it | Inspect the entry status and canonical text inputs; add searchable wording; verify filters. |
| Semantic helper returns no results | Reindex incomplete, provider unreachable, no active indexed content, wrong profile, extension unavailable | Check API logs, provider quota/network, selected helper, status policy, and database extension. Schedule reindex. |
| New PostgreSQL content is temporarily missing | Durable indexing job has not completed or is retrying | Check API logs and provider availability; allow the queue to reconcile. |
| Results are broadly related but not precise | Entries or chunks mix topics; no Content Type filter; limit too high | Split content, reduce chunk size, add filter, lower limit. |
| Correct hit is ranked below irrelevant hits | Weak title/text, embedding model mismatch, multilingual/domain limitations | Improve source wording, evaluate another embedding model, add metadata filtering or a reranker in a custom helper. |
| Model ignores correct retrieved text | Prompt does not delimit context, instructions permit model memory, too much context, prompt injection | Strengthen grounding instructions, separate question/context, reduce weak hits, treat retrieved text as untrusted data. |
| Direct search API fails but workflow action shows a warning | Different error surfaces | The action wraps known configuration/availability errors; the direct endpoint can return an HTTP error. Inspect both response and logs. |
### Inspect retrieval before the LLM
During debugging, temporarily stop the workflow after `retrieve_rag_content` and inspect:
* `hits.length`;
* each hit's `contentId` and `title`;
* `text` returned to the model;
* `source`;
* optional scores;
* `warning`.
Only tune generation after retrieval reliably returns the intended evidence.
### Security checklist
* Treat retrieved content as untrusted data and tell the model not to follow instructions inside it.
* Keep system and developer instructions separate from retrieved text.
* Do not index secrets or content that the workflow's users are not allowed to see.
* Remember that the official helper filters are status and Content Type, not per-user authorization.
* Review embedding-provider retention, regional processing, and incident-response terms.
* Keep **Index only active content** enabled unless there is a reviewed reason not to.
* Log content IDs and helper names for audits without unnecessarily logging full confidential text.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.hexabot.ai/retrieval-augmented-generation/tune-and-troubleshoot.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.