Techniques Catalog 📚¶

RAGLib provides a comprehensive collection of RAG techniques, each implementing the unified RAGTechnique.apply() interface. This catalog showcases all available techniques organized by category.

Updating the Techniques Index¶

Techniques Index Update

The complete techniques catalog is generated from the RAGLib registry. To update the catalog:

Linux/macOSWindows

Using Make

make docs-generate    # Generate techniques index
make docs            # Build docs (includes generation)  
make docs-serve      # Build and serve locally

Manual Generation

python tools/generate_techniques_index.py

Using PowerShell

.\build.ps1 docs-generate    # Generate techniques index
.\build.ps1 docs            # Build docs (includes generation)
.\build.ps1 docs-serve      # Build and serve locally

Using Batch Script

.\build.bat docs-generate    # Generate techniques index
.\build.bat docs            # Build docs (includes generation)
.\build.bat docs-serve      # Build and serve locally

Manual Generation

python tools\generate_techniques_index.py

The content appears in docs/techniques_generated.md and is included in the documentation build.

Do Not Edit Index Files

The file docs/techniques_generated.md is created by the generation script. Manual edits will be overwritten when the index is regenerated.

Technique Categories 🗂️¶

RAG techniques are organized into the following categories:

Chunking:material-search: Sparse Retrieval Dense Retrieval Retrieval Enhancement Reranking Generation:material-orchestra: Orchestration

Split documents into processable segments

Purpose: Prepare documents for embedding and retrieval
When to use: When working with large documents that need to be broken down
Output: Structured chunks with metadata

Find relevant information using lexical/sparse methods

Purpose: Retrieve documents using term-based matching (BM25, TF-IDF, lexical matching)
When to use: When exact term matching is important, or as a baseline/fallback for dense retrieval
Output: Ranked list of documents with sparse similarity scores
Techniques: BM25, TF-IDF, Lexical Matching, SPLADE, Lexical Transformer

Find relevant information using dense vector methods

Purpose: Locate semantically similar information using embedding-based matching
When to use: Primary retrieval method for semantic similarity; works with any embedder
Output: Ranked list of relevant documents/chunks with dense similarity scores
Techniques: FAISS retrieval, Dual Encoder retrieval, ColBERT token-level matching

Advanced techniques that extend basic retrieval capabilities

Purpose: Improve retrieval through multi-step approaches and query enhancement
When to use: When single-query retrieval needs enhancement or multiple perspectives
Output: Enhanced retrieval results through query expansion or multi-vector approaches
Techniques: Multi-Query Retrieval, Multi-Vector Retrieval

Improve retrieval quality through reordering

Purpose: Refine initial retrieval results
When to use: When you need higher precision in retrieved results
Output: Reordered list of documents with improved relevance scores

Produce final answers using retrieved context

Purpose: Generate human-readable responses
When to use: Final step in RAG pipelines to create answers
Output: Generated text based on retrieved context

Coordinate multiple techniques in workflows

Purpose: Chain multiple techniques together
When to use: Building complex RAG pipelines
Output: Results from multi-step processing

Available Techniques 🛠️¶

The following sections contain the complete catalog of techniques available in RAGLib. This content is generated from the technique registry.

Techniques Index¶

Total Techniques: 22 Categories: 5

Chunking¶

content_aware_chunker¶

Content-aware chunking that respects text structure and natural boundaries

Property	Value
Version	`1.0.0`
Class	`ContentAwareChunker`
Module	`raglib.techniques.content_aware_chunker`
Dependencies	None

document_specific_chunker¶

Document-specific chunking that adapts to document type

Property	Value
Version	`1.0.0`
Class	`DocumentSpecificChunker`
Module	`raglib.techniques.document_specific_chunker`
Dependencies	None

fixed_size_chunker¶

Fixed-size text chunking with overlap support

Property	Value
Version	`1.0.0`
Class	`FixedSizeChunker`
Module	`raglib.techniques.fixed_size_chunker`
Dependencies	None

parent_document_chunker¶

Parent document retrieval with small-to-large chunk mapping

Property	Value
Version	`1.0.0`
Class	`ParentDocumentChunker`
Module	`raglib.techniques.parent_document_chunker`
Dependencies	None

propositional_chunker¶

Propositional chunking based on semantic propositions

Property	Value
Version	`1.0.0`
Class	`PropositionalChunker`
Module	`raglib.techniques.propositional_chunker`
Dependencies	None

recursive_chunker¶

Recursive chunking with hierarchical text splitting

Property	Value
Version	`1.0.0`
Class	`RecursiveChunker`
Module	`raglib.techniques.recursive_chunker`
Dependencies	None

semantic_chunker¶

Semantic similarity-based chunking with configurable embedder

Property	Value
Version	`1.0.0`
Class	`SemanticChunker`
Module	`raglib.techniques.semantic_chunker`
Dependencies	None

sentence_window_chunker¶

Sentence-based windowing with configurable window size and overlap

Property	Value
Version	`1.0.0`
Class	`SentenceWindowChunker`
Module	`raglib.techniques.sentence_window_chunker`
Dependencies	None

Reranking¶

crossencoder_rerank¶

Cross-encoder re-ranking using pairwise (query, document) scoring

Property	Value
Version	`1.0.0`
Class	`CrossEncoderReRanker`
Module	`raglib.techniques.crossencoder_rerank`
Dependencies	None

mmr¶

Maximal Marginal Relevance re-ranking for balancing relevance and diversity

Property	Value
Version	`1.0.0`
Class	`MMRReRanker`
Module	`raglib.techniques.mmr`
Dependencies	None

Core Retrieval¶

colbert_retriever¶

ColBERT-style late interaction dense retrieval

Property	Value
Version	`1.0.0`
Class	`ColBERTRetriever`
Module	`raglib.techniques.colbert_retriever`
Dependencies	None

dense_retriever¶

Production-friendly dense retriever with optional adapters fallback.

Property	Value
Version	`1.0.0`
Class	`DenseRetriever`
Module	`raglib.techniques.dense_retriever`
Dependencies	None

dual_encoder¶

Dual-encoder dense retrieval with separate query/doc encoding

Property	Value
Version	`1.0.0`
Class	`DualEncoder`
Module	`raglib.techniques.dual_encoder`
Dependencies	None

faiss_retriever¶

FAISS-based dense retriever with efficient similarity search

Property	Value
Version	`1.0.0`
Class	`FAISSRetriever`
Module	`raglib.techniques.faiss_retriever`
Dependencies	None

multi_vector_retriever¶

Multi-vector dense retrieval with document segmentation

Property	Value
Version	`1.0.0`
Class	`MultiVectorRetriever`
Module	`raglib.techniques.multi_vector_retriever`
Dependencies	None

Retrieval Enhancement¶

hyde¶

Generate hypothetical documents to improve retrieval

Property	Value
Version	`1.0.0`
Class	`HyDE`
Module	`raglib.techniques.hyde`
Dependencies	None

multi_query_retriever¶

Multi-query retrieval with result fusion

Property	Value
Version	`1.0.0`
Class	`MultiQueryRetriever`
Module	`raglib.techniques.multi_query_retriever`
Dependencies	None

Sparse Retrieval¶

bm25¶

BM25 ranking function for text retrieval with in-memory indexing

Property	Value
Version	`1.0.0`
Class	`BM25`
Module	`raglib.techniques.bm25`
Dependencies	None

lexical_matcher¶

Lexical matching retrieval with configurable matching modes

Property	Value
Version	`1.0.0`
Class	`LexicalMatcher`
Module	`raglib.techniques.lexical_matcher`
Dependencies	None

lexical_transformer¶

Transformer-aware lexical retrieval with attention weighting

Property	Value
Version	`1.0.0`
Class	`LexicalTransformer`
Module	`raglib.techniques.lexical_transformer`
Dependencies	None

splade¶

SPLADE sparse-dense hybrid retrieval with term expansion

Property	Value
Version	`1.0.0`
Class	`Splade`
Module	`raglib.techniques.splade`
Dependencies	None

tfidf¶

TF-IDF retrieval with cosine similarity scoring

Property	Value
Version	`1.0.0`
Class	`TfIdf`
Module	`raglib.techniques.tfidf`
Dependencies	None

Adding New Techniques¶

To add a new technique to RAGLib:

Chunking Techniques¶

ContentAwareChunker¶

Description: Content-aware chunking that respects text structure and natural boundaries
Category: chunking
Dependencies: None
Parameters: max_chunk_size, min_chunk_size, overlap

DocumentSpecificChunker¶

Description: Document-specific chunking that adapts to document type
Category: chunking
Dependencies: None
Parameters: max_chunk_size, min_chunk_size, overlap

FixedSizeChunker¶

Description: Fixed-size text chunking with overlap support
Category: chunking
Dependencies: None
Parameters: chunk_size, overlap

ParentDocumentChunker¶

Description: Parent document retrieval with small-to-large chunk mapping
Category: chunking
Dependencies: None
Parameters: small_chunk_size, large_chunk_size, overlap

PropositionalChunker¶

Description: Propositional chunking based on semantic propositions
Category: chunking
Dependencies: None
Parameters: max_chunk_size, min_chunk_size, overlap, max_propositions_per_chunk

RecursiveChunker¶

Description: Recursive chunking with hierarchical text splitting
Category: chunking
Dependencies: None
Parameters: chunk_size, overlap, separators

SemanticChunker¶

Description: Semantic similarity-based chunking with configurable embedder
Category: chunking
Dependencies: None
Parameters: embedder, chunk_size, overlap, similarity_threshold, dim

SentenceWindowChunker¶

Description: Sentence-based windowing with configurable window size and overlap
Category: chunking
Dependencies: None
Parameters: window_size, overlap_sentences

Retrieval Techniques¶

DenseRetriever¶

Description: Dense retrieval using semantic embeddings
Category: retrieval
Dependencies: faiss
Parameters: top_k, similarity_threshold

BM25SimpleRetriever¶

Description: Simple BM25-based sparse retrieval
Category: retrieval
Dependencies: None
Parameters: top_k, k1, b

Reranking Techniques¶

CrossEncoderReRanker¶

Description: Rerank documents using a cross-encoder model
Category: reranking
Dependencies: transformers, torch
Parameters: model_name, top_k

MMR (Maximal Marginal Relevance)¶

Description: Diversify results using Maximal Marginal Relevance
Category: reranking
Dependencies: None
Parameters: diversity_lambda, top_k

Technique Discovery

Use the TechniqueRegistry.list_techniques() method to programmatically discover available techniques at runtime.

Using Techniques 🚀¶

Basic Usage Pattern¶

All techniques follow the same consistent pattern:

from raglib.registry import TechniqueRegistry

# Discover available techniques
available = TechniqueRegistry.list_techniques()
print(f"Available techniques: {list(available.keys())}")

# Get technique class
TechniqueClass = TechniqueRegistry.get("technique_name")

# Initialize with configuration
technique = TechniqueClass(param1=value1, param2=value2)

# Apply technique
result = technique.apply(input_data, mode="some_mode")

# Check results
if result.success:
    output = result.payload
    print(f"Success: {output}")
else:
    print(f"Error: {result.error}")

Finding Techniques by Category¶

from raglib.registry import TechniqueRegistry

# Get all chunking techniques
chunking_techniques = TechniqueRegistry.list_by_category("chunking")

# Get all retrieval techniques  
retrieval_techniques = TechniqueRegistry.list_by_category("retrieval")

# Print technique information
for name, meta in chunking_techniques.items():
    print(f"{name}: {meta.description}")

Find all chunking techniques¶

chunking_techniques = TechniqueRegistry.find_by_category("chunking")

List available techniques¶

for name, technique_class in chunking_techniques.items(): meta = technique_class.meta print(f"{name}: {meta.description}")

### Technique Metadata

Each technique provides metadata through its `meta` attribute:

```python
technique_class = TechniqueRegistry.get("fixed_size_chunker")
meta = technique_class.meta

print(f"Name: {meta.name}")
print(f"Category: {meta.category}")
print(f"Description: {meta.description}")
print(f"Version: {meta.version}")
print(f"Dependencies: {meta.dependencies}")

Chunking Techniques¶

Chunking techniques split documents into smaller, processable segments.

Common Parameters¶

chunk_size: Maximum size of each chunk (in characters or tokens)
overlap: Number of characters/tokens to overlap between chunks
separator: Text used to split documents

Usage Example¶

from raglib.techniques import FixedSizeChunker

chunker = FixedSizeChunker(chunk_size=512, overlap=50)
documents = ["Long document text here..."]

result = chunker.apply(documents)
chunks = result.payload["chunks"]

Retrieval Techniques¶

Retrieval techniques find relevant information from knowledge bases.

Common Parameters¶

top_k: Number of results to return
similarity_threshold: Minimum similarity score for results
embedder: Embedder adapter for encoding queries
vectorstore: Vector store adapter for similarity search

Usage Example¶

from raglib.techniques import DenseRetriever
from raglib.adapters import InMemoryVectorStore, DummyEmbedder

retriever = DenseRetriever(
    embedder=DummyEmbedder(),
    vectorstore=InMemoryVectorStore()
)

# Index documents
chunks = ["chunk1", "chunk2", "chunk3"]
retriever.apply(chunks, mode="index")

# Retrieve relevant chunks
query = "search query"
result = retriever.apply(query, mode="retrieve", top_k=3)
relevant_chunks = result.payload["chunks"]

Reranking Techniques¶

Reranking techniques improve the quality of retrieved results.

Common Parameters¶

model: Model used for reranking
top_k: Number of results to return after reranking
cross_encoder: Whether to use cross-encoder architecture

Usage Example¶

from raglib.techniques import CrossEncoderReRanker

reranker = CrossEncoderReRanker(model="cross-encoder/ms-marco-TinyBERT-L-2-v2")

query = "search query"
candidates = ["candidate1", "candidate2", "candidate3"]

result = reranker.apply(query=query, candidates=candidates, top_k=2)
reranked_candidates = result.payload["reranked_chunks"]

Generation Techniques¶

Generation techniques produce final answers using retrieved context.

Common Parameters¶

model: Language model to use for generation
temperature: Sampling temperature for generation
max_tokens: Maximum number of tokens to generate
prompt_template: Template for formatting prompts

Usage Example¶

from raglib.techniques import HyDE

# HyDE for query expansion
hyde = HyDE(llm_adapter=llm_adapter, temperature=0.1)

query = "What is RAG?"
context = ["RAG stands for Retrieval-Augmented Generation..."]

result = generator.apply(query=query, context=context)
answer = result.payload["answer"]

Orchestration Techniques¶

Orchestration techniques coordinate multiple techniques in complex workflows.

Usage Example¶

from raglib.architectures import FiDPipeline

pipeline = FiDPipeline(
    chunker_name="fixed_size_chunker",
    retriever_name="dense_retriever", 
    generator_name="llm_generator"
)

documents = ["document1", "document2"]
query = "question"

result = pipeline.apply(documents=documents, query=query)
answer = result.payload["answer"]

Custom Techniques¶

Creating a New Technique¶

To create a custom technique:

Create a new file in raglib/techniques/your_technique.py
Implement the technique following the RAGTechnique interface
Register the technique using the @TechniqueRegistry.register decorator
Add tests in tests/test_your_technique.py

Example:

from raglib.core import RAGTechnique, TechniqueMeta, TechniqueResult
from raglib.registry import TechniqueRegistry

@TechniqueRegistry.register
class MyCustomTechnique(RAGTechnique):
    meta = TechniqueMeta(
        name="my_custom_technique",
        category="custom",
        description="A custom technique for special processing",
        version="1.0.0",
        dependencies=[]
    )

    def __init__(self, param1=None):
        super().__init__(self.meta)
        self.param1 = param1

    def apply(self, input_data, **kwargs):
        try:
            # Your processing logic here
            processed_data = self._process(input_data)

            return TechniqueResult(
                success=True,
                payload={"result": processed_data}
            )
        except Exception as e:
            return TechniqueResult(
                success=False,
                error=str(e)
            )

    def _process(self, data):
        # Implementation details
        return data

Best Practices¶

One technique per file: Keep each technique in its own file
Clear metadata: Provide comprehensive TechniqueMeta information
Error handling: Always return TechniqueResult with proper error handling
Documentation: Include docstrings and type hints
Testing: Add comprehensive tests for your technique
Dependencies: Use optional dependencies for external libraries

For detailed contribution guidelines, see our Contributing Guide.