A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

Fully neural approach for text chunking

🍱 Semantically create chunks from large document for passing to LLM workflows

RAG boilerplate with semantic/propositional chunking, hybrid search (BM25 + dense), LLM reranking, query enhancement agents, CrewAI orchestration, Qdrant vector search, Redis/Mongo sessioning, Celery ingestion pipeline, Gradio UI, and an evaluation suite (Hit-Rate, MRR, hybrid configs).

Turn speech into semantic paragraphs in real time, on a CPU (no GPU) — single-pass streaming chunker + Moonshine STT + a live mic demo. Plus a transcribe/chunk/summarize API.

Rust CLI implementing the Recursive Language Model (RLM) pattern for Claude Code. Process documents 100x larger than context windows through intelligent chunking, SQLite persistence, and recursive sub-LLM orchestration.

Token-aware, LangChain-compatible semantic chunker with PDF, markdown, and layout support

🍶 llm-distillery ⇢ use LLMs to run map-reduce summarization tasks on large documents until a target token size is met.

Semantic Chunking is a Python library for segmenting text into meaningful chunks using embeddings from Sentence Transformers.

Advanced semantic text chunking with custom structural markers, whole-text coherence preservation, and flexible token management. Features async processing, LangChain integration, and dynamic drift detection. Ideal for RAG systems, augmented text processing, and domain-specific document analysis.

A research framework tA research framework to evaluate how document parsing quality determines downstream RAG performance.o evaluate how document parsing quality de

Advanced local-first RAG system powered by Ollama and LangGraph. Optimized for high-performance sLLM orchestration featuring adaptive intent routing, semantic chunking, intelligent hybrid search (FAISS + BM25), and real-time thought streaming. Includes integrated PDF analysis and secure vector caching.

Sementic chunking algorithm in (mostly) Go

AI-native data annotation pipeline using Dynamic In-Context Learning (ICL) routing. Leverages RAG to retrieve semantically relevant few-shot examples for long-context document chunking and targeted LLM prompting.

HR Policy Assistant (RAG-based Chatbot) A conversational AI assistant for employees to query company HR policies. Built with LangChain and Qdrant, it semantically ingests HR documents, retrieves relevant policy information, reranks results with BM25/MMR, and delivers precise LLM-generated responses.Cloud-based vector storage ensure quick responses.

A high-performance Retrieval-Augmented Generation pipeline for technical Q&A workloads. Combines hybrid retrieval (dense + BM25), query expansion, Reciprocal Rank Fusion (RRF), and cross-encoder re-ranking to improve retrieval precision and answer grounding. Evaluated with Ragas, showing measurable gains in context recall and faithfulness.

A hands-on guide to RAG techniques using LangGraph.

High-performance, zero-dependency Markdown parser and semantic chunking tree for RAG and LLM agent contexts.

Cutting-edge semantic text processing system that uses hierarchical clustering and advanced language models to automatically organize and summarize large volumes of text.

A modular RAG pipeline for automated document processing using Semantic Chunking and Qdrant Vector Database.