Your AI assistant handles thousands of conversations daily. But do you know what users actually need?
Kura is an open-source library for understanding chat data through machine learning, inspired by Anthropic's CLIO. It automatically clusters conversations to reveal patterns, pain points, and opportunities hidden in your data.
The Hidden Cost of Not Understanding Your Users
Every day, your AI assistant or chatbot has thousands of conversations. Within this data lies critical intelligence:
- 80% of support tickets might stem from the same 5 unclear features
- Key feature requests repeated by hundreds of users in different ways
- Revenue opportunities from unmet needs you didn't know existed
- Critical failures affecting user trust that go unreported
Manually reviewing conversations doesn't scale. Traditional analytics miss semantic meaning. Kura bridges this gap.
Kura transforms unstructured conversation data into structured insights:
10,000 conversations → AI Analysis → 20 clear patterns
- Automatic Intent Discovery: Find what users actually want (not what they say)
- Failure Pattern Detection: Identify where your AI falls short before users complain
- Feature Priority Insights: See which missing features impact the most users
- Semantic Clustering: Group by meaning, not keywords
- Privacy-First Design: Analyze patterns without exposing individual conversations
Challenge: 50,000 weekly conversations, unknown pain points Discovery: 35% of conversations about shipping clustered into 3 issues Result: Fixed root causes, reduced support volume by 40%
Challenge: Users struggling but not reporting specific issues Discovery: 2,000+ conversations revealed 5 consistently confusing APIs Result: Targeted doc improvements, 60% reduction in those queries
Challenge: 30% of trials not converting, unclear why Discovery: Clustering revealed 3 missing integration requests Result: Built integrations, trial conversion increased 18%
uv pip install kuraKura is perfect when you have:
- 100+ conversations to analyze (scales to millions)
- A need to understand user patterns, not individual conversations
- Unstructured conversation data from chatbots, support systems, or AI assistants
- Questions like "What are users struggling with?" or "What features are they requesting?"
Kura might not be the best fit if:
- You have fewer than 100 conversations (manual review might be faster)
- You need real-time analysis (Kura is designed for batch processing)
- You only need keyword-based search (use traditional search tools instead)
- You require conversation-level sentiment analysis (Kura focuses on patterns)
- Feature Discovery: Find the features users ask for in their own words
- Pain Point Analysis: Identify friction in user journeys
- Roadmap Prioritization: Quantify impact of potential improvements
- Support Deflection: Find common issues to create better docs/FAQs
- Escalation Patterns: Identify conversations that lead to churn
- Success Patterns: Discover what makes users successful
- Prompt Engineering: Find where prompts fail or confuse users
- Model Evaluation: Understand model performance beyond metrics
- Training Data: Identify gaps in model knowledge
- Behavioral Insights: Understand user segments by conversation patterns
- Trend Analysis: Track how user needs evolve over time
- ROI Measurement: Connect conversation patterns to business outcomes
import asyncio
from rich.console import Console
from kura.cache import DiskCacheStrategy
from kura.summarisation import summarise_conversations, SummaryModel
from kura.cluster import generate_base_clusters_from_conversation_summaries, ClusterDescriptionModel
from kura.meta_cluster import reduce_clusters_from_base_clusters, MetaClusterModel
from kura.dimensionality import reduce_dimensionality_from_clusters, HDBUMAP
from kura.visualization import visualise_pipeline_results
from kura.types import Conversation
from kura.checkpoints import JSONLCheckpointManager
async def main():
console = Console()
# Define Models
summary_model = SummaryModel(
console=console,
cache=DiskCacheStrategy(cache_dir="./.summary"), # Uses disk-based caching
)
cluster_model = ClusterDescriptionModel(console=console) # Uses K-means by default
meta_cluster_model = MetaClusterModel(console=console)
dimensionality_model = HDBUMAP()
# Define Checkpoints
checkpoint_manager = JSONLCheckpointManager("./checkpoints", enabled=True)
# Load conversations from Hugging Face dataset
conversations = Conversation.from_hf_dataset(
"ivanleomk/synthetic-gemini-conversations", split="train"
)
# Process through the pipeline step by step
summaries = await summarise_conversations(
conversations, model=summary_model, checkpoint_manager=checkpoint_manager
)
clusters = await generate_base_clusters_from_conversation_summaries(
summaries, model=cluster_model, checkpoint_manager=checkpoint_manager
)
reduced_clusters = await reduce_clusters_from_base_clusters(
clusters, model=meta_cluster_model, checkpoint_manager=checkpoint_manager
)
projected_clusters = await reduce_dimensionality_from_clusters(
reduced_clusters,
model=dimensionality_model,
checkpoint_manager=checkpoint_manager,
)
# Visualize results
visualise_pipeline_results(projected_clusters, style="basic")
if __name__ == "__main__":
asyncio.run(main())- Loads 190 real programming conversations from Hugging Face
- Summarizes each conversation into a concise task description (with caching!)
- Clusters similar conversations using MiniBatch K-means for speed
- Organizes clusters into a hierarchy for easy navigation
- Visualizes the results in your terminal
Programming Assistance (190 conversations)
├── Data Analysis & Visualization (38 conversations)
│ ├── R Programming for statistical analysis (12 conversations)
│ ├── Tableau dashboard creation (10 conversations)
│ └── Python data manipulation with pandas (16 conversations)
├── Web Development (45 conversations)
│ ├── React component development (20 conversations)
│ ├── API integration issues (15 conversations)
│ └── CSS styling and responsive design (10 conversations)
├── Machine Learning (32 conversations)
│ ├── Model training with TensorFlow (18 conversations)
│ └── Data preprocessing challenges (14 conversations)
└── ... (more clusters)
Total processing time: 21.9s (2.1s with cache!)
Checkpoints saved to: ./checkpoints/
See CONTRIBUTING.md for development setup, testing, and contribution guidelines.
Kura is under active development. If you face any issues or have suggestions, please feel free to open an issue or a PR. For more details on the technical implementation, check out this walkthrough of the code.