chatbot-evaluation

Tune your AI Agent to best meet its KPI with a cyclic process of analyze, improve and simulate

Exploratory data analysis and interactive model-understanding and evaluation tool for chatbot training data and feedback

An open-source framework for robust, LLM-powered testing and tracing of conversational AI applications.

AI chatbot evaluation benchmark for mental health and suicide prevention with rule-based ethical alignment, inclusivity scoring, and sentiment analysis. Research paper submitted to AI & Society (Springer Nature) under review.

Evaluation results and experimental data for TRACER, demonstrating its effectiveness in discovering chatbot functionalities and detecting errors with coverage analysis and mutation testing.

This is a repository for a Jupyter based tool to calculate Greedy Matching, Vector Extrema and Average Embedding evaluation metrics for generative AI chatbots

Customer-service auto-reply quality evaluation pipeline | LLM-as-a-Judge (DeepSeek) + TF-IDF fallback | 4 weighted metrics with human-annotation consistency check (90%)

Privacy-aware chatbot evaluation pipeline for Human-AI Interaction Design

Trustworthy AI evaluation tool for testing chatbot safety, reliability, hallucination behavior, privacy risk, and instruction-following quality.

Open benchmark defining emotional connection in AI characters and companions as 8 observable behaviors: listening, attunement, curiosity, honest validation, continuity, character consistency, emotional range and user agency.

Automated LLM evaluation framework for a fictional ShopMate customer support chatbot using golden datasets, keyword matching, ROUGE-L, LLM-as-Judge, prompt variants, and model comparisons.

First publication & part of Master’s thesis at HCMUS: End-to-end automatic evaluation framework for retrieval-augmented chatbots (accepted at ACIIDS 2026)

Self-hosted LLM/Agent batch evaluation platform with OpenAI-compatible API testing, streaming response parsing, LLM-as-a-Judge scoring, Docker deployment, and CSV/XLSX export.

Structured evaluation of 30 support bot conversations measuring containment, escalation rate, intent accuracy, and CSAT correlation, LLM-assisted qualitative coding, edge cases, and recommendations.

This NTU project presents a comprehensive approach to predicting human preferences for large language model (LLM) outputs in a multi-model ranking scenario. We combine knowledge distillation techniques with novel NLP-based preference modeling that integrates logical consistency and human-aligned desirability metrics.