toxicity-detection

Local AI text classifier for Vietnamese: scam/phishing detection, emotion and sentiment analysis, toxicity and spam checks. Django + Hugging Face, runs on CPU, no API key.

It is a trained Deep Learning model to predict different level of toxic comments. Toxicity like threats, obscenity, insults, and identity-based hate.

Detecting prompt toxicity from a small LLM's internal logits and embeddings, no separate moderation model needed. 99.18% AUPRC with a 64-neuron MLP.

Telegram bot that detects toxic comments based on Perspective API

Detecting Toxic comments using machine learning

Production-grade AI safety middleware — prompt injection detection, PII masking (Indian Aadhaar/PAN), toxicity filtering, structured outputs via FastAPI

“Built a multi-label toxic comment classification system using TF-IDF features and a One-Vs-Rest Logistic Regression model, evaluated with F1-score and ROC-AUC to handle class imbalance.”

Russian text toxicity detection.

Toxicity detection in a conversation or phases.

Interactive Flask-based dashboard for visualizing data with Chart.js, AJAX, and Bootstrap. Integrates LLM chatbot (Hugging Face) to answer questions in real-time, showcasing trends and toxicity analysis from Reddit, and 4chan.

A multilingual text analysis system that performs sentiment analysis and toxicity detection with detoxified text generation.

A GitHub Action that hides abusive issue, pull request and discussion comments with the Unprofane profanity and toxicity API

Stop prompt injections in 20ms. The safety toolkit every LLM app needs. No API keys, no complex setup, just `pip install llm-guard` and you're protected.

Post-Processing AI Quality API

Natural language processing project combining semantic embeddings and toxicity detection to analyze Brexit-related immigration discourse.

A novel approach to news summarization that promotes positive and constructive narratives by integrating sentiment and toxicity control into the summarisation process.

A comprehensive evaluation of LLM trustworthiness (toxicity, bias, and privacy) using the DecodingTrust framework on Microsoft Phi-2.

Preference optimization for context-aware toxicity detection using Reddit community feedback.

Top AI Content Moderation (Opensource) 🌟 Star if you like it! 🌟