GoEmotions Emotion Classification

A reproducible five-class emotion-classification study based on the GoEmotions dataset. The project compares TF-IDF machine-learning baselines with a fine-tuned BERT model and publishes the verified test metrics, analysis figures, and training configuration.

Originally developed as a university project in 2025 and refactored, rerun, tested, and documented for public release in 2026.

• Five target emotions: joy, sadness, anger, fear, and surprise
• Official GoEmotions train, validation, and test splits
• Ambiguous multi-label examples excluded from the single-label task
• Logistic Regression, Linear SVM, Multinomial Naive Bayes, and BERT
• Reproducible command-line training and evaluation scripts
• Verified on an NVIDIA GeForce RTX 4050 Laptop GPU
• Automated JSON metrics, confusion matrices, and Matplotlib report figures
• Test suite: 3 passed

The final test set contains 478 examples. BERT correctly classified 421 examples.

BERT improved test accuracy by approximately 9.2 percentage points and Macro F1 by approximately 9.0 percentage points over the strongest classical baseline.

The lowest BERT F1 score was observed for surprise. The confusion matrix shows that surprise examples were most often confused with anger and fear.

The original GoEmotions task is multi-label. This repository defines a focused single-label experiment:

1. Keep examples with exactly one original emotion label.
2. Keep only examples labeled as joy, sadness, anger, fear, or surprise.
3. Preserve the official train, validation, and test split membership.

The source dataset is downloaded at runtime from google-research-datasets/go_emotions. The dataset card lists the dataset under Apache License 2.0 and documents the Reddit source, annotation process, and known bias considerations.

• TF-IDF word unigrams and bigrams
• Maximum 20,000 features
• Logistic Regression
• Linear Support Vector Machine
• Multinomial Naive Bayes

• google-bert/bert-base-uncased
• Maximum sequence length: 128
• Batch size: 8
• Learning rate: 2e-5
• Epochs: 3
• Best-checkpoint selection based on validation Macro F1
• Test evaluation performed after training

The recorded BERT training runtime was approximately 155.8 seconds on the RTX 4050 system.

Validation Macro F1 increased across the three epochs, while validation loss rose after the first epoch. This is treated as a mild overfitting signal rather than hidden: the report discusses both trends.

.
├── artifacts/
│   └── metrics/                 # Compact verified JSON results
├── assets/                      # Published plots and confusion matrices
├── docs/
│   └── original_project_notes.md
├── reports/
│   └── GoEmotions_Technical_Report_TR.pdf
├── results/
│   └── README.md                # Local generated-result policy
├── scripts/
│   ├── collect_report_inputs.py
│   ├── generate_report_figures.py
│   ├── profile_dataset.py
│   ├── system_info.py
│   ├── train_bert.py
│   └── train_classical.py
├── src/goemotions_classifier/
│   ├── constants.py
│   ├── data.py
│   └── metrics.py
├── tests/
│   └── test_data.py
├── .gitignore
├── LICENSE
├── pyproject.toml
└── README.md

Python 3.10-3.12 is recommended.

python -m venv .venv

Windows PowerShell:

.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e ".[bert,dev]"

macOS/Linux:

source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ".[bert,dev]"

For CUDA acceleration, install a CUDA-enabled PyTorch build appropriate for the local NVIDIA driver before installing the project extras.

python scripts/system_info.py
python scripts/profile_dataset.py
python scripts/train_classical.py
python scripts/train_bert.py --epochs 3 --batch-size 8
pytest -q

Generate compact report inputs:

python scripts/collect_report_inputs.py

Regenerate the published Matplotlib figures from the compact JSON metrics:

python scripts/generate_report_figures.py

Local training outputs are written under results/ and ignored by Git because they can include model checkpoints and serialized pipelines. The repository commits only:

• Compact verified JSON metrics under artifacts/metrics/
• Selected publication figures under assets/
• The final technical report under reports/

The trained BERT checkpoint is intentionally not committed to GitHub.

• Official train, validation, and test splits are preserved.
• Ambiguous multi-label examples are not forced into an arbitrary class.
• Classical-model preprocessing is kept separate from BERT tokenization.
• Multiclass ROC-AUC is calculated with a one-vs-rest formulation.
• BERT logits are converted to probabilities before probability-based metrics.
• Seeds, run configuration, system information, and outputs are recorded.
• Metrics are reported from a fresh rerun rather than copied from the original report.

• The task is restricted to five emotions and does not represent the full GoEmotions label space.
• Removing multi-label examples simplifies the task and discards emotionally mixed comments.
• The dataset consists of English Reddit comments and may reflect platform, demographic, annotation, and language biases.
• Emotion predictions should not be treated as objective psychological assessments.
• This model is unsuitable for high-stakes decisions such as hiring, insurance, education assessment, or clinical diagnosis.
• Linear SVM decision scores are useful for ranking but are not calibrated probabilities; ROC-AUC is therefore treated as a secondary metric.

• Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., & Ravi, S. (2020). GoEmotions: A Dataset of Fine-Grained Emotions. Proceedings of ACL 2020.
• Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019.

Project source code is released under the MIT License. The GoEmotions dataset and pretrained models retain their own licenses and terms.