MAML for Scientific Claim Detection

Model-Agnostic Meta-Learning Applied to Few-Shot Scientific Claim Identification

Author: Mohammad Asadolahi — Senior Agentic AI Engineer

Focus: Agentic AI Architectures In The Wild

---

• Summary
• Motivation & Problem Statement
• Architecture
• MAML Algorithm
• Datasets
• Project Structure
• Quick Start
• Hyperparameters
• References
• Citation

This project implements a Model-Agnostic Meta-Learning (MAML) framework for few-shot scientific claim detection — the task of identifying whether a sentence in a scientific document constitutes a factual claim. The system uses bi-level optimization over heterogeneous source tasks to learn an initialization that enables rapid adaptation to new scientific domains with limited labeled data.

The model uses a SciBERT encoder with a custom classification head, trained via MAML's bi-level gradient procedure with 2 source tasks and a target task (CoDiE dataset).

Scientific claim detection is a critical component of automated fact-checking, systematic review pipelines, and knowledge extraction systems. Traditional supervised approaches struggle when:

1. Labeled data is scarce — Annotating scientific claims requires domain expertise and is prohibitively expensive
2. Domains shift rapidly — New scientific fields emerge faster than training data can be curated
3. Class imbalance is severe — Claims constitute a small fraction of sentences in scientific papers

Approach: Instead of training a single model on a fixed dataset, this project uses MAML to learn a meta-initialization — a set of model parameters that can be quickly fine-tuned to any new scientific domain with just a handful of labeled examples.

$$\theta^* = \arg\min_\theta \sum_{i=1}^{N} \mathcal{L}_{\mathcal{T}_i}\left(f_{\theta_i'}\right) \quad \text{where} \quad \theta_i' = \theta - \alpha \nabla_\theta \mathcal{L}_{\mathcal{T}_i}(f_\theta)$$

The system combines a SciBERT encoder (pre-trained on scientific papers from Semantic Scholar) with a custom classification head, optimized via MAML's bi-level gradient procedure.

┌─────────────────────────────────────────────────────────────────┐
│                     SciBERT Encoder                              │
│  Input IDs → Token Embeddings → 12× Transformer Layers → [CLS] │
└───────────────────────────┬─────────────────────────────────────┘
                            │ 768-dim [CLS] representation
                            ▼
┌─────────────────────────────────────────────────────────────────┐
│                  Custom Classification Head                      │
│  Dropout(0.5) → Linear(768→128) → Dropout(0.8) → BatchNorm(128)│
│  → Linear(128→1) → BatchNorm(1) → Sigmoid                      │
└───────────────────────────┬─────────────────────────────────────┘
                            │
                            ▼
                   P(claim) ∈ [0, 1]

The aggressive dropout (0.8) in the classification head is designed to prevent overfitting during the inner-loop adaptation — a common failure mode in meta-learning with large language models.

MAML optimizes for learning ability rather than task performance. The meta-objective finds parameters $\theta$ such that one or a few gradient steps on a new task produces maximally effective task-specific parameters.

For each source task $\mathcal{T}_i$, clone the meta-model and adapt on the support set:

$$\theta_i' = \theta - \alpha \sum_{t=1}^{10} \nabla_\theta \mathcal{L}_{\text{support}}^{(t)}(f_\theta)$$

• K = 192 support samples per task
• 10 inner gradient steps with AdamW
• Learning rate $\alpha = 2 \times 10^{-5}$

Evaluate each adapted model on the target query set and aggregate gradients:

$$\theta \leftarrow \theta - \beta \frac{1}{N} \sum_{i=1}^{N} \nabla_\theta \mathcal{L}_{\text{query}}(f_{\theta_i'})$$

• K = 32 query samples from the target domain
• Learning rate $\beta = 2 \times 10^{-5}$
• N = 2 source tasks per meta-step

for step in range(meta_steps):
    gradients = []
    query_batch = sample(target_train, k=32)

    for task in source_tasks:
        # INNER LOOP — Task-specific adaptation
        fast_model = deepcopy(meta_model)
        support = sample(task, k=192)

        for _ in range(10):  # Inner gradient steps
            loss = BCE(fast_model(support), support.labels)
            fast_model.update(lr=2e-5)

        # Evaluate adapted model on target query
        query_loss = BCE(fast_model(query_batch), query_batch.labels)
        gradients.append(query_loss.grad)

    # OUTER LOOP — Meta-parameter update
    meta_model.grad = mean(gradients)
    meta_model.update(lr=2e-5)

The framework uses data from multiple heterogeneous scientific corpora. Two are used as source tasks for meta-training, and one as the target task for few-shot evaluation. Datasets are not included in this repository.

Raw Data → HTML/Tag Stripping → Sentence Splitting (WtP) → Length Filtering
    → Tokenization (SciBERT Tokenizer) → Episodic Batching → MAML Training

MAML-Scientific-Claim-Detection/
│
├── MAML Scientific Claim detection.ipynb   # Original research notebook
│
├── src/                                     # Source code
│   ├── __init__.py
│   ├── data_loader.py                       # Multi-source dataset pipeline
│   ├── model.py                             # SciBERT + Classification head
│   ├── maml_learner.py                      # MAML bi-level optimization engine
│   ├── train.py                             # End-to-end training script
│   └── evaluate.py                          # Evaluation suite
│
├── assets/                                  # Figures & visualizations
│   └── architecture.png                     # System architecture diagram
│
├── datasets/                                # Data directory (not included)
│   ├── train_labels.json
│   ├── test_labels.json
│   ├── validation_labels.json
│   ├── claims_train.jsonl
│   ├── claims_dev.jsonl
│   ├── claims_test.jsonl
│   ├── final_ann.csv
│   ├── outof_chunked.csv
│   └── full/                                # Brat annotation files
│       ├── A01.txt ... A40.txt
│       └── A01.ann ... A40.ann
│
├── models/                                  # Saved checkpoints
├── generate_plots.py                        # Visualization generation script
└── README.md

# Python 3.8+
pip install torch transformers scikit-learn pandas numpy matplotlib
pip install wtpsplit sentence_splitter  # For sentence segmentation

# Full MAML meta-training pipeline
python -m src.train \
    --dataset_root datasets \
    --save_dir models \
    --steps 10000 \
    --k_support 192 \
    --k_query 32 \
    --inner_lr 2e-5 \
    --outer_lr 2e-5 \
    --inner_steps 10 \
    --hidden_size 128

# Evaluation with metrics & confusion matrices
python -m src.evaluate \
    --model_path models/best_model.pt \
    --dataset_root datasets

python generate_plots.py

1. Finn, C., Abbeel, P., & Levine, S. (2017). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. ICML 2017. [Paper]

2. Beltagy, I., Lo, K., & Cohan, A. (2019). SciBERT: A Pretrained Language Model for Scientific Text. EMNLP 2019. [Paper]

3. Wadden, D., et al. (2020). Fact or Fiction: Verifying Scientific Claims. EMNLP 2020. [Paper]

4. Minixhofer, B., et al. (2023). Where's the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence Segmentation. ACL 2023. [GitHub]

@software{asadolahi2024maml_claim,
    title     = {MAML for Scientific Claim Detection: 
                 Few-Shot Meta-Learning with SciBERT},
    author    = {Asadolahi, Mohammad},
    year      = {2024},
    publisher = {GitHub},
    url       = {https://github.com/MohammadAsadolahi/MAML-Scintific-Claim-Detection-using-Meta-Learning}
}