Periodically scans major news publishers worldwide and reports how often they preserve embedded photo metadata (EXIF, IPTC IIM, IPTC XMP, C2PA). A project of IPTC.
See SPEC.md for the full specification.
crawler/ Python crawler (CLI: pmd-crawler)
config/publishers/ Per-country YAML publisher lists ({cc}.yaml)
data/runs/ Crawl results, one directory per monthly run (Parquet)
site/ Astro static site (deployed to GitHub Pages)
.github/ CI + scheduled crawl workflows
# One-time setup
python3 -m venv .venv
.venv/bin/pip install -e './crawler[dev]'
.venv/bin/pre-commit install # ruff + whitespace checks on every commit
brew install exiftool # macOS — or apt-get install exiftool
# Smoke-test discovery against one site
.venv/bin/pmd-crawler smoke-test --site-id ap
# Run a small crawl (one or a few sites)
.venv/bin/pmd-crawler run \
--site-id ap,reuters,bbc-news,ny-times,le-monde \
--output data/runs/$(date -u +%Y-%m-%d)-smoke
# Export the latest run to JSON for the site
.venv/bin/python crawler/scripts/export_for_site.py
# Dev-preview the static site
cd site && npm install && npm run dev- The monthly crawl is triggered by
.github/workflows/crawl.yml(cron, 02:00 UTC on the 1st) and commits the run todata/runs/. - The static site is rebuilt and deployed to GitHub Pages by
.github/workflows/build-site.ymlwheneversite/ordata/runs/changes onmain. - Deployed at https://metawatch.iptc.org/.
- Code: MIT (see LICENSE).
- Data: CC BY 4.0.