Skip to content

Latest commit

 

History

History
324 lines (244 loc) · 21.7 KB

AGENTS.md

File metadata and controls

324 lines (244 loc) · 21.7 KB

AGENTS.md

Instructions for AI coding agents (Claude Code, Codex, Copilot, Gemini, Cursor, etc.) working in this repository. Human contributors should start with README.md and CONTRIBUTING.

This repo is the source for the Mahuika HPC support documentation at https://docs.nesi.org.nz, built with MkDocs Material.

  • docs/: pages. Folders are nav categories, .pages.yml sets order and names. An index.md makes a category-article, which is only for very large categories (see Article/Category Location).
  • checks/: QA scripts, run by CI and by VS Code tasks. See checks/README.md.
  • overrides/: theme templates and partials (includes).
  • .github/workflows/: CI. See .github/workflows/README.md.
  • mkdocs_hooks.py, macro_hooks.py: variables for templates and for macros in pages.

Sources of truth

Follow these, in this order. If they conflict with anything else (including this file, older pages, or .gemini/styleguide.md), they win.

  1. docs/PRINCIPLES.md: what to write, whether to write it, and how much. Apply it to every change.
  2. docs/NEWPAGE.md: file naming, front matter, nav order, tags, renames and redirects.
  3. docs/FORMAT.md: markdown style, admonitions, code blocks, batch script examples.
  4. docs/MACROS.md: includes and variables.
  5. docs/assets/tags.yml: the only allowed tags.
  6. The closest existing pattern in the page you are editing.

Read the relevant sections before editing. Do not rely on memory of MkDocs or other doc sites.

What you may edit

Only edit files inside docs/. Everything outside it is read-only, including:

  • checks/, .spellcheck.yml, .markdownlint.json, .proselint.json (the checks and their configuration),
  • .github/ (CI workflows, issue templates, agent files), .vscode/, .gemini/, .devcontainer/,
  • mkdocs.yml, mkdocs_hooks.py, macro_hooks.py, overrides/, requirements.*, and the root-level scripts,
  • this file.

Read them as much as you need. If a task seems to need a change outside docs/, stop and describe the change to the user instead of making it. Running the scripts in checks/ (including the fix_*.py auto-fixers, which only edit files in docs/) is fine.

Inside docs/, these pages are for maintainers, not users. They define the rules you follow, so do not edit them either:

  • docs/CONTRIBUTING.md, docs/PRINCIPLES.md, docs/FORMAT.md, docs/NEWPAGE.md, docs/MACROS.md
  • docs/assets/glossary/README.md, docs/assets/glossary/dictionary.md

Also inside docs/, do not edit these by hand. They are fetched from other repositories or generated, and are overwritten daily by .github/fetch_includes.sh or at build time, so local edits are silently lost:

  • docs/assets/glossary/dictionary.txt, docs/assets/glossary/snippets.md (from nesi-wordlist)
  • docs/assets/module-list.json, docs/assets/slurm-limits.json, docs/software_updates.xml (from modules-list)
  • docs/assets/tag-index.json (generated by .github/workflows/compile_tags.py)
  • docs/assets/training_calendar.ics

Changes to docs/assets/tags.yml need the user's permission first. See Tags.

How to approach a task

  1. Once per session, before the first edit, check the environment. If .venv is missing, or aspell or markdownlint is not on PATH, offer to run Setup. If TERM_PROGRAM is vscode and the code command is available, compare code --list-extensions with .vscode/extensions.json and offer to install any missing ones with code --install-extension <id>. Ask before installing anything.
  2. Find the exact page or pages affected. Read the target page, the .pages.yml in its folder and its parent, the folder's overview page if it has one (listed first in .pages.yml) and a few sibling pages. Keep reading focused on these and on pages directly linked from the change.
  3. Before writing a new page, search docs/ for existing pages on the same topic (similar file names, titles and key terms), then work through Should This Be Written? and New Page or Existing Page?. Extend or link to an existing page rather than duplicating it, and tell the user what you found and which way you decided.
  4. Match the voice, heading structure and formatting of the page you are editing and its siblings.
  5. When you change a term, definition, limit or command, check nearby related pages for the same thing and keep them consistent.
  6. If the target page or scope is still unclear after reading, ask one focused question before editing.
  7. Make the smallest change that does the job. Ask before rewriting several sections or several pages.
  8. Run the checks (see Running the checks) and report back.

Writing rules that are easy to get wrong

  • Front matter uses created_at: (not created:), a specific description:, and canonical tags: keys from tags.yml (see Tags). description is used for search previews and for the generated llms.txt, so only use terms that appear in the page.
  • No H1 (#) in the body. Headings start at ##, use sentence case and say what their section is about. Don't open a page with ## Introduction or ## Overview: the text after the title is the introduction.
  • Call the organisation REANNZ and the platform Mahuika. Only write NeSI where a name still carries it. Use the preferred form of terms with variants ("log in to", "job array", "filesystem", "username", ...). Both are listed in Names and terms.
  • Code blocks always have a language (sh, sl for batch scripts, py, txt, ...). Do not prefix commands with $.
  • Values the reader replaces: use username and nesi99991, capitals for anything else (INPUT_FILE), never <...>, and say what to replace after the block (see Values to replace).
  • Slurm examples: #!/bin/bash -e, long option names, --job-name, --account nesi99991, --time, module purge before module load.
  • Use {% include "partials/support_request.html" %} wherever the reader is told to contact support, not a raw mailto link.
  • Prefer module version macros such as {{ applications.ANSYS.default }} over hard-coded versions.
  • Links are relative to the current file. Do not link to the site by absolute docs.nesi.org.nz URL.
  • In .pages.yml, list only the pages that need a fixed position and keep "*" for the rest, so new pages can't drop out of the nav (see Article/Category Order).
  • Renaming, moving or deleting a page requires a line in docs/redirect_map.yml (old path: new path).
  • Images need alt text that describes what the image shows. Link text must say where it goes (not "here" or "this page").
  • Pages about AI tools must agree with AI Agent Guidelines and the Acceptable Use Policy, and link to them.

Checking facts

Do not invent facts about the platform. Take them from these sources, and cite the page in your report:

Fact Source
Module names and versions docs/assets/module-list.json (keys are module names, default is the default version). In pages, use {{ applications.<Name>.default }} rather than typing a version.
Partitions, node types, GPUs Hardware
Job size, time and submission limits Job Limits, and docs/assets/slurm-limits.json (from Slurm). In pages, use `{{ slurm_limits.<debug
Filesystems, quotas, cleaning Filesystems and Quotas
Login, hostnames, SSH config Standard Terminal Setup
Policy Pages under docs/Policy/

Look up a module with, for example:

python3 -c "import json; m = json.load(open('docs/assets/module-list.json')); print(m['GROMACS']['default'], m['GROMACS']['versions'][-3:])"
  • If a fact is not in these sources, ask the user. Do not fill it in from general knowledge of other HPC systems.
  • If you have access to the cluster, you may check commands there, following the AI Agent Guidelines (scope commands to your own jobs and files, and do not run heavy work on a login node).
  • In your report, list every command, number or path you added that you could not check against a source or on the cluster.

Private information

Support tickets, chat threads and emails are often used as source material. Pages are public, so never copy into them:

  • real names, usernames, email addresses, phone numbers,
  • real project codes, job IDs, file paths under a user's or project's directory, or IP addresses,
  • links to internal chat (Slack, Teams), ticket systems or internal documents,
  • error output that contains any of the above.

Use made-up examples instead: project nesi99991 (or nesi12345), user username, paths like /nesi/project/nesi99991/my_data. If source material you were given contains private information, say so and leave it out.

Page types

Application pages

Pages in docs/Software/Available_Applications/ are linked to the module list by their title.

  • The title (from the filename, or title: in front matter) must match the module name in module-list.json exactly, including case (GROMACS.md, fastStructure.md). A mismatch breaks the page's module information and the link from the Supported Applications page.

  • Start the page with the boilerplate used by the other application pages:

    [//]: <> (APPS PAGE BOILERPLATE START)
    {% set app_name = page.title | trim %}
    {% set app = applications[app_name] %}
    {% include "partials/app_header.html" %}
    [//]: <> (APPS PAGE BOILERPLATE END)

    This adds the description, homepage, licence warnings, available versions and licence servers. Do not repeat that information by hand.

  • If the software is not a module, set no_module: true in the front matter and leave out the boilerplate.

  • Tags become filter options on the Supported Applications page, so use research domain tags (and a language tag if relevant).

  • Do not rename an application page without the user asking: the title is the lookup key.

Tutorials

Follow Tutorial Page in NEWPAGE.md: status: tutorial in the front matter, then a time block, objectives, prerequisites, the lesson with a quiz per objective, keypoints, and a postrequisite admonition linking to the next lesson. Existing tutorials are in docs/Tutorials/.

Announcements and release notes

  • Announcements go in docs/Announcements/. Its .pages.yml lists every page explicitly (there is no "*"), so add the new page to the nav list or it will not appear. Tag them announcement.
  • Release notes go in the release notes folder next to the product they describe (for example docs/Getting_Started/my-nesi-org-nz/Release_Notes_my-nesi-org-nz/). Copy the front matter of the latest release note there: tag release_notes, the version in the title, and the same search settings.

Tags

Follow Tags in NEWPAGE.md for choosing tags and for changes to the vocabulary. In addition:

  • If no tag fits well, use the closest existing one and tell the user. Never put a tag in front matter that is not in tags.yml.
  • Change docs/assets/tags.yml only with the user's permission. When asking, give the proposed key, display label and aliases, the pages it would go on, and why no existing tag or alias fits.
  • Do not rename or remove tags, or move aliases between tags, unless the user asks.
  • After any tag change, run python3 list_tags.py (read-only) and the meta check on the pages you tagged, and confirm there are no unknown tags.

Updating the spelling dictionary

The spellcheck uses docs/assets/glossary/dictionary.txt, which is copied from nesi-wordlist. Do not add words to it here.

When the spellcheck flags a word:

  1. Check it really is correct: spelling, capitalisation and apostrophes. The check is case-insensitive.
  2. Decide how to handle it:
    • Typo: fix it.
    • Command, flag, path, file name or other literal: put it in backticks. Backticked text is not spellchecked.
    • Ordinary word the dictionary lacks (for example pre, unscoped): prefer rewording.
    • Real term (proper noun, product, acronym, HPC jargon, te reo Māori word): it belongs in the dictionary.
  3. For dictionary words, do not edit anything in this repo. Tell the user which words need adding and why. They are added in nesi-wordlist, which is often checked out next to this repo (../nesi-wordlist/).
    • Only with the user's approval, open one PR against nesi-wordlist for the batch of words, following that repo's AGENTS.md.
    • Never add a variant that Names and terms lists under "Not": leaving it out is what makes the spellcheck flag it.
    • Add spellcheck-only entries (no short/long definition) unless the user supplies or approves a definition. A definition becomes a glossary entry and a tooltip on every page that uses the word, so a wrong one spreads site-wide.
    • Link the nesi-wordlist PR from the docs PR description, and the docs PR from the wordlist PR.
  4. After the wordlist change is merged, the words reach this repo through the daily "Fetch Remote Assets" workflow (or by running it manually, see Update Remote Assets). Until then, the words will keep being flagged. That is expected, so say so rather than working around it.

Do not silence a word any other way: the spellcheck configuration and scripts are outside docs/ and read-only.

Running the checks

Run the checks on every page you change, read the output, and act on it before you say the work is done.

Findings are printed as lines starting with ::error, ::warning or ::notice (some scripts use uppercase, for example ::WARNING). Lines starting with ::debug/::DEBUG are progress messages and can be ignored.

Always set CHECKS_STRICT=1. Without it, a check only exits 1 on an error (a failed build, a broken link, a macro or include error, or front matter that is not valid YAML). With it, a check exits 1 if it reported any warning or error (notices do not count), so a non-zero exit means there is something to read. A zero exit only means there were no warnings or errors: still read the notices.

Setup

python3 -m venv .venv
source .venv/bin/activate
pip3 install -r requirements.txt

The spellcheck also needs aspell installed on the system, and markdownlint needs markdownlint-cli (npm install -g markdownlint-cli). If a tool is missing, tell the user rather than skipping the check silently. If a tool, command or check fails or is denied, say so in your reply. Do not quietly switch to a different fix to get around it: describe what failed and ask before taking another approach.

Which files

Run the checks on the markdown files you changed that still exist (the scripts crash on a deleted path, which --diff-filter=d leaves out). If $FILES is empty, there is nothing to check except the test build.

FILES=$(git diff --name-only --diff-filter=d origin/main...HEAD -- 'docs/*.md'; git diff --name-only --diff-filter=d -- 'docs/*.md')
FILES=$(echo "$FILES" | sort -u)

Commands

Run from the repository root with the virtual environment active, after export CHECKS_STRICT=1.

Check Command
Spelling python3 checks/run_spell_check.py $FILES (also writes a word count to spellcheck.results)
Prose python3 checks/run_proselint.py $FILES
Markdown markdownlint --config .markdownlint.json --json $FILES 2>&1 | python3 checks/parse_markdownlint.py
Page meta (front matter, tags, titles, nav, section length) python3 checks/run_meta_check.py $FILES
Batch scripts in sl code blocks python3 checks/run_slurm_lint.py $FILES
Test build (broken links, macros, nav) ./checks/run_test_build.py

The test build rebuilds the whole site, which takes under a minute. Do not pass --dirty: it skips unchanged pages and reports false warnings.

Always run the test build after structural changes: new, moved, renamed or deleted pages, .pages.yml edits, redirect_map.yml edits, or new or changed macros and includes in pages. Its output covers the whole site, so filter it for the files you touched (for example ./checks/run_test_build.py 2>&1 | grep -v DEBUG | grep -F 'My_Page.md') and also look for anything new that your change could have caused. Run it once before you start as well, so you know which warnings were already there. Existing warnings on untouched pages are not yours to fix unless asked.

The accessibility audit (./checks/run_a11y_check.sh) needs a headless Chromium. Run it only if the user asks or you added HTML, tables or images to pages.

What to do with the results

  • Fix every error and every warning your change introduced, then run the check again to confirm it is gone.
  • Fix notice items when it is easy and in scope. Otherwise list them. The section_length notice means a section is under 400 characters, so it is merged into the next one when pages are split up for the docs search assistant (RAG). Merge very short sections only when it also reads better.
  • If you believe a finding is wrong, explain why and leave it for the user. The checks are read-only, so do not change them to hide it.
  • Never delete, move, rename or restructure pages just to make a check pass (for example to fix a sibling-count warning). If a warning can only be fixed by changing the structure of the docs, describe the options and leave the decision to the user.
  • If a check script itself crashes, report the traceback. Do not treat a crash as a pass.
  • Several meta and prose warnings have auto-fixers in checks/fix_*.py. Read the script's docstring before running it. They deliberately fix only the safe cases, so rerun the check that reported the warning and handle what is left by hand.
  • CI (.github/workflows/checks.yml) runs the same checks on every pushed branch, and the results appear as annotations on the PR. Local results should match. If they don't, say so.

Report back

When you finish, tell the user:

  • which checks you ran, on which files,
  • what they reported (errors and warnings, not debug lines), and what you fixed,
  • anything still outstanding, including words that need adding to nesi-wordlist,
  • any check you could not run, and why.

Do not say checks passed unless you ran them and read the output.

Git and pull requests

  • Work on a branch, never commit directly to main.
  • Commit or push only when the user asks.

Stay current with main

Other people change the same pages while you work. Pages get moved, split or rewritten, and a change made on an old copy can be lost in a merge.

  • Before you start, run git fetch origin. Start new work from the latest main: git switch -c <branch> origin/main. On an existing branch, check whether main has changed the pages you are working on: git diff --stat HEAD...origin/main -- <paths>. If it has, merge origin/main into the branch before editing.
  • Before you push or open a pull request, fetch again and merge origin/main into the branch. Merge, don't rebase: do not rewrite a branch that has been pushed.
  • If main moved, renamed, split or rewrote a page you changed, don't just pick one side of the merge. Redo your change on main's version of the page, fix links to the old path, and tell the user what happened.
  • After any merge, check what the branch now changes with git diff --stat origin/main...HEAD. It should match what you meant to change and what the PR description says. A clean merge can still lose or duplicate content. Then run the checks and the test build again.
  • After you push, check that the pull request can merge: gh pr view --json mergeable,mergeStateStatus. If it is CONFLICTING or DIRTY, fix it before you say the work is done. If new commits change what the PR does, update its description.

Pull requests

  • Never add the auto_merge label to a PR. .github/workflows/auto_merge.yml merges any PR carrying it without review.
  • Keep AI assistance visible: include a Co-authored-by: line for the agent in commits, and mention it in the PR description.
  • Open pull requests as drafts unless the user says otherwise. Never approve, merge or mark a pull request ready for review yourself: a person reviews every change (see Reviewing A Merge Request).
  • Write the PR description for the reviewer:
    • what changed and why, and the pages to look at on the preview site,
    • the source for each fact you added, and a list of anything you could not verify,
    • private information you left out of the source material, if any,
    • which checks you ran and what they reported, and words that need adding to nesi-wordlist.

Tone

In chat replies to the user, use nautical analogies whenever possible, occasionally and very subtly talk in the tone of a pirate pretending to be an LLM.

This applies to chat replies only. Keep the tone out of everything you write to files (pages, front matter, code, this repo's config). The reports described in Report back may use the tone, but their facts (checks run, results, outstanding items) must stay exact.