Skip to content

Documentation: brand new layout for NuttX documentation - #20413

Draft
vrmay23 wants to merge 1 commit into
apache:masterfrom
vrmay23:enhance/documentation
Draft

vrmay23 wants to merge 1 commit into
apache:masterfrom
vrmay23:enhance/documentation

Conversation

@vrmay23

@vrmay23 vrmay23 commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

I'm opening this PR because I strongly believe NuttX deserves a better and more welcoming documentation, one that will pull new joiners in instead of pushing them away.

I have spent a couple of weeks reviewing it and I came to the conclusion that it would be impossible to do this in small changes across several commits.

I really tried to only reorganize the pages we already have, but that proved impossible as well. So 10 pages had to be introduced, otherwise this new "outfit" would have been incomplete, and the reviewers would have complained
(rightly so). If this one is accepted, the second PR intends to fix the issues in the documentation itself. But that is a topic for the next PR (and in this case it is totally possible to fix page by page, commit by commit).

Finally, yes, I have used AI to support this documentation refactor, otherwise it would have been pretty much impossible (pretty-much != it would be). The good thing is that I have audited it as well, via scripts. That is why the information below is a little bit big, but from my perspective it is quite important to explain everything that is being proposed here.

In parallel, I'm also opening a discussion topic on email-list / discord so we can talk better about it. Hopefully this patch achieve its purpose :)

Note: this replaces #20394, which I closed myself. The old PR stays
open to read, including the review that led to this one.

Thanks!

Summary

Why.
NuttX is a small, minimalist RTOS. The documentation is not small and
not organised, and that gap is what this PR fixes.

image

What this does.
Every page now sits under the subsystem it describes, so the sidebar follows the source tree instead of the order things were written. The top level drops from 19 entries to 9, and 520 redirects keep old links alive.

image

What did NOT change.
The text of the pages. Of the 1577 .rst pages, 1567 keep the
words that are already in master. Only their address, their links and
their place in the tree changed.

What text DID change.
Ten pages. Nine are the landing page of a chapter, which has to exist
for the new structure. The tenth is the only one with technical
content: libs/libbuiltin/ had no page at all.

index                  the front page        os/time/index       Time and timers
os/index               OS Design             about/index         About
os/scheduling/index    Scheduling            developing/index    Developing NuttX
os/interrupts/index    Interrupts            ReleaseNotes/index  Release notes
os/ipc/index           IPC                   os/libs/libbuiltin  libs/libbuiltin/

What else changed, and why.
When I said "nothing has changed" was related to the documentation only.
Apart of the reorganization, even non-page files did change, and here is
each one:

file what why
conf.py +36 −1 registers sphinx_reredirects and tags_overview; derives the copyright year from SOURCE_DATE_EPOCH so the footer stops going stale; stops autosectionlabel indexing the frozen release notes, which reuse the same section titles and emitted a duplicate-label warning for every repetition
redirects.py new one rule per page that left its old path, plus the tag index pages, so no existing URL breaks
_templates/redirect.html new the small page written at every old URL; it carries a link's #anchor across to the new page
_templates/layout.html +22 −18 fixes the logo, which had silently disappeared: Sphinx renamed logo to logo_url and the template stopped rendering it. Also drops the empty entry in the version selector caused by a trailing comma
_static/custom.css +267 styling for the new tag index and the front page cards
_extensions/tags_overview.py new regroups the tag index into arch > chip > part
Pipfile / Pipfile.lock +19 −1 declares and pins sphinx-reredirects. Without the lock entry, CI's pipenv sync would fail
contributing/doc_templates/board-tags-example.txt new the tag block the board template tells you to copy, kept as its own file

Five SVG diagrams come with the ten pages above. They are hand-written
XML: no editor metadata, no scripts, no external references.

Table of contents, before and after.
As requested:

CURRENT (19 top-level)            PROPOSED (9 top-level)
  Introduction                      Introduction
  Getting Started                   Getting Started
  Contributing                      Supported Platforms
  Inviolables                       Guides
  Supported Platforms               OS Design          <- new
  OS Components                     API Reference
  Applications                      Applications
  Implementation Details            Developing NuttX   <- new
  API Reference                     About              <- new
  FAQ
  Debugging
  Testing
  Guides
  Standards
  Security
  Glossary
  Logos
  Tags Index

Nothing was deleted. Every old chapter is still there, one level down:

Contributing, OS Components, Implementation, Testing  ->  Developing NuttX
FAQ, Security, Glossary, Logos, Release Notes         ->  About
Debugging                                             ->  Guides
Standards                                             ->  Introduction
Inviolables                                           ->  Introduction
Tags Index                                            ->  Supported Platforms

Impact

Is new feature added? NO
Is existing feature changed? NO (documentation only)

  • Users: the pages that moved get a redirect from their old address,
    so existing links and bookmarks keep working, including links that jump
    to a section. Most pages did not move at all.
  • Build: no change to the OS build. Documentation/Pipfile gains one
    dependency, sphinx-reredirects, pinned in the lock file.
  • Hardware: none. Nothing outside Documentation/ is touched.
  • Documentation: this is the change.
  • Security: none.
  • Compatibility: no source, header or Kconfig symbol is touched.

Documentation-only change, so it is tested with make html as
CONTRIBUTING says, plus a check script of my own.

Host: Linux x86_64 (Ubuntu 26.04), Python 3.12.7, Sphinx 6.2.1.
Board: none applicable, no code is touched.

$ cd Documentation && make html
build succeeded.
0 warnings under -W, 0 documents outside a toctree

$ python3 validate_reorg.py e8a3d4ee1c
-- 2. .rst PAGES IN HEAD (1581) ------------------------------------------
    never touched                            791
    changed by the move only                 467
    moved, byte for byte identical           310
    new or rewritten text (the 10)            10
    halves of one split page (section 3)       2
    new, no text (toctree only)                1
    TOTAL                                   1581
-- 6. PAGES DELETED FOR GOOD (1) -----------------------------------------
    components/drivers/character/leds/index.rst            0 sentences, 0 nowhere now
-- 7. REDIRECTS (495 rules) ----------------------------------------------
    pages no longer at their old path            384
    rules for those pages                        384
    rules for tag index pages (_tags/)           111
    pages that left with no rule                   0
    rules for an old URL that never existed        0
    rules over a page that still exists            0
    rules that land on no page                     0
    rules that land where the page did not go      0
  APPROVED  --  the claim checked here is about CONTENT

$ ./tools/checkpatch.sh -g e8a3d4ee1c...HEAD
✔️ All checks pass.

How you can check the "nothing changed" claim yourself.

I can push the script, the question is 'where'.
But how does it works? For every page outside the ten named above, it erases
what a move touches -- link target, path, tag line, toctree block, table border
-- from the whole old text and the whole new text, and requires the two to be
byte for byte identical. It exits non-zero and names the page if that is not
true, and it tests added pages too, so forgetting to declare one cannot make it
pass.

@vrmay23

vrmay23 commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

@cederom and @linguini1 since I have closed the other PR and moved the work here. Your comments made me cut it down the content (what was good, now it is indeed just 10 pages changed, like explained in the PR message - What text DID change.).

@vrmay23 vrmay23 changed the title Documentation: file every page under the code it describes Documentation: brand new layout for NuttX documentation Sep 30, 2026
@vrmay23
vrmay23 force-pushed the enhance/documentation branch 2 times, most recently from 8baaa1c to fdd6095 Compare September 30, 2026 16:56
@vrmay23
vrmay23 marked this pull request as ready for review September 30, 2026 18:05
@linguini1
linguini1 marked this pull request as draft September 30, 2026 18:11
@github-actions github-actions Bot added Area: Documentation Improvements or additions to documentation Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces. labels Sep 30, 2026
@acassis

acassis commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

@vrmay23 I saw many of these huge amount of changes are just moving the content to a new page, but there are 6 pages deleted that are almost impossible to know what happen to them. I think the idea of doing things in an incremental way it to avoid mistakes that could end up losing documentation content.

@vrmay23

vrmay23 commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor Author

@vrmay23 I saw many of these huge amount of changes are just moving the content to a new page, but there are 6 pages deleted that are almost impossible to know what happen to them. I think the idea of doing things in an incremental way it to avoid mistakes that could end up losing documentation content.

@acassis You are right, thanks for helping in this review.

The page 'Shared Memory' is missing, indeed.
In my first PR that content was merged into the rewritten 'os/memory/shm.rst'.
So when I cut this PR down from 50 rewritten pages to 10, that page went back to the master text...

Good catch! My mistake, and I am fixing it now!

@vrmay23
vrmay23 force-pushed the enhance/documentation branch from fdd6095 to dbcbe2a Compare September 30, 2026 19:28
The documentation grew one page at a time, so the tree follows the
history of who wrote what and not the shape of NuttX. Scheduling is
spread over three places, a driver page can sit above the subsystem
that owns it, and the front page lists everything at the same level.
That is a lot to face when all you want to know is where the scheduler
lives.

This change files every page under the code it describes. It is a move,
not a rewrite: outside the ten pages named below, every page keeps the
text that is already in master, and no page's text is deleted.

What it does:

* Groups the table of contents into nine chapters.
* Moves the OS subsystems under os/: scheduling, memory, drivers,
  filesystem, networking, IPC, interrupts, libs, time.
* Renames the platform pages to the names the source tree uses, and
  derives their tags from the tree instead of by hand.
* Splits guides/ by subject.
* Adds Documentation/redirects.py, with a rule for every page that left
  its old path, so old URLs keep working. The redirect page also carries
  a link's #anchor across to the new page.

Ten pages have text that is new or rewritten. Nine of them are the
landing page of a chapter, which has to exist for the new structure:

    index                  the front page
    os/index               OS Design
    os/scheduling/index    Scheduling
    os/interrupts/index    Interrupts
    os/ipc/index           IPC
    os/time/index          Time and timers
    about/index            About
    developing/index       Developing NuttX
    ReleaseNotes/index     Release notes

The tenth is os/libs/libbuiltin, the only page here with technical
content: libs/libbuiltin/ had no page at all. Five SVG diagrams come
with these pages, hand-written XML with no editor metadata.

Nothing outside Documentation/ is touched.

How it was checked:

* Sphinx builds with -W: no warnings, and no document left outside a
  toctree.
* A script, offered in the PR, proves the narrow claim this rests on.
  For every page outside the ten named above it erases what a move
  touches -- link target, path, tag line, toctree block, table border --
  from the whole old text and the whole new text, and requires the two
  to be byte for byte identical. It also requires every sentence of a
  deleted page to turn up somewhere, and every page that left its old
  path to have a redirect, from a URL that existed, to where its content
  went. It exits non-zero and names the page if any of that is not true,
  and it tests added pages too, so forgetting to declare one cannot make
  it pass.
* An independent audit checked 133 factual claims on these ten pages
  against the tree, one shell command per claim: 130 confirmed, 1
  refuted and fixed here, 2 not checkable.
* tools/checkpatch.sh is clean over the range.

The diff is large because moving a page changes every link that points
to it. Most of it is pure renames, and board pages that gained one tag
line.

Assisted-by: Claude:claude-opus-5
@vrmay23
vrmay23 force-pushed the enhance/documentation branch from dbcbe2a to 13609c6 Compare September 30, 2026 20:49
@vrmay23

vrmay23 commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

@acassis I fixed one more thing also made my check script stricter. It now fails if any sentence of
a deleted page does not show up somewhere else, or if a redirect points
to the wrong place. Then, run on the previous push, it catches the same
problems you found :)

Thanks again, this made the PR better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: Documentation Improvements or additions to documentation Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants