Decide the fate of undocumented script behavior of some modules

There are three dozens of standard modules that can be called via python -m and their documentation doesn't mention it. They can be grouped into five categories:

kind of smoke tests:
- codecs: ~~performs stdin:latin1 → utf-8 → latin1 → stdout passthrough~~ edit: it just wraps stdin and stdout then just exits the script (gh-93096: Remove python -m codecs #94233)
- curses.has_key: "Compare the output of this implementation and the ncurses has_key, on platforms where has_key is already available"
- pprint: measures performance (gh-92546: Move pprint benchmark into pyperformance #94613 → Add a benchmark based on python -m pprint pyperformance#222)
- random: evaluates output statistics of supported generators
full-fledged crossplatform utils for admin-like users and small automation:
- asyncio: like python but allows to use await in top-level script code
- cProfile, profile: runs a script under the profiler
- encodings.rot_13: a stream converter
- filecmp: a crossplatform file comparison utility
- fileinput: prints specified files one by another annotating lines with their source
- http.server: makes a directory available as a site; useful to quickly test a static site with relative links
- mimetypes: useful for batch processing of files (maybe) (gh-93096: Make mimetypes CLI tool public #93097)
- modulefinder: the objdump but for Python source files
- netrc: prints content of .netrc for a current user
- pdb
- platform: returns a single line like Windows-10-10.0.19044-SP0; can be useful in automation
- quopri: a stream converter
- tabnanny
- wsgiref.simple_server - the same as http.server but for APIs; pases a single request and exits
both:
- base64:
  - a stream converter
  - base64 -t encodes/decodes Aladdin:open sesame and tests if the result is the same as the original (gh-93096: Remove python -m base64 -t #94230)
demos with no real world application:
- curses.textpad: shows an input area; when a user closes it, prints the text back
- ftplib: a simple one-pass FTP downloader (uses ~/.netrc for login)
- ~~getopt: just passes arguments to getopt()~~ The module is no longer maintained after gh-106535: Soft deprecate the getopt module #105735
- imaplib: sending emails to a dead end has no sence but can be used to check if a email client works or got broken
- shlex: parses stdin using shlex() and prints the list into stdout
- smtplib: a simple e-mail client
- xmlrpc.server: serves a datetime service
complex matter; better leave untouched:
- idlelib.*
- tkinter.*
- turtledemo.*
- pstats

Eggs and to-be-removed modules aren't listed.

We need to decide what to do with all these undocumented categories.

I propose the following:

move smoke tests into test module with deduplication
for full-fledged utils, add Command-Line Usage into the docs like in https://docs.python.org/3/library/ast.html#command-line-usage or https://docs.python.org/3/library/trace.html#command-line-usage
move demos into the docs of the corresponding module

Linked PRs

pprint also has a rudimentary performance test. See #92546 for a bug, where it wasn't being tested. If we really care about this performance metric, it should be moved to a proper benchmark.

@ericvsmith

@ericvsmith Thank you, I've added it to the list.

gh-93096: Remove `python -m codecs` (gh-94233)

gh-93096: Remove `python -m base64 -t` (gh-94230)

gh-92546: Move pprint benchmark into pyperformance (GH-94613)

pythongh-93096: Port a bunch of stdlib cli to ArgumentParser

@picnixz

@picnixz based on your comment i suggest to get rid of t and v flags in pickle and pickletools modules and put tests in common place

I wasn’t aware of this ticket!
Script feature of stdlib modules have been discussed a few times in various PRs, as well as this forum thread: https://discuss.python.org/t/command-line-interface-for-the-random-module/51304

I suggest that discuss is a better place to reach a decision that a ticket.

For deciding whether to expose the cli or not is probably worth a DPO thread, but deciding whether to remove or not the internal options meant for self-testing may be discussed in an issue directly I think (as we would just move self-tests/doctests to unittests and/or run them directly).

[3.12] gh-93096: Load doctests in `test_itertools` (GH-131133) (#131137)

[3.13] gh-93096: Load doctests in `test_itertools` (GH-131133) (#131136)

ping @hugovk as author of --test flag in random cli

In discuss you suggested opening a new issue but I think this issue is more appropriate. Since the discussion was closed I suggest that if anyone has objections, they write here, but I have not seen any objections yet

If you don't mind such a change, then I will open a PR

sorry, i didn't wait for you because i found that we already removed such thing for pprint module (PR #94613)
these tests are moving to pyperformance, see #131144

gh-93096: Make `mimetypes` CLI tool public (#93097)

gh-93096: Remove CLI interface for `difflib` (#131099)

gh-93096: Update and document `pickle` CLI (#131097)

@vstinner

We have modules that don't have the -h/--help flag, for example:

fileinput (no docs, run demo)
ftplib (no docs, have help message, but it only prints without any flags)
filecmp (no docs)
poplib (no docs, run demo)
pyclbr (no docs)
quopri (no docs)
tabnanny (no docs)
site (have docs and help message, but it only prints when there is error)

The problem is that these CLIs expect some arguments, but nowhere is it said which ones! (no docs, no help messages)
I don't suggest adding documentation for all of them, as this would make them public and this could raise other questions.
I think that users to not have to look into source code, it would be convenient to have a help message and a flag to print it

I don't think help message is required for asyncio, this, turtledemo or something similar

@vstinner does it make sense to open issue to send PRs?

EDIT: I think this will add consistency to the behavior of CLI modules

gh-93096: Update and document `pickletools` CLI (#131273)

pythongh-93096: Make `mimetypes` CLI tool public (python#93097)

pythongh-93096: Remove CLI interface for `difflib` (python#131099)

pythongh-93096: Update and document `pickle` CLI (python#131097)

pythongh-93096: Update and document `pickletools` CLI (python#131273)

gh-93096: fix `test_mimetypes.test_guess_type_conflicting_with_mimety…

gh-93096: fix `test_mimetypes.test_invocation_error` tests on iOS ARM…

pythongh-93096: fix `test_mimetypes.test_invocation_error` tests on i…

Navigation Menu

Repository navigation

Decide the fate of undocumented script behavior of some modules #93096

Linked PRs

13 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Decide the fate of undocumented script behavior of some modules #93096

Description

Linked PRs

Activity

ericvsmith commented on May 23, 2022

arhadthedev commented on May 23, 2022

donbarbos commented on Mar 9, 2025

merwok commented on Mar 10, 2025

picnixz commented on Mar 10, 2025

13 remaining items

donbarbos commented on Mar 12, 2025

donbarbos commented on Mar 16, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions