argparse does not honor default argument for nargs=argparse.REMAINDER argument

@mblahay

BPO	35495
Nosy	@mblahay

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-12-14.15:59:40.287>
labels = ['type-bug', 'library', '3.9']
title = 'argparse does not honor default argument for nargs=argparse.REMAINDER argument'
updated_at = <Date 2019-07-18.21:03:41.175>
user = 'https://bugs.python.org/rgov'

bugs.python.org fields:

activity = <Date 2019-07-18.21:03:41.175>
actor = 'mblahay'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-12-14.15:59:40.287>
creator = 'rgov'
dependencies = []
files = []
hgrepos = []
issue_num = 35495
keywords = []
message_count = 17.0
messages = ['331837', '332567', '341826', '341844', '341938', '341939', '342115', '342122', '342124', '342129', '342150', '342196', '342370', '343603', '343710', '345916', '348129']
nosy_count = 4.0
nosy_names = ['bethard', 'paul.j3', 'rgov', 'mblahay']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'test needed'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue35495'
versions = ['Python 3.9']

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('things', nargs=argparse.REMAINDER, default=['nothing'])
parser.parse_args([])
>>> Namespace(things=[])

Since there were no unparsed arguments remaining, the default setting for things should have been honored. However it silently ignores this setting.

If there's a reason why this wouldn't be desirable, it should raise an exception that the options aren't compatible.

argparse.REMAINDER matches an empty list of arguments, just like '?' and '*'. So they are always 'filled', even by parse_args([]).

'?' and '*' have some special handling of defaults in this case, see in

argparse.ArgumentParser._get_values

the two

    value = action.default

REMAINDER has its own section in the function that does nothing with the default.

I think it should be left as is.

Ryan, what are the exact steps to reproduce the problem? This is what I get when I run the code you included:

>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('things', nargs=argparse.REMAINDER, default=['nothing'])
_StoreAction(option_strings=[], dest='things', nargs='...', const=None, default=['nothing'], type=None, choices=None, help=None, metavar=None)
>>> parser.parse_args([])
Namespace(things=[])
>>> Namespace(things=[])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'Namespace' is not defined

Just don’t run the last line which is just an echoing of the output of
parser.parse_args() repeated. The Namespace type would need to be imported
if you really wanted to but there’s no point.

On Tuesday, May 7, 2019, Michael Blahay <report@bugs.python.org> wrote:

>
> Michael Blahay <mblahay@gmail.com> added the comment:
>
> Ryan, what are the exact steps to reproduce the problem? This is what I
> get when I run the code you included:
>
> >>> import argparse
> >>> parser = argparse.ArgumentParser()
> >>> parser.add_argument('things', nargs=argparse.REMAINDER,
> default=['nothing'])
> _StoreAction(option_strings=[], dest='things', nargs='...', const=None,
> default=['nothing'], type=None, choices=None, help=None, metavar=None)
> >>> parser.parse_args([])
> Namespace(things=[])
> >>> Namespace(things=[])
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> NameError: name 'Namespace' is not defined
>
>

nosy: +mblahay

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue35495\>

Okay, so the expected output after running parse.parse_args([]) is Namespace(['nothing'])

Ryan, I have reviewed the documentation at https://docs.python.org/3/library/argparse.html#nargs and must admit that there is not a definitive answer that I can see regarding the defined behavior should there be no command line arguments that are in fact remaining. One could certainly argue that the empty list is the expression of the fact that no remaining arguments could be found. One can also argue that when seeking the remaining arguments, a list that may be zero to many elements in size, that by definition there cannot be a default.

Can you cite any documentation that would support your claim?

For the purpose of facilitating continuing conversation, here are two tests that contrast the use of * versus REMAINDER

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('foo', nargs=1,default=['none'])
parser.add_argument('bar', nargs=argparse.REMAINDER,default=['nothing'])
parser.add_argument('baz', nargs='*', default=['nada'])
parser.parse_args('a b c'.split())

Out[7]: Namespace(bar=['b', 'c'], baz=['nada'], foo=['a'])

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('foo', nargs=1,default=['none'])
parser.add_argument('baz', nargs='*', default=['nada'])
parser.add_argument('bar', nargs=argparse.REMAINDER,default=['nothing'])
parser.parse_args('a b c'.split())

Out[8]: Namespace(bar=[], baz=['b', 'c'], foo=['a'])

You can see that * and REMAINDER do differ in functionality when they are the last defined argument.

Here is another take on the issue, this time illustrated through the lens of optional arguments.

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', nargs=1,default=['none'])
parser.add_argument('--baz', nargs='*', default=['nada'])
parser.add_argument('--bar', nargs=argparse.REMAINDER,default=['nothing'])
parser.parse_args('--foo a --bar b --baz c'.split())

Out[9]: Namespace(bar=['b', '--baz', 'c'], baz=['nada'], foo=['a'])

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', nargs=1,default=['none'])
parser.add_argument('--baz', nargs='*', default=['nada'])
parser.add_argument('--bar', nargs=argparse.REMAINDER,default=['nothing'])
parser.parse_args('--foo a --baz b --bar c'.split())

Out[10]: Namespace(bar=['c'], baz=['b'], foo=['a'])

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo', nargs=1,default=['none'])
parser.add_argument('--baz', nargs='*', default=['nada'])
parser.add_argument('--bar', nargs=argparse.REMAINDER,default=['nothing'])
parser.parse_args([])

Out[11]: Namespace(bar=['nothing'], baz=['nada'], foo=['none'])

It is important to note that when an optional argument is not present then the default is always used, including for one using nargs=argparse.REMAINDER. In all three tests, bar is the argument using REMAIDER. In the first test, one can see that when bar isn't the last argument then anything else, including other arguments, are swept up as being arguments of bar. This greedy behavior for REMAINDER is something that * does not share (test 2).

With the optional arguments, the determination about whether to use the default value is made based on whether the flag is present or not. When positional arguments are involved, the need for the defaults seems to in part be determined based on whether the argument exists. The fact that * and REMAINDER are zero-to-many in nature add some ambiguity into the situation. For the *, it seems that the positional argument only exists if there is at least one actual argument value that it can consume.

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('foo', nargs=1,default=['none'])
#parser.add_argument('bar', nargs=argparse.REMAINDER,default=['nothing'])
parser.add_argument('baz', nargs='*', default=['nada'])
parser.parse_args('a b'.split())

Out[25]: Namespace(baz=['b'], foo=['a'])

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('foo', nargs=1,default=['none'])
#parser.add_argument('bar', nargs=argparse.REMAINDER,default=['nothing'])
parser.add_argument('baz', nargs='*', default=['nada'])
parser.parse_args('a'.split())

Out[26]: Namespace(baz=['nada'], foo=['a'])

Mean while, the REMAINDER option makes the argument act as if it exists regardless of whether an actual argument value exists.

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('foo', nargs=1,default=['none'])
parser.add_argument('bar', nargs=argparse.REMAINDER,default=['nothing'])
#parser.add_argument('baz', nargs='*', default=['nada'])
parser.parse_args('a b'.split())

Out[27]: Namespace(bar=['b'], foo=['a'])

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('foo', nargs=1,default=['none'])
parser.add_argument('bar', nargs=argparse.REMAINDER,default=['nothing'])
#parser.add_argument('baz', nargs='*', default=['nada'])
parser.parse_args('a'.split())

Out[28]: Namespace(bar=[], foo=['a'])

To conclude, * and REMAINDER perform similar, but different, roles when used with positional arguments. With edge cases like the ones laid out above, it can be hard to conceptualize what the exact behavior should be. I will recommend that the documentation be updated to convey the following message: "When used with positional arguments, REMAINDER will never use the designated default value list. It will instead return an empty list if there are no values for the argument to consume. If the use of default values is desired, then * must be used."

Much detail has been provided regarding why the default is ignored when user the REMAINDER option. The desire to add an exception has not. Is there anyone that can provide guidance on whether the combination of:

Positional Argument
nargs=REMAINDER
default=something
should raise an exception upon execution of the add_argument method?

At the start of parse_known_args, all defaults (except SUPPRESS ones) are placed in the namespace:

        # add any action defaults that aren't present
        for action in self._actions:
            if action.dest is not SUPPRESS:
                if not hasattr(namespace, action.dest):
                    if action.default is not SUPPRESS:
                        setattr(namespace, action.dest, action.default)

at the end of _parse_known_args there's a conditional expression that cleans up remaining defaults that are strings, by passing them through the 'type` callable:

    setattr(namespace, action.dest,
        self._get_value(action, action.default))

Read the comments to see why this default setting is done in two parts.

The pattern of defaults in msg342122 with optionals is consistent with that. If the argument is not provided, the default appears.

In the first example of that message, the REMAINDER is given all the remaining strings including the '--bar', so there is nothing left to trigger the '--bar' optional argument, and it retains the default.

The difference for positionals is due to how the '*' and '...' are handled in _getvalues. Both may be filled with an empty list of values.

    # when nargs='*' on a positional, if there were no command-line
    # args, use the default if it is anything other than None
    elif (not arg_strings and action.nargs == ZERO_OR_MORE and
          not action.option_strings):
        if action.default is not None:
            value = action.default
        else:
            value = arg_strings
        self._check_value(action, value)

    ....

    # REMAINDER arguments convert all values, checking none
    elif action.nargs == REMAINDER:
        value = [self._get_value(action, v) for v in arg_strings]

In the case of '*', the default is, effectively, placed back on the namespace. REMAINDER does not - the empty list is put in the namespace.

In _get_positional_kwargs,

        # mark positional arguments as required if at least one is
        # always required
        if kwargs.get('nargs') not in [OPTIONAL, ZERO_OR_MORE]:
            kwargs['required'] = True
        if kwargs.get('nargs') == ZERO_OR_MORE and 'default' not in kwargs:
            kwargs['required'] = True

That last conditional is a little puzzling, but I suspect it has to do with mutually_exclusive_groups, A '*' positional can be a member of a group if it has a default.

Anyways, we could add a test at this point like:

        if kwargs.get('nargs') == REMAINDER and 'default' in kwargs:
            msg = _("'default' is not allowed with a REMAINDER positional")
            raise TypeError(msg)

But reviewing the code I notice another difference. 'choices' are not honored for REMAINDER. That makes a lot of sense. REMAINDER is supposed to be a catch all, documented as something that might be passed on to another parser. This parser shouldn't be doing anything with those values.

The documentation reads:

argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:

I think REMAINDER has another quirk. It doesn't work as the first (and only?) argument. There should be a bug/issue to that effect.

https://bugs.python.org/issue17050, argparse.REMAINDER doesn't work as first argument

In https://bugs.python.org/issue17050#msg315716, I suggest removing REMAINDER from the docs. We can leave the code as is, in case anyone is using still using it. But it is probably too much work to make the code and docs match, both for that issue, and for this.

'*' plus '--' gives almost the same behavior. So does parse_known_args.

Ryan, last chance, do you have any feedback?

Thanks Michael for all of the examples. After reading them all, I concur that "it can be hard to conceptualize what the exact behavior should be." A documentation change is warranted, at the least.

However the argparse documentation, while great, is dense and it would be easy to overlook a simple comment. And I think the point that is being raised isn't merely a suggestion on how to design a good CLI, but a pitfall that makes the behavior of code non-obvious---it's not something someone would necessarily consult the documentation for while reviewing code.

(By the way, I'm considering CLIs like docker run and ssh which take an optional command to execute, and when absent, fall back on default behavior.)

So I would prefer a code change that makes it harder to write code that hits this corner case. Potential solutions would be either

(a) making a positional REMAINDER arg with a default value an error, as in Paul's proposed change; or
(b) making a default value with a positional REMAINDER arg 'just work'

I think (a) is the most reasonable. The exception can recommend the use of nargs='*' instead, which makes it actionable. And it is unlikely that the exception would be buried down some untested code path that would end up in released code.

(Perhaps it's also worth raising an exception when adding a positional argument after a nargs>1 positional argument already exists, but that's another issue.)

Need some help searching github to determine the blast radius of the proposed changes. How does one look for instances of argparse.REMAINDER that are used with a default value?

Ryan, I like option A as well, but it is a breaking change. Unlike in a compiled language where we could output a warning, making the proposed change could bring some software to a grinding halt. For now I'm going to make the documentation change and this issue can stay open for if there is a major version change that would allow such a breaking change.

@savannahostrowski

@savannahostrowski I tested this against 3.14 and the issue still exists:

Python 3.14.4 (main, Apr  7 2026, 13:13:20) [Clang 17.0.0 (clang-1700.6.4.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import argparse
... parser = argparse.ArgumentParser()
... parser.add_argument('things', nargs=argparse.REMAINDER, default=['nothing'])
... parser.parse_args([])
...
Namespace(things=[])

To summarize the problem: when a positional uses nargs=REMAINDER, its default is ignored -- parse_args() returns [] instead of the default.

Going through the previous discussion, REMAINDER is an undocumented legacy feature (source). It still exists in the source, but all mentions were removed from the docs in GH-18661 and GH-103586

As for the fix, there are two possible options, copying the answer from the comment above:
(a) making a positional REMAINDER arg with a default value an error, as in Paul's proposed change; or
(b) making a default value with a positional REMAINDER arg 'just work'

Since the issue is quite old and REMAINDER is a legacy feature, I'd assume we can close this issue? If you'd prefer to ship a fix instead, let me know which and I can look into implementing a) or b).

@clytaemnestra

Thanks for the thorough investigation, @clytaemnestra!

While the default is indeed ignored here, REMAINDER is still widely used (thousands of usages from a cursory search on GitHub), and both possible fixes carry real backwards compatibility risk that I'm not sure is worth it. Making it an error would break programs that currently pass a default harmlessly, and honoring the default would silently change results for anyone relying on the current [].

Since far more code could break than would be helped, I'd rather leave the behavior stable. It's a real rough edge; if there's broader demand or it comes up in larger parsing work, we can revisit.

Navigation Menu

Repository navigation

argparse does not honor default argument for nargs=argparse.REMAINDER argument #79676

2 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

argparse does not honor default argument for nargs=argparse.REMAINDER argument #79676

Description

Activity

rgov commented on Dec 14, 2018

paulj3 commented on Dec 26, 2018

mblahay commented on May 7, 2019

rgov commented on May 8, 2019

mblahay commented on May 8, 2019

mblahay commented on May 8, 2019

mblahay commented on May 10, 2019

mblahay commented on May 10, 2019

mblahay commented on May 10, 2019

mblahay commented on May 10, 2019

paulj3 commented on May 11, 2019

2 remaining items

mblahay commented on May 27, 2019

rgov commented on May 27, 2019

mblahay commented on Jun 17, 2019

mblahay commented on Jul 18, 2019

clytaemnestra commented on Jun 1, 2026

savannahostrowski commented on Jun 9, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions