Repository navigation
Document when to use Union[str, unicode] vs AnyStr #871
Description
This comes up pretty often, and it would be useful to have some documentation in the CONTRIBUTING.md file (or at least a link to this documentation).
Here's a starting point for what to document (there probably are other things that we should mention):
- If a function accepts both
strand ascii-onlyunicodearguments, usually the best type to use isUnion[str, unicode](orUnion[str, Text]in a 2and3 stub). - Use
AnyStrif you have two or more types in a signature that must agree on whether they arestrorunicode. (It would also be nice to give an example where this is important.) - You can also use
AnyStrin invariant positions in generic type arguments. For example,List[AnyStr]is generally better thanList[Union[str, unicode]](also explain why). However, often it's even better to use a covariant type such asIterableorSequence. In that case the union variant is preferable if the container may contain a mix ofstrandunicode. For example,Iterable[Union[str, unicode]]is fine if the iterable may contain a mix ofstrandunicodevalues. - Try to avoid using
Union[str, unicode]in a return type, since it means that every call site will have to deal with bothstrandunicodevalues. It may be fine to use this if the return type is sufficiently unpredictable. - Similarly, try to avoid using
Union[str, unicode]as an attribute type -- again code using this attribute would have to deal with bothstrandunicodevalues.
Activity
lincolnq commentedon Feb 10, 2017
(I'm new to this project so please correct me if I've asked this question in the wrong place.)
The xml.etree module documents the return type of tostring() in python 3 as either str (if the value of the
encodingparameter is"unicode") or bytes otherwise. (https://docs.python.org/3/library/xml.etree.elementtree.html)However, the tostring stub documents it returning simply str.
This is a blocker to usage in python 3 since the return type is just wrong for many use cases. Should this stubfile be changed to return AnyStr, or a Union, or something else entirely? (Am happy to submit a PR to do this but I'm not sure which you would do.)
JukkaL commentedon Feb 10, 2017
@lincolnq In cases like these typeshed typically uses an
Anyreturn type. The return type depends on the value of an argument, and PEP 484 isn't expressive enough to represent this.Add str/unicode docs to CONTRIBUTING.md
JelleZijlstra commentedon Mar 16, 2017
I started writing documentation to address this issue; what I have so far is at master...JelleZijlstra:patch-10 (heavily based on Jukka's comments here).
However, I don't understand what Jukka is referring to in his paragraph about type arguments: "For example,
List[AnyStr]is generally better thanList[Union[str, unicode]]". I would think that you use the former if you want a list that contains either str or unicode but not both, and the latter if you can accept a mixed list. But I don't see how that relates to invariance versus covariance. @JukkaL can you clarify?srittau commentedon Sep 17, 2020
I think we have sufficient documentation when to use
str,bytes, andTextnow.fix _GenericAlias import (python#871)