declaring exceptions in stubs

Should methods in stubs be allowed to declare that they can throw exceptions?

When Jukka and I discussed this, we came up with the syntax ideas

def f():
  raise IOError()
  raise EOFError()

and

def f():
  typing.raises(IOError, EOFError)

So can we close this issue as won't fix?

@matthiaskramm

Few comments for readers which come here at later times from search/links:

Having some notion of what builtin/stdlib types, in terms of exceptions, can or cannot do, helps with that. That's why we currently still allow them in the stubs of typeshed (https://github.com/JukkaL/typeshed/blob/master/python3/time.py)

The link above now refers to https://github.com/python/typeshed/blob/master/stdlib/2and3/time.pyi . You won't find any raise's there, or in its history browsable via github, because the file was renamed along the way. Doing so requires real git log --follow, and leads to
python/typeshed@337abed .

The commit by @matthiaskramm is done from @google.com email address, which should clarify what is meant by:

The way our type inferencer works ...

Indeed, he (at this time) is listed as no.2 contributor to https://github.com/google/pytype/graphs/contributors .

The nature of the changes in that commit is illustrated by:

-def gmtime(*args, **kwargs) -> tuple:
-    raise OSError()
+def gmtime(secs: Union[float, None] = None) -> struct_time: ...

I.e., there were no argument annotations, but were exception annotations, and this commit added arg annotations at the expense of losing raise annotations. As of today's typeshed repo, there're no raise annotations in it.

Again, the discussion in this ticket is along the lines of "nobody against exception annotations, but neither they're a subject of currently intended typing usage". The "currently intended typing usage" is from 2015 however. It would be very said if all this work didn't lead from mere linters to real code generation optimizers. Actually, it quite naturally already does with efforts like https://github.com/mypyc/mypyc, though even that is so far keeps being disguised as being an ends to linting needs again ("Currently our focus is on making mypy faster through compilation.")

Overall, I hope people interested in efficient code generation (which requires typing annotations, including those beyond needed for mere linting tools) won't be kept pushed away from Python into other languages, like it was before.

Oh, and 2 cents from me on the actual syntax:

def f():
  raise IOError()
  raise EOFError()

Those parens look like noise to me (in type annotations, again), better to keep it simple:

def f():
  raise IOError
  raise EOFError

(That's of course valid Python code, the semantics being is that "raise" statement automatically instantiates an exception object, if it's passed an exception type.)

I also find interesting that the time.py annotations prior to the commit listed above had:

-def strftime(a: str, *args, **kwargs) -> unicode:
-    raise MemoryError()

Annotating for MemoryError would seem to be useless - it's archetypal kind of unchecked exception: anything which can allocate, can throw MemoryError. And in CPython, (almost) anything can allocate indeed. But some functions/methods are still would, consider for example a function which just returns l[0] of a non-empty list. And is some Python implementations, e.g. MicroPython, no-allocation property is important, and some functions are purposely designed to never allocate.

Marking functions which may allocate is however somehow backwards, because majority of functions do that. What's interesting is annotating those few which purposely were made no-alloc.

The same is true for exceptions too - annotating individual exceptions raised is tedious is only so much useful even for optimized (native) code generation. However, a few functions which purposely made no-throw are much more interesting to annotate, because they allow to remove some exception handling overheads when dealing with them (and removing these overheads might be the reason they were made no-throw in the first place).

Is there a corresponding discussion on the typeshed tracker? We would be interested in doing static analysis on exceptions. We can definitely declare them outside of stubs but stubs seem like the natural place to do this.

typeshed does not support features that are not officially supported by (at least candidate) PEPs, Python documentation, or some other form of official document. It's not the right place to discuss this.

Checking exceptions is very useful for public interfaces. Being unaware of what exceptions may be thrown, it's impossible to "ask for forgiveness" ... because you simply don't know what to catch. And many exceptions (ValueError) are so generic, that it's impossible to know whether the error you want to catch is being caught. Support for typing is a nice tool.

Food for thought (syntax-wise): now that PEP 593 is accepted it could be possible to use Annotated with an alias to encode this.

@earonesty

Adding to what @earonesty said: Many functions in libraries (I am looking at you, requests) can raise a whole lot of different exceptions and these are often not documented. I often find myself using the REPL to try to replicate all possible circumstances that I can come up with, just to figure out what exceptions the function can raise.

This is fine for little hacky scripts or when working on scientific code or something. But it sucks when writing production code that needs to be reliable.

Also, @gvanrossum, I can come up with a few possible scenarios that could use this information that are not mandatory type checking. For example, a type checker could implement a function where you supply the type checker with the code base and a reference to a function, and the type checker then tells you all possible exceptions that could be raised by that function. Here it does not matter if an exception in the output is useful (e.g. something like a TimeoutException) or not (like a MemoryError) because this information is only presented to the programmer, so they can filter out the nonsense from the real information.

Also, adding optional type checking to a type checker does not hurt any user who does not want it, because they can just not enable it.

Pylint provides an interesting solution for anyone that still requires the exception handling.

The missing-raises-doc is a pylint message for all no documented exception raised. It's a nice solution to catch exceptions using the linter.

@juansedo

@juansedo does it check for functions that are raised indirectly, by a function used inside the function?

When a function raises an exception explicitly that's already pretty self documenting.

refactor: [#66087] convert get_ldap_dn to returning None instead of r…

Navigation Menu

Repository navigation

declaring exceptions in stubs #71

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

declaring exceptions in stubs #71

Description

Activity

gvanrossum commented on Apr 14, 2015

pfalcon commented on Jan 7, 2019

pfalcon commented on Jan 7, 2019

dkgi commented on Jan 7, 2019

srittau commented on Jan 8, 2019

earonesty commented on Dec 10, 2019

jstasiak commented on Dec 10, 2019

Dakkaron commented on Feb 18, 2020

juansedo commented on Jun 12, 2024

truh commented on Jun 13, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions