Skip to content

wsgiref package totally broken #48968

Closed
Closed
wsgiref package totally broken#48968
Labels
release-blockerstdlibStandard Library Python modules in the Lib/ directorytype-crashA hard crash of the interpreter, possibly with a core dump

Description

@hdima
hdima
mannequin
opened on Dec 22, 2008
BPO 4718
Nosy @pjeby, @pitrou, @benjaminp
Files
  • wsgiref.patch: WSGI 1.0+ fixes for wsgiref (version 2)
  • wsgiref2.patch: WSGI 1.0+ fixes for wsgiref (with wsgi.input.read() argument check)
  • wsgiref-bb.patch
  • no_proxy.patch: patch for test_urllib
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2009-01-31.23:30:45.350>
created_at = <Date 2008-12-22.11:30:56.870>
labels = ['library', 'type-crash', 'release-blocker']
title = 'wsgiref package totally broken'
updated_at = <Date 2009-01-31.23:30:45.332>
user = 'https://bugs.python.org/hdima'

    bugs.python.org fields:

    activity = <Date 2009-01-31.23:30:45.332>
actor = 'benjamin.peterson'
assignee = 'none'
closed = True
closed_date = <Date 2009-01-31.23:30:45.350>
closer = 'benjamin.peterson'
components = ['Library (Lib)']
creation = <Date 2008-12-22.11:30:56.870>
creator = 'hdima'
dependencies = []
files = ['12447', '12454', '12573', '12577']
hgrepos = []
issue_num = 4718
keywords = ['patch']
message_count = 45.0
messages = ['78171', '78172', '78188', '78192', '78193', '78194', '78196', '78207', '78214', '78215', '78229', '78232', '78235', '78236', '78237', '78241', '78251', '78261', '78269', '78279', '78280', '78292', '78294', '78298', '78304', '78348', '78349', '78351', '78352', '78353', '78354', '78612', '78738', '78787', '78990', '78996', '78997', '78999', '79000', '79002', '79003', '79005', '79013', '79020', '80889']
nosy_count = 5.0
nosy_names = ['pje', 'hdima', 'pitrou', 'benjamin.peterson', 'grahamd']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue4718'
versions = ['Python 3.0']

    Activity

    1. hdima commented on Dec 22, 2008

      @hdima
      hdimamannequin
      on Dec 22, 2008
      MannequinAuthor

      It seems the wsgiref package was copied from Python 2.* without any
      modifications. There are already 3 issues about that but they only
      describe a part of the problem so I decided to start a new one. The
      issues was:

      http://bugs.python.org/issue3348
      http://bugs.python.org/issue3795
      http://bugs.python.org/issue4522

      The attached patch fix the problem with the following changes:

      • Fixed headers handling in wsgiref/simple_server.py;

      • Fixed encoding problems. Now WSGI applications must return iterable
        with bytes but start_response() allow status and headers as bytes and as
        strings. "wsgi.input" file-like now use BytesIO instead of StringIO;

      • Fixed tests;

      • Updated documentation examples;

    2. hdima
      added
      stdlibStandard Library Python modules in the Lib/ directory
      type-crashA hard crash of the interpreter, possibly with a core dump
      on Dec 22, 2008

    3. pitrou commented on Dec 22, 2008

      @pitrou
      pitrou
      on Dec 22, 2008
      Member

      Phillip, do you have time to take a look at it? We really *must* fix
      wsgiref in py3k...

    4. pitrou commented on Dec 22, 2008

      @pitrou
      pitrou
      on Dec 22, 2008
      Member

      FYI, instead of trying to do exhaustive type checking in _check_type(),
      you can just rely on duck typing and catch the TypeError:

      >>> str(b"a", "utf-8")
'a'
>>> str(bytearray(b"a"), "utf-8")
'a'
>>> str(memoryview(b"a"), "utf-8")
'a'
>>> str(1, "utf-8")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: coercing to str: need string or buffer, int found

    5. pjeby commented on Dec 22, 2008

      @pjeby
      pjebymannequin
      on Dec 22, 2008
      Mannequin

      If you want to change to using bytes, you're going to have to take it to
      the Web-SIG and hash out a revision to PEP-333, which at the moment
      requires the use of strings, period.

      This has nothing to do with the desirability of bytes vs. strings; I am
      sure that if Python had bytes from day 1, bytes would've been the way to
      go with it. But simply changing the reference library is not the way to
      change the spec.

      In the meantime, as far as I'm aware, there are two other patches
      pending to address these issues, but I'm not in a position to assess
      their readiness/correctness since I have yet to even download Py3K. In
      principle, I approve their approaches, so if someone else can handle the
      code review, those fixes could in principle be put in without changing
      the PEP. But to put *this* patch in, the PEP would have to be changed.

    6. pitrou commented on Dec 22, 2008

      @pitrou
      pitrou
      on Dec 22, 2008
      Member

      If you want to change to using bytes, you're going to have to take it
      to the Web-SIG and hash out a revision to PEP-333, which at the moment
      requires the use of strings, period.

      What was called str in 2.x has become the bytes object in py3k.
      What was called unicode in 2.x has become str in py3k.
      (roughly)

      Given the meaning of the term "string" and its possible acceptions have
      dramatically changed between 2.x and py3k, how does this patch violate
      the PEP more than any other?

      Actually, the PEP says:

          HTTP does not directly support Unicode, and neither does this
    interface. All encoding/decoding must be handled by the
    application; all strings passed to or from the server must be
    standard Python *byte strings*, not Unicode objects.
    [emphasis mine]

      So, not accepting bytes in py3k is clearly a violation of the PEP!

    7. hdima commented on Dec 22, 2008

      @hdima
      hdimamannequin
      on Dec 22, 2008
      MannequinAuthor

      Antoine Pitrou wrote:

      FYI, instead of trying to do exhaustive type checking in _check_type(),
      you can just rely on duck typing and catch the TypeError:

      Good point! I'll update the patch soon.

    8. pitrou
      added
      release-blocker
      on Dec 22, 2008

    9. hdima commented on Dec 22, 2008

      @hdima
      hdimamannequin
      on Dec 22, 2008
      MannequinAuthor

      Antoine Pitrou wrote:

      > If you want to change to using bytes, you're going to have to take it
      > to the Web-SIG and hash out a revision to PEP-333, which at the moment
      > requires the use of strings, period.

      What was called str in 2.x has become the bytes object in py3k.
      What was called unicode in 2.x has become str in py3k.
      (roughly)

      Agreed, moreover it's time for Python 3.0.1 and we need to decide -
      apply a patch or just remove wsgiref completely for now. Keep wsgiref
      just as nonworking piece of code is the worse solution which can made
      questionable all WSGI effort.

      Given that old str has been replaced by bytes in Python 3 I think the
      patch is a correct implementation of the PEP from the Python 3's point
      of view. To avoid confusion note about the meaning of the term *string*
      can be added to the PEP later.

    10. hdima commented on Dec 22, 2008

      @hdima
      hdimamannequin
      on Dec 22, 2008
      MannequinAuthor

      New version of the patch:

      • Now only Unicode strings are allowed as status and headers because
        allowing bytes leads to big changes in wsgiref.validate and
        wsgiref.handlers;

    11. pjeby commented on Dec 22, 2008

      @pjeby
      pjebymannequin
      on Dec 22, 2008
      Mannequin

      At 03:37 PM 12/22/2008 +0000, Antoine Pitrou wrote:

      So, not accepting bytes in py3k is clearly a violation of the PEP!

      On the contrary. Please read the two paragraphs *after* the one you quoted.

    12. pjeby commented on Dec 22, 2008

      @pjeby
      pjebymannequin
      on Dec 22, 2008
      Mannequin

      To be quite clear: this change requires discussion on the Web-SIG and an
      appropriate revision of the PEP. Ideally, the patch should include the
      necessary PEP revision.

      The Web-SIG discussion regarding a switch to bytes should also take into
      consideration the effects of running 2to3 on existing WSGI applications
      and/or servers. Will their code be converted to use bytes, or Unicode?

      The previous choice to use Unicode was based on source compatibility
      across Python implementations, so this shouldn't be thrown out on the
      basis of simple expediency.

    13. hdima commented on Dec 23, 2008

      @hdima
      hdimamannequin
      on Dec 23, 2008
      MannequinAuthor

      OK, I've attached PEP-333 compatible fixes for wsgiref. I think there is
      only one problem remains:

      • wsgiref expects io.BytesIO as input and output streams because of
        http.server module. I didn't find any restrictions on data returned by
        read() method of the "wsgi.input" stream in the PEP. Maybe I've missed
        some details?

    14. pitrou commented on Dec 23, 2008

      @pitrou
      pitrou
      on Dec 23, 2008
      Member

      Please read the two paragraphs *after* the one you quoted.

      I don't see anything forbidding bytes objects in those two paragraphs.

    15. 22 remaining items

    16. pitrou commented on Jan 3, 2009

      @pitrou
      pitrou
      on Jan 3, 2009
      Member

      It is committed now in py3k and the 3.0 maintenance branch. Thanks all
      for your participation!

    17. pitrou
      closed this as completedon Jan 3, 2009

    18. pitrou commented on Jan 3, 2009

      @pitrou
      pitrou
      on Jan 3, 2009
      Member

      Reopening, the patch actually produces failures when run with "python
      -bb", that is there are comparisons between str and bytes.

      See the errors at the end of
      http://www.python.org/dev/buildbot/3.x.stable/ppc%20Debian%20unstable%203.0/builds/26/step-test/0

    19. pitrou
      reopened this on Jan 3, 2009

    20. pitrou commented on Jan 3, 2009

      @pitrou
      pitrou
      on Jan 3, 2009
      Member

      People, does this patch look ok to you?

    21. hdima commented on Jan 3, 2009

      @hdima
      hdimamannequin
      on Jan 3, 2009
      MannequinAuthor

      Antoine Pitrou wrote:

      People, does this patch look ok to you?

      Oh, didn't know about -bb.
      The patch looks OK for me.

    22. pitrou commented on Jan 3, 2009

      @pitrou
      pitrou
      on Jan 3, 2009
      Member

      Le samedi 03 janvier 2009 à 20:24 +0000, Dmitry Vasiliev a écrit :

      Dmitry Vasiliev <dima@hlabs.spb.ru> added the comment:

      Antoine Pitrou wrote:
      > People, does this patch look ok to you?

      Oh, didn't know about -bb.

      Well, it's meant to catch potential bugs. str and bytes always compare
      unequal in py3k (i.e. "a" != b"a"), so it's good to use -bb when porting
      some stuff from 2.x.

      There's another problem in that buildbot failure with the environment
      variable "NO_PROXY". We'll see if it's still there after the patch.

    23. hdima commented on Jan 3, 2009

      @hdima
      hdimamannequin
      on Jan 3, 2009
      MannequinAuthor

      Antoine Pitrou wrote:

      There's another problem in that buildbot failure with the environment
      variable "NO_PROXY". We'll see if it's still there after the patch.

      Strange error and it seems there is only part of the traceback. I've
      already seen such "partially displayed" Python 3 traceback and error
      actually can be in very different place.

    24. pitrou commented on Jan 3, 2009

      @pitrou
      pitrou
      on Jan 3, 2009
      Member

      Strange error and it seems there is only part of the traceback. I've
      already seen such "partially displayed" Python 3 traceback and error
      actually can be in very different place.

      If you can reproduce such a problem, please open a bug.

    25. hdima commented on Jan 3, 2009

      @hdima
      hdimamannequin
      on Jan 3, 2009
      MannequinAuthor

      Antoine Pitrou wrote:

      > Strange error and it seems there is only part of the traceback. I've
      > already seen such "partially displayed" Python 3 traceback and error
      > actually can be in very different place.

      If you can reproduce such a problem, please open a bug.

      OK, I'll try to reproduce.

    26. hdima commented on Jan 3, 2009

      @hdima
      hdimamannequin
      on Jan 3, 2009
      MannequinAuthor

      Attached patch for test_urllib, possible source of the "NO_PROXY" problem.

    27. pitrou commented on Jan 3, 2009

      @pitrou
      pitrou
      on Jan 3, 2009
      Member

      Nice catch! I've committed the two patches and we'll see whether it
      makes the buildbots feel better.

    28. benjaminp commented on Jan 31, 2009

      @benjaminp
      benjaminp
      on Jan 31, 2009
      Contributor

      I assume the buildbots were placated?

    29. benjaminp
      closed this as completedon Jan 31, 2009
    30. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      release-blockerstdlibStandard Library Python modules in the Lib/ directorytype-crashA hard crash of the interpreter, possibly with a core dump

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions