Skip to content

cgitb sends a bogus HTTP header if the app crashes before finishing headers #52950

Closed
Closed
cgitb sends a bogus HTTP header if the app crashes before finishing headers#52950
Labels
3.8 (EOL)end of lifeeasystdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

Description

@stutzbach
stutzbach
mannequin
opened on May 13, 2010
BPO 8704
Nosy @orsenthil, @cthart

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2010-05-13.14:12:50.822>
labels = ['easy', '3.8', 'type-bug', 'library']
title = 'cgitb sends a bogus HTTP header if the app crashes before finishing headers'
updated_at = <Date 2022-03-15.14:23:55.676>
user = 'https://bugs.python.org/stutzbach'

bugs.python.org fields:

activity = <Date 2022-03-15.14:23:55.676>
actor = 'cthart'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2010-05-13.14:12:50.822>
creator = 'stutzbach'
dependencies = []
files = []
hgrepos = []
issue_num = 8704
keywords = ['easy']
message_count = 9.0
messages = ['105633', '105690', '105705', '193189', '294027', '364186', '376811', '389031', '415246']
nosy_count = 10.0
nosy_names = ['orsenthil', 'stutzbach', 'igs', 'meatballhat', 'ysj.ray', 'p0lar_bear', '\xd0\x90\xd1\x80\xd1\x82\xd1\x83\xd1\x80 \xd0\x9a\xd0\xbb\xd0\xb5\xd1\x81\xd1\x83\xd0\xbd', 'coyot', 'Ryan Tu', 'cthart']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8704'
versions = ['Python 3.8']

Activity

  1. stutzbach commented on May 13, 2010

    @stutzbach
    stutzbachmannequin
    on May 13, 2010
    MannequinAuthor

    If the CGI script crashes before finishing the headers, cgitb will emit invalid HTTP headers before showing the error message. Below are HTTP headers I received, captured with a packet sniffer. Note the "<--: spam".

    HTTP/1.1 200 OK
    Date: Thu, 13 May 2010 14:00:42 GMT
    Server: Apache/2.2.9
    <!--: spam
    Vary: Accept-Encoding
    Cache-Control: max-age=0
    Expires: Thu, 13 May 2010 14:00:42 GMT
    Set-Cookie: ref=; path=/; HttpOnly
    Transfer-Encoding: chunked
    Content-Type: text/html

    That string it emitted by cgitb.reset(), which is trying to reset the browser to a sane state so the error message will be shown. The problem can be easily fixed by having cgitb.reset() emit two CRLF pairs first, to ensure that we're done with the headers and emitting content:

    • return '''<!--: spam
      + return '''\r\n\r\n<!--: spam
  2. stutzbach
    added
    stdlibStandard Library Python modules in the Lib/ directory
    type-bugAn unexpected behavior, bug, or error
    easy
    on May 13, 2010

  3. ysjray commented on May 14, 2010

    @ysjray
    ysjraymannequin
    on May 14, 2010
    Mannequin

    Yes, I saw the "<!--: spam" string in headers, but it seems that this string doesn't make problems. The displaying page is correct.

    But after I apply the changes you mentioned:

    • return '''<!--: spam
      + return '''\r\n\r\n<!--: spam

    I got text/plain output, and the response headers are like this:

    Date	Fri, 14 May 2010 07:30:03 GMT
Server	Apache/2.2.15 (Unix)
Keep-Alive	timeout=5, max=100
Connection	Keep-Alive
Transfer-Encoding	chunked
Content-Type	text/plain

    And the content is like this:

    <!--: spam
    Content-Type: text/html

    <body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> -->
    <body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> --> -->

    ......

    So the hole page is not displayed correctly!

    Is there any problem with me?

  4. stutzbach commented on May 14, 2010

    @stutzbach
    stutzbachmannequin
    on May 14, 2010
    MannequinAuthor

    It displays correctly in some browsers, yes, but not everything that speaks HTTP is a browser. For example, the invalid header makes C#'s WebRequest throw an exception.

    I hadn't noticed the 'Content-Type' on the next line of the string output by reset(). That does make things more complicated.

    We could put the "Content-Type: text/html" first, but the downside is that it will be output as visible content if a script crashes after the headers have been emitted.

    I'm not sure if that's better or worse than emitting an invalid header.

  5. p0larbear commented on Jul 16, 2013

    @p0larbear
    p0larbearmannequin
    on Jul 16, 2013
    Mannequin

    I get similar results if my CGI script sends a Content-Type header of anything besides "text/html", e.g. print('Content-Type: text/json').

  6. ghost commented on May 20, 2017

    @ghost
    ghost
    on May 20, 2017

    Apache started strict check of headers ch
    aracters to be valid recently. That causes it fail on "<--: spam".

    [Sat May 20 13:09:23.056673 2017] [http:error] [pid 26379] [client 12.34.567.41:60988] AH02429: Response header name '<!--' contains invalid characters, aborting request, referer: http://example.com/

    http://apache-http-server.18135.x6.nabble.com/Bug-60863-New-Apache-proxy-2-4-25-can-disable-header-check-Set-Cookie-td5036120.html

    The workaround is to put:
    HttpProtocolOptions Unsafe
    line into your apache .conf

  7. RyanTu commented on Mar 14, 2020

    @RyanTu
    RyanTumannequin
    on Mar 14, 2020
    Mannequin

    #Maybe not a good solution
    I do not know the should we delete the code in cgitb.py or adjust the configration of apache httpd. My solution is deleting some code as follows:

            return '''
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> -->
<body bgcolor="#f0f0f8"><font color="#f0f0f8" size="-5"> --> -->
</font> </font> </font> </script> </object> </blockquote> </pre>
</table> </table> </table> </table> </table> </font> </font> </font>'''

    Then it works very well, and it has good view.Anyone know what is the situation in ngix?

  8. RyanTu
    added
    3.8 (EOL)end of life
    on Mar 14, 2020

  9. igs commented on Sep 12, 2020

    @igs
    igsmannequin
    on Sep 12, 2020
    Mannequin

    As mentioned above standard Apache does not accept the extra characters anymore and produces '500 internal error'. So the normal behaviour of this module makes things worse in most cases instead of being helpful.

  10. coyot commented on Mar 18, 2021

    @coyot
    coyotmannequin
    on Mar 18, 2021
    Mannequin

    Ran into this also, got:

    AH02429: Response header name '<!--' contains invalid characters, aborting request

  11. cthart commented on Mar 15, 2022

    @cthart
    cthartmannequin
    on Mar 15, 2022
    Mannequin
    1. This module is scheduled to be removed by Python 3.13 (although I preseonally am of the opinion that it is a useful module and would like to see it brought up-to-date).
    2. Is reset() even necessary anymore? Can't the same results be achieved with CSS since we are in the third decade of the 2000s after all?
  12. ezio-melotti
    transferred this issue fromon Apr 10, 2022

  13. hugovk commented on Apr 11, 2022

    @hugovk
    hugovk
    on Apr 11, 2022
    Member

    Yep, deprecated in 3.11 and removed in 3.13: see PEP 594 – Removing dead batteries from the standard library, #91217 and #32410.

    @cthart Some good news: a fork has been made by @jackrosenthal:

    Contributions are accepted, but should be focused on bug fixes instead of new features or major refactoring.

    Because there's no maintainer for this module in CPython, perhaps it would be better to close this issue and instead put energy into the fork?

  14. AlexWaygood commented on Apr 12, 2022

    @AlexWaygood
    AlexWaygood
    on Apr 12, 2022
    Member

    Because there's no maintainer for this module in CPython, perhaps it would be better to close this issue and instead put energy into the fork?

    I think so. I'm going to close this issue for now, but I'm happy to reopen if somebody wants to persuasively argue for why this issue should be fixed in CPython despite the deprecation.

    Cc. @cthart

  15. AlexWaygood
    closed this as completedon Apr 12, 2022

  16. cthart commented on Apr 12, 2022

    @cthart
    cthart
    on Apr 12, 2022

    Good call. A fork available via GitHub / PyPi is a perfect solution for this library. My interest is only in the cgitb part, which I believe could be modernised to use CSS for displaying Python stack traces in a marked up format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.8 (EOL)end of lifeeasystdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions