[Windows] cgi handling of POSTed files is broken in Windows

@gvanrossum

BPO	8077
Nosy	@gvanrossum, @birkenfeld, @pfmoore, @orsenthil, @tjguk, @MLModel, @bitdancer, @PierreQuentel, @zware, @zooba
PRs	bpo-8077 Fix CGI Handling of POST on Windows. #25652
Files	cgi-post-broken.zip: zipped demo of problem http-server.patch: Patch for http.server.CGIHTTPRequestHandler

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/orsenthil'
closed_at = None
created_at = <Date 2010-03-06.01:43:39.134>
labels = ['type-bug', '3.10', 'OS-windows']
title = '[Windows] cgi handling of POSTed files is broken in Windows'
updated_at = <Date 2021-04-27.14:24:08.871>
user = 'https://github.com/MLModel'

bugs.python.org fields:

activity = <Date 2021-04-27.14:24:08.871>
actor = 'vstinner'
assignee = 'orsenthil'
closed = False
closed_date = None
closer = None
components = ['Windows']
creation = <Date 2010-03-06.01:43:39.134>
creator = 'MLModel'
dependencies = []
files = ['16465', '25440']
hgrepos = []
issue_num = 8077
keywords = ['patch']
message_count = 5.0
messages = ['100509', '101211', '107405', '107960', '159838']
nosy_count = 11.0
nosy_names = ['gvanrossum', 'georg.brandl', 'paul.moore', 'ggenellina', 'orsenthil', 'tim.golden', 'MLModel', 'r.david.murray', 'quentel', 'zach.ware', 'steve.dower']
pr_nums = ['25652']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue8077'
versions = ['Python 3.10']

I am reluctant to post this because (a) I might have made some dumb mistake in my code, simple as it is, and (b) the problem might be well-known or even hopeless because the cgi module may not be WSGI compliant, but if this isn't going to be fixed then at least the problem should be described in the cgi module documentation.

I run an HTTPServer with a CGIHTTPRequestHandler.
I open an HTML file with a trivial form that simply POSTs an upload of a single file.
I browse, select a file, and click submit.
The web request never completes -- the browser just waits for a response.
Interrupting the server with ^C^C produces a backtrace that indicates it is stuck trying to decode the file contents.

The attached zip file contains:
cgi-server.py
cgi-post.html
cgi-bin/cgi-test.py
exception.txt
The latter is the backtrace output from when I interrupt the server.

This is on OS X 10.5 with either Python3.1 or Python3.2 from the repository.

This doesn't look like a documentation bug to me - handling of uploaded files via CGI *should* work, even if CGI is not the best way to do that.

The example works for me if I make this change:

--- Lib/cgi.py	(revision 81862)
+++ Lib/cgi.py	(working copy)
@@ -608,7 +608,7 @@
         parser = email.parser.FeedParser()
         # Create bogus content-type header for proper multipart parsing
         parser.feed('Content-Type: %s; boundary=%s\r\n\r\n' % (self.type, ib))
-        parser.feed(self.fp.read())
+        parser.feed(self.fp.read(self.length))
         full_msg = parser.close()
         # Get subparts
         msgs = full_msg.get_payload()

However this seems iffy to me because the content length presumably counts bytes whereas self.fp seems to be a text file, but since most HTTP clients don't close the connection, without some kind of boundary on the read() call it just hangs forever.

Also someone pointed out to me offline that this change may be needed, separately (though I haven't confirmed this yet):

--- Lib/cgi.py	(revision 81862)
+++ Lib/cgi.py	(working copy)
@@ -233,6 +233,7 @@
         lines = []
         while 1:
             line = fp.readline()
+            line = line.decode()
             if not line:
                 terminator = lastpart # End outer loop
                 break

Could this be related to bpo-4953?

There are 2 different problems :

handling of data by cgi.FieldStorage (bpo-4953) : fixed since version 3.2
in http.server.CGIHTTPRequestHandler, for POST requests on Windows, before opening the subprocess in run_cgi() all data is read by a *single* call to self.rfile.read(). If not all bytes are read by this single call, which is the case for a large file upload, only the read data are processed

The attached patch modifies http.server :

if all data are read in the first call to self.rfile.read() (that is, if its length is equal to the Content-length header), process them
if not, store all data in a temporary file (not in memory) and set the stdin argument of subprocess.Popen to this temporary file

With this patch, the tests provided by Mitchell all work on my PC (Windows XP Pro SP3)

cgi module is deprecated as per PEP 594 hence closing.

Navigation Menu

Repository navigation

[Windows] cgi handling of POSTed files is broken in Windows #52324

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[Windows] cgi handling of POSTed files is broken in Windows #52324

Description

Activity

MLModel commented on Mar 6, 2010

ggenellina commented on Mar 17, 2010

gvanrossum commented on Jun 9, 2010

gvanrossum commented on Jun 16, 2010

PierreQuentel commented on May 3, 2012

kumaraditya303 commented on Apr 15, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions