Repository navigation
httplib/http.client in method _tunnel used HTTP/1.0 CONNECT method #66897
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/orsenthil'
closed_at = None
created_at = <Date 2014-10-23.06:21:23.450>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'httplib/http.client in method _tunnel used HTTP/1.0 CONNECT method'
updated_at = <Date 2021-11-07.18:37:03.856>
user = 'https://github.com/vo-va'bugs.python.org fields:
activity = <Date 2021-11-07.18:37:03.856>
actor = 'eric.araujo'
assignee = 'orsenthil'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2014-10-23.06:21:23.450>
creator = 'vova'
dependencies = []
files = ['36996', '38808']
hgrepos = []
issue_num = 22708
keywords = ['patch']
message_count = 6.0
messages = ['229856', '229876', '229915', '239921', '239952', '241237']
nosy_count = 6.0
nosy_names = ['orsenthil', 'r.david.murray', 'berker.peksag', 'martin.panter', 'demian.brecht', 'vova']
pr_nums = ['742', '8305']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22708'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']
Activity
vo-va commentedon Oct 23, 2014
At my workplace I have to use corporate Internet proxy server with AD/domain/ntlm authorization. I use local cntlm proxy server to authorize myself on corporate proxy. Programs are send requests to cntlm proxy without any authorization information. Cntlm proxy communicate with corporate proxy and handle all authorization stuff and return response to programs.
But programs which use httplib, like pip, and want to open https url can't work in my network scheme. Because to open https connection httplib send to cntlm proxy
"CONNECT encrypted.google.com:443 HTTP/1.0\r\n"
HTTP/1.0 does not assume persistent connection so corporate proxy return http response 407 (need authorization) and close connection. Cntlm proxy detect closed connection and return http response 407 to pip/httplib which can't handle this response or begin ntlm negotiation, throw exception
and close.
So I suggest change HTTP CONNECT method to
"CONNECT %s:%d HTTP/1.1\r\n"
This change allow cntlm proxy keep alive connection to corporate proxy do all authorization stuff and return proper response.
And also in header of httplib is stated what it is "HTTP/1.1 client library"
bitdancer commentedon Oct 23, 2014
See bpo-21224 and bpo-9740. It's not 100% clear to me from reading those issues, but it *sounds* like the support is there, you just have to turn it on. So I'm closing this issue as not a bug...either it already works (and you just have to set your code to use 1.1) or this is a duplicate of one of those two issues, and you should comment on whichever one is more appropriate.
Your point about PIP is a good one, though...that might be worth a specific issue for 1.1 support in pip, if it doesn't already have a way to switch it on.
vo-va commentedon Oct 24, 2014
The issue http://bugs.python.org/issue21224 is about http server implementations. The issue http://bugs.python.org/issue9740 is more relevant for what I talking about, but not exactly.
Look, in this line
https://hg.python.org/cpython/file/6a2f74811240/Lib/http/client.py#l786
http protocol version is setted and in this line
https://hg.python.org/cpython/file/6a2f74811240/Lib/http/client.py#l1036
variable used to send http method (GET, PUT etc) and it work for direct connection or proxy with http connections. But if required to use CONNECT method through proxy (usually used for https connection) will be used _tunnel() method from http.client (py3k) or httplib (py2.7)
https://hg.python.org/cpython/file/6a2f74811240/Lib/http/client.py#l871
and here version off http is hardcoded to HTTP/1.0
PIP use urllib3, but urllib3 for actual network working (work with socket and send request) use httplib or http.client. So I think it would be better to make changes in httplib than override _tunnel() method in urllib3.
P.S. I'm not sure about rules how to open/close issues, so I open this issue again. I'm sorry if this causes some inconvenience.
demianbrecht commentedon Apr 2, 2015
Thanks for the report and follow up Vova. I've come across this line and it puzzled me as well as to why it's hardcoded. Rather than the hardcoded approach in your patch, I've attached a patch that uses _http_vsn_str which is consistent with other areas of the library.
Additionally, I noticed that a failure will occur if the destination address contains characters outside of ASCII range, so I've added support for IDN.
I assume that the patch should be committed to tip and maintenance branches as both issues are bugs (there is IDN support elsewhere in the library).
vadmium commentedon Apr 3, 2015
Patch looks good to me. I’m not an expert on non-ASCII domain names, but enabling HTTP 1.1 should be okay, at least for new versions of Python. CONNECT for HTTP 1.1 is described at <https://tools.ietf.org/html/rfc7231#section-4.3.6\>.
Also, see the code review about removing an comment about HTTP 1.1 blessing.
vo-va commentedon Apr 16, 2015
I don't know why I hardcoded version of HTTP, it was almost 6 months ago. Maybe because it just POC, or maybe because HTTP 1.0 is not allowed CONNECT method. Thank you for your patch and also for IDN support.
Qubits01 commentedon Dec 26, 2022
Sad to see the patch is not implemented 8 years after the issue was opened.
gh-66897: Upgrade HTTP CONNECT to protocol HTTP/1.1 (#8305)
arhadthedev commentedon Apr 5, 2023
gh-8305 is merged so closing as done.
pythongh-66897: Upgrade HTTP CONNECT to protocol HTTP/1.1 (python#8305)
pythongh-66897: Upgrade HTTP CONNECT to protocol HTTP/1.1 (python#8305)