New Upstream Release - requests
Ready changes
Summary
Merged new upstream version: 2.31.0+dfsg (was: 2.28.1+dfsg).
Diff
diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
index 617f2df3..1e7dba23 100644
--- a/.github/workflows/codeql-analysis.yml
+++ b/.github/workflows/codeql-analysis.yml
@@ -14,8 +14,15 @@ on:
schedule:
- cron: '0 23 * * 0'
+permissions:
+ contents: read
+
jobs:
analyze:
+ permissions:
+ actions: read # for github/codeql-action/init to get workflow details
+ contents: read # for actions/checkout to fetch code
+ security-events: write # for github/codeql-action/autobuild to send a status report
name: Analyze
runs-on: ubuntu-latest
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
index c69aa7fc..df275c51 100644
--- a/.github/workflows/lint.yml
+++ b/.github/workflows/lint.yml
@@ -1,16 +1,20 @@
name: Lint code
-on:
- push:
- pull_request:
+on: [push, pull_request]
+
+permissions:
+ contents: read
jobs:
lint:
runs-on: ubuntu-20.04
+ timeout-minutes: 10
steps:
- uses: actions/checkout@v3
- name: Set up Python
- uses: actions/setup-python@v3
+ uses: actions/setup-python@v4
+ with:
+ python-version: "3.x"
- name: Run pre-commit
uses: pre-commit/action@v3.0.0
diff --git a/.github/workflows/run-tests.yml b/.github/workflows/run-tests.yml
index 7b0e3c03..c4159508 100644
--- a/.github/workflows/run-tests.yml
+++ b/.github/workflows/run-tests.yml
@@ -2,6 +2,9 @@ name: Tests
on: [push, pull_request]
+permissions:
+ contents: read
+
jobs:
build:
runs-on: ${{ matrix.os }}
@@ -9,14 +12,13 @@ jobs:
strategy:
fail-fast: false
matrix:
- python-version: ["3.7", "3.8", "3.9", "3.10", "3.11-dev"]
- os: [ubuntu-18.04, macOS-latest, windows-latest]
+ python-version: ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12-dev", "pypy-3.8", "pypy-3.9"]
+ os: [ubuntu-22.04, macOS-latest, windows-latest]
include:
- # pypy-3.7 on Mac OS currently fails trying to compile
- # brotlipy. Moving pypy3 to only test linux.
+ # pypy-3.7 on Windows and Mac OS currently fails trying to compile
+ # cryptography. Moving pypy-3.7 to only test linux.
- python-version: pypy-3.7
os: ubuntu-latest
- experimental: false
steps:
- uses: actions/checkout@v2
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index cac5ddcc..5b915dc3 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -9,7 +9,7 @@ repos:
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/PyCQA/isort
- rev: 5.10.1
+ rev: 5.12.0
hooks:
- id: isort
- repo: https://github.com/psf/black
@@ -22,7 +22,7 @@ repos:
hooks:
- id: pyupgrade
args: [--py37-plus]
-- repo: https://gitlab.com/pycqa/flake8
- rev: 4.0.1
+- repo: https://github.com/PyCQA/flake8
+ rev: 6.0.0
hooks:
- id: flake8
diff --git a/HISTORY.md b/HISTORY.md
index 307f92a4..bbe6dd42 100644
--- a/HISTORY.md
+++ b/HISTORY.md
@@ -6,6 +6,64 @@ dev
- \[Short description of non-trivial change.\]
+2.31.0 (2023-05-22)
+-------------------
+
+**Security**
+- Versions of Requests between v2.3.0 and v2.30.0 are vulnerable to potential
+ forwarding of `Proxy-Authorization` headers to destination servers when
+ following HTTPS redirects.
+
+ When proxies are defined with user info (https://user:pass@proxy:8080), Requests
+ will construct a `Proxy-Authorization` header that is attached to the request to
+ authenticate with the proxy.
+
+ In cases where Requests receives a redirect response, it previously reattached
+ the `Proxy-Authorization` header incorrectly, resulting in the value being
+ sent through the tunneled connection to the destination server. Users who rely on
+ defining their proxy credentials in the URL are *strongly* encouraged to upgrade
+ to Requests 2.31.0+ to prevent unintentional leakage and rotate their proxy
+ credentials once the change has been fully deployed.
+
+ Users who do not use a proxy or do not supply their proxy credentials through
+ the user information portion of their proxy URL are not subject to this
+ vulnerability.
+
+ Full details can be read in our [Github Security Advisory](https://github.com/psf/requests/security/advisories/GHSA-j8r2-6x86-q33q)
+ and [CVE-2023-32681](https://nvd.nist.gov/vuln/detail/CVE-2023-32681).
+
+
+2.30.0 (2023-05-03)
+-------------------
+
+**Dependencies**
+- ⚠️ Added support for urllib3 2.0. ⚠️
+
+ This may contain minor breaking changes so we advise careful testing and
+ reviewing https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html
+ prior to upgrading.
+
+ Users who wish to stay on urllib3 1.x can pin to `urllib3<2`.
+
+2.29.0 (2023-04-26)
+-------------------
+
+**Improvements**
+
+- Requests now defers chunked requests to the urllib3 implementation to improve
+ standardization. (#6226)
+- Requests relaxes header component requirements to support bytes/str subclasses. (#6356)
+
+2.28.2 (2023-01-12)
+-------------------
+
+**Dependencies**
+
+- Requests now supports charset\_normalizer 3.x. (#6261)
+
+**Bugfixes**
+
+- Updated MissingSchema exception to suggest https scheme rather than http. (#6188)
2.28.1 (2022-06-29)
-------------------
@@ -42,7 +100,7 @@ dev
cert verification. All Requests 2.x versions before 2.28.0 are affected. (#6074)
- Fixed urllib3 exception leak, wrapping `urllib3.exceptions.SSLError` with
`requests.exceptions.SSLError` for `content` and `iter_content`. (#6057)
-- Fixed issue where invalid Windows registry entires caused proxy resolution
+- Fixed issue where invalid Windows registry entries caused proxy resolution
to raise an exception rather than ignoring the entry. (#6149)
- Fixed issue where entire payload could be included in the error message for
JSONDecodeError. (#6036)
diff --git a/Makefile b/Makefile
index 5e8fe3cc..f74dc42d 100644
--- a/Makefile
+++ b/Makefile
@@ -2,7 +2,7 @@
init:
pip install -r requirements-dev.txt
test:
- # This runs all of the tests, on both Python 2 and Python 3.
+ # This runs all of the tests on all supported Python versions.
tox -p
ci:
pytest tests --junitxml=report.xml
diff --git a/debian/changelog b/debian/changelog
index 93d60e87..ab28cc4d 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,10 +1,14 @@
-requests (2.28.1+dfsg-1.1) UNRELEASED; urgency=low
+requests (2.31.0+dfsg-1) UNRELEASED; urgency=low
+ [ Max-Julian Pogner ]
* Non-maintainer upload.
* Update upstream homepage, as the previous url would direct browsers
to a non-existant webserver.
- -- Max-Julian Pogner <max-julian@pogner.at> Fri, 16 Dec 2022 00:35:11 +0100
+ [ Debian Janitor ]
+ * New upstream release.
+
+ -- Max-Julian Pogner <max-julian@pogner.at> Sat, 24 Jun 2023 19:28:02 -0000
requests (2.28.1+dfsg-1) unstable; urgency=medium
diff --git a/debian/patches/0001-Remove-remote-images-traking-code-and-ads.patch b/debian/patches/0001-Remove-remote-images-traking-code-and-ads.patch
index 0650103a..5b9359f3 100644
--- a/debian/patches/0001-Remove-remote-images-traking-code-and-ads.patch
+++ b/debian/patches/0001-Remove-remote-images-traking-code-and-ads.patch
@@ -10,10 +10,10 @@ Subject: Remove remote images, traking code and ads
docs/index.rst | 17 ------------
5 files changed, 1 insertion(+), 137 deletions(-)
-diff --git a/docs/_templates/hacks.html b/docs/_templates/hacks.html
-index eca5dff..196abbe 100644
---- a/docs/_templates/hacks.html
-+++ b/docs/_templates/hacks.html
+Index: requests.git/docs/_templates/hacks.html
+===================================================================
+--- requests.git.orig/docs/_templates/hacks.html
++++ requests.git/docs/_templates/hacks.html
@@ -24,57 +24,3 @@
div.highlight pre {margin-right: -30px;}
}
@@ -72,10 +72,10 @@ index eca5dff..196abbe 100644
- }
- );
-</script>
-diff --git a/docs/_templates/sidebarintro.html b/docs/_templates/sidebarintro.html
-index 45d57b9..029c8c4 100644
---- a/docs/_templates/sidebarintro.html
-+++ b/docs/_templates/sidebarintro.html
+Index: requests.git/docs/_templates/sidebarintro.html
+===================================================================
+--- requests.git.orig/docs/_templates/sidebarintro.html
++++ requests.git/docs/_templates/sidebarintro.html
@@ -5,11 +5,6 @@
</p>
@@ -94,10 +94,10 @@ index 45d57b9..029c8c4 100644
-<div id="native-ribbon">
</div>
-diff --git a/docs/_templates/sidebarlogo.html b/docs/_templates/sidebarlogo.html
-index 56d6109..5a88a69 100644
---- a/docs/_templates/sidebarlogo.html
-+++ b/docs/_templates/sidebarlogo.html
+Index: requests.git/docs/_templates/sidebarlogo.html
+===================================================================
+--- requests.git.orig/docs/_templates/sidebarlogo.html
++++ requests.git/docs/_templates/sidebarlogo.html
@@ -1,30 +1,6 @@
<p>
- <iframe src="https://ghbtns.com/github-btn.html?user=psf&repo=requests&type=watch&count=true&size=large"
@@ -129,10 +129,10 @@ index 56d6109..5a88a69 100644
- <li><a href="https://github.com/psf/requests/issues">Issue Tracker</a></li>
-</ul>
-
-diff --git a/docs/conf.py b/docs/conf.py
-index edbd72b..1f5b050 100644
---- a/docs/conf.py
-+++ b/docs/conf.py
+Index: requests.git/docs/conf.py
+===================================================================
+--- requests.git.orig/docs/conf.py
++++ requests.git/docs/conf.py
@@ -128,7 +128,7 @@ html_theme_options = {
"show_powered_by": False,
"github_user": "requests",
@@ -142,10 +142,10 @@ index edbd72b..1f5b050 100644
"show_related": False,
"note_bg": "#FFF59C",
}
-diff --git a/docs/index.rst b/docs/index.rst
-index dbcaa55..31c774d 100644
---- a/docs/index.rst
-+++ b/docs/index.rst
+Index: requests.git/docs/index.rst
+===================================================================
+--- requests.git.orig/docs/index.rst
++++ requests.git/docs/index.rst
@@ -8,23 +8,6 @@ Requests: HTTP for Humans™
Release v\ |version|. (:ref:`Installation <install>`)
diff --git a/docs/api.rst b/docs/api.rst
index 83eb5878..34959dd6 100644
--- a/docs/api.rst
+++ b/docs/api.rst
@@ -36,6 +36,7 @@ Exceptions
.. autoexception:: requests.ConnectTimeout
.. autoexception:: requests.ReadTimeout
.. autoexception:: requests.Timeout
+.. autoexception:: requests.JSONDecodeError
Request Sessions
diff --git a/docs/user/advanced.rst b/docs/user/advanced.rst
index 2a91401a..c664a83d 100644
--- a/docs/user/advanced.rst
+++ b/docs/user/advanced.rst
@@ -656,10 +656,10 @@ certificates trusted by Requests can be found with::
from requests.utils import DEFAULT_CA_BUNDLE_PATH
print(DEFAULT_CA_BUNDLE_PATH)
-You override this default certificate bundle by setting the standard
-``curl_ca_bundle`` environment variable to another file path::
+You override this default certificate bundle by setting the ``REQUESTS_CA_BUNDLE``
+(or ``CURL_CA_BUNDLE``) environment variable to another file path::
- $ export curl_ca_bundle="/usr/local/myproxy_info/cacert.pem"
+ $ export REQUESTS_CA_BUNDLE="/usr/local/myproxy_info/cacert.pem"
$ export https_proxy="http://10.10.1.10:1080"
$ python
@@ -717,10 +717,9 @@ If ``chardet`` is installed, ``requests`` uses it, however for python3
library is an LGPL-licenced dependency and some users of requests
cannot depend on mandatory LGPL-licensed dependencies.
-When you install ``request`` without specifying ``[use_chardet_on_py3]]`` extra,
+When you install ``requests`` without specifying ``[use_chardet_on_py3]`` extra,
and ``chardet`` is not already installed, ``requests`` uses ``charset-normalizer``
-(MIT-licensed) to guess the encoding. For Python 2, ``requests`` uses only
-``chardet`` and is a mandatory dependency there.
+(MIT-licensed) to guess the encoding.
The only time Requests will not guess the encoding is if no explicit charset
is present in the HTTP headers **and** the ``Content-Type``
diff --git a/docs/user/install.rst b/docs/user/install.rst
index d0d454d3..7fa9a606 100644
--- a/docs/user/install.rst
+++ b/docs/user/install.rst
@@ -22,7 +22,7 @@ Requests is actively developed on GitHub, where the code is
You can either clone the public repository::
- $ git clone git://github.com/psf/requests.git
+ $ git clone https://github.com/psf/requests.git
Or, download the `tarball <https://github.com/psf/requests/tarball/main>`_::
diff --git a/docs/user/quickstart.rst b/docs/user/quickstart.rst
index 7fac5ce7..464e4f5f 100644
--- a/docs/user/quickstart.rst
+++ b/docs/user/quickstart.rst
@@ -177,7 +177,7 @@ server, you can access ``r.raw``. If you want to do this, make sure you set
<urllib3.response.HTTPResponse object at 0x101194810>
>>> r.raw.read(10)
- '\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'
+ b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'
In general, however, you should use a pattern like this to save what is being
streamed to a file::
@@ -237,7 +237,7 @@ dictionary of data will automatically be form-encoded when the request is made::
>>> payload = {'key1': 'value1', 'key2': 'value2'}
- >>> r = requests.post("https://httpbin.org/post", data=payload)
+ >>> r = requests.post('https://httpbin.org/post', data=payload)
>>> print(r.text)
{
...
diff --git a/pyproject.toml b/pyproject.toml
index 996bf14c..d3ab7bd9 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ src_paths = ["requests", "test"]
honor_noqa = true
[tool.pytest.ini_options]
-addopts = "-p no:warnings --doctest-modules"
+addopts = "--doctest-modules"
doctest_optionflags = "NORMALIZE_WHITESPACE ELLIPSIS"
minversion = "6.2"
testpaths = [
diff --git a/requests/__init__.py b/requests/__init__.py
index 7ac8e297..300a16c5 100644
--- a/requests/__init__.py
+++ b/requests/__init__.py
@@ -66,10 +66,10 @@ def check_compatibility(urllib3_version, chardet_version, charset_normalizer_ver
# Check urllib3 for compatibility.
major, minor, patch = urllib3_version # noqa: F811
major, minor, patch = int(major), int(minor), int(patch)
- # urllib3 >= 1.21.1, <= 1.26
- assert major == 1
- assert minor >= 21
- assert minor <= 26
+ # urllib3 >= 1.21.1
+ assert major >= 1
+ if major == 1:
+ assert minor >= 21
# Check charset_normalizer for compatibility.
if chardet_version:
@@ -80,8 +80,8 @@ def check_compatibility(urllib3_version, chardet_version, charset_normalizer_ver
elif charset_normalizer_version:
major, minor, patch = charset_normalizer_version.split(".")[:3]
major, minor, patch = int(major), int(minor), int(patch)
- # charset_normalizer >= 2.0.0 < 3.0.0
- assert (2, 0, 0) <= (major, minor, patch) < (3, 0, 0)
+ # charset_normalizer >= 2.0.0 < 4.0.0
+ assert (2, 0, 0) <= (major, minor, patch) < (4, 0, 0)
else:
raise Exception("You need either charset_normalizer or chardet installed")
diff --git a/requests/__version__.py b/requests/__version__.py
index e725ada6..5063c3f8 100644
--- a/requests/__version__.py
+++ b/requests/__version__.py
@@ -5,10 +5,10 @@
__title__ = "requests"
__description__ = "Python HTTP for Humans."
__url__ = "https://requests.readthedocs.io"
-__version__ = "2.28.1"
-__build__ = 0x022801
+__version__ = "2.31.0"
+__build__ = 0x023100
__author__ = "Kenneth Reitz"
__author_email__ = "me@kennethreitz.org"
__license__ = "Apache 2.0"
-__copyright__ = "Copyright 2022 Kenneth Reitz"
+__copyright__ = "Copyright Kenneth Reitz"
__cake__ = "\u2728 \U0001f370 \u2728"
diff --git a/requests/_internal_utils.py b/requests/_internal_utils.py
index 7dc9bc53..f2cf635e 100644
--- a/requests/_internal_utils.py
+++ b/requests/_internal_utils.py
@@ -14,9 +14,11 @@ _VALID_HEADER_NAME_RE_STR = re.compile(r"^[^:\s][^:\r\n]*$")
_VALID_HEADER_VALUE_RE_BYTE = re.compile(rb"^\S[^\r\n]*$|^$")
_VALID_HEADER_VALUE_RE_STR = re.compile(r"^\S[^\r\n]*$|^$")
+_HEADER_VALIDATORS_STR = (_VALID_HEADER_NAME_RE_STR, _VALID_HEADER_VALUE_RE_STR)
+_HEADER_VALIDATORS_BYTE = (_VALID_HEADER_NAME_RE_BYTE, _VALID_HEADER_VALUE_RE_BYTE)
HEADER_VALIDATORS = {
- bytes: (_VALID_HEADER_NAME_RE_BYTE, _VALID_HEADER_VALUE_RE_BYTE),
- str: (_VALID_HEADER_NAME_RE_STR, _VALID_HEADER_VALUE_RE_STR),
+ bytes: _HEADER_VALIDATORS_BYTE,
+ str: _HEADER_VALIDATORS_STR,
}
diff --git a/requests/adapters.py b/requests/adapters.py
index d3b2d5bb..78e3bb6e 100644
--- a/requests/adapters.py
+++ b/requests/adapters.py
@@ -22,7 +22,6 @@ from urllib3.exceptions import ProxyError as _ProxyError
from urllib3.exceptions import ReadTimeoutError, ResponseError
from urllib3.exceptions import SSLError as _SSLError
from urllib3.poolmanager import PoolManager, proxy_from_url
-from urllib3.response import HTTPResponse
from urllib3.util import Timeout as TimeoutSauce
from urllib3.util import parse_url
from urllib3.util.retry import Retry
@@ -194,7 +193,6 @@ class HTTPAdapter(BaseAdapter):
num_pools=connections,
maxsize=maxsize,
block=block,
- strict=True,
**pool_kwargs,
)
@@ -485,63 +483,19 @@ class HTTPAdapter(BaseAdapter):
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
- if not chunked:
- resp = conn.urlopen(
- method=request.method,
- url=url,
- body=request.body,
- headers=request.headers,
- redirect=False,
- assert_same_host=False,
- preload_content=False,
- decode_content=False,
- retries=self.max_retries,
- timeout=timeout,
- )
-
- # Send the request.
- else:
- if hasattr(conn, "proxy_pool"):
- conn = conn.proxy_pool
-
- low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
-
- try:
- skip_host = "Host" in request.headers
- low_conn.putrequest(
- request.method,
- url,
- skip_accept_encoding=True,
- skip_host=skip_host,
- )
-
- for header, value in request.headers.items():
- low_conn.putheader(header, value)
-
- low_conn.endheaders()
-
- for i in request.body:
- low_conn.send(hex(len(i))[2:].encode("utf-8"))
- low_conn.send(b"\r\n")
- low_conn.send(i)
- low_conn.send(b"\r\n")
- low_conn.send(b"0\r\n\r\n")
-
- # Receive the response from the server
- r = low_conn.getresponse()
-
- resp = HTTPResponse.from_httplib(
- r,
- pool=conn,
- connection=low_conn,
- preload_content=False,
- decode_content=False,
- )
- except Exception:
- # If we hit any problems here, clean up the connection.
- # Then, raise so that we can handle the actual exception.
- low_conn.close()
- raise
+ resp = conn.urlopen(
+ method=request.method,
+ url=url,
+ body=request.body,
+ headers=request.headers,
+ redirect=False,
+ assert_same_host=False,
+ preload_content=False,
+ decode_content=False,
+ retries=self.max_retries,
+ timeout=timeout,
+ chunked=chunked,
+ )
except (ProtocolError, OSError) as err:
raise ConnectionError(err, request=request)
diff --git a/requests/api.py b/requests/api.py
index 2f71aaed..cd0b3eea 100644
--- a/requests/api.py
+++ b/requests/api.py
@@ -106,7 +106,7 @@ def post(url, data=None, json=None, **kwargs):
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
- :param json: (optional) json data to send in the body of the :class:`Request`.
+ :param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
@@ -121,7 +121,7 @@ def put(url, data=None, **kwargs):
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
- :param json: (optional) json data to send in the body of the :class:`Request`.
+ :param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
@@ -136,7 +136,7 @@ def patch(url, data=None, **kwargs):
:param url: URL for the new :class:`Request` object.
:param data: (optional) Dictionary, list of tuples, bytes, or file-like
object to send in the body of the :class:`Request`.
- :param json: (optional) json data to send in the body of the :class:`Request`.
+ :param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`.
:param \*\*kwargs: Optional arguments that ``request`` takes.
:return: :class:`Response <Response>` object
:rtype: requests.Response
diff --git a/requests/models.py b/requests/models.py
index 3cd49f5b..617a4134 100644
--- a/requests/models.py
+++ b/requests/models.py
@@ -438,7 +438,7 @@ class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
if not scheme:
raise MissingSchema(
f"Invalid URL {url!r}: No scheme supplied. "
- f"Perhaps you meant http://{url}?"
+ f"Perhaps you meant https://{url}?"
)
if not host:
diff --git a/requests/sessions.py b/requests/sessions.py
index 6cb3b4da..dbcf2a7b 100644
--- a/requests/sessions.py
+++ b/requests/sessions.py
@@ -324,7 +324,9 @@ class SessionRedirectMixin:
except KeyError:
username, password = None, None
- if username and password:
+ # urllib3 handles proxy authorization for us in the standard adapter.
+ # Avoid appending this to TLS tunneled requests where it may be leaked.
+ if not scheme.startswith('https') and username and password:
headers["Proxy-Authorization"] = _basic_auth_str(username, password)
return new_proxies
diff --git a/requests/utils.py b/requests/utils.py
index ad535838..a367417f 100644
--- a/requests/utils.py
+++ b/requests/utils.py
@@ -25,7 +25,12 @@ from . import certs
from .__version__ import __version__
# to_native_string is unused here, but imported here for backwards compatibility
-from ._internal_utils import HEADER_VALIDATORS, to_native_string # noqa: F401
+from ._internal_utils import ( # noqa: F401
+ _HEADER_VALIDATORS_BYTE,
+ _HEADER_VALIDATORS_STR,
+ HEADER_VALIDATORS,
+ to_native_string,
+)
from .compat import (
Mapping,
basestring,
@@ -1031,20 +1036,23 @@ def check_header_validity(header):
:param header: tuple, in the format (name, value).
"""
name, value = header
+ _validate_header_part(header, name, 0)
+ _validate_header_part(header, value, 1)
- for part in header:
- if type(part) not in HEADER_VALIDATORS:
- raise InvalidHeader(
- f"Header part ({part!r}) from {{{name!r}: {value!r}}} must be "
- f"of type str or bytes, not {type(part)}"
- )
-
- _validate_header_part(name, "name", HEADER_VALIDATORS[type(name)][0])
- _validate_header_part(value, "value", HEADER_VALIDATORS[type(value)][1])
+def _validate_header_part(header, header_part, header_validator_index):
+ if isinstance(header_part, str):
+ validator = _HEADER_VALIDATORS_STR[header_validator_index]
+ elif isinstance(header_part, bytes):
+ validator = _HEADER_VALIDATORS_BYTE[header_validator_index]
+ else:
+ raise InvalidHeader(
+ f"Header part ({header_part!r}) from {header} "
+ f"must be of type str or bytes, not {type(header_part)}"
+ )
-def _validate_header_part(header_part, header_kind, validator):
if not validator.match(header_part):
+ header_kind = "name" if header_validator_index == 0 else "value"
raise InvalidHeader(
f"Invalid leading whitespace, reserved character(s), or return"
f"character(s) in header {header_kind}: {header_part!r}"
diff --git a/requirements-dev.txt b/requirements-dev.txt
index 6edee417..d6263737 100644
--- a/requirements-dev.txt
+++ b/requirements-dev.txt
@@ -1,11 +1,12 @@
-e .[socks]
pytest>=2.8.0,<=6.2.5
pytest-cov
-pytest-httpbin==1.0.0
+pytest-httpbin==2.0.0
pytest-mock==2.0.0
httpbin==0.7.0
trustme
wheel
+cryptography<40.0.0; python_version <= '3.7' and platform_python_implementation == 'PyPy'
# Flask Stack
Flask>1.0,<2.0
diff --git a/setup.cfg b/setup.cfg
index 33af66eb..bf21c81c 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -5,7 +5,7 @@ provides-extra =
use_chardet_on_py3
requires-dist =
certifi>=2017.4.17
- charset_normalizer>=2,<3
+ charset_normalizer>=2,<4
idna>=2.5,<4
urllib3>=1.21.1,<1.27
diff --git a/setup.py b/setup.py
index 23977ed7..01235457 100755
--- a/setup.py
+++ b/setup.py
@@ -59,13 +59,13 @@ if sys.argv[-1] == "publish":
sys.exit()
requires = [
- "charset_normalizer>=2,<3",
+ "charset_normalizer>=2,<4",
"idna>=2.5,<4",
- "urllib3>=1.21.1,<1.27",
+ "urllib3>=1.21.1,<3",
"certifi>=2017.4.17",
]
test_requirements = [
- "pytest-httpbin==0.0.7",
+ "pytest-httpbin==2.0.0",
"pytest-cov",
"pytest-mock",
"pytest-xdist",
@@ -94,7 +94,7 @@ setup(
package_data={"": ["LICENSE", "NOTICE"]},
package_dir={"requests": "requests"},
include_package_data=True,
- python_requires=">=3.7, <4",
+ python_requires=">=3.7",
install_requires=requires,
license=about["__license__"],
zip_safe=False,
diff --git a/tests/__init__.py b/tests/__init__.py
index 04385be1..c8561a08 100644
--- a/tests/__init__.py
+++ b/tests/__init__.py
@@ -2,9 +2,13 @@
import warnings
-from urllib3.exceptions import SNIMissingWarning
+try:
+ from urllib3.exceptions import SNIMissingWarning
-# urllib3 sets SNIMissingWarning to only go off once,
-# while this test suite requires it to always fire
-# so that it occurs during test_requests.test_https_warnings
-warnings.simplefilter("always", SNIMissingWarning)
+ # urllib3 1.x sets SNIMissingWarning to only go off once,
+ # while this test suite requires it to always fire
+ # so that it occurs during test_requests.test_https_warnings
+ warnings.simplefilter("always", SNIMissingWarning)
+except ImportError:
+ # urllib3 2.0 removed that warning and errors out instead
+ SNIMissingWarning = None
diff --git a/tests/test_requests.py b/tests/test_requests.py
index 5b4c3f53..b420c44d 100644
--- a/tests/test_requests.py
+++ b/tests/test_requests.py
@@ -48,6 +48,7 @@ from requests.models import PreparedRequest, urlencode
from requests.sessions import SessionRedirectMixin
from requests.structures import CaseInsensitiveDict
+from . import SNIMissingWarning
from .compat import StringIO
from .utils import override_environ
@@ -646,6 +647,26 @@ class TestRequests:
assert sent_headers.get("Proxy-Authorization") == proxy_auth_value
+
+ @pytest.mark.parametrize(
+ "url,has_proxy_auth",
+ (
+ ('http://example.com', True),
+ ('https://example.com', False),
+ ),
+ )
+ def test_proxy_authorization_not_appended_to_https_request(self, url, has_proxy_auth):
+ session = requests.Session()
+ proxies = {
+ 'http': 'http://test:pass@localhost:8080',
+ 'https': 'http://test:pass@localhost:8090',
+ }
+ req = requests.Request('GET', url)
+ prep = req.prepare()
+ session.rebuild_proxies(prep, proxies)
+
+ assert ('Proxy-Authorization' in prep.headers) is has_proxy_auth
+
def test_basicauth_with_netrc(self, httpbin):
auth = ("user", "pass")
wrong_auth = ("wronguser", "wrongpass")
@@ -974,6 +995,10 @@ class TestRequests:
r = requests.get(httpbin(), cert=".")
assert r.status_code == 200
+ @pytest.mark.skipif(
+ SNIMissingWarning is None,
+ reason="urllib3 2.0 removed that warning and errors out instead",
+ )
def test_https_warnings(self, nosan_server):
"""warnings are emitted with requests.get"""
host, port, ca_bundle = nosan_server
@@ -1747,6 +1772,31 @@ class TestRequests:
with pytest.raises(InvalidHeader):
requests.get(httpbin("get"), headers=invalid_header)
+ def test_header_with_subclass_types(self, httpbin):
+ """If the subclasses does not behave *exactly* like
+ the base bytes/str classes, this is not supported.
+ This test is for backwards compatibility.
+ """
+
+ class MyString(str):
+ pass
+
+ class MyBytes(bytes):
+ pass
+
+ r_str = requests.get(httpbin("get"), headers={MyString("x-custom"): "myheader"})
+ assert r_str.request.headers["x-custom"] == "myheader"
+
+ r_bytes = requests.get(
+ httpbin("get"), headers={MyBytes(b"x-custom"): b"myheader"}
+ )
+ assert r_bytes.request.headers["x-custom"] == b"myheader"
+
+ r_mixed = requests.get(
+ httpbin("get"), headers={MyString("x-custom"): MyBytes(b"myheader")}
+ )
+ assert r_mixed.request.headers["x-custom"] == b"myheader"
+
@pytest.mark.parametrize("files", ("foo", b"foo", bytearray(b"foo")))
def test_can_send_objects_with_files(self, httpbin, files):
data = {"a": "this is a string"}
diff --git a/tox.ini b/tox.ini
index 2e4fd90d..546c7371 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,5 +1,5 @@
[tox]
-envlist = py{37,38,39,310}-{default, use_chardet_on_py3}
+envlist = py{37,38,39,310,311}-{default, use_chardet_on_py3}
[testenv]
deps = -rrequirements-dev.txt
More details
Historical runs
- failed: FAILED tests/test_requests.py::TestRequests::test_header_with_subclass_types
- nothing-to-do: Last upstream version 2.27.1 already imported. Import a snapshot by specifying --snapshot.
- quilt-refresh-error: An error occurred refreshing quilt patch 0001-Remove-remote-images-traking-code-and-ads.patch: Applying patch debian/patches/0001-Remove-remote-images-traking-code-and-ads.patch patching file docs/_templates/hacks.html patching file docs/_templates/sidebarintro.html Hunk #1 FAILED at 1. Hunk #2 succeeded at 75 (offset 6 lines). 1 out of 2 hunks FAILED -- rejects in file docs/_templates/sidebarintro.html patching file docs/_templates/sidebarlogo.html patching file docs/conf.py patching file docs/index.rst Hunk #1 FAILED at 8. 1 out of 1 hunk FAILED -- rejects in file docs/index.rst Patch debian/patches/0001-Remove-remote-images-traking-code-and-ads.patch does not apply (enforce with -f)