Skip to content

Commit

Permalink
🔖 Release 3.8.0 (#144)
Browse files Browse the repository at this point in the history 
**Added**
- Support for HTTP Trailers.
- Help script now yield warnings if update are available for each sub
dependencies.

**Fixed**
- Setting a list of Resolver.

**Changed**
- urllib3-future lower bound version is raised to 2.9.900 (for http
trailer support).
- relax strict kwargs passing in Session adapters (required for some
plugins).
  • Loading branch information
@Ousret
Ousret authored Sep 24, 2024
1 parent 8956008 commit b2bdac6
Show file tree
Hide file tree
Showing 13 changed files with 221 additions and 51 deletions.
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,4 @@ repos:
- id: mypy
args: [--check-untyped-defs]
exclude: 'tests/|noxfile.py'
additional_dependencies: ['charset_normalizer', 'urllib3.future>=2.7.904', 'wassima>=1.0.1', 'idna', 'kiss_headers']
additional_dependencies: ['charset_normalizer', 'urllib3.future>=2.9.900', 'wassima>=1.0.1', 'idna', 'kiss_headers']
14 changes: 14 additions & 0 deletions HISTORY.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,20 @@
Release History
===============

3.8.0 (2024-09-24)
------------------

**Added**
- Support for HTTP Trailers.
- Help script now yield warnings if update are available for each sub dependencies.

**Fixed**
- Setting a list of Resolver.

**Changed**
- urllib3-future lower bound version is raised to 2.9.900 (for http trailer support).
- relax strict kwargs passing in Session adapters (required for some plugins).

3.7.2 (2024-07-09)
------------------

Expand Down
61 changes: 33 additions & 28 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,34 +12,36 @@ Niquests, is the “**Safest**, **Fastest[^10]**, **Easiest**, and **Most advanc
<details>
<summary>👆 <b>Look at the feature table comparison</b> against <i>requests, httpx and aiohttp</i>!</summary>

| Feature | niquests | requests | httpx | aiohttp |
|--------------------------------------------------------------------------|:-------------------------:|:----------------------------------:|:-----------------------------:|----------------------|
| `HTTP/1.1` | ✅ | ✅ | ✅ | ✅ |
| `HTTP/2` | ✅ | ❌ | ✅[^7] | ❌ |
| `HTTP/3 over QUIC` | ✅ | ❌ | ❌ | ❌ |
| `Synchronous` | ✅ | ✅ | ✅ | ❌ |
| `Asynchronous` | ✅ | ❌ | ✅ | ✅ |
| `Thread Safe` | ✅ | ✅ | ❌[^5] | _N/A_[^1] |
| `Task Safe` | ✅ | _N/A_[^2] | ✅ | ✅ |
| `OS Trust Store` | ✅ | ❌ | ❌ | ❌ |
| `Multiplexing` | ✅ | ❌ | _Limited_[^3] | ❌ |
| `DNSSEC` | ✅[^11] | ❌ | ❌ | ❌ |
| `Customizable DNS Resolution` | ✅ | ❌ | ❌ | ✅ |
| `DNS over HTTPS` | ✅ | ❌ | ❌ | ❌ |
| `DNS over QUIC` | ✅ | ❌ | ❌ | ❌ |
| `DNS over TLS` | ✅ | ❌ | ❌ | ❌ |
| `Multiple DNS Resolver` | ✅ | ❌ | ❌ | ❌ |
| `Network Fine Tuning & Inspect` | ✅ | ❌ | _Limited_[^6] | _Limited_[^6] |
| `Certificate Revocation Protection` | ✅ | ❌ | ❌ | ❌ |
| `Session Persistence` | ✅ | ✅ | ✅ | ✅ |
| `In-memory Certificate CA & mTLS` | ✅ | ❌ | _Limited_[^4] | _Limited_[^4] |
| `SOCKS 4/5 Proxies` | ✅ | ✅ | ✅ | ❌ |
| `HTTP/HTTPS Proxies` | ✅ | ✅ | ✅ | ✅ |
| `TLS-in-TLS Support` | ✅ | ✅ | ✅ | ✅ |
| `Direct HTTP/3 Negotiation` | ✅[^9] | N/A[^8] | N/A[^8] | N/A[^8] |
| `Happy Eyeballs` | ✅ | ❌ | ❌ | ✅ |
| `Package / SLSA Signed` | ✅ | ❌ | ❌ | ✅ |
| `HTTP/2 with prior knowledge (h2c)` | ✅ | ❌ | ✅ | ❌ |
| Feature | niquests | requests | httpx | aiohttp |
|-------------------------------------|:--------------:|:---------:|:-------------:|---------------|
| `HTTP/1.1` | ✅ | ✅ | ✅ | ✅ |
| `HTTP/2` | ✅ | ❌ | ✅[^7] | ❌ |
| `HTTP/3 over QUIC` | ✅ | ❌ | ❌ | ❌ |
| `Synchronous` | ✅ | ✅ | ✅ | ❌ |
| `Asynchronous` | ✅ | ❌ | ✅ | ✅ |
| `Thread Safe` | ✅ | ✅ | ❌[^5] | _N/A_[^1] |
| `Task Safe` | ✅ | _N/A_[^2] | ✅ | ✅ |
| `OS Trust Store` | ✅ | ❌ | ❌ | ❌ |
| `Multiplexing` | ✅ | ❌ | _Limited_[^3] | ❌ |
| `DNSSEC` | ✅[^11] | ❌ | ❌ | ❌ |
| `Customizable DNS Resolution` | ✅ | ❌ | ❌ | ✅ |
| `DNS over HTTPS` | ✅ | ❌ | ❌ | ❌ |
| `DNS over QUIC` | ✅ | ❌ | ❌ | ❌ |
| `DNS over TLS` | ✅ | ❌ | ❌ | ❌ |
| `Multiple DNS Resolver` | ✅ | ❌ | ❌ | ❌ |
| `Network Fine Tuning & Inspect` | ✅ | ❌ | _Limited_[^6] | _Limited_[^6] |
| `Certificate Revocation Protection` | ✅ | ❌ | ❌ | ❌ |
| `Session Persistence` | ✅ | ✅ | ✅ | ✅ |
| `In-memory Certificate CA & mTLS` | ✅ | ❌ | _Limited_[^4] | _Limited_[^4] |
| `SOCKS 4/5 Proxies` | ✅ | ✅ | ✅ | ❌ |
| `HTTP/HTTPS Proxies` | ✅ | ✅ | ✅ | ✅ |
| `TLS-in-TLS Support` | ✅ | ✅ | ✅ | ✅ |
| `Direct HTTP/3 Negotiation` | ✅[^9] | N/A[^8] | N/A[^8] | N/A[^8] |
| `Happy Eyeballs` | ✅ | ❌ | ❌ | ✅ |
| `Package / SLSA Signed` | ✅ | ❌ | ❌ | ✅ |
| `HTTP/2 with prior knowledge (h2c)` | ✅ | ❌ | ✅ | ❌ |
| `Post-Quantum Security` | _Limited_[^12] | ❌ | ❌ | ❌ |
| `HTTP Trailers` | ✅ | ❌ | ❌ | ❌ |
</details>

<details>
Expand Down Expand Up @@ -148,6 +150,7 @@ Niquests is ready for the demands of building scalable, robust and reliable HTTP
- HTTP/2 with prior knowledge
- Object-oriented headers
- Multi-part File Uploads
- Post-Quantum Security
- Chunked HTTP Requests
- Fully type-annotated!
- SOCKS Proxy Support
Expand All @@ -158,6 +161,7 @@ Niquests is ready for the demands of building scalable, robust and reliable HTTP
- Happy Eyeballs
- Multiplexed!
- Thread-safe!
- Trailers!
- DNSSEC!
- Async!

Expand Down Expand Up @@ -198,3 +202,4 @@ Niquests is a highly improved HTTP client that is based (forked) on Requests. Th
[^9]: you must use a custom DNS resolver so that it can preemptively connect using HTTP/3 over QUIC when remote is compatible.
[^10]: performance measured when leveraging a multiplexed connection with or without uses of any form of concurrency as of July 2024. The research compared `httpx`, `requests`, `aiohttp` against `niquests`. See https://github.com/Ousret/niquests-stats
[^11]: enabled when using a custom DNS resolver.
[^12]: available only when using HTTP/3 over QUIC and that the remote server support also the same post-quantum key-exchange algorithm. Also, the `qh3` installed version must be >= 1.1.
2 changes: 2 additions & 0 deletions docs/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ Niquests is ready for today's web.
- Familiar `dict`–like Cookies
- Object-oriented headers
- Multi-part File Uploads
- Post-Quantum Security
- Chunked HTTP Requests
- Fully type-annotated!
- SOCKS Proxy Support
Expand All @@ -88,6 +89,7 @@ Niquests is ready for today's web.
- Happy Eyeballs
- Multiplexed!
- Thread-safe!
- Trailers!
- DNSSEC!
- Async!

Expand Down
22 changes: 22 additions & 0 deletions docs/user/advanced.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1485,3 +1485,25 @@ Here is a basic example of how you would proceed::
for chunk in r.iter_content():
# do anything you want with chunk
print(r.download_progress.total) # this actually contain the amt of bytes (raw) downloaded from the socket.


HTTP Trailers
-------------

.. note:: Available since Niquests 3.8+

HTTP response may contain one or several trailer headers. Those special headers are received
after the reception of the body. Before this, those headers were unreachable and dropped silently.

Quoted from Mozilla MDN: "The Trailer response header allows the sender to include additional fields
at the end of chunked messages in order to supply metadata that might be dynamically generated while the
message body is sent, such as a message integrity check, digital signature, or post-processing status."

For example, we retrieve our trailers this way::

>>> url = 'https://httpbingo.org/trailers?foo=baz'
>>> r = niquests.get(url)
>>> r.trailers # output: {'foo': 'baz'}


.. warning:: The ``trailers`` property is only filled when the response has been consumed entirely. The server only send them after finishing sending the body. By default, ``trailers`` is an empty CaseInsensibleDict.
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ dynamic = ["version"]
dependencies = [
"charset_normalizer>=2,<4",
"idna>=2.5,<4",
"urllib3.future>=2.8.902,<3",
"urllib3.future>=2.9.900,<3",
"wassima>=1.0.1,<2",
"kiss_headers>=2,<4",
]
Expand Down
4 changes: 2 additions & 2 deletions src/niquests/__version__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@
__url__: str = "https://niquests.readthedocs.io"

__version__: str
__version__ = "3.7.2"
__version__ = "3.8.0"

__build__: int = 0x030702
__build__: int = 0x030800
__author__: str = "Kenneth Reitz"
__author_email__: str = "[email protected]"
__license__: str = "Apache-2.0"
Expand Down
54 changes: 44 additions & 10 deletions src/niquests/help.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@
import idna
import wassima

from . import RequestException
from . import RequestException, HTTPError
from . import __version__ as niquests_version
from . import get
from . import Session
from ._compat import HAS_LEGACY_URLLIB3

if HAS_LEGACY_URLLIB3 is True:
Expand Down Expand Up @@ -142,26 +142,60 @@ def info():
}


def main() -> None:
"""Pretty-print the bug information as JSON."""
pypi_session = Session()


def check_update(package_name: str, actual_version: str) -> None:
"""
Small and concise utility to check for updates.
"""
try:
response = get("https://pypi.org/pypi/niquests/json")
package_info = response.json()
response = pypi_session.get(f"https://pypi.org/pypi/{package_name}/json")
package_info = response.raise_for_status().json()

if (
isinstance(package_info, dict)
and "info" in package_info
and "version" in package_info["info"]
):
if package_info["info"]["version"] != niquests_version:
if package_info["info"]["version"] != actual_version:
warnings.warn(
f"You are using Niquests {niquests_version} and PyPI yield version ({package_info['info']['version']}) as the stable one. "
"We invite you to install this version as soon as possible. Run `python -m pip install niquests -U`.",
f"You are using {package_name} {actual_version} and PyPI yield version ({package_info['info']['version']}) as the stable one. "
f"We invite you to install this version as soon as possible. Run `python -m pip install {package_name} -U`.",
UserWarning,
)
except (RequestException, JSONDecodeError):
except (RequestException, JSONDecodeError, HTTPError):
pass


PACKAGE_TO_CHECK_FOR_UPGRADE = {
"niquests": niquests_version,
"urllib3-future": urllib3.__version__,
"qh3": qh3.__version__ if qh3 is not None else None,
"jh2": jh2.__version__,
"h11": h11.__version__,
"charset-normalizer": charset_normalizer.__version__,
"wassima": wassima.__version__,
"idna": idna.__version__,
}


def main() -> None:
"""Pretty-print the bug information as JSON."""
for package, actual_version in PACKAGE_TO_CHECK_FOR_UPGRADE.items():
if actual_version is None:
continue
check_update(package, actual_version)

if __legacy_urllib3_version__ is not None:
warnings.warn(
"urllib3-future is installed alongside (legacy) urllib3. This may cause compatibility issues."
"Some (Requests) 3rd parties may be bound to urllib3, therefor the plugins may wrongfully invoke"
"urllib3 (legacy) instead of urllib3-future. To remediate this, run "
"`python -m pip uninstall -y urllib3 urllib3-future`, then run `python -m pip install urllib3-future`.",
UserWarning,
)

print(json.dumps(info(), sort_keys=True, indent=2))


Expand Down
24 changes: 24 additions & 0 deletions src/niquests/models.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -976,6 +976,10 @@ def __init__(self) -> None:
#: value of a ``'Content-Encoding'`` response header.
self.headers: CaseInsensitiveDict = CaseInsensitiveDict()

#: Case-insensitive Dictionary of Response Trailer Headers.
#: This can only be filled after response consumption.
self.trailers: CaseInsensitiveDict = CaseInsensitiveDict()

#: File-like object representation of response (for advanced usage).
#: Use of ``raw`` requires that ``stream=True`` be set on the request.
#: This requirement does not apply for use internally to Requests.
Expand Down Expand Up @@ -1245,6 +1249,9 @@ def generate() -> typing.Generator[bytes, None, None]:
break
yield chunk

if self.raw is not None and hasattr(self.raw, "trailers"):
self.trailers = CaseInsensitiveDict(self.raw.trailers)

self._content_consumed = True

if self._content_consumed and isinstance(self._content, bool):
Expand Down Expand Up @@ -1347,6 +1354,15 @@ def oheaders(self) -> Headers:
return headers
return parse_it(self.headers)

@property
def otrailers(self) -> Headers:
"""
Retrieve trailers as they were objects. There is no need to parse headers yourself.
"""
if self.raw:
return parse_it(self.raw.trailers)
return parse_it(self.trailers)

@property
def content(self) -> bytes | None:
"""Content of the response, in bytes."""
Expand Down Expand Up @@ -1374,6 +1390,8 @@ def content(self) -> bytes | None:
raise RequestsSSLError(e)

self._content_consumed = True
if self.raw is not None and hasattr(self.raw, "trailers"):
self.trailers = CaseInsensitiveDict(self.raw.trailers)
# don't need to release the connection; that's been handled by urllib3
# since we exhausted the data.
return self._content
Expand Down Expand Up @@ -1662,6 +1680,9 @@ async def generate() -> (

yield chunk

if self.raw is not None and hasattr(self.raw, "trailers"):
self.trailers = CaseInsensitiveDict(self.raw.trailers)

self._content_consumed = True

if self._content_consumed and isinstance(self._content, bool):
Expand Down Expand Up @@ -1761,6 +1782,9 @@ async def content(self) -> bytes | None: # type: ignore[override]
except SSLError as e:
raise RequestsSSLError(e)

if self.raw is not None and hasattr(self.raw, "trailers"):
self.trailers = CaseInsensitiveDict(self.raw.trailers)

self._content_consumed = True
# don't need to release the connection; that's been handled by urllib3
# since we exhausted the data.
Expand Down
Loading

0 comments on commit b2bdac6

Please sign in to comment.