Feature or enhancement

CookieJar.add_cookie_header() gets slower with every stored domain, even when only one cookie matches the request:

import timeit, urllib.request
from http.cookiejar import CookieJar, Cookie

def cookie(name, domain):
    return Cookie(0, name, "v", None, False, domain, True, False, "/", True, False, None, True, None, None, {})

for n in (10, 100, 1000, 10000):
    jar = CookieJar()
    for i in range(n):
        jar.set_cookie(cookie(f"c{i}", f"site{i}.example"))
    req = urllib.request.Request("https://site0.example/")
    t = min(timeit.repeat(lambda: jar.add_cookie_header(req), number=200, repeat=3)) / 200
    print(f"{n:>6} domains: {t*1e6:8.1f} us")

    10 domains:     37.9 us
   100 domains:    320.8 us
  1000 domains:   3086.9 us
 10000 domains:  30450.6 us

Each call:

1. Runs DefaultCookiePolicy.domain_return_ok() for every stored domain, and each call re-parses the request URL via eff_request_host().
2. Runs clear_expired_cookies(), which iterates every stored cookie.

Possible improvements, in increasing size:

1. Compute eff_request_host() once per request instead of once per domain.
2. With DefaultCookiePolicy, look up only the domain keys that can match the request host (its dot-suffixes) instead of iterating all domains.
3. Track the earliest expiry in set_cookie() and skip the sweep until it has passed.

HTTPX2 ships 2 and 3 as a CookieJar subclass, which passes test_http_cookiejar: pydantic/httpx2#1285

I'm happy to send a PR.

Linked PRs

• gh-160027: Use urlsplit() instead of urlparse() in http.cookiejar.request_host() #160680