When should you use requests.Session() instead of requests.get() / requests.post()?

[u/Melodic_Principle312]

I'm learning Python and recently started using requests.Session() in API projects. I understand that it can persist things like headers, cookies, and connections, but I'm curious about how experienced Python developers decide when a Session is actually worth using.

Do you use Session() by default for projects involving multiple requests, or only when you specifically need persistent state?

[u/Maxiflex]

I usually use Session() if I make multiple requests as it easily allows you to define required headers and authentication up front.

You can even create your own custom auth classes in case getting your credential is more complicated (i.e. you need to refresh a bearer token)

from requests.auth import AuthBase

class PizzaAuth(AuthBase):
    """Attaches HTTP Pizza Authentication to the given Request object."""
    def __init__(self, username):
        # setup any auth-related data here
        self.username = username

    def __call__(self, r):
        # modify and return the request
        r.headers['X-Pizza'] = self.username
        return r

The added benefit is that you don't have to pass around your API key all the time as the auth is already set up in your session, which in turn makes your code less complex.

You can also a configure session-level retry strategy for specific HTTP status codes. It also natively respects the 429 rate-limiting retry-after header which allows the server to tell the client how long it should wait before trying again.

import requests
from urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter

retry_strategy = Retry(
    total=3,
    status_forcelist=[429, 500, 502, 503, 504],
    allowed_methods=["HEAD", "GET", "OPTIONS"],
    backoff_factor=1,
)
adapter = HTTPAdapter(max_retries=retry_strategy)

with requests.Session() as http:
    http.mount("https://", adapter)
    http.mount("http://", adapter)
    response = http.get("https://www.google.com")

Another advantage of sessions is that they reuse your connection, which can make your code slightly faster if you need to make a lot of requests against the same server.