Sometimes, we want to avoid exceeding max retries with URL in requests with Python.

In this article, we’ll look at how to avoid exceeding max retries with URL in requests with Python.

How to avoid exceeding max retries with URL in requests with Python?

To avoid exceeding max retries with URL in requests with Python, we can make our requests with the requests module.

For instance, we write:

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

url = 'http://example.com'
session = requests.Session()
retry = Retry(connect=3, backoff_factor=0.5)
adapter = HTTPAdapter(max_retries=retry)
session.mount('http://', adapter)
session.mount('https://', adapter)

session.get(url)

We use the Retry class to set the max number of retries.

total is the total retries to allow.

backoff_factor is the factor to apply between attempts.

Then we can use the returned object as the value of max_retries parameter of the HTTPAdapter constructor.

And we use the adapter for the requests with http and https requests by calling session.mount.

Finally, we make our GET request with session.get.

Conclusion

To avoid exceeding max retries with URL in requests with Python, we can make our requests with the requests module.