Skip to content

Commit 4c12e0d

Browse files
author
Cursor
committed
Add extension class documentation for aiohttp and requests modules
Document the internal extension classes (ProxyTCPConnector, ProxyClientRequest, ProxyClientResponse, HTTPProxyHeaderAdapter, ProxySession) to improve API coverage and help users understand how the proxy header capture mechanism works.
1 parent e70bf3d commit 4c12e0d

2 files changed

Lines changed: 155 additions & 7 deletions

File tree

‎docs/aiohttp.rst‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,3 +101,75 @@ The ``ProxyClientSession`` works just like the standard ``ClientSession`` and su
101101

102102
All standard aiohttp request methods are supported: ``get``, ``post``, ``put``, ``delete``, ``patch``, ``head``, and ``options``.
103103

104+
Extension Classes
105+
-----------------
106+
107+
The ``python_proxy_headers.aiohttp_proxy`` module provides several extension classes that work together to capture and expose proxy response headers. These classes extend aiohttp's internal classes.
108+
109+
ProxyClientSession
110+
~~~~~~~~~~~~~~~~~~
111+
112+
The main entry point for using proxy headers with aiohttp. This class extends ``aiohttp.ClientSession`` and automatically configures the session to use the other extension classes.
113+
114+
.. code-block:: python
115+
116+
from python_proxy_headers.aiohttp_proxy import ProxyClientSession
117+
118+
async with ProxyClientSession() as session:
119+
async with session.get('https://example.com', proxy="http://PROXYHOST:PORT") as r:
120+
proxy_ip = r.headers.get('X-ProxyMesh-IP')
121+
122+
The ``ProxyClientSession`` constructor accepts all the same arguments as ``aiohttp.ClientSession``, and automatically sets:
123+
124+
* ``connector`` to ``ProxyTCPConnector()``
125+
* ``response_class`` to ``ProxyClientResponse``
126+
* ``request_class`` to ``ProxyClientRequest``
127+
128+
ProxyTCPConnector
129+
~~~~~~~~~~~~~~~~~
130+
131+
Extends ``aiohttp.TCPConnector`` to capture proxy response headers during HTTPS tunnel establishment. This class overrides the ``_create_proxy_connection`` method to:
132+
133+
1. Send the CONNECT request with custom proxy headers
134+
2. Capture the proxy's response headers from the CONNECT response
135+
3. Store them on the protocol object for later retrieval
136+
137+
When establishing an HTTPS connection through a proxy, the connector:
138+
139+
* Creates a CONNECT request to the proxy server
140+
* Includes any custom proxy headers you've specified
141+
* Captures the proxy's response headers (e.g., ``X-ProxyMesh-IP``)
142+
* Stores them so they can be merged into the final response
143+
144+
You typically don't need to use this class directly - it's automatically configured when using ``ProxyClientSession``.
145+
146+
ProxyClientRequest
147+
~~~~~~~~~~~~~~~~~~
148+
149+
Extends ``aiohttp.ClientRequest`` to transfer proxy headers from the connection protocol to the response object. This class overrides the ``send`` method to check if the connection's protocol has captured proxy headers and attaches them to the response.
150+
151+
This class is used internally by ``ProxyClientSession`` and typically doesn't need to be used directly.
152+
153+
ProxyClientResponse
154+
~~~~~~~~~~~~~~~~~~~
155+
156+
Extends ``aiohttp.ClientResponse`` to merge proxy response headers into the response's headers property. This class overrides the ``headers`` property to:
157+
158+
1. Check if proxy headers were captured during tunnel establishment
159+
2. If present, merge them with the target server's response headers
160+
3. Return a combined ``CIMultiDictProxy`` containing both sets of headers
161+
162+
This allows you to access proxy response headers (like ``X-ProxyMesh-IP``) directly from the response object's ``headers`` property, alongside the target server's response headers.
163+
164+
How It Works
165+
~~~~~~~~~~~~
166+
167+
The extension classes work together in the following flow:
168+
169+
1. **ProxyClientSession** creates a session configured with all the extension classes
170+
2. **ProxyTCPConnector** intercepts the CONNECT request/response during tunnel establishment and captures proxy headers
171+
3. **ProxyClientRequest** transfers the captured headers from the protocol to the response object
172+
4. **ProxyClientResponse** merges the proxy headers into the response's ``headers`` property
173+
174+
This allows proxy response headers to be transparently available in your application without any special handling.
175+

‎docs/requests.rst‎

Lines changed: 83 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -108,18 +108,94 @@ The exact headers available depend on your proxy provider. Check your proxy prov
108108
Session Support
109109
---------------
110110

111-
You can also use the adapter with requests Sessions for better connection pooling and cookie handling:
111+
For better connection pooling and cookie handling, you can use ``ProxySession``:
112112

113113
.. code-block:: python
114114
115-
from python_proxy_headers import requests_adapter
115+
from python_proxy_headers.requests_adapter import ProxySession
116+
117+
with ProxySession(proxy_headers={'X-ProxyMesh-Country': 'US'}) as session:
118+
r = session.get('https://api.example.com', proxies=proxies)
119+
proxy_ip = r.headers.get('X-ProxyMesh-IP')
120+
121+
Or you can manually mount the adapter to a standard requests Session:
122+
123+
.. code-block:: python
124+
125+
from python_proxy_headers.requests_adapter import HTTPProxyHeaderAdapter
116126
import requests
117127
118128
session = requests.Session()
119-
session.mount('http://', requests_adapter.HTTPAdapter())
120-
session.mount('https://', requests_adapter.HTTPAdapter())
129+
session.mount('http://', HTTPProxyHeaderAdapter(proxy_headers={'X-ProxyMesh-Country': 'US'}))
130+
session.mount('https://', HTTPProxyHeaderAdapter(proxy_headers={'X-ProxyMesh-Country': 'US'}))
121131
122-
r = session.get('https://api.example.com',
123-
proxies=proxies,
124-
proxy_headers={'X-ProxyMesh-Country': 'US'})
132+
r = session.get('https://api.example.com', proxies=proxies)
133+
proxy_ip = r.headers.get('X-ProxyMesh-IP')
134+
135+
Extension Classes
136+
-----------------
137+
138+
The ``python_proxy_headers.requests_adapter`` module provides extension classes that build on top of the ``urllib3_proxy_manager`` module to integrate proxy header support with the requests library.
139+
140+
HTTPProxyHeaderAdapter
141+
~~~~~~~~~~~~~~~~~~~~~~
142+
143+
Extends ``requests.adapters.HTTPAdapter`` to use our custom ``ProxyHeaderManager`` for proxy connections. This adapter enables both sending custom proxy headers and receiving proxy response headers.
144+
145+
.. code-block:: python
146+
147+
from python_proxy_headers.requests_adapter import HTTPProxyHeaderAdapter
148+
149+
adapter = HTTPProxyHeaderAdapter(proxy_headers={'X-ProxyMesh-Country': 'US'})
150+
151+
**Constructor Parameters:**
152+
153+
* ``proxy_headers`` (dict, optional): A dictionary of custom headers to send to the proxy server. These headers will be included in the CONNECT request when establishing HTTPS tunnel connections.
154+
155+
The adapter overrides the ``proxy_manager_for`` method to:
156+
157+
1. Check if the proxy URL is already cached in ``self.proxy_manager``
158+
2. For SOCKS proxies, delegate to the parent class
159+
3. For HTTP/HTTPS proxies, create a ``ProxyHeaderManager`` from the ``urllib3_proxy_manager`` module with the custom proxy headers
160+
161+
This ensures that all proxy connections made through the adapter will include your custom headers and capture proxy response headers.
162+
163+
ProxySession
164+
~~~~~~~~~~~~
165+
166+
Extends ``requests.Session`` with pre-configured ``HTTPProxyHeaderAdapter`` instances for both HTTP and HTTPS connections. This provides a convenient way to create a session that automatically handles proxy headers.
167+
168+
.. code-block:: python
169+
170+
from python_proxy_headers.requests_adapter import ProxySession
171+
172+
with ProxySession(proxy_headers={'X-ProxyMesh-Country': 'US'}) as session:
173+
session.proxies = {
174+
'http': 'http://PROXYHOST:PORT',
175+
'https': 'http://PROXYHOST:PORT'
176+
}
177+
r = session.get('https://api.example.com')
178+
proxy_ip = r.headers.get('X-ProxyMesh-IP')
179+
180+
**Constructor Parameters:**
181+
182+
* ``proxy_headers`` (dict, optional): A dictionary of custom headers to send to the proxy server.
183+
184+
The ``ProxySession`` automatically mounts ``HTTPProxyHeaderAdapter`` instances for both ``http://`` and ``https://`` URL schemes, so all requests through the session will use proxy header support.
185+
186+
Helper Functions
187+
----------------
188+
189+
The module provides convenience functions that mirror the standard ``requests`` API:
190+
191+
* ``request(method, url, proxy_headers=None, **kwargs)`` - Make a request with the specified method
192+
* ``get(url, proxy_headers=None, **kwargs)`` - Make a GET request
193+
* ``post(url, proxy_headers=None, **kwargs)`` - Make a POST request
194+
* ``put(url, proxy_headers=None, **kwargs)`` - Make a PUT request
195+
* ``patch(url, proxy_headers=None, **kwargs)`` - Make a PATCH request
196+
* ``delete(url, proxy_headers=None, **kwargs)`` - Make a DELETE request
197+
* ``head(url, proxy_headers=None, **kwargs)`` - Make a HEAD request
198+
* ``options(url, proxy_headers=None, **kwargs)`` - Make an OPTIONS request
199+
200+
Each function creates a temporary ``ProxySession`` with the specified ``proxy_headers`` and makes the request. All standard requests parameters (``params``, ``data``, ``json``, ``headers``, ``cookies``, ``auth``, ``proxies``, etc.) are supported.
125201

0 commit comments

Comments
 (0)