Skip to content

Commit b81109a

Browse files
proxymeshproxymesh
authored
Merge pull request #2 from proxymesh/docs/update-docs
Add extension class documentation for aiohttp and requests modules
2 parents e70bf3d + 4c12e0d commit b81109a

2 files changed

Lines changed: 155 additions & 7 deletions

File tree

‎docs/aiohttp.rst‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,3 +101,75 @@ The ``ProxyClientSession`` works just like the standard ``ClientSession`` and su
101101

102102
All standard aiohttp request methods are supported: ``get``, ``post``, ``put``, ``delete``, ``patch``, ``head``, and ``options``.
103103

104+
Extension Classes
105+
-----------------
106+
107+
The ``python_proxy_headers.aiohttp_proxy`` module provides several extension classes that work together to capture and expose proxy response headers. These classes extend aiohttp's internal classes.
108+
109+
ProxyClientSession
110+
~~~~~~~~~~~~~~~~~~
111+
112+
The main entry point for using proxy headers with aiohttp. This class extends ``aiohttp.ClientSession`` and automatically configures the session to use the other extension classes.
113+
114+
.. code-block:: python
115+
116+
from python_proxy_headers.aiohttp_proxy import ProxyClientSession
117+
118+
async with ProxyClientSession() as session:
119+
async with session.get('https://example.com', proxy="http://PROXYHOST:PORT") as r:
120+
proxy_ip = r.headers.get('X-ProxyMesh-IP')
121+
122+
The ``ProxyClientSession`` constructor accepts all the same arguments as ``aiohttp.ClientSession``, and automatically sets:
123+
124+
* ``connector`` to ``ProxyTCPConnector()``
125+
* ``response_class`` to ``ProxyClientResponse``
126+
* ``request_class`` to ``ProxyClientRequest``
127+
128+
ProxyTCPConnector
129+
~~~~~~~~~~~~~~~~~
130+
131+
Extends ``aiohttp.TCPConnector`` to capture proxy response headers during HTTPS tunnel establishment. This class overrides the ``_create_proxy_connection`` method to:
132+
133+
1. Send the CONNECT request with custom proxy headers
134+
2. Capture the proxy's response headers from the CONNECT response
135+
3. Store them on the protocol object for later retrieval
136+
137+
When establishing an HTTPS connection through a proxy, the connector:
138+
139+
* Creates a CONNECT request to the proxy server
140+
* Includes any custom proxy headers you've specified
141+
* Captures the proxy's response headers (e.g., ``X-ProxyMesh-IP``)
142+
* Stores them so they can be merged into the final response
143+
144+
You typically don't need to use this class directly - it's automatically configured when using ``ProxyClientSession``.
145+
146+
ProxyClientRequest
147+
~~~~~~~~~~~~~~~~~~
148+
149+
Extends ``aiohttp.ClientRequest`` to transfer proxy headers from the connection protocol to the response object. This class overrides the ``send`` method to check if the connection's protocol has captured proxy headers and attaches them to the response.
150+
151+
This class is used internally by ``ProxyClientSession`` and typically doesn't need to be used directly.
152+
153+
ProxyClientResponse
154+
~~~~~~~~~~~~~~~~~~~
155+
156+
Extends ``aiohttp.ClientResponse`` to merge proxy response headers into the response's headers property. This class overrides the ``headers`` property to:
157+
158+
1. Check if proxy headers were captured during tunnel establishment
159+
2. If present, merge them with the target server's response headers
160+
3. Return a combined ``CIMultiDictProxy`` containing both sets of headers
161+
162+
This allows you to access proxy response headers (like ``X-ProxyMesh-IP``) directly from the response object's ``headers`` property, alongside the target server's response headers.
163+
164+
How It Works
165+
~~~~~~~~~~~~
166+
167+
The extension classes work together in the following flow:
168+
169+
1. **ProxyClientSession** creates a session configured with all the extension classes
170+
2. **ProxyTCPConnector** intercepts the CONNECT request/response during tunnel establishment and captures proxy headers
171+
3. **ProxyClientRequest** transfers the captured headers from the protocol to the response object
172+
4. **ProxyClientResponse** merges the proxy headers into the response's ``headers`` property
173+
174+
This allows proxy response headers to be transparently available in your application without any special handling.
175+

‎docs/requests.rst‎

Lines changed: 83 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -108,18 +108,94 @@ The exact headers available depend on your proxy provider. Check your proxy prov
108108
Session Support
109109
---------------
110110

111-
You can also use the adapter with requests Sessions for better connection pooling and cookie handling:
111+
For better connection pooling and cookie handling, you can use ``ProxySession``:
112112

113113
.. code-block:: python
114114
115-
from python_proxy_headers import requests_adapter
115+
from python_proxy_headers.requests_adapter import ProxySession
116+
117+
with ProxySession(proxy_headers={'X-ProxyMesh-Country': 'US'}) as session:
118+
r = session.get('https://api.example.com', proxies=proxies)
119+
proxy_ip = r.headers.get('X-ProxyMesh-IP')
120+
121+
Or you can manually mount the adapter to a standard requests Session:
122+
123+
.. code-block:: python
124+
125+
from python_proxy_headers.requests_adapter import HTTPProxyHeaderAdapter
116126
import requests
117127
118128
session = requests.Session()
119-
session.mount('http://', requests_adapter.HTTPAdapter())
120-
session.mount('https://', requests_adapter.HTTPAdapter())
129+
session.mount('http://', HTTPProxyHeaderAdapter(proxy_headers={'X-ProxyMesh-Country': 'US'}))
130+
session.mount('https://', HTTPProxyHeaderAdapter(proxy_headers={'X-ProxyMesh-Country': 'US'}))
121131
122-
r = session.get('https://api.example.com',
123-
proxies=proxies,
124-
proxy_headers={'X-ProxyMesh-Country': 'US'})
132+
r = session.get('https://api.example.com', proxies=proxies)
133+
proxy_ip = r.headers.get('X-ProxyMesh-IP')
134+
135+
Extension Classes
136+
-----------------
137+
138+
The ``python_proxy_headers.requests_adapter`` module provides extension classes that build on top of the ``urllib3_proxy_manager`` module to integrate proxy header support with the requests library.
139+
140+
HTTPProxyHeaderAdapter
141+
~~~~~~~~~~~~~~~~~~~~~~
142+
143+
Extends ``requests.adapters.HTTPAdapter`` to use our custom ``ProxyHeaderManager`` for proxy connections. This adapter enables both sending custom proxy headers and receiving proxy response headers.
144+
145+
.. code-block:: python
146+
147+
from python_proxy_headers.requests_adapter import HTTPProxyHeaderAdapter
148+
149+
adapter = HTTPProxyHeaderAdapter(proxy_headers={'X-ProxyMesh-Country': 'US'})
150+
151+
**Constructor Parameters:**
152+
153+
* ``proxy_headers`` (dict, optional): A dictionary of custom headers to send to the proxy server. These headers will be included in the CONNECT request when establishing HTTPS tunnel connections.
154+
155+
The adapter overrides the ``proxy_manager_for`` method to:
156+
157+
1. Check if the proxy URL is already cached in ``self.proxy_manager``
158+
2. For SOCKS proxies, delegate to the parent class
159+
3. For HTTP/HTTPS proxies, create a ``ProxyHeaderManager`` from the ``urllib3_proxy_manager`` module with the custom proxy headers
160+
161+
This ensures that all proxy connections made through the adapter will include your custom headers and capture proxy response headers.
162+
163+
ProxySession
164+
~~~~~~~~~~~~
165+
166+
Extends ``requests.Session`` with pre-configured ``HTTPProxyHeaderAdapter`` instances for both HTTP and HTTPS connections. This provides a convenient way to create a session that automatically handles proxy headers.
167+
168+
.. code-block:: python
169+
170+
from python_proxy_headers.requests_adapter import ProxySession
171+
172+
with ProxySession(proxy_headers={'X-ProxyMesh-Country': 'US'}) as session:
173+
session.proxies = {
174+
'http': 'http://PROXYHOST:PORT',
175+
'https': 'http://PROXYHOST:PORT'
176+
}
177+
r = session.get('https://api.example.com')
178+
proxy_ip = r.headers.get('X-ProxyMesh-IP')
179+
180+
**Constructor Parameters:**
181+
182+
* ``proxy_headers`` (dict, optional): A dictionary of custom headers to send to the proxy server.
183+
184+
The ``ProxySession`` automatically mounts ``HTTPProxyHeaderAdapter`` instances for both ``http://`` and ``https://`` URL schemes, so all requests through the session will use proxy header support.
185+
186+
Helper Functions
187+
----------------
188+
189+
The module provides convenience functions that mirror the standard ``requests`` API:
190+
191+
* ``request(method, url, proxy_headers=None, **kwargs)`` - Make a request with the specified method
192+
* ``get(url, proxy_headers=None, **kwargs)`` - Make a GET request
193+
* ``post(url, proxy_headers=None, **kwargs)`` - Make a POST request
194+
* ``put(url, proxy_headers=None, **kwargs)`` - Make a PUT request
195+
* ``patch(url, proxy_headers=None, **kwargs)`` - Make a PATCH request
196+
* ``delete(url, proxy_headers=None, **kwargs)`` - Make a DELETE request
197+
* ``head(url, proxy_headers=None, **kwargs)`` - Make a HEAD request
198+
* ``options(url, proxy_headers=None, **kwargs)`` - Make an OPTIONS request
199+
200+
Each function creates a temporary ``ProxySession`` with the specified ``proxy_headers`` and makes the request. All standard requests parameters (``params``, ``data``, ``json``, ``headers``, ``cookies``, ``auth``, ``proxies``, etc.) are supported.
125201

0 commit comments

Comments
 (0)