|
| 1 | +# python-proxy-headers Outreach Drafts |
| 2 | + |
| 3 | +This document contains draft text for reaching out to library maintainers to add links to python-proxy-headers. |
| 4 | + |
| 5 | +--- |
| 6 | + |
| 7 | +## 1. HTTPX (PR Ready) |
| 8 | + |
| 9 | +**Status:** Branch created, ready to submit PR |
| 10 | +**Fork:** https://github.com/proxymesh/httpx |
| 11 | +**Branch:** `docs/add-python-proxy-headers` |
| 12 | +**Target:** https://github.com/encode/httpx |
| 13 | + |
| 14 | +### PR Title |
| 15 | +``` |
| 16 | +Add python-proxy-headers to Third Party Packages |
| 17 | +``` |
| 18 | + |
| 19 | +### PR Description |
| 20 | +``` |
| 21 | +## Summary |
| 22 | +
|
| 23 | +Add python-proxy-headers to the Third Party Packages documentation. |
| 24 | +
|
| 25 | +python-proxy-headers is a library that enables: |
| 26 | +- Sending custom headers to proxy servers during HTTPS CONNECT requests |
| 27 | +- Receiving and accessing proxy response headers from the CONNECT response |
| 28 | +
|
| 29 | +This is useful for proxy services like ProxyMesh that use custom headers for country selection (X-ProxyMesh-Country) and provide metadata like assigned IP addresses (X-ProxyMesh-IP). |
| 30 | +
|
| 31 | +- GitHub: https://github.com/proxymesh/python-proxy-headers |
| 32 | +- PyPI: https://pypi.org/project/python-proxy-headers/ |
| 33 | +- Docs: https://python-proxy-headers.readthedocs.io/ |
| 34 | +``` |
| 35 | + |
| 36 | +### To Submit PR |
| 37 | +```bash |
| 38 | +cd /home/jacob/proxymesh-httpx |
| 39 | +gh pr create --repo encode/httpx --title "Add python-proxy-headers to Third Party Packages" --body "## Summary |
| 40 | +
|
| 41 | +Add python-proxy-headers to the Third Party Packages documentation. |
| 42 | +
|
| 43 | +python-proxy-headers is a library that enables: |
| 44 | +- Sending custom headers to proxy servers during HTTPS CONNECT requests |
| 45 | +- Receiving and accessing proxy response headers from the CONNECT response |
| 46 | +
|
| 47 | +This is useful for proxy services like ProxyMesh that use custom headers for country selection (X-ProxyMesh-Country) and provide metadata like assigned IP addresses (X-ProxyMesh-IP). |
| 48 | +
|
| 49 | +- GitHub: https://github.com/proxymesh/python-proxy-headers |
| 50 | +- PyPI: https://pypi.org/project/python-proxy-headers/ |
| 51 | +- Docs: https://python-proxy-headers.readthedocs.io/" |
| 52 | +``` |
| 53 | + |
| 54 | +--- |
| 55 | + |
| 56 | +## 2. Requests (Manual Fork Required) |
| 57 | + |
| 58 | +**Status:** Fork failed due to token permissions - need to fork manually |
| 59 | +**Target:** https://github.com/psf/requests |
| 60 | +**File to edit:** `docs/community/recommended.rst` |
| 61 | + |
| 62 | +### Steps |
| 63 | +1. Go to https://github.com/psf/requests and click "Fork" |
| 64 | +2. Clone your fork: |
| 65 | + ```bash |
| 66 | + git clone https://github.com/proxymesh/requests.git |
| 67 | + cd requests |
| 68 | + git config user.email "cursor@proxymesh.com" |
| 69 | + git config user.name "Cursor" |
| 70 | + ``` |
| 71 | +3. Create branch: |
| 72 | + ```bash |
| 73 | + git checkout -b docs/add-python-proxy-headers |
| 74 | + ``` |
| 75 | +4. Edit `docs/community/recommended.rst` and add before the last entry: |
| 76 | + ```rst |
| 77 | + python-proxy-headers |
| 78 | + --------------------- |
| 79 | +
|
| 80 | + `python-proxy-headers <https://github.com/proxymesh/python-proxy-headers>`_ |
| 81 | + enables sending custom headers to proxy servers and receiving proxy response |
| 82 | + headers from HTTPS CONNECT requests. This is useful for proxy services that |
| 83 | + use custom headers for features like country selection or session management. |
| 84 | +
|
| 85 | + .. code-block:: python |
| 86 | +
|
| 87 | + from python_proxy_headers import requests_adapter |
| 88 | + |
| 89 | + response = requests_adapter.get( |
| 90 | + 'https://api.ipify.org', |
| 91 | + proxies={'https': 'http://proxy:8080'}, |
| 92 | + proxy_headers={'X-ProxyMesh-Country': 'US'} |
| 93 | + ) |
| 94 | + print(response.headers.get('X-ProxyMesh-IP')) |
| 95 | + ``` |
| 96 | +5. Commit and push: |
| 97 | + ```bash |
| 98 | + git add docs/community/recommended.rst |
| 99 | + git commit -m "Add python-proxy-headers to Recommended Packages" |
| 100 | + git push -u origin docs/add-python-proxy-headers |
| 101 | + ``` |
| 102 | +6. Create PR to psf/requests |
| 103 | + |
| 104 | +### PR Title |
| 105 | +``` |
| 106 | +Add python-proxy-headers to Recommended Packages |
| 107 | +``` |
| 108 | + |
| 109 | +### PR Description |
| 110 | +``` |
| 111 | +## Summary |
| 112 | +
|
| 113 | +Add python-proxy-headers to the Recommended Packages documentation. |
| 114 | +
|
| 115 | +python-proxy-headers enables sending custom headers to proxy servers and receiving |
| 116 | +proxy response headers from HTTPS CONNECT requests. This fills a gap in the requests |
| 117 | +library where proxy headers from the CONNECT response are not normally accessible. |
| 118 | +
|
| 119 | +Use cases: |
| 120 | +- Proxy services that use custom headers for country/region selection |
| 121 | +- Accessing metadata from proxy servers (e.g., assigned IP address) |
| 122 | +- Session management with rotating proxies |
| 123 | +
|
| 124 | +- GitHub: https://github.com/proxymesh/python-proxy-headers |
| 125 | +- PyPI: https://pypi.org/project/python-proxy-headers/ |
| 126 | +- Docs: https://python-proxy-headers.readthedocs.io/ |
| 127 | +``` |
| 128 | + |
| 129 | +--- |
| 130 | + |
| 131 | +## 3. urllib3 (Discord/Discussion) |
| 132 | + |
| 133 | +**Status:** No third-party page exists - contact maintainers |
| 134 | +**Discord:** https://discord.gg/urllib3 |
| 135 | +**Discussions:** https://github.com/urllib3/urllib3/discussions |
| 136 | + |
| 137 | +### Discord Message |
| 138 | +``` |
| 139 | +Hi! I maintain python-proxy-headers, a library that extends urllib3 (and other |
| 140 | +HTTP libraries) to support: |
| 141 | +- Sending custom headers to proxy servers during HTTPS CONNECT |
| 142 | +- Receiving and accessing proxy response headers from the CONNECT response |
| 143 | +
|
| 144 | +This is useful for proxy services like ProxyMesh that communicate via custom |
| 145 | +headers (e.g., X-ProxyMesh-Country for country selection, X-ProxyMesh-IP for |
| 146 | +the assigned IP). |
| 147 | +
|
| 148 | +GitHub: https://github.com/proxymesh/python-proxy-headers |
| 149 | +Docs: https://python-proxy-headers.readthedocs.io/ |
| 150 | +
|
| 151 | +Would you be interested in linking to this from the urllib3 docs or README? |
| 152 | +I noticed there's a "Who uses urllib3?" section - perhaps a section for |
| 153 | +extensions/ecosystem projects would be valuable? |
| 154 | +``` |
| 155 | + |
| 156 | +### GitHub Discussion Title |
| 157 | +``` |
| 158 | +Request: Add ecosystem/extensions section to documentation |
| 159 | +``` |
| 160 | + |
| 161 | +### GitHub Discussion Body |
| 162 | +``` |
| 163 | +Hi urllib3 maintainers! |
| 164 | +
|
| 165 | +I maintain python-proxy-headers, a library that extends urllib3 to support |
| 166 | +sending custom headers to proxy servers and receiving proxy response headers |
| 167 | +from HTTPS CONNECT requests. |
| 168 | +
|
| 169 | +This functionality isn't available in standard urllib3 because the CONNECT |
| 170 | +response headers are not exposed to the caller. |
| 171 | +
|
| 172 | +**Use cases:** |
| 173 | +- Proxy services that use custom headers for country/region selection |
| 174 | +- Accessing metadata from proxy servers (e.g., assigned IP address) |
| 175 | +- Session management with rotating proxies |
| 176 | +
|
| 177 | +**Links:** |
| 178 | +- GitHub: https://github.com/proxymesh/python-proxy-headers |
| 179 | +- PyPI: https://pypi.org/project/python-proxy-headers/ |
| 180 | +- Docs: https://python-proxy-headers.readthedocs.io/ |
| 181 | +
|
| 182 | +**Request:** |
| 183 | +Would you consider: |
| 184 | +1. Adding a link in the README or docs? |
| 185 | +2. Creating an "Ecosystem" or "Extensions" section for third-party projects? |
| 186 | +
|
| 187 | +I'm happy to submit a PR if you point me to the right location. |
| 188 | +
|
| 189 | +Thanks for maintaining such a foundational library! |
| 190 | +``` |
| 191 | + |
| 192 | +--- |
| 193 | + |
| 194 | +## 4. pycurl (Mailing List) |
| 195 | + |
| 196 | +**Status:** No third-party page - contact via mailing list |
| 197 | +**Mailing List:** https://lists.haxx.se/listinfo/curl-and-python |
| 198 | +**Archive:** https://curl.haxx.se/mail/list.cgi?list=curl-and-python |
| 199 | + |
| 200 | +### Email Subject |
| 201 | +``` |
| 202 | +[ANN] python-proxy-headers: Proxy header support for pycurl |
| 203 | +``` |
| 204 | + |
| 205 | +### Email Body |
| 206 | +``` |
| 207 | +Hi curl-and-python community, |
| 208 | +
|
| 209 | +I'd like to announce python-proxy-headers, a library that provides convenient |
| 210 | +proxy header support for pycurl (and other Python HTTP libraries). |
| 211 | +
|
| 212 | +The library provides: |
| 213 | +- set_proxy_headers(): Helper to set CURLOPT_PROXYHEADER and CURLOPT_HEADEROPT |
| 214 | +- HeaderCapture: Class to capture and parse headers from both proxy CONNECT |
| 215 | + response and origin server response |
| 216 | +- High-level convenience functions (get, post, etc.) for simple use cases |
| 217 | +
|
| 218 | +Example usage with existing pycurl code: |
| 219 | +
|
| 220 | + import pycurl |
| 221 | + from python_proxy_headers.pycurl_proxy import set_proxy_headers, HeaderCapture |
| 222 | +
|
| 223 | + c = pycurl.Curl() |
| 224 | + c.setopt(pycurl.URL, 'https://httpbin.org/ip') |
| 225 | + c.setopt(pycurl.PROXY, 'http://proxy:8080') |
| 226 | +
|
| 227 | + # Add custom headers to send to the proxy |
| 228 | + set_proxy_headers(c, {'X-ProxyMesh-Country': 'US'}) |
| 229 | +
|
| 230 | + # Capture response headers |
| 231 | + capture = HeaderCapture(c) |
| 232 | +
|
| 233 | + c.perform() |
| 234 | +
|
| 235 | + # Access proxy CONNECT response headers |
| 236 | + print(capture.proxy_headers) # {'X-ProxyMesh-IP': '1.2.3.4', ...} |
| 237 | + c.close() |
| 238 | +
|
| 239 | +Links: |
| 240 | +- GitHub: https://github.com/proxymesh/python-proxy-headers |
| 241 | +- PyPI: https://pypi.org/project/python-proxy-headers/ |
| 242 | +- Docs: https://python-proxy-headers.readthedocs.io/en/latest/pycurl.html |
| 243 | +
|
| 244 | +Would the maintainers consider linking to this from the pycurl documentation |
| 245 | +or examples directory? |
| 246 | +
|
| 247 | +Best regards, |
| 248 | +ProxyMesh Team |
| 249 | +``` |
| 250 | + |
| 251 | +--- |
| 252 | + |
| 253 | +## 5. CloudScraper (GitHub Issue) |
| 254 | + |
| 255 | +**Status:** Contact maintainer via GitHub issue |
| 256 | +**URL:** https://github.com/VeNoMouS/cloudscraper/issues/new |
| 257 | + |
| 258 | +### Issue Title |
| 259 | +``` |
| 260 | +Request: Add python-proxy-headers to README/documentation |
| 261 | +``` |
| 262 | + |
| 263 | +### Issue Body |
| 264 | +``` |
| 265 | +Hi @VeNoMouS, |
| 266 | +
|
| 267 | +I maintain python-proxy-headers, which provides proxy header support for |
| 268 | +cloudscraper (and other Python HTTP libraries). |
| 269 | +
|
| 270 | +## What it does |
| 271 | +
|
| 272 | +When using cloudscraper with HTTPS through a proxy, the library enables: |
| 273 | +- Sending custom headers to the proxy server (e.g., `X-ProxyMesh-Country`) |
| 274 | +- Receiving proxy response headers from the CONNECT response (e.g., `X-ProxyMesh-IP`) |
| 275 | +
|
| 276 | +This is useful for proxy services like ProxyMesh that use custom headers for |
| 277 | +country selection and provide metadata about the connection. |
| 278 | +
|
| 279 | +## Usage |
| 280 | +
|
| 281 | +```python |
| 282 | +from python_proxy_headers.cloudscraper_proxy import create_scraper |
| 283 | +
|
| 284 | +# Drop-in replacement for cloudscraper.create_scraper() |
| 285 | +scraper = create_scraper( |
| 286 | + proxy_headers={'X-ProxyMesh-Country': 'US'}, |
| 287 | + browser='chrome' |
| 288 | +) |
| 289 | +
|
| 290 | +scraper.proxies = {'https': 'http://proxy:8080'} |
| 291 | +response = scraper.get('https://example.com') |
| 292 | +
|
| 293 | +# Access proxy response headers |
| 294 | +print(response.headers.get('X-ProxyMesh-IP')) |
| 295 | +``` |
| 296 | + |
| 297 | +All cloudscraper features (Cloudflare bypass, stealth mode, etc.) are preserved. |
| 298 | + |
| 299 | +## Links |
| 300 | + |
| 301 | +- GitHub: https://github.com/proxymesh/python-proxy-headers |
| 302 | +- PyPI: https://pypi.org/project/python-proxy-headers/ |
| 303 | +- Docs: https://python-proxy-headers.readthedocs.io/en/latest/cloudscraper.html |
| 304 | + |
| 305 | +## Request |
| 306 | + |
| 307 | +Would you consider adding a link to python-proxy-headers in the README or |
| 308 | +creating an "Integrations" section? Happy to submit a PR if you prefer. |
| 309 | + |
| 310 | +Thanks for maintaining cloudscraper! |
| 311 | +``` |
| 312 | +
|
| 313 | +--- |
| 314 | +
|
| 315 | +## 6. AutoScraper (GitHub Issue) |
| 316 | +
|
| 317 | +**Status:** Contact maintainer via GitHub issue |
| 318 | +**URL:** https://github.com/alirezamika/autoscraper/issues/new |
| 319 | +
|
| 320 | +### Issue Title |
| 321 | +``` |
| 322 | +Request: Add python-proxy-headers integration to README |
| 323 | +``` |
| 324 | +
|
| 325 | +### Issue Body |
| 326 | +``` |
| 327 | +Hi @alirezamika, |
| 328 | + |
| 329 | +I maintain python-proxy-headers, which provides proxy header support for |
| 330 | +autoscraper (and other Python HTTP libraries). |
| 331 | + |
| 332 | +## What it does |
| 333 | + |
| 334 | +When using autoscraper with proxies, the library enables: |
| 335 | +- Sending custom headers to the proxy server during requests |
| 336 | +- Useful for proxy services that support custom headers for features like |
| 337 | + country selection or session management |
| 338 | + |
| 339 | +## Usage |
| 340 | + |
| 341 | +```python |
| 342 | +from python_proxy_headers.autoscraper_proxy import ProxyAutoScraper |
| 343 | + |
| 344 | +# Drop-in replacement for AutoScraper |
| 345 | +scraper = ProxyAutoScraper(proxy_headers={'X-ProxyMesh-Country': 'US'}) |
| 346 | + |
| 347 | +result = scraper.build( |
| 348 | + url='https://example.com', |
| 349 | + wanted_list=['Example text'], |
| 350 | + request_args={'proxies': {'https': 'http://proxy:8080'}} |
| 351 | +) |
| 352 | +``` |
| 353 | + |
| 354 | +All autoscraper features (rule learning, saving/loading, etc.) are preserved. |
| 355 | + |
| 356 | +## Links |
| 357 | + |
| 358 | +- GitHub: https://github.com/proxymesh/python-proxy-headers |
| 359 | +- PyPI: https://pypi.org/project/python-proxy-headers/ |
| 360 | +- Docs: https://python-proxy-headers.readthedocs.io/en/latest/autoscraper.html |
| 361 | + |
| 362 | +## Request |
| 363 | + |
| 364 | +Would you consider mentioning python-proxy-headers in the README, perhaps in |
| 365 | +the proxy usage section? Happy to submit a PR if you prefer. |
| 366 | + |
| 367 | +Thanks for creating autoscraper! |
| 368 | +``` |
| 369 | +
|
| 370 | +--- |
| 371 | +
|
| 372 | +## Summary |
| 373 | +
|
| 374 | +| Library | Method | Status | |
| 375 | +|---------|--------|--------| |
| 376 | +| httpx | PR | Branch ready at `proxymesh/httpx:docs/add-python-proxy-headers` | |
| 377 | +| requests | PR | Need to manually fork first | |
| 378 | +| urllib3 | Discord/Discussion | Draft message ready | |
| 379 | +| pycurl | Mailing list | Draft email ready | |
| 380 | +| cloudscraper | GitHub Issue | Draft issue ready | |
| 381 | +| autoscraper | GitHub Issue | Draft issue ready | |
0 commit comments