ProxyBroker2

🚀 Production-Ready Async Proxy Management

The modernized, actively-maintained successor to ProxyBroker (v0.3.2, abandoned 2018) — Python 3.10-3.14, comprehensive test coverage, first-class IPv6.

ProxyBroker2 is an open source tool that asynchronously finds public proxies from 50+ sources, validates them against judge servers, and can operate as a rotating proxy server.

Features

Finds more than 7000 working proxies from ~50 sources.
Support protocols: HTTP(S), SOCKS4/5. Also CONNECT method to ports 80 and 23 (SMTP).
Proxies may be filtered by type, anonymity level, response time, country and status in DNSBL.
Work as a proxy server that distributes incoming requests to external proxies. With automatic proxy rotation.
All proxies are checked to support Cookies and Referer (and POST requests if required).
Automatically removes duplicate proxies.
Is asynchronous.

What's New in ProxyBroker2

ProxyBroker2 is the actively-maintained successor to the original ProxyBroker (v0.3.2), which was abandoned in 2018. We've modernized the stack and added significant new capabilities while keeping the Python API compatible.

🚀 What you get over v0.3.2

Active maintenance — regular security updates and feature work
Modern Python support — Python 3.10-3.14 (v0.3.2 was Python 3.5+ only)
Zero critical bugs — fixed deadlocks, memory leaks, signal handler leaks, race conditions that existed in v0.3.2
First-class IPv6 — RFC 5952 canonical form, SOCKS5 ATYP=0x04, bracketed URI authority. v0.3.2 was IPv4-only.
Custom providers — drop YAML/JSON config files into a folder, no code required (see Custom Proxy Providers below)
Modern async patterns — updated from deprecated asyncio APIs; uses asyncio.run, get_running_loop, etc.
Comprehensive testing — contract-based test suite that protects user-visible behavior across refactors (v0.3.2 had minimal testing)
Better documentation — auto-generated API reference + hand-written guides on ReadTheDocs

🛠️ Core capabilities

Protocols: HTTP, HTTPS, SOCKS4, SOCKS5, CONNECT:80, CONNECT:25
Anonymity detection: Transparent / Anonymous / High
50+ proxy sources discovered asynchronously
Proxy server mode with automatic rotation and failover
Smart filtering by country, protocol, anonymity level, response time, DNSBL

Docker

Docker Hub https://hub.docker.com/r/bluet/proxybroker2

# Pull the image
$ docker pull bluet/proxybroker2

# Run with --help to see all options
$ docker run --rm bluet/proxybroker2 --help

Requirements

Python 3.10-3.14

That's it — all Python dependencies are installed automatically by pip / Poetry / uv.

For building the standalone executable (see pyinstaller build), you'll also need upx and binutils.

Installation

⚠️ WARNING: Do not install from PyPI. Both proxybroker (the abandoned v0.3.2) and the proxybroker2 name on PyPI have been squatted by unrelated/unauthorized publishers (see #186, pypi/support#10315). Install only from this GitHub repository.

📦 Install ProxyBroker2

Install the latest version (via pip):

# Install latest from default branch
$ pip install -U git+https://github.com/bluet/proxybroker2.git

For a specific stable release, append @<tag> from the Releases page — e.g.:

$ pip install -U git+https://github.com/bluet/proxybroker2.git@<tag>

Or install with uv (supported as alternative)

uv reads the same git URL pip does — under the hood, uv invokes the project's build backend (poetry-core) to produce a wheel with full PEP 621 metadata:

# Add to a uv-managed project
uv add "proxybroker2 @ git+https://github.com/bluet/proxybroker2.git"

# Or install into the current venv
uv pip install git+https://github.com/bluet/proxybroker2.git

Use pre-built Docker image

$ docker pull bluet/proxybroker2

Build bundled one-file executable with pyinstaller

Requirements

Supported Operating System: Windows, Linux, MacOS

On UNIX-like systems (Linux / macOSX / BSD)

Install these tools

upx
objdump (this tool is usually in the binutils package)

$ sudo apt install -y upx-ucl binutils # On Ubuntu / Debian

Build

pip install pyinstaller \
&& pip install . \
&& mkdir -p build \
&& cd build \
&& pyinstaller --onefile --name proxybroker --add-data "../proxybroker/data:data" --workpath ./tmp --distpath . --clean ../py2exe_entrypoint.py \
&& rm -rf tmp *.spec

The executable is now in the build directory

Quick Start

After installation, you can immediately start finding proxies:

# Find 5 working HTTP proxies
$ python -m proxybroker find --types HTTP --limit 5

# Find 10 US HTTP proxies
$ python -m proxybroker find --types HTTP --countries US --limit 10

# Run local proxy server on port 8888
$ python -m proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS

Run python -m proxybroker --help for full options.

Usage

CLI Examples

Find

Find and show 10 HTTP(S) proxies from United States with the high level of anonymity:

$ python -m proxybroker find --types HTTP HTTPS --lvl High --countries US --strict -l 10

Grab

Find and save to a file 10 US proxies (without a check):

$ python -m proxybroker grab --countries US --limit 10 --outfile ./proxies.txt

Serve

Run a local proxy server that distributes incoming requests to a pool of found HTTP(S) proxies with the high level of anonymity:

$ python -m proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl High --min-queue 5

Run python -m proxybroker --help for more information on the options available. Run python -m proxybroker <command> --help for more information on a command.

Basic code example

Find and show 10 working HTTP(S) proxies:

import asyncio
from proxybroker import Broker

async def show(proxies):
    while True:
        proxy = await proxies.get()
        if proxy is None:
            break
        print("Found proxy: %s" % proxy)

async def main():
    proxies = asyncio.Queue()
    broker = Broker(proxies)
    await asyncio.gather(
        broker.find(types=["HTTP", "HTTPS"], limit=10),
        show(proxies)
    )

if __name__ == "__main__":
    asyncio.run(main())

More examples.

🆕 Custom Proxy Providers

Add your own proxy sources without modifying the codebase. Two paths:

Docker (no coding required)

Drop YAML/JSON config files into a folder, bind-mount it to /configs:

# ~/proxy-sources/my_list.yaml
name: My Internal Source
type: simple
url: https://my-server.example.com/proxies.txt
format: text
protocols: [HTTP, HTTPS]

docker run --rm \
  -v ~/proxy-sources:/configs \
  bluet/proxybroker2 \
  find --types HTTP --limit 10

The CLI also accepts --provider-dir PATH (repeatable) and reads $PROXYBROKER_PROVIDER_DIR if neither is set.

Python API

from proxybroker import SimpleProvider, Broker

class MyProvider(SimpleProvider):
    domain = "mysite.com"

    def __init__(self):
        super().__init__(
            url="http://mysite.com/proxies.txt",
            format='text',
            proto=('HTTP', 'HTTPS')
        )

broker = Broker(providers=[MyProvider()])

Full documentation on custom providers

🔬 Testing Philosophy

ProxyBroker2 implements a comprehensive contract-based testing strategy that ensures reliability while enabling innovation:

✅ What We Test (Stable Public Contracts)

User-visible behavior - "Does proxy finding work?" vs internal algorithms
API signatures - Method parameters and return types users depend on
Protocol support - HTTP, HTTPS, SOCKS4/5 compatibility
Error contracts - Exception types and error handling behavior

❌ What We Don't Test (Flexible Implementation)

Internal algorithms - Allow optimization without breaking tests
Exact protocol bytes - Enable protocol improvements and IPv6 support
Provider specifics - Adapt to website changes without test failures
Performance metrics - Implementation details that can evolve

This approach protects your code from breaking changes while allowing ProxyBroker2 to continuously improve its internals.

Proxy information per requests

HTTP

Check X-Proxy-Info header in response.

$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://httpbin.org/get
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET http://httpbin.org/get HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< X-Proxy-Info: 174.138.42.112:8080
< Date: Mon, 04 May 2020 03:39:40 GMT
< Content-Type: application/json
< Content-Length: 304
< Server: gunicorn/19.9.0
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
< X-Cache: MISS from ADM-MANAGER
< X-Cache-Lookup: MISS from ADM-MANAGER:880
< Connection: keep-alive
<
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Cache-Control": "max-age=259200",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.58.0",
    "X-Amzn-Trace-Id": "Root=1-5eaf8e7c-6a1162a1387a1743a49063f4"
  },
  "origin": "...",
  "url": "http://httpbin.org/get"
}
* Connection #0 to host 127.0.0.1 left intact

HTTPS

We are not able to modify HTTPS traffic to inject custom header once they start being encrypted. A X-Proxy-Info will be sent to client after HTTP/1.1 200 Connection established but not sure how clients can read it.

(env) bluet@ocisly:~/workspace/proxybroker2$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v https://httpbin.org/get
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to httpbin.org:443
> CONNECT httpbin.org:443 HTTP/1.1
> Host: httpbin.org:443
> User-Agent: curl/7.58.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
< X-Proxy-Info: 207.148.22.139:8080
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
...
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5560b2e93580)
> GET /get HTTP/2
> Host: httpbin.org
> User-Agent: curl/7.58.0
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200
< date: Mon, 04 May 2020 03:39:35 GMT
< content-type: application/json
< content-length: 256
< server: gunicorn/19.9.0
< access-control-allow-origin: *
< access-control-allow-credentials: true
<
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.58.0",
    "X-Amzn-Trace-Id": "Root=1-5eaf8e77-efcb353b0983ad6a90f8bdcd"
  },
  "origin": "...",
  "url": "https://httpbin.org/get"
}
* Connection #0 to host 127.0.0.1 left intact

HTTP API

Get info of proxy been used for retrieving specific url

For HTTP, it's easy.

$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://proxycontrol/api/history/url:http://httpbin.org/get
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET http://proxycontrol/api/history/url:http://httpbin.org/get HTTP/1.1
> Host: proxycontrol
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 34
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
{"proxy": "..."}

For HTTPS, we're not able to know encrypted payload (request), so only hostname can be used.

$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://proxycontrol/api/history/url:httpbin.org:443
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET http://proxycontrol/api/history/url:httpbin.org:443 HTTP/1.1
> Host: proxycontrol
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 34
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
<
{"proxy": "..."}
* Connection #0 to host 127.0.0.1 left intact

Remove specific proxy from queue

$ http_proxy=http://127.0.0.1:8888 https_proxy=http://127.0.0.1:8888 curl -v http://proxycontrol/api/remove/PROXY_IP:PROXY_PORT
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8888 (#0)
> GET http://proxycontrol/api/remove/... HTTP/1.1
> Host: proxycontrol
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 204 No Content
<
* Connection #0 to host 127.0.0.1 left intact

Migration from ProxyBroker v0.3.2

If you're upgrading from the original ProxyBroker v0.3.2, here are the key changes:

🚨 Breaking Changes

Python Version

# v0.3.2: Python 3.5.3+ supported
# v2.0.0+: Python 3.10+ required
python --version  # Must be 3.10+

Installation

# v0.3.2: pip install proxybroker
# v2.0.0+: Install from GitHub (original is abandoned, PyPI name is squatted)
pip uninstall proxybroker  # Remove old version if installed
pip install git+https://github.com/bluet/proxybroker2.git

CLI Usage

# v0.3.2: proxybroker find --types HTTP --limit 10
# v2.0.0+: python -m proxybroker find --types HTTP --limit 10

✅ API Compatibility

The Python API remains largely compatible:

# This code works in both versions
import asyncio
from proxybroker import Broker

async def main():
    proxies = asyncio.Queue()
    broker = Broker(proxies)
    await broker.find(types=['HTTP'], limit=10)

asyncio.run(main())  # Modern async pattern

For what ProxyBroker2 adds over v0.3.2, see What's New in ProxyBroker2 above.

Documentation

📚 Complete Documentation: https://proxybroker2.readthedocs.io/

Auto-generated API reference (from docstrings)
Hand-written guides: installation, tutorials, architecture
Hosted on ReadTheDocs in HTML / PDF / downloadable formats

TODO

See open issues for active work and ideas:

Authentication features (SOCKS5 auth, HTTP Basic auth)
TLS hardening (cert pinning, ECH, TLS 1.3-only)
All open issues

Contributing

We welcome contributions! The project has comprehensive test coverage and development tooling.

Development Setup

Fork it: https://github.com/bluet/proxybroker2/fork

Install Poetry if you don't have it (canonical dev tool):

curl -sSL https://install.python-poetry.org | python3 -

Clone and setup:
```
git clone https://github.com/yourusername/proxybroker2.git
cd proxybroker2
poetry install  # Installs runtime + dev deps (including ruff, pytest)
```
Alternative: use uv as a command runner

Poetry remains the canonical dev tool — poetry.lock is the lockfile CI tests against. This repo currently uses Poetry metadata ([tool.poetry]) without a [project] table, so uv sync is not supported here. If you prefer uv ergonomics, install deps with Poetry first and then point uv at Poetry's environment (plain uv run would look for a [project] table and use its own environment instead):
```
poetry install
poetry run pytest                                                   # canonical
uv run --no-project --python "$(poetry env info --executable)" pytest   # same env, uv as the runner
```
Full uv migration remains deferred until snyk supports uv.lock natively (snyk-python-plugin#251).

Development Workflow

Create your feature branch: git checkout -b my-new-feature

Make changes and format:

# Auto-format code (required before commit)
poetry run ruff check . --fix && poetry run ruff format .

# Run tests to ensure everything works
poetry run pytest tests/ -v

Commit with Conventional Commits format:

git commit -m "feat: add SOCKS5 authentication support"
git commit -m "fix: resolve memory leak in proxy pool"
git commit -m "docs: update installation instructions"

Push to the branch: git push origin my-new-feature
Submit a pull request!

What contributors handle manually

The only tools you need to install yourself — everything else (ruff, pytest, pytest-asyncio, etc.) comes from poetry install:

Poetry 2.x — canonical dependency manager (uv supported as alternative, see #105)
pre-commit (optional) — run pre-commit install after poetry install to auto-format/lint on every commit

For project architecture and design conventions, see CLAUDE.md.

License

Licensed under the Apache License, Version 2.0 — see LICENSE.

This product includes GeoLite2 data created by MaxMind, available from http://www.maxmind.com.

Refs

constverum#161 — original PR thread where ProxyBroker2 emerged from the abandoned upstream.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

_a5r0n 💻	_{C.M. Yang} 💻 🤔 👀	_{Ivan Villareal} 💻	_Quancore 💻	_Felipe 🤔	_{vincentinttsh} 💻 👀	_Ziloka 💻
_hms5232 💻	_{Stefan Machmeier} 💻	_steven 📖

This project follows the all-contributors specification. Contributions of any kind welcome!

Name		Name	Last commit message	Last commit date
.github		.github
.trunk		.trunk
.vscode		.vscode
docs		docs
examples		examples
proxybroker		proxybroker
tests		tests
.all-contributorsrc		.all-contributorsrc
.editorconfig		.editorconfig
.gitignore		.gitignore
.gitleaks.toml		.gitleaks.toml
.gitmessage		.gitmessage
.isort.cfg		.isort.cfg
.pre-commit-config.yaml		.pre-commit-config.yaml
.semgrepignore		.semgrepignore
.whitesource		.whitesource
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
Dockerfile		Dockerfile
LICENSE		LICENSE
MANIFEST.in		MANIFEST.in
README.md		README.md
docker-build.sh		docker-build.sh
environment.yml		environment.yml
poetry.lock		poetry.lock
py2exe_entrypoint.py		py2exe_entrypoint.py
pyproject.toml		pyproject.toml
readthedocs.yml		readthedocs.yml
setup.cfg		setup.cfg
setup.py		setup.py
sonar-project.properties		sonar-project.properties
sweep.yaml		sweep.yaml

Latest commit

History

Folders and files

Repository files navigation

ProxyBroker2

Features

What's New in ProxyBroker2

🚀 What you get over v0.3.2

🛠️ Core capabilities

Docker

Requirements

Installation

📦 Install ProxyBroker2

Use pre-built Docker image

Build bundled one-file executable with pyinstaller

Requirements

Build

Quick Start

Usage

CLI Examples

Find

Grab

Serve

Basic code example

🆕 Custom Proxy Providers

Docker (no coding required)

Python API

🔬 Testing Philosophy

✅ What We Test (Stable Public Contracts)

❌ What We Don't Test (Flexible Implementation)

Proxy information per requests

HTTP

HTTPS

HTTP API

Get info of proxy been used for retrieving specific url

Remove specific proxy from queue

Migration from ProxyBroker v0.3.2

🚨 Breaking Changes

✅ API Compatibility

Documentation

TODO

Contributing

Development Setup

Development Workflow

What contributors handle manually

License

Refs

Contributors ✨

Star History

About

Topics

Resources

Stars

Watchers

Forks

Releases

Sponsor this project

Packages

Used by

Contributors

Languages