Web Data Gathering: How Modern Teams Collect Public Market Insights
Discover how companies use residential proxies to gather public data, analyze market trends, and conduct business research smoothly and reliably.
A simple, step-by-step developer guide on configuring rotating and authenticated proxies in Playwright, Selenium, and Python Requests with clean, copy-paste code examples.
Setting up proxies in your automation scripts is straightforward. Whether you need a fresh rotating IP for every web request or a sticky IP that holds the same session, this guide shows you the exact code required for three popular tools: Python Requests, Playwright, and Selenium.
ITN PROXY provides single-gateway endpoints that handle rotation and geo-targeting directly through your authentication string:
resi.itnproxy.com80801080customer_id (gives a fresh IP on every connection)customer_id-country-us (routes through the selected country)customer_id-session-mysession123-sessTime-30 (holds the same IP for up to 30 minutes)Requests is the most widely used HTTP library in Python. You can pass your proxy URL directly into the proxies dictionary.
import requests
# 1. Define proxy host, port, and credentials
PROXY_HOST = "resi.itnproxy.com"
PROXY_PORT = "8080"
USERNAME = "customer_id-country-us"
PASSWORD = "password_secret"
# 2. Build the proxy URL dictionary
proxies = {
"http": f"http://{USERNAME}:{PASSWORD}@{PROXY_HOST}:{PROXY_PORT}",
"https": f"http://{USERNAME}:{PASSWORD}@{PROXY_HOST}:{PROXY_PORT}",
}
# 3. Send a test request
try:
response = requests.get("https://ipinfo.io/json", proxies=proxies, timeout=15)
print("Response Status:", response.status_code)
print("Current Proxy IP:", response.json())
except requests.exceptions.RequestException as e:
print("Connection error:", e)Playwright has native, built-in proxy support with automatic authentication at both the browser launch and context levels.
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
# Configure proxy credentials in the browser context
context = browser.new_context(
proxy={
"server": "http://resi.itnproxy.com:8080",
"username": "customer_id-country-us",
"password": "password_secret",
}
)
page = context.new_page()
page.goto("https://ipinfo.io/json")
print("Playwright Output:", page.inner_text("body"))
browser.close()const { chromium } = require('playwright');
(async () => {
const browser = await chromium.launch({ headless: true });
// Set up proxy configuration
const context = await browser.newContext({
proxy: {
server: 'http://resi.itnproxy.com:8080',
username: 'customer_id-country-us',
password: 'password_secret',
},
});
const page = await context.newPage();
await page.goto('https://ipinfo.io/json');
const bodyText = await page.textContent('body');
console.log('Playwright IP Data:', bodyText);
await browser.close();
})();In standard Selenium, pass the proxy server directly using the --proxy-server argument in ChromeOptions.
pip install selenium webdriver-managerfrom selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
# 1. Configure standard Chrome options with proxy server
options = Options()
options.add_argument("--proxy-server=http://resi.itnproxy.com:8080")
options.add_argument("--headless=new")
# 2. Launch standard Selenium Chrome Driver
driver = webdriver.Chrome(
service=Service(ChromeDriverManager().install()),
options=options
)
try:
driver.get("https://ipinfo.io/json")
print("Selenium Output:", driver.find_element("tag name", "body").text)
finally:
driver.quit()--proxy-server Chrome options.Senior Network Infrastructure Engineer
Specializing in distributed network architectures, high-throughput edge routing, cloud infrastructure, and large-scale data systems.
Instant access to 100M+ real peer residential IPs across 190+ countries with city/ASN targeting and unlimited concurrency.
Continue exploring proxies, anti-bot strategies, and web scraping architectures.
Discover how companies use residential proxies to gather public data, analyze market trends, and conduct business research smoothly and reliably.
A friendly, comprehensive look at how residential proxy networks work, how requests are routed across global ISPs, and how teams choose between rotating and sticky sessions.
Learn how digital marketing agencies and brands verify localized advertisements, check creative layouts, and ensure brand safety across global regions.