跳到正文
Esc

文章

什么是代理?写给新手的大白话指南:工作原理、类型和用途什么是 4G 代理?移动代理如何工作、优缺点和适用场景Rotating proxies explained: rotation types and how to change a proxy IP with a linkSOCKS5 vs HTTP proxies: what's the difference and which should you use?Proxy vs VPN: what's the difference, and when should you use each?Static vs rotating proxies: how they differ, pros and cons, and when to use eachDedicated vs shared proxies: how they differ, and what "dedicated" means for a 4G proxyIPv4 vs IPv6 proxies: how they differ, why IPv6 proxies are cheap, and when they don't workHow do websites detect proxies? 8 signals anti-bot systems look atProxy glossary A to Z: 35 terms you'll meet when buying and using proxies, explained simplyProxy vs antidetect browser: how they differ, and do you need a proxy if you have antidetect?What is a residential proxy? How it works, pros and cons, and when to use oneWhat is a datacenter proxy? Pros and cons, why it's cheap, and when it's still the right choiceHow to find your IP address on a computer or phone: public IP, local IP, and your IP behind a proxy为什么总是出现验证码?9 个常见原因,以及你的 IP 起了什么作用reCAPTCHA、hCaptcha 与 Cloudflare Turnstile 对比:各自如何决定谁要验证什么是 botting?好机器人、坏机器人,以及你自己的自动化属于哪一类How to get a Singapore IP address: VPN, proxy, cloud server or SIM, and how to check itProxies for Facebook: managing several accounts more reliably with 4G IPsProxies for TikTok and TikTok Shop: when you need a Singapore IP and how to use it properlyProxies for advertising: checking and managing ads aimed at SingaporeProxies for e-commerce sellers: see marketplaces like a buyer, track competitors, keep store logins steadyProxies for web scraping: which type to choose, how to rotate, and how to play fairDo you need a proxy to buy tickets on ThaiTicketMajor, Ticketmelon or SISTIC?Scraping Google and Bing search results with a proxy, and when to use the API insteadHow to scrape without getting blocked: pace, sessions and clean IPsProxies for price monitoring: tracking competitor and retail prices across regionsScraping anti-bot and anti-proxy websites (Cloudflare, Akamai, DataDome): what works, and the limitsIs web scraping legal? Robots.txt, terms, personal data and copyright (Vietnam, Singapore and beyond)What are 4G proxies used for? 10 jobs mobile proxies do well, and the ones to avoid4G proxies for testing apps and websites as a Singapore mobile user: QA, ads, payments, bot rulesProxies for Instagram: managing brand and client accounts steadily with 4G IPsUsing a proxy to check flight and hotel prices as a Singapore user: when it helps and what to knowResearching the Singapore market with a proxy: see search, ads, marketplaces and competitors as a local userProxies for LinkedIn: managing B2B accounts and reaching the Singapore market the right wayProxies for X (Twitter): managing brand and client accounts steadily and within the rulesProxies for airdrops: why farming wallets through proxies gets disqualified, and how to join airdrops safelyProxies for seeding, bought likes and fake reviews: why they get caught, the risks, and lasting ways to growProxies for flash sales and limited drops: what a proxy can and can't do, and how to shop sales properlyProxies for large-scale, multi-country scraping: choosing proxy types, system design and real costsProxies for Netflix, YouTube and foreign streaming: why they usually fail, cost a lot, and what to do insteadSelling to Singapore from Vietnam: choosing channels, GST, shipping, payments and the tools you needSelling on TikTok Shop Singapore from abroad: seller requirements, workable routes and what a proxy can doThe Singapore online shopping market: who shops, where, how they pay and the big sale seasonsA workflow for agencies managing client accounts with proxies and antidetect: inventory, access, offboardingLocal SEO in Singapore: checking Google and Google Maps rankings as a Singapore user, the right wayProxies for Google Ads: when you need one, the official tools to use first, and the policy lines to knowRoblox lagging, high ping: does a proxy help, and the fixes that really reduce Roblox lagMultiple Roblox accounts (alts): what the rules allow and forbid, whether you need a proxy, and IP bansLag and high ping on Singapore game servers: do proxies or VPNs lower ping, and what actually helpsDiscord proxy: what it can and can't do, how to set it up, and managing communities the right wayShopee and Lazada Singapore proxies: see what Singapore buyers see, run your shop, track competitorsProxies for online business (MMO): which jobs need one, which type to pick and what to avoid机器人与自动化用代理:每个 IP 一个身份、礼貌的节奏,以及出现验证码时该怎么办How to set up a proxy on Windows, macOS, iPhone and Android (with a username and password)How to add a proxy to GoLogin, AdsPower, Hidemyacc, GenLogin and MultiloginRotate a proxy IP automatically with an API: curl, Python and Node.js examplesHow to check a proxy: IP, country, carrier, speed and DNS and WebRTC leaksCommon proxy errors and how to fix them: 407, timeouts, ERR_PROXY_CONNECTION_FAILEDScraping JavaScript-rendered sites with a headless browser and a proxy403, 429 and 503 errors when scraping: causes and how to handle them properly (with Python code)Using a proxy with Scrapy: authenticated setup, polite throttling, error fixes and rotating between runsUsing a proxy with curl and wget: HTTP, SOCKS5, authentication, environment variables and debuggingUsing a proxy in Python: requests, httpx and aiohttp with HTTP, SOCKS5 and authenticationSetting up a proxy in Chrome and Edge: a browser-only proxy without changing the whole computerSingapore proxy for Telegram: set up SOCKS5 on desktop, iPhone and Android, and how it differs from MTProtoSlow 4G proxy? How to measure speed and latency properly, and what really makes a proxy fasterUsing proxies safely: protect your proxy password, rotation link, accounts and dataUsing Proxifier for software without proxy support: SOCKS5 setup, per-app rules and DNSRoblox proxies for developers: calling Roblox APIs from HttpService, Open Cloud, and a safe self-hosted relayProtecting your Roblox account: cookie theft, "free Robux", malicious extensions, and how to stay safeHow to set up a proxy in AdsPower: HTTP or SOCKS5, where to paste it, checking the Singapore IP and rotatingHow to add a proxy to GoLogin: HTTP or SOCKS5, where to enter it, checking the Singapore IP and rotatingHow to add a proxy to GenLogin: HTTP or SOCKS5, where to paste it, checking the Singapore IP and rotatingHow to add a proxy to Hidemium: HTTP or SOCKS5, where to enter it, checking the Singapore IP and rotatingHow to add a proxy to BitBrowser: HTTP or SOCKS5, where to enter it, checking the Singapore IP and rotating验证码一直出现或无限循环:8 种解决办法,按这个顺序逐个试移动代理、住宅代理与数据中心代理对比:4G、家庭 IP 与服务器 IP,以及如何选择新加坡代理:什么时候需要新加坡 IP,以及该选哪种类型干净代理与黑名单:干净 IP 到底指什么,以及为什么“永不被封”是个神话4G rotating proxies: how mobile IP rotation works, what clean rotation means and how to choose oneHow to tell a real 4G proxy from a fake: check the carrier, IP type, latency and rotationFree vs paid proxies: the real risks of free proxies, and when paying is worth itDedicated 4G proxies, not shared: why a proxy only you use matters, and how to check it yourselfHow many proxies do I need? Working out the number for accounts, teams and scrapingStatic IP proxies for allowlists, APIs and remote access: 5 options and how to choose the right oneVietnam or Singapore proxy? Choosing which country's IP fits your workSingtel, M1 and StarHub proxies: which Singapore carrier a 4G proxy runs on, and how to check it干净代理与验证码:IP 信誉如何决定你会遇到多少验证脏代理的真实代价:验证码、重试、浪费的流量和被审查的账号Singapore SOCKS5 proxy: when to use it, how to set it up, and how to check it5G proxy vs 4G proxy: what changes, what doesn't, and whether 5G is worth paying more forCheap 4G proxies: $4 gets you 200GB, and how to compare mobile proxy prices properly before you buyRefer a friend and earn 10% on every proxy order: how the 65Proxy referral program worksWhy choose 65Proxy? 8 reasons you can verify, and when you should go elsewhere如何在 65Proxy 购买 4G 代理:注册、充值、付款、连接和轮换,一步一步来Where to buy a reliable proxy? 12 things to check before paying any provider在 1 小时试用中测试 4G 代理:逐分钟检查清单用 USDT 购买代理:通过 TRC20 或 BEP20 为新加坡 4G 代理付款,一步一步来4G proxy with 500GB: what the plan includes, how far 500GB goes, and how bandwidth is counted4G 移动代理指南

机器人与自动化用代理:每个 IP 一个身份、礼貌的节奏,以及出现验证码时该怎么办

按用途 阅读约 6 分钟

本文目录
  1. 01 代理对自动化有什么用,又没有什么用
  2. 02 比 IP 更重要的五条规则
  3. 03 出现验证时,先停下来读懂它
  4. 04 示例:遇到验证就停下的礼貌抓取程序
  5. 05 负责任地轮换 IP
  6. 06 需要多少个代理,多少流量
  7. 07 不该自动化的事
  8. 08 65Proxy
  9. 09 常见问题

脚本、定时任务、测试套件和浏览器自动化发出的流量,都不是由人一下一下手动点出来的,这正是它们存在的意义。本文讲的是如何通过代理运行这类正当的自动化,让它保持稳定:代理能带来什么,比 IP 更重要的五条规则,网站用验证码回应时该怎么办,以及一个能检测到验证并干净地停下、而不是硬闯下去的 Python 示例。本文不是教你绕过网站的规则,也帮不上这方面的忙。

代理对自动化有什么用,又没有什么用

有帮助的方面:

  • 地理位置。看到新加坡访客所看到的页面,或者从新加坡移动网络做测试。
  • 隔离。给每个真实账号或每项任务分配各自的地址,这样一个会话的活动不会和另一个混在一起。
  • 更好的起点。移动 IP 比数据中心 IP 更不容易一见面就被标记,这意味着在你自己的行为被计入之前,遇到的验证更少。参见 干净代理与验证码。

没有帮助的方面:

  • 许可。如果网站禁止自动化,代理并不会让你的脚本变得被允许。
  • 行为。速度、重复性,以及看起来像自动化的浏览器,都是独立于 IP 被读取的。
  • 验证码和验证。它们检查的是这次访问,而不只是地址。我们不会把代理当作攻破它们的办法来卖。

关于机器人是什么、哪几类受欢迎的概览,见 什么是 botting。

比 IP 更重要的五条规则

  1. 每个 IP 和会话只用一个身份。让 Cookie、user agent 和账号始终与它们最初使用的 IP 绑在一起。不要同时让许多账号通过同一个地址运行,也不要把一个账号随意在不同地址之间切换。
  2. 人的节奏,加上随机抖动。页面之间隔几秒,并带有随机变化,并行连接要少。固定的间隔本身就是一个信号。更多细节见 如何抓取而不被封。
  3. 被要求退让时就退让。遇到 429 时,等待 Retry-After 给出的时间,然后放慢速度。不要在紧凑的循环里重试。各个状态码的含义见 抓取时的 403、429 和 503 错误。
  4. 表明身份并使用缓存。在合适的情况下使用清晰的 User-Agent 并附上联系地址,不要重复下载没有变化的页面。
  5. 网站说不时就停下。反复出现的验证、拦截页面或者要求停止的请求,都是网站的决定。请尊重它。见下一节。

出现验证时,先停下来读懂它

你收到的结果 通常意味着什么 该怎么做
429 Too Many Requests 你太快了 等待 Retry-After,然后放慢速度运行
第一个请求就返回 403,而同一网络上的浏览器却能进去 你的程序缺少浏览器会发送的内容,或者这种 IP 类型被拒绝 检查请求头并手动测试 IP;参见 常见代理错误
一个“正在验证你是真人”的页面 网站想要做一次浏览器检查 如果网站允许自动化的话,用真实的浏览器并保持人的节奏
偶尔出现一个验证码 你的访问处在灰色地带 放慢速度,保持会话一致,检查 IP
一个接一个的验证码 网站不想要这种流量 停止运行。改为申请 API 或权限

有两件事看起来像是解决办法,其实不是:

  • 轮换 IP 后以同样的速度继续。网站会在新地址上看到同样的节奏。轮换是针对被限制的 IP,而不是无视验证的办法。
  • 使用验证码破解服务。它违背了这项检查的初衷,违反大多数网站的条款,也会增加你的法律风险。我们建议不要这样做;参见 不该越过的界线。

如果遇到循环的是真人而不是脚本,分步骤的做法见 验证码一直出现或陷入循环:8 种解决办法。

示例:遇到验证就停下的礼貌抓取程序

这段 Python 草稿通过你的代理,以人的节奏加载页面,遵守 Retry-After,并且在验证出现的那一刻停止整个运行。代理的详细信息来自环境变量,千万不要把它们写进代码里。

Python
import os
import random
import time

import requests

PROXY = os.environ["PROXY"]  # http://USER:PASS@HOST:PORT
HEADERS = {"User-Agent": "my-price-checker/1.0 (contact: [email protected])"}
MAX_TRIES = 3


class Challenged(Exception):
    """The site asked for a human. Stop the run; don't try to solve it."""


def looks_like_challenge(resp):
    # Cloudflare marks challenge responses with this header; other vendors differ.
    if resp.headers.get("cf-mitigated") == "challenge":
        return True
    body = resp.text[:20000].lower()
    return any(word in body for word in ("captcha", "verify you are human", "unusual traffic"))


def retry_after(resp, default=60):
    value = resp.headers.get("Retry-After", "")
    return int(value) if value.isdigit() else default


def polite_get(session, url):
    for attempt in range(1, MAX_TRIES + 1):
        resp = session.get(url, timeout=30)
        if looks_like_challenge(resp):
            raise Challenged(url)
        if resp.status_code == 429:
            time.sleep(retry_after(resp) * attempt)
            continue
        resp.raise_for_status()
        return resp
    raise RuntimeError(f"still rate limited after {MAX_TRIES} tries: {url}")


def run(urls):
    session = requests.Session()
    session.headers.update(HEADERS)
    session.proxies = {"http": PROXY, "https": PROXY}
    for url in urls:
        try:
            page = polite_get(session, url)
        except Challenged:
            print("Challenge page. Stopping the run:", url)
            break
        print(url, len(page.text))
        time.sleep(random.uniform(3, 8))  # a human pace, with jitter

关于这段草稿的说明:关键词检查也会匹配到一个只是谈论验证码的普通页面,所以请针对你的目标调整这些标记词。对于大量依赖 JavaScript 的页面,请使用真实的浏览器,而不是 requests;参见 抓取 JavaScript 网站。Python 的通用设置见 在 Python 中使用代理。

负责任地轮换 IP

4G 代理通过调用它的轮换链接来轮换,而每个 65Proxy 代理允许这样做的频率最多是每 5 分钟一次。请在IP 被限制时才轮换,而不是在每个请求之前轮换,并且在轮换之后开启一个全新的会话(新的 Cookie),因为网站刚刚看到的是一位新访客。调用链接并等待新 IP 的代码见 通过 API 轮换代理 IP。

需要多少个代理,多少流量

  • 每个并发身份一个代理。计算同时运行的账号或任务数,而不是存在的全部。我需要多少个代理逐步讲解了这一点。
  • 按页面大小预估流量。用一个典型页面的体积乘以你要加载的页面数。被验证的加载和重复的加载也会消耗额度。500GB 能用多久演示了如何测量。

不该自动化的事

  • 对你不拥有的账号执行操作,或者任何形式的虚假账号。
  • 虚假互动、评论和粉丝。原因见 代理用于种草和造势。
  • 绕过购买限制和排队,这一点 抢购与限量发售用代理 有说明。
  • 替你玩游戏的游戏机器人。
  • 在没有法律依据的情况下收集个人数据。

65Proxy

65Proxy 销售用于正当工作的新加坡 4G 移动代理:每个订单一个代理,自带登录账号和轮换链接,支持 HTTP 和 SOCKS5,定价 1 天 $4、7 天 $13、1 个月 $32(每个含 200GB),另有带宽更大的月付套餐(500GB $40、1TB $65)。请先通过免费试用,在你真正的目标上试用一小时。我们不会把它们当作攻破验证码、封禁或限制的办法来卖。

常见问题

代理能让我的机器人无法被检测吗?

不能。代理改变的是网站看到的地址。节奏、浏览器信号和行为是被单独读取的,而且网站可以在任何时候对任何访客发起验证。

每分钟多少个请求才安全?

没有通用的数字。先从每隔几秒一个请求开始,观察 429 和验证所占的比例,出现时就放慢速度。有些网站会在文档或 API 条款中写明限制,请遵守它。

我应该在每个请求上都轮换 IP 吗?

不应该。这会破坏会话,看起来不自然,而且每个代理有 5 分钟的轮换限制,所以本来也做不到。请在 IP 被限制时再轮换。

我的脚本遇到验证码时应该怎么做?

停止、记录下来,并提醒你。然后检查你的节奏,查看是否有 API,或者申请访问权限。上面的示例展示了大致的做法。

我可以把 Playwright 或 Selenium 与这个代理一起用吗?

可以。两者都支持带用户名和密码的代理;Playwright 和 Puppeteer 的设置见 通过 API 轮换代理 IP。节奏和验证方面的规则同样适用。