首选_selenium源码通读·10|webdriver/common/proxy.pyProxy类分析

篇首语：本文由编程笔记#小编为大家整理，主要介绍了selenium源码通读·10 |webdriver/common/proxy.py-Proxy类分析相关的知识，希望对你有一定的参考价值。

·10 |webdriver/common/proxy.py-Proxy类分析

• 1 源码路径
• 2 功能说明
• 3 引入原因
• 4 代理类型
• 5 代理类型模式或属性
• 6 各种代理实现
• 7 实例说明

selenium/webdriver/common/proxy.py


• 使用代理来规避一些反爬虫策略&＃xff1b;

• selenium webdriver测试或进行爬虫时&＃xff0c;如果未在目标服务的白名单内&＃xff0c;随着爬取的频率增多会被禁止访问目标服务;
• 目标服务的反爬虫策略相对完善的情况下&＃xff0c;容易将selenium的爬取给禁止;
• 采用分布式selenium爬取方式时&＃xff0c;也会很容易的被禁止爬取&＃xff1b;

综上&＃xff0c;所以Proxy类提供了一些反爬虫策略&＃xff0c;

• 源码&＃xff1a;

class ProxyType:
"""
Set of possible types of proxy.
Each proxy type has 2 properties:
&＃39;ff_value&＃39; is value of Firefox profile preference,
&＃39;string&＃39; is id of proxy type.
"""
DIRECT &＃61; ProxyTypeFactory.make(0, &＃39;DIRECT&＃39;) # Direct connection, no proxy (default on Windows).
MANUAL &＃61; ProxyTypeFactory.make(1, &＃39;MANUAL&＃39;) # Manual proxy settings (e.g., for httpProxy).
PAC &＃61; ProxyTypeFactory.make(2, &＃39;PAC&＃39;) # Proxy autoconfiguration from URL.
RESERVED_1 &＃61; ProxyTypeFactory.make(3, &＃39;RESERVED1&＃39;) # Never used.
AUTODETECT &＃61; ProxyTypeFactory.make(4, &＃39;AUTODETECT&＃39;) # Proxy autodetection (presumably with WPAD).
SYSTEM &＃61; ProxyTypeFactory.make(5, &＃39;SYSTEM&＃39;) # Use system settings (default on Linux).
UNSPECIFIED &＃61; ProxyTypeFactory.make(6, &＃39;UNSPECIFIED&＃39;) # Not initialized (for internal use).


• 说明&＃xff1a;

class ProxyTypeFactory:
"""
Factory for proxy types.
"""
&＃64;staticmethod
def make(ff_value, string):
return &＃39;ff_value&＃39;: ff_value, &＃39;string&＃39;: string


• 源码&＃xff1a;

class Proxy(object):
"""
Proxy contains information about proxy type and necessary proxy settings.
"""
proxyType &＃61; ProxyType.UNSPECIFIED
autodetect &＃61; False
ftpProxy &＃61; &＃39;&＃39;
httpProxy &＃61; &＃39;&＃39;
noProxy &＃61; &＃39;&＃39;
proxyAutoconfigUrl &＃61; &＃39;&＃39;
sslProxy &＃61; &＃39;&＃39;
socksProxy &＃61; &＃39;&＃39;
socksUsername &＃61; &＃39;&＃39;
socksPassword &＃61; &＃39;&＃39;


• 说明&＃xff1a;每种代理&＃xff0c;都是通过两个方法来实现&＃xff0c;先返回代理设置&＃xff0c;然后再对代理进行具体的值设置&＃xff0c;如下&＃xff1a;

&＃64;property
def http_proxy(self):
"""
Returns http proxy setting.
"""
return self.httpProxy
&＃64;http_proxy.setter
def http_proxy(self, value):
"""
Sets http proxy setting.
:Args:
- value: The http proxy value.
"""
self._verify_proxy_type_compatibility(ProxyType.MANUAL)
self.proxyType &＃61; ProxyType.MANUAL
self.httpProxy &＃61; value

7 实例说明

from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
profile &＃61; FirefoxProfile()
# 设置成手动代理
profile.set_preference("network.proxy.type", 1)
# 设置代理ip
profile.set_preference("network.proxy.http", "ip")
# 设置代理端口
profile.set_preference("network.proxy.http_port", port)
# 协议默认共用此ip、端口
profile.set_preference("network.proxy.share_proxy_settings", True)
# 启动配置
driver&＃61; webdriver.Firefox(profile)
# 打开浏览器
driver.get(&＃39;xxxxx&＃39;)


『全栈测试技术&＃xff0c;分享&＃xff0c;共勉&＃xff0c;共进&＃xff0c;提升』