U jipG@sddlmZddlZddlZddlZddlmZmZddlm Z m Z ddlm Z ddl m Z mZmZddlZddlmZdd lmZdd lmZdd lmZd d dgZeeZdZdZedeZ ddZ!e"e!e e"e!e dZ#e e dZ$Gdd d eZ%Gdd d e%Z&ddZ'dS))absolute_importN)HTTPHeaderDictRecentlyUsedContainer)HTTPConnectionPoolHTTPSConnectionPool)port_by_scheme)LocationValueError MaxRetryErrorProxySchemeUnknown)urljoin)RequestMethods) parse_url)Retry PoolManager ProxyManagerproxy_from_url)Zkey_fileZ cert_fileZ cert_reqsZca_certsZ ssl_versionZ ca_cert_dirZ ssl_contextZ key_password)Z key_schemeZkey_hostZkey_portZ key_timeoutZ key_retriesZ key_strictZ key_blockZkey_source_addressZ key_key_fileZkey_key_passwordZ key_cert_fileZ key_cert_reqsZ key_ca_certsZkey_ssl_versionZkey_ca_cert_dirZkey_ssl_contextZ key_maxsizeZ key_headersZ key__proxyZkey__proxy_headersZkey_socket_optionsZkey__socks_optionsZkey_assert_hostnameZkey_assert_fingerprintZkey_server_hostnamePoolKeycCs|}|d|d<|d|d<dD],}||kr,||dk r,t||||<q,|d}|dk rxt||d<t|D]}|||d|<q|j D]}||krd||<q|f|S)a Create a pool key out of a request context dictionary. According to RFC 3986, both the scheme and host are case-insensitive. Therefore, this function normalizes both before constructing the pool key for an HTTPS request. If you wish to change this behaviour, provide alternate callables to ``key_fn_by_scheme``. :param key_class: The class to use when constructing the key. This should be a namedtuple with the ``scheme`` and ``host`` keys at a minimum. :type key_class: namedtuple :param request_context: A dictionary-like object that contain the context for a request. :type request_context: dict :return: A namedtuple that can be used as a connection pool key. :rtype: PoolKey schemehost)headers_proxy_headersZ_socks_optionsNZsocket_optionsZkey_) copylower frozensetitemsgettuplelistkeyspop_fields)Z key_classrequest_contextcontextkeyZ socket_optsZfieldr%5/usr/lib/python3/dist-packages/urllib3/poolmanager.py_default_key_normalizerDs    r'httphttpsc@sxeZdZdZdZdddZddZdd Zdd d Zd d Z dddZ ddZ dddZ d ddZ ddZd!ddZdS)"ra$ Allows for arbitrary requests while transparently keeping track of necessary connection pools for you. :param num_pools: Number of connection pools to cache before discarding the least recently used pool. :param headers: Headers to include with all requests, unless other headers are given explicitly. :param \**connection_pool_kw: Additional parameters are used to create fresh :class:`urllib3.connectionpool.ConnectionPool` instances. Example:: >>> manager = PoolManager(num_pools=2) >>> r = manager.request('GET', 'http://google.com/') >>> r = manager.request('GET', 'http://google.com/mail') >>> r = manager.request('GET', 'http://yahoo.com/') >>> len(manager.pools) 2 N cKs~t||d|krR|d}t|tsR|dk }tj|dd}||_|}||d<||_t|ddd|_ t |_ t |_ dS)NretriesFredirectcSs|SN)close)pr%r%r&z&PoolManager.__init__..)Z dispose_func) r __init__ isinstancerfrom_intraise_on_redirectrconnection_pool_kwrpoolspool_classes_by_schemekey_fn_by_scheme)self num_poolsrr8r,r7r%r%r&r4s  zPoolManager.__init__cCs|Sr/r%r<r%r%r& __enter__szPoolManager.__enter__cCs |dS)NF)clear)r<exc_typeZexc_valZexc_tbr%r%r&__exit__szPoolManager.__exit__cCs^|j|}|dkr|j}dD]}||dq |dkrPtD]}||dq>|||f|S)a Create a new :class:`ConnectionPool` based on host, port, scheme, and any additional pool keyword arguments. If ``request_context`` is provided, it is provided as keyword arguments to the pool class used. This method is used to actually create the connection pools handed out by :meth:`connection_from_url` and companion methods. It is intended to be overridden for customization. N)rrportr))r:r8rr SSL_KEYWORDS)r<rrrCr"Zpool_clsr$kwr%r%r& _new_pools  zPoolManager._new_poolcCs|jdS)z Empty our store of pools and direct them all to close. This will not affect in-flight connections, but they will not be re-used after completion. N)r9r@r>r%r%r&r@szPoolManager.clearr)cCsT|s td||}|pd|d<|s:t|dd}||d<||d<||S)a Get a :class:`ConnectionPool` based on the host, port, and scheme. If ``port`` isn't given, it will be derived from the ``scheme`` using ``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is provided, it is merged with the instance's ``connection_pool_kw`` variable and used to create the new connection pool, if one is needed. zNo host specified.r)rPrCr)r _merge_pool_kwargsrrrconnection_from_context)r<rrCr pool_kwargsr"r%r%r&connection_from_hosts   z PoolManager.connection_from_hostcCs,|d}|j|}||}|j||dS)z Get a :class:`ConnectionPool` based on the request context. ``request_context`` must at least contain the ``scheme`` key and its value must be a key in ``key_fn_by_scheme`` instance variable. rr")rr;connection_from_pool_key)r<r"rZpool_key_constructorpool_keyr%r%r&rIs  z#PoolManager.connection_from_contextc Csl|jjZ|j|}|r*|W5QRS|d}|d}|d}|j||||d}||j|<W5QRX|S)z Get a :class:`ConnectionPool` based on the provided pool key. ``pool_key`` should be a namedtuple that only contains immutable objects. At a minimum it must have the ``scheme``, ``host``, and ``port`` fields. rrrCrL)r9lockrrF)r<rNr"ZpoolrrrCr%r%r&rM s  z$PoolManager.connection_from_pool_keycCs t|}|j|j|j|j|dS)a Similar to :func:`urllib3.connectionpool.connection_from_url`. If ``pool_kwargs`` is not provided and a new pool needs to be constructed, ``self.connection_pool_kw`` is used to initialize the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs`` is provided, it is used instead. Note that if a new pool does not need to be created for the request, the provided ``pool_kwargs`` are not used. )rCrrJ)rrKrrCr)r<urlrJur%r%r&connection_from_url!s zPoolManager.connection_from_urlc CsV|j}|rR|D]:\}}|dkrHz ||=WqPtk rDYqPXq|||<q|S)a Merge a dictionary of override values for self.connection_pool_kw. This does not modify self.connection_pool_kw and returns a new dict. Any keys in the override dictionary with a value of ``None`` are removed from the merged dictionary. N)r8rrKeyError)r<overrideZbase_pool_kwargsr$valuer%r%r&rH1s   zPoolManager._merge_pool_kwargsTc Kst|}|j|j|j|jd}d|d<d|d<d|krD|j|d<|jdk rj|jdkrj|j||f|}n|j||j f|}|o| }|s|St ||}|j dkrd }d|d <t |d|d<|d |j} t| tstj| |d } | jr<||s %s)!rrKrrCrrrproxyurlopenZ request_uriZget_redirect_locationr ZstatusrZ_prepare_for_method_changerr,r5rr6Zremove_headers_on_redirectZ is_same_hostrsixZiterkeysrr Z incrementr r7Z drain_connloginfo) r<methodrPr.rErQZconnrVZredirect_locationr,rheaderr%r%r&rXEsN       zPoolManager.urlopen)r+N)N)Nr)N)N)N)T)__name__ __module__ __qualname____doc__rWr4r?rBrFr@rKrIrMrRrHrXr%r%r%r&rs     csHeZdZdZdfdd Zdfdd Zdd d Zdfd d ZZS)rax Behaves just like :class:`PoolManager`, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs. :param proxy_url: The URL of the proxy to be used. :param proxy_headers: A dictionary containing headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication. Example: >>> proxy = urllib3.ProxyManager('http://localhost:3128/') >>> r1 = proxy.request('GET', 'http://google.com/') >>> r2 = proxy.request('GET', 'http://httpbin.org/') >>> len(proxy.pools) 1 >>> r3 = proxy.request('GET', 'https://httpbin.org/') >>> r4 = proxy.request('GET', 'https://twitter.com/') >>> len(proxy.pools) 3 r+Nc st|trd|j|j|jf}t|}|jsFt|jd}|j|d}|jdkrZt |j||_ |pfi|_ |j |d<|j |d<t t |j||f|dS)Nz %s://%s:%irG)rCr(_proxyr)r5rrrrCrrr_replacer rW proxy_headerssuperrr4)r< proxy_urlr=rrdr8rWrC __class__r%r&r4s"       zProxyManager.__init__r)csD|dkr tt|j||||dStt|j|jj|jj|jj|dS)Nr*)rJ)rerrKrWrrCr)r<rrCrrJrgr%r&rKs  z!ProxyManager.connection_from_hostcCs0ddi}t|j}|r||d<|r,|||S)z Sets headers needed by proxies: specifically, the Accept and Host headers. Only sets headers not provided by the user. ZAcceptz*/*ZHost)rnetlocupdate)r<rPrZheaders_rir%r%r&_set_proxy_headerss  zProxyManager._set_proxy_headersTc sNt|}|jdkr0|d|j}||||d<tt|j||fd|i|S)z@Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.r)rr.)rrrrrkrerrX)r<r\rPr.rErQrrgr%r&rXs  zProxyManager.urlopen)r+NN)Nr)N)N)T) r^r_r`rar4rKrkrX __classcell__r%r%rgr&rs cKstfd|i|S)Nrf)r)rPrEr%r%r&rs)(Z __future__r collections functoolsZlogging _collectionsrrZconnectionpoolrrr exceptionsr r r rYZsix.moves.urllib.parser Zrequestr Zutil.urlrZ util.retryr__all__Z getLoggerr^rZrDZ _key_fields namedtuplerr'partialr;r:rrrr%r%r%r&s6          6    a