13.2.3. keypirinha_net module

The keypirinha_net module offers some network utility features frequently required by the plugin developer.

13.2.3.1. Reference

class keypirinha_net.UrllibOpener(opener)

Bases: object

A replacement to urllib.request.OpenerDirector to apply user’s global settings if needed and to ensure reasonable default timeout value (i.e. too long hanging due to bad connection).

This class is not meant to be used directly, see build_urllib_opener().

open(fullurl, *args, data=None, timeout=None, **kwargs)

This overrides urllib.request.OpenerDirector.open() in order to apply the timeout value automatically from Keypirinha’s settings. Forcing a timeout value here is still allowed but discouraged.

keypirinha_net.build_urllib_opener(proxies=None, ssl_check_hostname=None, extra_handlers=[], extra_pre_handlers=[])

A replacement to urllib.request.build_opener() that takes care of using current user’s global settings (Keypirinha and/or system’s) regarding network connections, by inserting and configuring one or several connection handlers (derived from urllib.request.BaseHandler).

Examples:

# example 1
opener = build_urllib_opener()
with opener.open("http://httpbin.org/user-agent") as response:
    print(response.read())

# example 2: HTTP proxy
proxies = {'http': "http://proxy_user:proxy_pass@127.0.0.1:8080"}
opener = build_urllib_opener(proxies)
with opener.open("http://httpbin.org/ip") as response:
    print(response.read())
Parameters

  • proxies (dict) –

    A dictionary of proxies to pass to the constructor of urllib.request.ProxyHandler, if any. Notes:

    • None (default; recommended) means proxies configured by the user at Keypirinha’s level, or by default at system level will be used.

    • An empty dictionary (i.e. {}) means no proxy will be configured regardless of user or machine settings. Note that going against user’s will is against Keypirinha’s design policy!

    • See the notes below about SOCKS proxies.

    • See proxies_list_to_dict() if you need to convert a list of proxies URLs into a dictionary.

  • extra_handlers (list) – A list/tuple of extra handlers to append to the final handlers chain before passing it to urllib.request.build_opener().

  • extra_pre_handlers (list) –

    A list/tuple of extra handlers to prepend to the final handlers chain before passing it to urllib.request.build_opener().

    CAUTION: This parameter is here for convenience and you should use it only if you know what you are doing as it may interfere with the handlers added by this function.

  • ssl_check_hostname (bool) –

    Should the hostname be checked against received security certificate? This argument is equivalent to tweaking with ssl.SSLContext.check_hostname and ssl.SSLContext.verify_mode.

    Default behavior of the urllib module (i.e. None value) is to check the hostname unless explicitely specified here (boolean), in which case this function will either add an urllib.request.HTTPSHandler handler with the appropriate arguments to the chain, or, if caller already added a urllib.request.HTTPSHandler handler (either in the extra_handlers or extra_pre_handlers list), it will be modified accordingly.

Returns

A urllib.request.OpenerDirector-compatible opener object.

Return type

UrllibOpener

Note

Notes about SOCKS proxy support:

  • Support for SOCKS proxy (v4 and v5) is experimental and uses the PySocks third-party module under the hood.

  • DNS requests do not go through the proxy server.

  • IPv6 connections through the proxy server are not supported.

  • Tests have shown that if proxies for several schemes have been specified, UNKNOWN_PROTOCOL SSL error may occurs under some circumstances. For that reason, if a SOCKS proxy is specified, it takes precedence over the other proxy servers that might be in the dictionary as well so they will be purely ignored by this function in favor of the SOCKS proxy.

keypirinha_net.proxies_to_dict(proxies)

Transform a list of proxy specification lines into a dictionary that is suitable to use with build_urllib_opener() and urllib.request.

This function is used by build_urllib_opener() to parse Keypirinha’s proxy global setting of the [network] section.

proxies may be an iterable or a string, in which case it is split first using \n as a separator and each line is trimmed with str.strip().

Each entry of proxies must be a string that complies to the following grammar:

prox_entry  ::= [ conn_scheme "=" ] proxy_url
conn_scheme ::= ( "http" | "https" | "socks" | "socks4" | "socks5")
proxy_url   ::= url_scheme "://" [ proxy_user ":" proxy_pass "@" ] netloc ":" proxy_port
proxy_port  ::= unsigned

If the conn_scheme part is not specified, url_scheme is assumed.

Raises

ValueError – proxies cannot be parsed

Note

If two proxies have the same scheme, the last one prevails.