httpx-html: Web Scraping for Humans!

This library intends to make scraping the web as simple and intuitive as possible.

When using this library you automatically get:

  • Full JavaScript support!

  • CSS Selectors (a.k.a jQuery-style, thanks to PyQuery).

  • XPath Selectors, for the faint of heart.

  • Mocked user-agent (like a real web browser).

  • Automatic following of redirects.

  • Connection–pooling and cookie persistence.

  • The httpx experience you know and love, with magical parsing abilities.

  • Async Support

Installation

$ pip install httpx-html

Only Python 3.6+ is supported.

Tutorial & Usage

Make a GET request to python.org, using httpx:

>>> from httpx_html import HTMLSession
>>> session = HTMLSession()

>>> r = session.get('https://python.org/')

Or want to try our async session:

>>> from httpx_html import AsyncHTMLSession
>>> asession = AsyncHTMLSession()

>>> r = await asession.get('https://python.org/')

But async is fun when fetching some sites at the same time:

>>> from httpx_html import AsyncHTMLSession
>>> asession = AsyncHTMLSession()

>>> async def get_pythonorg():
...    r = await asession.get('https://python.org/')

>>> async def get_reddit():
...    r = await asession.get('https://reddit.com/')

>>> async def get_google():
...    r = await asession.get('https://google.com/')

>>> session.run(get_pythonorg, get_reddit, get_google)

Grab a list of all links on the page, as–is (anchors excluded):

>>> r.html.links
{'//docs.python.org/3/tutorial/', '/about/apps/', 'https://github.com/python/pythondotorg/issues', '/accounts/login/', '/dev/peps/', '/about/legal/', '//docs.python.org/3/tutorial/introduction.html#lists', '/download/alternatives', 'http://feedproxy.google.com/~r/PythonInsider/~3/kihd2DW98YY/python-370a4-is-available-for-testing.html', '/download/other/', '/downloads/windows/', 'https://mail.python.org/mailman/listinfo/python-dev', '/doc/av', 'https://devguide.python.org/', '/about/success/#engineering', 'https://wiki.python.org/moin/PythonEventsCalendar#Submitting_an_Event', 'https://www.openstack.org', '/about/gettingstarted/', 'http://feedproxy.google.com/~r/PythonInsider/~3/AMoBel8b8Mc/python-3.html', '/success-stories/industrial-light-magic-runs-python/', 'http://docs.python.org/3/tutorial/introduction.html#using-python-as-a-calculator', '/', 'http://pyfound.blogspot.com/', '/events/python-events/past/', '/downloads/release/python-2714/', 'https://wiki.python.org/moin/PythonBooks', 'http://plus.google.com/+Python', 'https://wiki.python.org/moin/', 'https://status.python.org/', '/community/workshops/', '/community/lists/', 'http://buildbot.net/', '/community/awards', 'http://twitter.com/ThePSF', 'https://docs.python.org/3/license.html', '/psf/donations/', 'http://wiki.python.org/moin/Languages', '/dev/', '/events/python-user-group/', 'https://wiki.qt.io/PySide', '/community/sigs/', 'https://wiki.gnome.org/Projects/PyGObject', 'http://www.ansible.com', 'http://www.saltstack.com', 'http://planetpython.org/', '/events/python-events', '/about/help/', '/events/python-user-group/past/', '/about/success/', '/psf-landing/', '/about/apps', '/about/', 'http://www.wxpython.org/', '/events/python-user-group/665/', 'https://www.python.org/psf/codeofconduct/', '/dev/peps/peps.rss', '/downloads/source/', '/psf/sponsorship/sponsors/', 'http://bottlepy.org', 'http://roundup.sourceforge.net/', 'http://pandas.pydata.org/', 'http://brochure.getpython.info/', 'https://bugs.python.org/', '/community/merchandise/', 'http://tornadoweb.org', '/events/python-user-group/650/', 'http://flask.pocoo.org/', '/downloads/release/python-364/', '/events/python-user-group/660/', '/events/python-user-group/638/', '/psf/', '/doc/', 'http://blog.python.org', '/events/python-events/604/', '/about/success/#government', 'http://python.org/dev/peps/', 'https://docs.python.org', 'http://feedproxy.google.com/~r/PythonInsider/~3/zVC80sq9s00/python-364-is-now-available.html', '/users/membership/', '/about/success/#arts', 'https://wiki.python.org/moin/Python2orPython3', '/downloads/', '/jobs/', 'http://trac.edgewall.org/', 'http://feedproxy.google.com/~r/PythonInsider/~3/wh73_1A-N7Q/python-355rc1-and-python-348rc1-are-now.html', '/privacy/', 'https://pypi.python.org/', 'http://www.riverbankcomputing.co.uk/software/pyqt/intro', 'http://www.scipy.org', '/community/forums/', '/about/success/#scientific', '/about/success/#software-development', '/shell/', '/accounts/signup/', 'http://www.facebook.com/pythonlang?fref=ts', '/community/', 'https://kivy.org/', '/about/quotes/', 'http://www.web2py.com/', '/community/logos/', '/community/diversity/', '/events/calendars/', 'https://wiki.python.org/moin/BeginnersGuide', '/success-stories/', '/doc/essays/', '/dev/core-mentorship/', 'http://ipython.org', '/events/', '//docs.python.org/3/tutorial/controlflow.html', '/about/success/#education', '/blogs/', '/community/irc/', 'http://pycon.blogspot.com/', '//jobs.python.org', 'http://www.pylonsproject.org/', 'http://www.djangoproject.com/', '/downloads/mac-osx/', '/about/success/#business', 'http://feedproxy.google.com/~r/PythonInsider/~3/x_c9D0S-4C4/python-370b1-is-now-available-for.html', 'http://wiki.python.org/moin/TkInter', 'https://docs.python.org/faq/', '//docs.python.org/3/tutorial/controlflow.html#defining-functions'}

Grab a list of all links on the page, in absolute form (anchors excluded):

>>> r.html.absolute_links
{'https://github.com/python/pythondotorg/issues', 'https://docs.python.org/3/tutorial/', 'https://www.python.org/about/success/', 'http://feedproxy.google.com/~r/PythonInsider/~3/kihd2DW98YY/python-370a4-is-available-for-testing.html', 'https://www.python.org/dev/peps/', 'https://mail.python.org/mailman/listinfo/python-dev', 'https://www.python.org/doc/', 'https://www.python.org/', 'https://www.python.org/about/', 'https://www.python.org/events/python-events/past/', 'https://devguide.python.org/', 'https://wiki.python.org/moin/PythonEventsCalendar#Submitting_an_Event', 'https://www.openstack.org', 'http://feedproxy.google.com/~r/PythonInsider/~3/AMoBel8b8Mc/python-3.html', 'https://docs.python.org/3/tutorial/introduction.html#lists', 'http://docs.python.org/3/tutorial/introduction.html#using-python-as-a-calculator', 'http://pyfound.blogspot.com/', 'https://wiki.python.org/moin/PythonBooks', 'http://plus.google.com/+Python', 'https://wiki.python.org/moin/', 'https://www.python.org/events/python-events', 'https://status.python.org/', 'https://www.python.org/about/apps', 'https://www.python.org/downloads/release/python-2714/', 'https://www.python.org/psf/donations/', 'http://buildbot.net/', 'http://twitter.com/ThePSF', 'https://docs.python.org/3/license.html', 'http://wiki.python.org/moin/Languages', 'https://docs.python.org/faq/', 'https://jobs.python.org', 'https://www.python.org/about/success/#software-development', 'https://www.python.org/about/success/#education', 'https://www.python.org/community/logos/', 'https://www.python.org/doc/av', 'https://wiki.qt.io/PySide', 'https://www.python.org/events/python-user-group/660/', 'https://wiki.gnome.org/Projects/PyGObject', 'http://www.ansible.com', 'http://www.saltstack.com', 'https://www.python.org/dev/peps/peps.rss', 'http://planetpython.org/', 'https://www.python.org/events/python-user-group/past/', 'https://docs.python.org/3/tutorial/controlflow.html#defining-functions', 'https://www.python.org/community/diversity/', 'https://docs.python.org/3/tutorial/controlflow.html', 'https://www.python.org/community/awards', 'https://www.python.org/events/python-user-group/638/', 'https://www.python.org/about/legal/', 'https://www.python.org/dev/', 'https://www.python.org/download/alternatives', 'https://www.python.org/downloads/', 'https://www.python.org/community/lists/', 'http://www.wxpython.org/', 'https://www.python.org/about/success/#government', 'https://www.python.org/psf/', 'https://www.python.org/psf/codeofconduct/', 'http://bottlepy.org', 'http://roundup.sourceforge.net/', 'http://pandas.pydata.org/', 'http://brochure.getpython.info/', 'https://www.python.org/downloads/source/', 'https://bugs.python.org/', 'https://www.python.org/downloads/mac-osx/', 'https://www.python.org/about/help/', 'http://tornadoweb.org', 'http://flask.pocoo.org/', 'https://www.python.org/users/membership/', 'http://blog.python.org', 'https://www.python.org/privacy/', 'https://www.python.org/about/gettingstarted/', 'http://python.org/dev/peps/', 'https://www.python.org/about/apps/', 'https://docs.python.org', 'https://www.python.org/success-stories/', 'https://www.python.org/community/forums/', 'http://feedproxy.google.com/~r/PythonInsider/~3/zVC80sq9s00/python-364-is-now-available.html', 'https://www.python.org/community/merchandise/', 'https://www.python.org/about/success/#arts', 'https://wiki.python.org/moin/Python2orPython3', 'http://trac.edgewall.org/', 'http://feedproxy.google.com/~r/PythonInsider/~3/wh73_1A-N7Q/python-355rc1-and-python-348rc1-are-now.html', 'https://pypi.python.org/', 'https://www.python.org/events/python-user-group/650/', 'http://www.riverbankcomputing.co.uk/software/pyqt/intro', 'https://www.python.org/about/quotes/', 'https://www.python.org/downloads/windows/', 'https://www.python.org/events/calendars/', 'http://www.scipy.org', 'https://www.python.org/community/workshops/', 'https://www.python.org/blogs/', 'https://www.python.org/accounts/signup/', 'https://www.python.org/events/', 'https://kivy.org/', 'http://www.facebook.com/pythonlang?fref=ts', 'http://www.web2py.com/', 'https://www.python.org/psf/sponsorship/sponsors/', 'https://www.python.org/community/', 'https://www.python.org/download/other/', 'https://www.python.org/psf-landing/', 'https://www.python.org/events/python-user-group/665/', 'https://wiki.python.org/moin/BeginnersGuide', 'https://www.python.org/accounts/login/', 'https://www.python.org/downloads/release/python-364/', 'https://www.python.org/dev/core-mentorship/', 'https://www.python.org/about/success/#business', 'https://www.python.org/community/sigs/', 'https://www.python.org/events/python-user-group/', 'http://ipython.org', 'https://www.python.org/shell/', 'https://www.python.org/community/irc/', 'https://www.python.org/about/success/#engineering', 'http://www.pylonsproject.org/', 'http://pycon.blogspot.com/', 'https://www.python.org/about/success/#scientific', 'https://www.python.org/doc/essays/', 'http://www.djangoproject.com/', 'https://www.python.org/success-stories/industrial-light-magic-runs-python/', 'http://feedproxy.google.com/~r/PythonInsider/~3/x_c9D0S-4C4/python-370b1-is-now-available-for.html', 'http://wiki.python.org/moin/TkInter', 'https://www.python.org/jobs/', 'https://www.python.org/events/python-events/604/'}

Select an Element with a CSS Selector (learn more):

>>> about = r.html.find('#about', first=True)

Grab an Element’s text contents:

>>> print(about.text)
About
Applications
Quotes
Getting Started
Help
Python Brochure

Introspect an Element’s attributes (learn more):

>>> about.attrs
{'id': 'about', 'class': ('tier-1', 'element-1'), 'aria-haspopup': 'true'}

Render out an Element’s HTML:

>>> about.html
'<li aria-haspopup="true" class="tier-1 element-1 " id="about">\n<a class="" href="/about/" title="">About</a>\n<ul aria-hidden="true" class="subnav menu" role="menu">\n<li class="tier-2 element-1" role="treeitem"><a href="/about/apps/" title="">Applications</a></li>\n<li class="tier-2 element-2" role="treeitem"><a href="/about/quotes/" title="">Quotes</a></li>\n<li class="tier-2 element-3" role="treeitem"><a href="/about/gettingstarted/" title="">Getting Started</a></li>\n<li class="tier-2 element-4" role="treeitem"><a href="/about/help/" title="">Help</a></li>\n<li class="tier-2 element-5" role="treeitem"><a href="http://brochure.getpython.info/" title="">Python Brochure</a></li>\n</ul>\n</li>'

Crab an Element’s root tag name:

>>> about.tag
'li'

Show the line number that an Element’s root tag located in:

>>> about.lineno
249

Select an Element list within an Element:

>>> about.find('a')
[<Element 'a' href='/about/' title='' class=''>, <Element 'a' href='/about/apps/' title=''>, <Element 'a' href='/about/quotes/' title=''>, <Element 'a' href='/about/gettingstarted/' title=''>, <Element 'a' href='/about/help/' title=''>, <Element 'a' href='http://brochure.getpython.info/' title=''>]

Search for links within an element:

>>> about.absolute_links
{'http://brochure.getpython.info/', 'https://www.python.org/about/gettingstarted/', 'https://www.python.org/about/', 'https://www.python.org/about/quotes/', 'https://www.python.org/about/help/', 'https://www.python.org/about/apps/'}

Search for text on the page:

>>> r.html.search('Python is a {} language')[0]
programming

More complex CSS Selector example (copied from Chrome dev tools):

>>> r = session.get('https://github.com/')
>>> sel = 'body > div.application-main > div.jumbotron.jumbotron-codelines > div > div > div.col-md-7.text-center.text-md-left > p'

>>> print(r.html.find(sel, first=True).text)
GitHub is a development platform inspired by the way you work. From open source to business, you can host and review code, manage projects, and build software alongside millions of other developers.

XPath is also supported (learn more):

>>> r.html.xpath('a')
[<Element 'a' class='btn' href='https://help.github.com/articles/supported-browsers'>]

You can also select only elements containing certain text:

>>> r = session.get('http://python-requests.org/')
>>> r.html.find('a', containing='kenneth')
[<Element 'a' href='http://kennethreitz.com/pages/open-projects.html'>, <Element 'a' href='http://kennethreitz.org/'>, <Element 'a' href='https://twitter.com/kennethreitz' class=('twitter-follow-button',) data-show-count='false'>, <Element 'a' class=('reference', 'internal') href='dev/contributing/#kenneth-reitz-s-code-style'>]

JavaScript Support

Let’s grab some text that’s rendered by JavaScript:

>>> r = session.get('http://python-requests.org/')

>>> r.html.render()

>>> r.html.search('Python 2 will retire in only {months} months!')['months']
'<time>25</time>'

Or you can do this async also:

>>> r = asession.get('http://python-requests.org/')

>>> await r.html.arender()

>>> r.html.search('Python 2 will retire in only {months} months!')['months']
'<time>25</time>'

Note, the first time you ever run the render() method, it will download Chromium into your home directory (e.g. ~/.pyppeteer/). This only happens once. You may also need to install a few Linux packages to get pyppeteer working.

Using without httpx

You can also use this library without httpx:

>>> from httpx_html import HTML
>>> doc = """<a href='https://httpbin.org'>"""

>>> html = HTML(html=doc)
>>> html.links
{'https://httpbin.org'}

You can also render JavaScript pages without httpx:

# ^^ proceeding from above ^^
>>> script = """
        () => {
            return {
                width: document.documentElement.clientWidth,
                height: document.documentElement.clientHeight,
                deviceScaleFactor: window.devicePixelRatio,
            }
        }
    """
>>> val = html.render(script=script, reload=False)

>>> print(val)
{'width': 800, 'height': 600, 'deviceScaleFactor': 1}

>>> print(html.html)
<html><head></head><body><a href="https://httpbin.org"></a></body></html>

For using arender just pass async_=True to HTML.

# ^^ using above script ^^
>>> html = HTML(html=doc, async_=True)
>>> val = await html.arender(script=script, reload=False)
>>> print(val)
{'width': 800, 'height': 600, 'deviceScaleFactor': 1}

API Documentation

Main Classes

These classes are the main interface to httpx-html:

class httpx_html.HTML(*, session: Optional[BaseSession] = None, url: str = 'https://example.org/', html: _Html, default_encoding: Optional[str] = 'utf-8', async_: bool = False)[source]

An HTML document, ready for parsing.

Parameters

  • url – The URL from which the HTML originated, used for absolute_links.

  • html – HTML from which to base the parsing upon (optional).

  • default_encoding – Which encoding to default to.

All found links on page, in absolute form (learn more).

async arender(retries: int = 8, script: Optional[str] = None, wait: float = 0.2, scrolldown: bool = False, sleep: int = 0, reload: bool = True, timeout: Union[float, int] = 8.0, wait_until: Optional[Union[str, List[str]]] = None, keep_page: bool = False, cookies: list = [{}], send_cookies_session: bool = False)[source]

Async version of render. Takes same parameters.

property base_url

The base URL for the page. Supports the <base> tag (learn more).

property encoding

The encoding string to be used, extracted from the HTML and HTMLResponse headers.

find(selector: str = '*', *, containing: _Containing = None, clean: bool = False, first: bool = False, _encoding: str = None) → _Find

Given a CSS Selector, returns a list of Element objects or a single one.

Parameters

  • selector – CSS Selector to use.

  • clean – Whether or not to sanitize the found HTML of <script> and <style> tags.

  • containing – If specified, only return elements that contain the provided text.

  • first – Whether or not to return just the first result.

  • _encoding – The encoding format.

Example CSS Selectors:

  • a

  • a.someClass

  • a#someID

  • a[target=_blank]

See W3School’s CSS Selectors Reference for more details.

If first is True, only returns the first Element found.

property full_text

The full text content (including links) of the Element or HTML.

property html

Unicode representation of the HTML content

All found links on page, in as–is form.

property lxml

lxml representation of the Element or HTML.

next(fetch: bool = False, next_symbol: _NextSymbol = ['next', 'more', 'older']) → Optional[_Next][source]

Attempts to find the next page, if there is one. If fetch is True (default), returns HTML object of next page. If fetch is False, simply returns the next URL.

property pq

PyQuery representation of the Element or HTML.

property raw_html

Bytes representation of the HTML content.

render(retries: int = 8, script: Optional[str] = None, wait: float = 0.2, scrolldown: bool = False, sleep: int = 0, reload: bool = True, timeout: Union[float, int] = 8.0, wait_until: Optional[Union[str, List[str]]] = None, keep_page: bool = False, cookies: list = [{}], send_cookies_session: bool = False)[source]

Reloads the response in Chromium, and replaces HTML content with an updated version, with JavaScript executed.

Parameters

  • retries – The number of times to retry loading the page in Chromium.

  • script – JavaScript to execute upon page load (optional).

  • wait – The number of seconds to wait before loading the page, preventing timeouts (optional).

  • scrolldown – Integer, if provided, of how many times to page down.

  • sleep – Integer, if provided, of how many seconds to sleep after initial render.

  • reload – If False, content will not be loaded from the browser, but will be provided from memory.

  • timeout – Timeout for rendering, in seconds.

  • wait_until – When to consider the page loaded. Acceptable values are: load (default) domcontentloaded, networkidle0, networkidle1.

  • keep_page – If True will allow you to interact with the browser page through r.html.page.

  • send_cookies_session – If True send HTMLSession.cookies convert.

  • cookies – If not empty send cookies.

If scrolldown is specified, the page will scrolldown the specified number of times, after sleeping the specified amount of time (e.g. scrolldown=10, sleep=1).

If just sleep is provided, the rendering will wait n seconds, before returning.

If script is specified, it will execute the provided JavaScript at runtime. Example:

script = """
    () => {
        return {
            width: document.documentElement.clientWidth,
            height: document.documentElement.clientHeight,
            deviceScaleFactor: window.devicePixelRatio,
        }
    }
"""

Returns the return value of the executed script, if any is provided:

>>> r.html.render(script=script)
{'width': 800, 'height': 600, 'deviceScaleFactor': 1}

Warning: the first time you run this method, it will download Chromium into your home directory (~/.pyppeteer).

search(template: str) → parse.Result

Search the Element for the given Parse template.

Parameters

template – The Parse template to use.

search_all(template: str) → _Result

Search the Element (multiple times) for the given parse template.

Parameters

template – The Parse template to use.

property text

The text content of the Element or HTML.

xpath(selector: str, *, clean: bool = False, first: bool = False, _encoding: Optional[str] = None) → _XPath

Given an XPath selector, returns a list of Element objects or a single one.

Parameters

  • selector – XPath Selector to use.

  • clean – Whether or not to sanitize the found HTML of <script> and <style> tags.

  • first – Whether or not to return just the first result.

  • _encoding – The encoding format.

If a sub-selector is specified (e.g. //a/@href), a simple list of results is returned.

See W3School’s XPath Examples for more details.

If first is True, only returns the first Element found.

class httpx_html.Element(*, element, url: _Url, default_encoding: Optional[str] = None)[source]

An element of HTML.

Parameters

  • element – The element from which to base the parsing upon.

  • url – The URL from which the HTML originated, used for absolute_links.

  • default_encoding – Which encoding to default to.

All found links on page, in absolute form (learn more).

property attrs

Returns a dictionary of the attributes of the Element (learn more).

property base_url

The base URL for the page. Supports the <base> tag (learn more).

property encoding

The encoding string to be used, extracted from the HTML and HTMLResponse headers.

find(selector: str = '*', *, containing: _Containing = None, clean: bool = False, first: bool = False, _encoding: str = None) → _Find

Given a CSS Selector, returns a list of Element objects or a single one.

Parameters

  • selector – CSS Selector to use.

  • clean – Whether or not to sanitize the found HTML of <script> and <style> tags.

  • containing – If specified, only return elements that contain the provided text.

  • first – Whether or not to return just the first result.

  • _encoding – The encoding format.

Example CSS Selectors:

  • a

  • a.someClass

  • a#someID

  • a[target=_blank]

See W3School’s CSS Selectors Reference for more details.

If first is True, only returns the first Element found.

property full_text

The full text content (including links) of the Element or HTML.

property html

Unicode representation of the HTML content

All found links on page, in as–is form.

property lxml

lxml representation of the Element or HTML.

property pq

PyQuery representation of the Element or HTML.

property raw_html

Bytes representation of the HTML content.

search(template: str) → parse.Result

Search the Element for the given Parse template.

Parameters

template – The Parse template to use.

search_all(template: str) → _Result

Search the Element (multiple times) for the given parse template.

Parameters

template – The Parse template to use.

property text

The text content of the Element or HTML.

xpath(selector: str, *, clean: bool = False, first: bool = False, _encoding: Optional[str] = None) → _XPath

Given an XPath selector, returns a list of Element objects or a single one.

Parameters

  • selector – XPath Selector to use.

  • clean – Whether or not to sanitize the found HTML of <script> and <style> tags.

  • first – Whether or not to return just the first result.

  • _encoding – The encoding format.

If a sub-selector is specified (e.g. //a/@href), a simple list of results is returned.

See W3School’s XPath Examples for more details.

If first is True, only returns the first Element found.

Utility Functions

httpx_html.user_agent(style: Optional[Mapping] = None) → str[source]

Returns an apparently legit user-agent, if not requested one of a specific style. Defaults to a Chrome-style User-Agent.

HTML Sessions

These sessions are for making HTTP requests:

class httpx_html.HTMLSession(*, mock_browser: bool = True, verify: bool = True, browser_args: list = ['--no-sandbox'], proxies: Optional[Mapping[str, str]] = None)[source]
property auth

Authentication class used when none is passed at the request-level.

See also [Authentication][0].

[0]: /quickstart/#authentication

property base_url

Base URL to use when sending requests with relative URLs.

build_request(method: str, url: Union[httpx.URL, str], *, content: Optional[Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]]] = None, data: Optional[dict] = None, files: Optional[Union[Mapping[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]], Sequence[Tuple[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]]]]] = None, json: Optional[Any] = None, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], …], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None) → httpx.Request

Build and return a request instance.

  • The params, headers and cookies arguments

are merged with any values set on the client. * The url argument is merged with any base_url set on the client.

See also: [Request instances][0]

[0]: /advanced/#request-instances

close() → None[source]

If a browser was created close it first.

property cookies

Cookie values to include when sending requests.

delete(url: Union[httpx.URL, str], *, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a DELETE request.

Parameters: See httpx.request.

get(url: Union[httpx.URL, str], *, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a GET request.

Parameters: See httpx.request.

head(url: Union[httpx.URL, str], *, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a HEAD request.

Parameters: See httpx.request.

property headers

HTTP headers to include when sending requests.

property is_closed

Check if the client being closed

options(url: Union[httpx.URL, str], *, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send an OPTIONS request.

Parameters: See httpx.request.

property params

Query parameters to include in the URL when sending requests.

patch(url: Union[httpx.URL, str], *, content: Optional[Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]]] = None, data: Optional[dict] = None, files: Optional[Union[Mapping[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]], Sequence[Tuple[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]]]]] = None, json: Optional[Any] = None, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a PATCH request.

Parameters: See httpx.request.

post(url: Union[httpx.URL, str], *, content: Optional[Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]]] = None, data: Optional[dict] = None, files: Optional[Union[Mapping[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]], Sequence[Tuple[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]]]]] = None, json: Optional[Any] = None, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a POST request.

Parameters: See httpx.request.

put(url: Union[httpx.URL, str], *, content: Optional[Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]]] = None, data: Optional[dict] = None, files: Optional[Union[Mapping[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]], Sequence[Tuple[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]]]]] = None, json: Optional[Any] = None, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a PUT request.

Parameters: See httpx.request.

request(*args, **kwargs) → httpx_html.session.HTMLResponse

Build and send a request.

Equivalent to:

`python request = client.build_request(...) response = client.send(request, ...) `

See Client.build_request(), Client.send() and [Merging of configuration][0] for how the various parameters are merged with client-level configuration.

[0]: /advanced/#merging-of-configuration

send(request: httpx.Request, *, stream: bool = False, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a request.

The request is sent as-is, unmodified.

Typically you’ll want to build one with Client.build_request() so that any client-level configuration is merged into the request, but passing an explicit httpx.Request() is supported as well.

See also: [Request instances][0]

[0]: /advanced/#request-instances

stream(method: str, url: Union[httpx.URL, str], *, content: Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]] = None, data: dict = None, files: Union[Mapping[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]], Sequence[Tuple[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]]]] = None, json: Any = None, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]] = None, cookies: Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → Iterator[httpx.Response]

Alternative to httpx.request() that streams the response body instead of loading it into memory at once.

Parameters: See httpx.request.

See also: [Streaming Responses][0]

[0]: /quickstart#streaming-responses

class httpx_html.AsyncHTMLSession(loop=None, workers=None, mock_browser: bool = True, *args, **kwargs)[source]

An async consumable session.

property auth

Authentication class used when none is passed at the request-level.

See also [Authentication][0].

[0]: /quickstart/#authentication

property base_url

Base URL to use when sending requests with relative URLs.

build_request(method: str, url: Union[httpx.URL, str], *, content: Optional[Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]]] = None, data: Optional[dict] = None, files: Optional[Union[Mapping[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]], Sequence[Tuple[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]]]]] = None, json: Optional[Any] = None, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], …], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None) → httpx.Request

Build and return a request instance.

  • The params, headers and cookies arguments

are merged with any values set on the client. * The url argument is merged with any base_url set on the client.

See also: [Request instances][0]

[0]: /advanced/#request-instances

async close() → None[source]

If a browser was created close it first.

property cookies

Cookie values to include when sending requests.

delete(url: Union[httpx.URL, str], *, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a DELETE request.

Parameters: See httpx.request.

get(url: Union[httpx.URL, str], *, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a GET request.

Parameters: See httpx.request.

head(url: Union[httpx.URL, str], *, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a HEAD request.

Parameters: See httpx.request.

property headers

HTTP headers to include when sending requests.

property is_closed

Check if the client being closed

options(url: Union[httpx.URL, str], *, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send an OPTIONS request.

Parameters: See httpx.request.

property params

Query parameters to include in the URL when sending requests.

patch(url: Union[httpx.URL, str], *, content: Optional[Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]]] = None, data: Optional[dict] = None, files: Optional[Union[Mapping[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]], Sequence[Tuple[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]]]]] = None, json: Optional[Any] = None, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a PATCH request.

Parameters: See httpx.request.

post(url: Union[httpx.URL, str], *, content: Optional[Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]]] = None, data: Optional[dict] = None, files: Optional[Union[Mapping[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]], Sequence[Tuple[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]]]]] = None, json: Optional[Any] = None, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a POST request.

Parameters: See httpx.request.

put(url: Union[httpx.URL, str], *, content: Optional[Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]]] = None, data: Optional[dict] = None, files: Optional[Union[Mapping[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]], Sequence[Tuple[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]]]]] = None, json: Optional[Any] = None, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Optional[Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]]] = None, cookies: Optional[Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a PUT request.

Parameters: See httpx.request.

request(*args, **kwargs) → httpx_html.session.HTMLResponse[source]

Partial original request func and run it in a thread.

run(*coros)[source]

Pass in all the coroutines you want to run, it will wrap each one in a task, run it and wait for the result. Return a list with all results, this is returned in the same order coros are passed in.

send(request: httpx.Request, *, stream: bool = False, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → httpx.Response

Send a request.

The request is sent as-is, unmodified.

Typically you’ll want to build one with Client.build_request() so that any client-level configuration is merged into the request, but passing an explicit httpx.Request() is supported as well.

See also: [Request instances][0]

[0]: /advanced/#request-instances

stream(method: str, url: Union[httpx.URL, str], *, content: Union[str, bytes, Iterable[bytes], AsyncIterable[bytes]] = None, data: dict = None, files: Union[Mapping[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]], Sequence[Tuple[str, Union[IO[bytes], bytes, Tuple[Optional[str], Union[IO[bytes], bytes]], Tuple[Optional[str], Union[IO[bytes], bytes], Optional[str]]]]]] = None, json: Any = None, params: Optional[Union[httpx.QueryParams, Mapping[str, Union[str, int, float, bool, None, Sequence[Optional[Union[str, int, float, bool]]]]], List[Tuple[str, Optional[Union[str, int, float, bool]]]], Tuple[Tuple[str, Optional[Union[str, int, float, bool]]], ...], str, bytes]] = None, headers: Union[httpx.Headers, Dict[str, str], Dict[bytes, bytes], Sequence[Tuple[str, str]], Sequence[Tuple[bytes, bytes]]] = None, cookies: Union[httpx.Cookies, http.cookiejar.CookieJar, Dict[str, str], List[Tuple[str, str]]] = None, auth: Union[Tuple[Union[str, bytes], Union[str, bytes]], Callable[[httpx.Request], httpx.Request], httpx.Auth, None, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, allow_redirects: Union[bool, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>, timeout: Union[float, None, Tuple[Optional[float], Optional[float], Optional[float], Optional[float]], httpx.Timeout, httpx._client.UseClientDefault] = <httpx._client.UseClientDefault object>) → Iterator[httpx.Response]

Alternative to httpx.request() that streams the response body instead of loading it into memory at once.

Parameters: See httpx.request.

See also: [Streaming Responses][0]

[0]: /quickstart#streaming-responses

Indices and tables