Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Unverified Commit 85034b49 authored by Noémi Ványi's avatar Noémi Ványi Committed by GitHub
Browse files

Remove `httpx` and use `requests` instead (#3305)

## What does this PR do?

This PR prepares for removing `httpx`, and reverts back to `requests`.

## Why is this change important?

`httpx` hasn't proven itself to be faster or better than `requests`. On the other
hand it has caused issues on Windows.

=============================================
Please update your environment to use requests instead of httpx.
=============================================
parent 210e59c6
Loading
Loading
Loading
Loading
+13 −51
Original line number Diff line number Diff line
@@ -130,12 +130,14 @@ Global Settings
       request_timeout : 2.0        # default timeout in seconds, can be override by engine
       # max_request_timeout: 10.0  # the maximum timeout in seconds
       useragent_suffix : ""        # informations like an email address to the administrator
       pool_connections : 100       # Maximum number of allowable connections, or None for no limits. The default is 100.
       pool_maxsize : 10            # Number of allowable keep-alive connections, or None to always allow. The default is 10.
       enable_http2: True           # See https://www.python-httpx.org/http2/
       pool_connections : 100       # Number of different hosts
       pool_maxsize : 10            # Number of simultaneous requests by host
   # uncomment below section if you want to use a proxy
   #    proxies:
   #        all://:
   #        http:
   #            - http://proxy1:8080
   #            - http://proxy2:8080
   #        https:
   #            - http://proxy1:8080
   #            - http://proxy2:8080
   # uncomment below section only if you have more than one network interface
@@ -143,7 +145,6 @@ Global Settings
   #    source_ips:
   #        - 1.1.1.1
   #        - 1.1.1.2
   #        - fe80::/126


``request_timeout`` :
@@ -156,46 +157,20 @@ Global Settings
  Suffix to the user-agent searx uses to send requests to others engines.  If an
  engine wish to block you, a contact info here may be useful to avoid that.

``keepalive_expiry``:
  Number of seconds to keep a connection in the pool. By default 5.0 seconds.

.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying
.. _requests proxies: https://requests.readthedocs.io/en/latest/user/advanced/#proxies
.. _PySocks: https://pypi.org/project/PySocks/

``proxies`` :
  Define one or more proxies you wish to use, see `httpx proxies`_.
  Define one or more proxies you wish to use, see `requests proxies`_.
  If there are more than one proxy for one protocol (http, https),
  requests to the engines are distributed in a round-robin fashion.

  - Proxy: `see <https://2.python-requests.org/en/latest/user/advanced/#proxies>`__.
  - SOCKS proxies are also supported: `see <https://2.python-requests.org/en/latest/user/advanced/#socks>`__

``source_ips`` :
  If you use multiple network interfaces, define from which IP the requests must
  be made. Example:

  * ``0.0.0.0`` any local IPv4 address.
  * ``::`` any local IPv6 address.
  * ``192.168.0.1``
  * ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses
  * ``fe80::60a2:1691:e5a2:ee1f``
  * ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network.
  * ``[ 192.168.0.1, fe80::/126 ]``

``retries`` :
  Number of retry in case of an HTTP error.
  On each retry, searx uses an different proxy and source ip.

``retry_on_http_error`` :
  Retry request on some HTTP status code.

  Example:

  * ``true`` : on HTTP status code between 400 and 599.
  * ``403`` : on HTTP status code 403.
  * ``[403, 429]``: on HTTP status code 403 and 429.

``enable_http2`` :
  Enable by default. Set to ``False`` to disable HTTP/2.

``max_redirects`` :
  30 by default. Maximum redirect before it is an error.
  be made. This parameter is ignored when ``proxies`` is set.


``locales:``
@@ -241,13 +216,6 @@ Engine settings
     api_key : 'apikey'
     disabled : True
     language : en_US
     #enable_http: False
     #enable_http2: False
     #retries: 1
     #retry_on_http_error: True # or 403 or [404, 429]
     #max_connections: 100
     #max_keepalive_connections: 10
     #keepalive_expiry: 5.0
     #proxies:
     #    http:
     #        - http://proxy1:8080
@@ -302,12 +270,6 @@ Engine settings
``display_error_messages`` : default ``True``
  When an engine returns an error, the message is displayed on the user interface.

``network``: optional
  Use the network configuration from another engine.
  In addition, there are two default networks:
  * ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses)
  * ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses)

.. note::

   A few more options are possible, but they are pretty specific to some
+1 −2
Original line number Diff line number Diff line
@@ -107,8 +107,7 @@ fi
export DOCS_BUILD

buildenv() {
    SEARX_DEBUG=1 pyenv.cmd python utils/build_env.py 2>&1 \
        | prefix_stdout "${_Blue}BUILDENV${_creset}  "
    SEARX_DEBUG=1 pyenv.cmd python utils/build_env.py 2>&1
    return "${PIPESTATUS[0]}"
}

+0 −1
Original line number Diff line number Diff line
@@ -17,4 +17,3 @@ sphinx-tabs==3.2.0
sphinxcontrib-programoutput==0.17
sphinx-autobuild==2021.3.14
linuxdoc==20211220
aiounittest==1.4.1
+4 −7
Original line number Diff line number Diff line
certifi==2022.5.18.1
Brotli==1.0.9
babel==2.9.1
certifi==2022.5.18.1
flask-babel==2.0.0
flask==2.1.1
jinja2==3.1.2
langdetect==1.0.9
lxml==4.9.0
pygments==2.8.0
python-dateutil==2.8.2
pyyaml==6.0
httpx[http2]==0.23.0
Brotli==1.0.9
uvloop==0.16.0; python_version >= '3.7'
uvloop==0.14.0; python_version < '3.7'
httpx-socks[asyncio]==0.7.4
langdetect==1.0.9
requests[socks]==2.28.1
setproctitle==1.2.2
+3 −5
Original line number Diff line number Diff line
@@ -20,12 +20,10 @@ from lxml import etree
from json import loads
from urllib.parse import urlencode

from httpx import HTTPError

from requests import RequestException

from searx import settings
from searx.data import ENGINES_LANGUAGES
from searx.network import get as http_get
from searx.poolrequests import get as http_get
from searx.exceptions import SearxEngineResponseException


@@ -154,5 +152,5 @@ def search_autocomplete(backend_name, query, lang):

    try:
        return backend(query, lang)
    except (HTTPError, SearxEngineResponseException):
    except (RequestException, SearxEngineResponseException):
        return []
Loading