Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Unverified Commit 4829a76a authored by Lukas van den Berk's avatar Lukas van den Berk Committed by GitHub
Browse files

Created new plugin type custom_results. Added new plugin bang_redirect (#2027) 

* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Refactored getting search query. Also changed bang operator to ! and is now working.

* Removed prints

* Removed temporary bangs_redirect.js file. Updated plugin documentation

* Added unit test for the bangs plugin

* Fixed a unit test and added 2 more for bangs plugin

* Changed back to default settings.yml

* Added myself to AUTHORS.rst

* Refacored working of custom plugin.

* Refactored _get_bangs_data from list to dict to improve search speed.

* Decoupled bangs plugin from webserver with redirect_url

* Refactored bangs unit tests

* Fixed unit test bangs. Removed dubbel parsing in bangs.py

* Removed a dumb print statement

* Refactored bangs plugin to core engine.

* Removed bangs plugin.

* Refactored external bangs unit tests from plugin to core.

* Removed custom_results/bangs documentation from plugins.rst

* Added newline in settings.yml so the PR stays clean.

* Changed searx/plugins/__init__.py back to the old file

* Removed newline search.py

* Refactored get_external_bang_operator from utils to external_bang.py

* Removed unnecessary import form test_plugins.py

* Removed _parseExternalBang and _isExternalBang from query.py

* Removed get_external_bang_operator since it was not necessary

* Simplified external_bang.py

* Simplified external_bang.py

* Moved external_bangs unit tests to test_webapp.py. Fixed return in search with external_bang

* Refactored query parsing to unicode to support python2

* Refactored query parsing to unicode to support python2

* Refactored bangs plugin to core engine.

* Refactored search parameter to search_query in external_bang.py
parent c21220c6

AUTHORS.rst

+1 −0
Original line number Diff line number Diff line
@@ -124,3 +124,4 @@ generally made searx better: 
- @CaffeinatedTech

- Robin Schneider @ypid

- @splintah

- Lukas van den Berk @lukasvdberk

docs/dev/plugins.rst

+8 −0
Original line number Diff line number Diff line
@@ -30,6 +30,14 @@ Example plugin 
       ctx['search'].suggestions.add('example')

       return True



Register your plugin

====================



To enable your plugin register your plugin in

searx > plugin > __init__.py.

And at the bottom of the file add your plugin like.

``plugins.register(name_of_python_file)``



Plugin entry points

===================

searx/data/bangs.json

0 → 100644
+75351 −0

File added.

Preview size limit exceeded, changes collapsed.

searx/external_bang.py

0 → 100644
+43 −0
Original line number Diff line number Diff line 
import json

from os.path import join



from searx import searx_dir



# bangs data coming from the following url convert to json with

# https://raw.githubusercontent.com/jivesearch/jivesearch/master/bangs/bangs.toml

# https://pseitz.github.io/toml-to-json-online-converter/

# NOTE only use the get_bang_url



bangs_data = {}

with open(join(searx_dir, 'data/bangs.json')) as json_file:

    for bang in json.load(json_file)['bang']:

        for trigger in bang["triggers"]:

            bangs_data[trigger] = {x: y for x, y in bang.items() if x != "triggers"}





def get_bang_url(search_query):

    """

    Redirects if the user supplied a correct bang search.

    :param search_query: This is a search_query object which contains preferences and the submitted queries.

    :return: None if the bang was invalid, else a string of the redirect url.

    """



    if search_query.external_bang:

        query = search_query.query.decode('utf-8', 'ignore')

        bang = _get_bang(search_query.external_bang)



        if bang and query:

            # TODO add region support.

            bang_url = bang["regions"]["default"]



            return bang_url.replace("{{{term}}}", query)

    return None





def _get_bang(user_bang):

    """

    Searches if the supplied user bang is available. Returns None if not found.

    :param user_bang: The parsed user bang. For example yt

    :return: Returns a dict with bangs data (check bangs_data.json for the structure)

    """

    return bangs_data.get(user_bang)

searx/query.py

+9 −2
Original line number Diff line number Diff line
@@ -44,10 +44,11 @@ class RawTextQuery(object): 
        self.engines = []

        self.languages = []

        self.timeout_limit = None

        self.external_bang = None

        self.specific = False



    # parse query, if tags are set, which

    # change the serch engine or search-language

    # change the search engine or search-language

    def parse_query(self):

        self.query_parts = []
@@ -120,6 +121,11 @@ class RawTextQuery(object): 
                        self.languages.append(lang)

                        parse_next = True



            # external bang

            if query_part[0:2] == "!!":

                self.external_bang = query_part[2:]

                parse_next = True

                continue

            # this force a engine or category

            if query_part[0] == '!' or query_part[0] == '?':

                prefix = query_part[1:].replace('-', ' ').replace('_', ' ')
@@ -178,7 +184,7 @@ class SearchQuery(object): 
    """container for all the search parameters (query, language, etc...)"""



    def __init__(self, query, engines, categories, lang, safesearch, pageno, time_range,

                 timeout_limit=None, preferences=None):

                 timeout_limit=None, preferences=None, external_bang=None):

        self.query = query.encode('utf-8')

        self.engines = engines

        self.categories = categories
@@ -188,6 +194,7 @@ class SearchQuery(object): 
        self.time_range = None if time_range in ('', 'None', None) else time_range

        self.timeout_limit = timeout_limit

        self.preferences = preferences

        self.external_bang = external_bang



    def __str__(self):

        return str(self.query) + ";" + str(self.engines)