Commit 823c490c authored Apr 30, 2023 by Markus Heiser

[mod] limiter: block requests from PetalBot

Block requests from PetalBlock.  Normally robots.txt is enough to stop
PetalBlock from making requests [1].  However, if SearXNG is offered below a
path (example.org/search), then the robots.txt is not available in the root
paths of the domain / subdomain.

[1] https://webmaster.petalsearch.com/site/petalbot



Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

parent 9967fbb7

searx/plugins/limiter.py

+1 −0

Original line number	Diff line number	Diff line
		@@ -37,6 +37,7 @@ block_user_agent = re.compile(
		# unmaintained Farside instances
		+ r'\|'
		+ re.escape(r'Mozilla/5.0 (compatible; Farside/0.1.0; +https://farside.link)')
		+ '\|.PetalBot.'
		+ r')'
		)