`SERPent`

SERP results scrapping

SERPent exposes an unified API to query SERP (Search Engine Result Pages) for a few common search engines, namely:

DuckDuckGo
Brave
Bing
Google Patents
arXiv
Google

The application uses the playwright library to control a headless web browser, to simulate normal user activity, to fool the anti-bot measures often present on those sites. See the /serp/ endpoints for search results scrapping.

Website sources scrapping

SERPent also exposes a few endpoints to scrap the contents of certain sources (patents, scholar). See the /scrap/ endpoints for supported website sources scrapping.