crawler-python

This was inspired by a project belonging to a dear friend of mine.

A python arxiv paper fetcher based on arxiv official API.

爬虫练手项目（几个音乐平台）

GooglePlayStore Crawler

Automation of IOC and POST collection and search for CTI purposes via X (formerly Twitter).

A web crawler that is easy to use and follows politeness policies.

Web crawler for emails

用python爬取豆瓣电影top250排行榜的榜单数据。

This set of scripts extracts official speech transcripts from PCOO (https://pcoo.gov.ph/presidential-speech/) and unofficial ones from Rappler using Selenium, BeautifulSoup, and Requests libraries

武汉理工大学校园网络爬虫项目，专注于自动化抓取校园网信息

马哥数据采集工具产品主页，汇总抖音、快手、小红书、微博、蒲公英和 YouTube 采集软件。

Crawl any site, starting from the sitemap and convert entire website into Markdown, making it easy for the LLMs to learn

In this project a webcrawler is developed to extract the cookies that are being set by websites using Selenium. This webcrawler searches for urls in your .csv file and looks for cookies in each one and exports these cookies along with their domain in an excel or csv file. it also intracts with websites to produce more possible cookies.

Crawler for probing popular domains for machine-readable, callable, commerce, and payment surfaces.

Zhi-Zhu is a multithreaded spidering script that recursively searches base webpages and all urls appearing in it, for specific (regex) words.

A powerful tool that crawls documentation websites and generates a clean, well-formatted markdown document. Built with FastAPI and support for multiple LLM providers (DeepSeek and Groq).

Python data science project developed js at the end of Unit 35 (Computer Science Module) of the Trybe's Web Development course

Automatically scraps and collects olx apartment data every 15 days

dysdera web crawler