crawling-python

🕷 Automatically detects changes to the official Telegram sites, beta clients, MTProto servers and mini apps

Enhance your Scrap Mechanic experience with advanced cheats, unlimited resources, and instant building hacks for 2026.

Scalable Python web scraping scripts for +40 popular domains

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

Extract data from websites using a fast, intelligent web scraping library designed for modern HTML structures and dynamic content.

List of libraries, tools and APIs for web scraping and data processing.

python기반 Webapp 개발을 위해 다양한 도구를 이용하는 것을 정리 합니다. ( html5, crawling, sqlalchemy, openai, flask 등)

🚀 OFFICIAL STARTER TEMPLATE FOR BOTASAURUS SCRAPING FRAMEWORK 🤖

Async Python web crawler with a live sci-fi terminal dashboard, robots.txt compliance, and built-in rate limiting.

Handle custom proxy headers when making HTTPS requests through proxies in scrapy

Terminal Port Crawler or TPC for short; it crawls ports based on what your looking for. TPC Comes in CLI and GUI flavors. Additionally it was built using python and textual for the flavor

A not very lightweight speedy web crawler built in python

FlexiCrawl is a simple site and file crawler, storing discovered pages and relationships in PostgreSQL. It exposes an API that can be consumed by the front-end.

Free Facebook pages MetaData Scraping Library - Unlimited Calls

Transform Web Content into LLM-Ready Data

API to parse tibia.com content into python objects.

A project about crawling data from websites (lyrics and category)

All the files necessary to create a free, locally hosted RAG chatbot

Python-based web crawling script with randomized intervals, user-agent rotation, and proxy server IP rotation to outsmart website bots and prevent blocking.