SSealbook
Sealbook / Open source / Bases de datos e ingeniería de datos

gnemoug/distribute_crawler

使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现

GitHub ↗

Licencia y mantenimiento

Sin licencia indicada: por defecto nadie puede reutilizar el código. Pregunta a los autores antes de usarlo en un producto.

Último cambio de código hace 9 años.

Proyectos similares

RepositorioEstrellasLenguajeLicenciaActividad
1supabase/supabaseThe Postgres development platform. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.★ 111.7kTypeScriptApache-2.0Activo
2django/djangoThe Web framework for perfectionists with deadlines.★ 91.4kPythonBSD-3-ClauseActivo
3doocs/advanced-java😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲：涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识★ 79.2kJavaCC-BY-SA-4.0Activo
4elastic/elasticsearchFree and Open Source, Distributed, RESTful Search Engine★ 78.3kJavaOtherActivo
5obsproject/obs-studioOBS Studio - Free and open source software for live streaming and screen recording★ 77.7kCGPL-2.0Activo
6redis/redisFor developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data stru★ 76.9kCOtherActivo
7pathwaycom/pathwayPython ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.★ 62.1kPythonOtherActivo
8meilisearch/meilisearchA lightning-fast search engine API bringing AI-powered hybrid search to your sites and applications.★ 59.6kRustOtherActivo