pip3 install scrapy
pip3 install bs4
pip3 install lxml
create new topic in kafka:
bin/kafka-topics.sh --create --partitions 1 --replication-factor 1 --topic x_news_1 --bootstrap-server localhost:9092
Run consumer kafka in package: xnews.kafkaconsumer.consumer.
cd to crawler folder and run command:
scrapy crawl news
This repo contains a powerful tool made using python which is used to visualize, analyse and finally assess the quality of the product depending upon the given observations
Impyute is a library of missing data imputation algorithms. This library was designed to be super lightweight, here's a sneak peak at what impyute can do.
AWS Glue ETL Code Samples This repository has samples that demonstrate various aspects of the new AWS Glue service, as well as various AWS Glue utilit
Rent Houses Germany - GCP Pipeline Project: The goal of the project is to extract data about house rentals in Germany, store, process and analyze it u
A Numba-based two-point correlation function (2PCF) calculator using a grid decomposition. Like Corrfunc, but written in Numba, with simplicity and hackability in mind.
Find relative paths from a project root directory Finding project directories in Python (data science) projects, just like there R here and rprojroot
This app is based on the Dash Enterprise App Gallery "Oil & Gas Wells" example. For more information and more apps see: Dash App Gallery See the Dash
A package to aggregate gridded data in xarray to polygons in geopandas using area-weighting from the relative area overlaps between pixels and polygons. Check out the binder link above for a sample c
PostQF Copyright © 2022 Ralph Seichter PostQF is a user-friendly Postfix queue data filter which operates on data produced by postqueue -j. See the ma
Lale is a Python library for semi-automated data science. Lale makes it easy to automatically select algorithms and tune hyperparameters of pipelines that are compatible with scikit-learn, in a type-
Pypeln Pypeln (pronounced as "pypeline") is a simple yet powerful Python library for creating concurrent data pipelines. Main Features Simple: Pypeln
TheMachineScraper 🐱👤 is a tool made purely for analysing machine data for any reason.
Stock-Market-AI-Analysis I dont mind anyone using this code but please give me credit A program that uses an API and a AI model to get info of stocks
Nebula is an extremely-fast end-to-end interactive big data analytics solution. Nebula is designed as a high-performance columnar data storage and tabular OLAP engine.
Falcon: Interactive Visual Analysis for Big Data Crossfilter millions of records without latencies. This project is work in progress and not documente
Analysing aggregated UK polling data The tweet collection & storage pipeline used in email-service is used to also collect tweets from @britainelects.
Quickstart Hyenas is a curde Hy handle written on top of Pandas API to allow for more elegant access to data-scientist's powerhouse that is Pandas. In
CaterApp is a cross platform, remotely data sharing tool created for sharing files in a quick and secured manner. It is aimed to integrate this tool with several more features including providing a U
Introduction This repository is to show you how to integrate Zeppelin with Airflow. The philosophy behind the ingtegration is to make the transition f
A part of HyRiver software stack that provides access to NHD+ V2 data through NLDI and WaterData web services
8 Oct 18, 2022
329 Dec 05, 2022
1.2k Jan 03, 2023
2 Jan 20, 2022
3 Aug 24, 2022
102 Nov 16, 2022
1 Nov 08, 2021
42 Nov 09, 2022
11 Nov 24, 2022
293 Dec 29, 2022
1.4k Dec 31, 2022
5 Dec 01, 2022
1 Dec 17, 2021
131 Dec 26, 2022
803 Dec 27, 2022
0 Dec 22, 2021
4 Sep 05, 2022
3 Jun 27, 2021
11 Dec 30, 2022
23 Dec 14, 2022