Persine is an automated tool to study and reverse-engineer algorithmic recommendation systems.

Overview

Documentation Status

Persine, the Persona Engine

Persine is an automated tool to study and reverse-engineer algorithmic recommendation systems. It has a simple interface and encourages reproducible results. You tell Persine to drive around YouTube and it gives back a spreadsheet of what else YouTube suggests you watch!

Persine => Pers[ona Eng]ine

For example!

People have suggested that if you watch a few lightly political videos, YouTube starts suggesting more and more extreme content – but does it really?

The theory is difficult to test since it involves a lot of boring clicking and YouTube already knows what you usually watch. Persine to the rescue!

  1. Persine starts a new fresh-as-snow Chrome
  2. You provide a list of videos to watch and buttons to click (like, dislike, "next up" etc)
  3. As it watches and clicks more and more, YouTube customizes and customizes
  4. When you're all done, Persine will save your winding path and the video/playlist/channel recommendations to nice neat CSV files.

Beyond analysis, these files can be used to repeat the experiment again later, seeing if recommendations change by time, location, user history, etc.

If you didn't quite get enough data, don't worry – you can resume your exploration later, picking up right where you left off. Since each "persona" is based on Chrome profiles, all your cookies and history will be safely stored until your next run.

An actual example

See Persine in action on Google Colab.

Includes a few examples for analysis, too.

Installation

pip install persine

Persine will automatically install Selenium and BeautifulSoup for browsing/scraping, pandas for data analysis, and pillow for processing screenshots.

You will need to manually install chromedriver to allow Selenium to control Chrome. See details here

Quickstart

In this example, we start a new session by visiting a YouTube video and clicking the "next up" video three times to see where it leads us. We then save the results for later analysis.

from persine import PersonaEngine

engine = PersonaEngine(headless=False)

with engine.persona() as persona:
    persona.run("https://www.youtube.com/watch?v=hZw23sWlyG0")
    persona.run("youtube:next_up#3")
    persona.history.to_csv("history.csv")
    persona.recommendations.to_csv("recs.csv")

We turn off headless mode because it's fun to watch!

More examples, more features, more everything

Find the complete documentation here

Owner
Jonathan Soma
baby data journo wrangler @ledeprogram + @littlecolumns, cat wrangler @cat-republic
Jonathan Soma
Use Jupyter Notebooks to demonstrate how to build a Recommender with Apache Spark & Elasticsearch

Recommendation engines are one of the most well known, widely used and highest value use cases for applying machine learning. Despite this, while there are many resources available for the basics of

International Business Machines 793 Dec 18, 2022
A recommendation system for suggesting new books given similar books.

Book Recommendation System A recommendation system for suggesting new books given similar books. Datasets Dataset Kaggle Dataset Notebooks goodreads-E

Sam Partee 2 Jan 06, 2022
Fast Python Collaborative Filtering for Implicit Feedback Datasets

Implicit Fast Python Collaborative Filtering for Implicit Datasets. This project provides fast Python implementations of several different popular rec

Ben Frederickson 3k Dec 31, 2022
QRec: A Python Framework for quick implementation of recommender systems (TensorFlow Based)

QRec is a Python framework for recommender systems (Supported by Python 3.7.4 and Tensorflow 1.14+) in which a number of influential and newly state-of-the-art recommendation models are implemented.

Yu 1.4k Dec 27, 2022
This library intends to be a reference for recommendation engines in Python

Crab - A Python Library for Recommendation Engines

Marcel Caraciolo 85 Oct 04, 2021
A PyTorch implementation of "Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information" (WSDM 2021)

FairGNN A PyTorch implementation of "Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information" (

31 Jan 04, 2023
A tensorflow implementation of the RecoGCN model in a CIKM'19 paper, titled with "Relation-Aware Graph Convolutional Networks for Agent-Initiated Social E-Commerce Recommendation".

This repo contains a tensorflow implementation of RecoGCN and the experiment dataset Running the RecoGCN model python train.py Example training outp

xfl15 30 Nov 25, 2022
ToR[e]cSys is a PyTorch Framework to implement recommendation system algorithms

ToR[e]cSys is a PyTorch Framework to implement recommendation system algorithms, including but not limited to click-through-rate (CTR) prediction, learning-to-ranking (LTR), and Matrix/Tensor Embeddi

LI, Wai Yin 90 Oct 08, 2022
A Library for Field-aware Factorization Machines

Table of Contents ================= - What is LIBFFM - Overfitting and Early Stopping - Installation - Data Format - Command Line Usage - Examples -

1.6k Dec 05, 2022
Movie Recommender System

Movie-Recommender-System Movie-Recommender-System is a web application using which a user can select his/her watched movie from list and system will r

1 Jul 14, 2022
Temporal Meta-path Guided Explainable Recommendation (WSDM2021)

Temporal Meta-path Guided Explainable Recommendation (WSDM2021) TMER Code of paper "Temporal Meta-path Guided Explainable Recommendation". Requirement

Yicong Li 13 Nov 30, 2022
Learning Fair Representations for Recommendation: A Graph-based Perspective, WWW2021

FairGo WWW2021 Learning Fair Representations for Recommendation: A Graph-based Perspective As a key application of artificial intelligence, recommende

lei 39 Oct 26, 2022
An Efficient and Effective Framework for Session-based Social Recommendation

SEFrame This repository contains the code for the paper "An Efficient and Effective Framework for Session-based Social Recommendation". Requirements P

Tianwen CHEN 23 Oct 26, 2022
Self-supervised Graph Learning for Recommendation

SGL This is our Tensorflow implementation for our SIGIR 2021 paper: Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian,and Xing

151 Dec 20, 2022
Cloud-based recommendation system

This project is based on cloud services to create data lake, ETL process, train and deploy learning model to implement a recommendation system.

Yi Ding 1 Feb 02, 2022
Real time recommendation playground

concierge A continuous learning collaborative filter1 deployed with a light web server2. Distributed updates are live (real time pubsub + delta traini

Mark Essel 16 Nov 07, 2022
Spark-movie-lens - An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset

A scalable on-line movie recommender using Spark and Flask This Apache Spark tutorial will guide you step-by-step into how to use the MovieLens datase

Jose A Dianes 794 Dec 23, 2022
Approximate Nearest Neighbors in C++/Python optimized for memory usage and loading/saving to disk

Annoy Annoy (Approximate Nearest Neighbors Oh Yeah) is a C++ library with Python bindings to search for points in space that are close to a given quer

Spotify 10.6k Jan 01, 2023
RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystems

RecSim NG, a probabilistic platform for multi-agent recommender systems simulation. RecSimNG is a scalable, modular, differentiable simulator implemented in Edward2 and TensorFlow. It offers: a power

Google Research 110 Dec 16, 2022
Cross-Domain Recommendation via Preference Propagation GraphNet.

PPGN Codes for CIKM 2019 paper Cross-Domain Recommendation via Preference Propagation GraphNet. Citation Please cite our paper if you find this code u

Information Retrieval Group, Wuhan University, China 20 Dec 15, 2022