BErt-like Neurophysiological Data Representation

Related tags

Data AnalysisBENDR
Overview

BENDR

BErt-like Neurophysiological Data Representation

A picture of Bender from Futurama

This repository contains the source code for reproducing, or extending the BERT-like self-supervision pre-training for EEG data from the article:

BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data

To run these scripts, you will need to use the DN3 project. We will try to keep this updated so that it works with the latest DN3 release. If you are just looking for the BENDR model, and don't need to reproduce the article results per se, BENDR will be (or maybe already is if I forgot to update it here) integrated into DN3, in which case I would start there.

Currently, we recommend version 0.2. Feel free to open an issue if you are having any trouble.

More extensive instructions are upcoming, but in essence you will need to either:

a)  Download the TUEG dataset and pre-train new encoder and contextualizer weights, _or_
b)  Use the [pre-trained model weights](https://github.com/SPOClab-ca/BENDR/releases/tag/v0.1-alpha)

Once you have a pre-trained model:

1) Add the paths of the pre-trained weights to configs/downstream.yml
2) Edit paths to local copies of your datasets in configs/downstream_datasets.yml
3) Run downstream.sh

Comments
  • about the loss function

    about the loss function

    Very appreciate for your contribution.i am really interested in the self training in EEG. The only question is about calculating loss function. In your paper, The calculation of the denominator uses cosine similarity between the output of the transformer and the 20 distractors and the input of the transformer. However, in the code, the calculation of the denominator uses cosine similarity between the input of the transformer and the 20 distractors, and the output of the transformer. In other word, the output and the input switch positions. Are both the calculation approaches the same? Or why did you change the calculation approache in the code? Thanks!

    opened by stickOverCarrot 2
  • About deploy downstream.yml and downstream_datasets.yml

    About deploy downstream.yml and downstream_datasets.yml

    Tranks for supplying your code. But when I follow your markdown, I meet some problems image

    This is my project files image

    This is my downstream.yml image

    This is my downstream_datasets.yml image

    opened by YoloEliwa 1
  • Pre-trained weights?

    Pre-trained weights?

    Not an issue per se, but you state the pre-trained weights for your paper are available in this repo, yet I have had a good look around and I haven't found them, nor a means of downloading them. Please can you let me know where I could find them? I'm really keen to try out this exciting architecture you've put together!

    opened by SgtWhiskeyjack 1
  • result_tracking module

    result_tracking module

    There's a reference that's in the module import: downstream.py from result_tracking import ThinkerwiseResultTracker that looks like some type of tracking code for experiments?

    opened by bencten 1
  • dropout should change

    dropout should change

    Iteration: 4%|▍ | 13/330 [00:36<16:00, 3.03s/batches, bac=0.5, Accuracy=0.51, loss=0.695, lr=1.47e-6]D:\Anaconda\envs\LGG\lib\site-packages\torch\nn\functional.py:1338: UserWarning: dropout2d: Received a 3D input to dropout2d and assuming that channel-wise 1D dropout behavior is desired - input is interpreted as shape (N, C, L), where C is the channel dim. This behavior will change in a future release to interpret the input as one without a batch dimension, i.e. shape (C, H, W). To maintain the 1D channel-wise dropout behavior, please switch to using dropout1d instead. warnings.warn("dropout2d: Received a 3D input to dropout2d and assuming that channel-wise "

    opened by zy2021314 0
  • A more detailed explanation

    A more detailed explanation

    We need to use your code for research, may I ask when you can provide detailed explanation, because we have some difficulties in understanding the code without detailed explanation.

    opened by EchizenMike 0
  • preload in downstream.yml

    preload in downstream.yml

    In the "downstream.yml" file, what is the function of the "preload"? What's mean if I specify "preload: True" or "preload: False"?

    Thank you in advance

    opened by frannfuri 0
Releases(v0.1-alpha)
Owner
GitHub Repository
Geospatial data-science analysis on reasons behind delay in Grab ride-share services

Grab x Pulis Detailed analysis done to investigate possible reasons for delay in Grab services for NUS Data Analytics Competition 2022, to be found in

Keng Hwee 6 Jun 07, 2022
API>local_db>AWS_RDS - Disclaimer! All data used is for educational purposes only.

APIlocal_dbAWS_RDS Disclaimer! All data used is for educational purposes only. ETL pipeline diagram. Aim of project By creating a fully working pipe

0 Apr 25, 2022
Leverage Twitter API v2 to analyze tweet metrics such as impressions and profile clicks over time.

Tweetmetric Tweetmetric allows you to track various metrics on your most recent tweets, such as impressions, retweets and clicks on your profile. The

Mathis HAMMEL 29 Oct 18, 2022
Implementation in Python of the reliability measures such as Omega.

OmegaPy Summary Simple implementation in Python of the reliability measures: Omega Total, Omega Hierarchical and Omega Hierarchical Total. Name Link O

Rafael Valero Fernández 2 Apr 27, 2022
Tokyo 2020 Paralympics, Analytics

Tokyo 2020 Paralympics, Analytics Thanks for checking out my app! It was built entirely using matplotlib and Tokyo 2020 Paralympics data. This applica

Petro Ivaniuk 1 Nov 18, 2021
A Pythonic introduction to methods for scaling your data science and machine learning work to larger datasets and larger models, using the tools and APIs you know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

This tutorial's purpose is to introduce Pythonistas to methods for scaling their data science and machine learning work to larger datasets and larger models, using the tools and APIs they know and lo

Coiled 102 Nov 10, 2022
wikirepo is a Python package that provides a framework to easily source and leverage standardized Wikidata information

Python based Wikidata framework for easy dataframe extraction wikirepo is a Python package that provides a framework to easily source and leverage sta

Andrew Tavis McAllister 35 Jan 04, 2023
A Python package for the mathematical modeling of infectious diseases via compartmental models

A Python package for the mathematical modeling of infectious diseases via compartmental models. Originally designed for epidemiologists, epispot can be adapted for almost any type of modeling scenari

epispot 12 Dec 28, 2022
Show you how to integrate Zeppelin with Airflow

Introduction This repository is to show you how to integrate Zeppelin with Airflow. The philosophy behind the ingtegration is to make the transition f

Jeff Zhang 11 Dec 30, 2022
OpenARB is an open source program aiming to emulate a free market while encouraging players to participate in arbitrage in order to increase working capital.

Overview OpenARB is an open source program aiming to emulate a free market while encouraging players to participate in arbitrage in order to increase

Tom 3 Feb 12, 2022
Using Python to scrape some basic player information from www.premierleague.com and then use Pandas to analyse said data.

PremiershipPlayerAnalysis Using Python to scrape some basic player information from www.premierleague.com and then use Pandas to analyse said data. No

5 Sep 06, 2021
A probabilistic programming library for Bayesian deep learning, generative models, based on Tensorflow

ZhuSuan is a Python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and

Tsinghua Machine Learning Group 2.2k Dec 28, 2022
songplays datamart provide details about the musical taste of our customers and can help us to improve our recomendation system

Songplays User activity datamart The following document describes the model used to build the songplays datamart table and the respective ETL process.

Leandro Kellermann de Oliveira 1 Jul 13, 2021
Top 50 best selling books on amazon

It's a dashboard that shows the detailed information about each book in the top 50 best selling books on amazon over the last ten years

Nahla Tarek 1 Nov 18, 2021
Statistical Rethinking course winter 2022

Statistical Rethinking (2022 Edition) Instructor: Richard McElreath Lectures: Uploaded Playlist and pre-recorded, two per week Discussion: Online, F

Richard McElreath 3.9k Dec 31, 2022
Statistical Rethinking: A Bayesian Course Using CmdStanPy and Plotnine

Statistical Rethinking: A Bayesian Course Using CmdStanPy and Plotnine Intro This repo contains the python/stan version of the Statistical Rethinking

Andrés Suárez 3 Nov 08, 2022
Exploratory Data Analysis for Employee Retention Dataset

Exploratory Data Analysis for Employee Retention Dataset Employee turn-over is a very costly problem for companies. The cost of replacing an employee

kana sudheer reddy 2 Oct 01, 2021
Minimal working example of data acquisition with nidaqmx python API

Data Aquisition using NI-DAQmx python API Based on this project It is a minimal working example for data acquisition using the NI-DAQmx python API. It

Pablo 1 Nov 05, 2021
Vectorizers for a range of different data types

Vectorizers for a range of different data types

Tutte Institute for Mathematics and Computing 69 Dec 29, 2022
A Python package for modular causal inference analysis and model evaluations

Causal Inference 360 A Python package for inferring causal effects from observational data. Description Causal inference analysis enables estimating t

International Business Machines 506 Dec 19, 2022