scAR (single-cell Ambient Remover) is a package for data denoising in single-cell omics.

Overview

scAR

scAR single-cell omics machine learning variational autoencoders denoising

scAR (single cell Ambient Remover) is a package for denoising multiple single cell omics data. It can be used for multiple tasks, such as, sgRNA assignment for scCRISPRseq, identity barcode assignment for cell indexing, protein denoising for CITE-seq, mRNA denoising for scRNAseq, and etc... It is built using probabilistic deep learning, illustrated as follows:

Table of Contents

Installation

Clone this repository,

$ git clone https://github.com/Novartis/scAR.git

Enter the cloned directory:

$ cd scAR

To install the dependencies, create a conda environment:

Please use scAR-gpu if you have an nvidia graphis card and the corresponging driver installed.

$ conda env create -f scAR-gpu.yml

or

Please use scAR-cpu if you don't have a graphis card availalble.

$ conda env create -f scAR-cpu.yml

To activate the scAR conda environment run:

$ conda activate scAR

Usage

There are two ways to run scAR.

  1. Use scAR API if you are Python users
>>> from scAR import model
>>> scarObj = model(adata.X.to_df(), empty_profile)
>>> scarObj.train()
>>> scarObj.inference()
>>> adata.layers["X_scAR_denoised"] = scarObj.native_counts
>>> adata.obsm["X_scAR_assignment"] = scarObj.feature_assignment  # feature assignment, e.g., sgRNAs, tags, and etc.. Only available in 'cropseq' mode

See the tutorials

  1. Run scAR from the command line
$ scar raw_count_matrix.pickle -t technology -e empty_profile.pickle -o output

raw_count_matrix.pickle, a pickle-formatted raw count matrix (MxN) with cells in rows and features in columns
empty_profile.pickle, a pickle-formatted feature frequencies (Nx1) in empty droplets
technology, a string, either 'scRNAseq' or 'CROPseq' or 'CITEseq'

Use scar --help command to see other optional arguments and parameters.

The output folder contains four (or five) files:

output
├── denoised_counts.pickle		# denoised count matrix
├── expected_noise_ratio.pickle	# estimated noise ratio
├── BayesFactor.pickle			# bayesian factor of ambient contamination
├── expected_native_freq.pickle	# estimated native frequencies
└── assignment.pickle			# feature assignment, e.g., sgRNAs, tags, and etc.. Gernerated under 'cropseq' mode

Dependencies

PyTorch 1.8 Python 3.8.6 torchvision 0.9.0 tqdm 4.62.3 scikit-learn 1.0.1

Resources

License

This project is licensed under the terms of License.
Copyright 2022 Novartis International AG.

Reference

If you use scAR in your research, please consider citing our manuscript,

@article {Sheng2022.01.14.476312,
	author = {Sheng, Caibin and Lopes, Rui and Li, Gang and Schuierer, Sven and Waldt, Annick and Cuttat, Rachel and Dimitrieva, Slavica and Kauffmann, Audrey and Durand, Eric and Galli, Giorgio G and Roma, Guglielmo and de Weck, Antoine},
	title = {Probabilistic modeling of ambient noise in single-cell omics data},
	elocation-id = {2022.01.14.476312},
	year = {2022},
	doi = {10.1101/2022.01.14.476312},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2022/01/14/2022.01.14.476312},
	eprint = {https://www.biorxiv.org/content/early/2022/01/14/2022.01.14.476312.full.pdf},
	journal = {bioRxiv}
}
Comments
  • Stochastic rounding to integers for downstream use in TotalVI/SCVI

    Stochastic rounding to integers for downstream use in TotalVI/SCVI

    Hi Caibin,

    I tried using scar's output as input for TotalVI/SCVI. As expected, those gave an error because the input is not integer anymore. I would suggest implementing stochastic rounding to integers as done in SoupX.

    Let me know if you're interested and I can find the time to implement it.

    Regards, Mikhael

    enhancement 
    opened by mdmanurung 9
  • BiocondaBot not triggered

    BiocondaBot not triggered

    Hi @fgypas , I made a new release v0.4.1 but bioconda somehow is not triggered upon the new release.

    In the new release, some codes related to building process have been refactored.

    • All information in setup.py (deleted) is integrated into setup.cfg.
    • An extra pyproject.toml file is added.

    I am wondering whether these affect the bioconda-recipes.

    Many thanks, Caibin

    opened by CaibinSh 7
  • New release

    New release

    Hi @fgypas ,

    I am making a new release. There are mainly three changes: 1) addition of a readthedocs; 2) code reformatting via black and pylint (pylint now can score >7, so I have increase the standard in the Action test from 0.5 to 6); 3) renaming 'scAR' to 'scar'.

    I have a couple of questions regarding whether these changes influence the bioconda recipe.

    • Will renaming package name (scAR) require modification in bioconda PR? All uppercase ('scAR') is changed to lowercase ('scar') in everywhere possible (inc. folder, environment, and etc.) But the repo name may stay as 'scAR' for a while, as renaming repo name requires permission from Nick.

    • Should we exclude the folder of datasets in the conda recipe? In addition, a folder, named 'datasets' contains >100 MBs data is added for the tutorial. Should we exclude it?

    question 
    opened by CaibinSh 3
  • Implementation in scvi-tools

    Implementation in scvi-tools

    Hi scAR team,

    I'm reaching out to gauge interest in having a mirror implementation in scvi-tools for scAR. Given the existing infrastructure in the scvi-tools repository, I was able to create a port of scAR quite easily as an external module. Of course, the implementation will link to this repository as the original and cites the paper in the docs. On top of that, the port would allow users of scvi-tools to use the pretrained scAR encoder for doublet detection using the solo model.

    Here's the pending pull request so you can check out what it would look like in the final implementation: https://github.com/scverse/scvi-tools/pull/1683

    Please let me know what you think!

    opened by ricomnl 2
  • Positive-valued denoising results for ADTs with raw 0 counts

    Positive-valued denoising results for ADTs with raw 0 counts

    Hi scar team!

    Thank you for developing this interesting package. I had a question about the resulting denoised values for CITE-seq experiments.

    I've noticed that some cells that originally have a 0 value for an ADT (as a raw count) will have a positive value (>0) for that ADT after the denoising procedure. Below, I show this case for the CD25 ADT in the 10xPBMC5k CITE-seq dataset (from the tutorial at https://scar-tutorials.readthedocs.io/en/latest/tutorials/scAR_tutorial_denoising_CITEseq.html).

    I'm a bit confused about how to best interpret these values and how they are occurring. Should these be set to 0 after the denoising procedure?

    Screen Shot 2022-05-25 at 1 16 37 AM question 
    opened by diegoalexespi 2
  • Sparsity values for mRNA decontamination?

    Sparsity values for mRNA decontamination?

    Hello,

    I was wondering what the recommendations for the sparsity value would be in denoising mRNA? Specifically if we don't know too much of the data besides UMI/nGenes in the cells etc.? I noticed its generally set at 1 for sgRNA decontamination, but what would the general recommended value be for mRNA?

    Thanks, Chang

    question 
    opened by cnk113 1
  • Number of training epochs + batch size

    Number of training epochs + batch size

    Dear scAR-Team,

    thank you for developing this package. I am currently exploring it and I would like to ask you

    1. how do you determine the number of epochs the user should use for feature_type = "mRNA"? In your tutorials you used 400 epochs and in your paper you mentioned that you fixed the epochs to 800. I applied it for various batch sizes (up to 1000) and noticed that the model is sensitive to it.

    2. I noticed that you use rather small batch-size - is scAR sensitive to the batch-size, it is just due to computational limitations or due to better perfromance?

    Thank you in advance!

    Best,

    question 
    opened by KalinNonchev 1
  • bump to version 0.3.2

    bump to version 0.3.2

    fix(*): changelog docs: adding docstring in documentation docs: adding Release notes in documentation docs: adding docstring in documentation test: adding semantic release refactor: further refactoring codes fix semantic release

    opened by CaibinSh 1
  • ask for permission of Webhooks

    ask for permission of Webhooks

    Hi @kliatsko ,

    We are currently refactoring and adding functionalities to scAR.

    Could you please grant the Webhooks permission for us to automate the documentation?

    Many thanks in advance. Best regards, Caibin on behalf of the scar team @fgypas @Tobias-Ternent @mr-nvs @AlexMTYZ.

    help wanted 
    opened by CaibinSh 1
  • New release

    New release

    • Additions of readthedocs
    • Code refactoring
    1. Renaming module names, e.g. changing "scAR" -> "scar"
    2. Renaming parameter names, e.g.

    changing "scRNAseq_tech" -> "feature_type" changing "model" -> "count_model" changing "scRNAseq_tech" -> "feature_type"

    • Black and Pylint re-formatting the code
    enhancement 
    opened by CaibinSh 1
  • Black github action

    Black github action

    Addition of black github action that runs on every push and every pull request. It shows in the stdout all the changes that need to be made (--diff), but returns exit code 0, even if errors are observed.

    opened by fgypas 1
Releases(v0.4.4)
  • v0.4.4(Aug 9, 2022)

    Documentation

    • Update dependency (03cf19e)
    • Update dependencies (9bd7f1c)
    • Update documentations (418996c)
    • Update dependencies (1bde351)
    • main: Add link to anndata and scanpy (8436e05)
    • main: Update dependencies (984df35)
    • main: Update documentation for .h5 file (2a309e0)
    • Add a link of binary installers (2faed3e)
    • Update documentations (e26a6e9)
    • Add competing methods (8564b2b)
    • scar: Add versionadded directives for parameter sparsity and round_to_int (33e35ca)
    • Update docs (a4da539)
    • Update introduction (a036b24)
    • Change readthedocs template (421e52f)
    • data_generator: Update docs (1f8f668)
    • data_generator: Re-style docs (afef9fb)
    • *: Re-style docs (2d550fa)

    Performance

    • main: Command line tool supports a new input: filtered_feature_bc_matrix.h5 (73bc13e)
    • setup: Add an error raise statement (f4fb1a8)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.3(Jun 15, 2022)

    Fix

    • setup: Fix a bug to allow sample reasonable numbers of droplets (ef6f7e4)
    • main: Fix a bug in main to set default NN number (794ff17)

    Documentation

    • main: Add scanpy as dependency (252a492)

    Performance

    • main: Set a separate batchsize_infer parameter for inference (8727f04)
    • setup: Add an option of random sampling droplets to speed up calculation (ce042dd)
    • setup: Enable manupulate large-scale emptydroplets (15f1840)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.2(Jun 7, 2022)

  • v0.4.1(May 19, 2022)

    What's Changed

    Feature

    • inference: add a round_to_int parameter to round the counts (float) for easy interpretation and better integration into other methods (#47) (902a2b9) (8694239)

    Build

    • setup: replace setup.py with setup.cfg and pyproject.toml (#51) (3dc999a)

    Chore

    Documentation

    • readthedocs: add scAR_logo image (#51) (c34f362)
    • tutorials: add ci=None to speed up plotting (#51) (902a2b9)

    Contributor

    @CaibinSh and @mdmanurung

    Full Changelog: https://github.com/Novartis/scar/compare/v0.4.0...v0.4.1

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(May 5, 2022)

  • v0.3.5(May 3, 2022)

  • v0.3.4(May 1, 2022)

  • v0.3.3(May 1, 2022)

  • v0.3.1(Apr 29, 2022)

  • v0.3.0(Apr 27, 2022)

    What's Changed

    Renaming module names, e.g. changing "scAR" -> "scar" Renaming parameter names, e.g.

    "scRNAseq_tech" -> "feature_type" "model" -> "count_model" "empty_profile" -> "ambient_profile" ...

    • Black and Pylint re-formatting the code
    • New release by @CaibinSh in https://github.com/Novartis/scAR/pull/26

    Contributor

    @CaibinSh @fgypas @mr-nvs @Tobias-Ternent

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.3...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.3(Apr 20, 2022)

    • Add integration test
    • Black formating
    • Bump version to 0.2.3

    Contributors: @fgypas , @mr-nvs and @CaibinSh

    What's Changed

    • Develop by @CaibinSh in https://github.com/Novartis/scAR/pull/19

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.2...v0.2.3

    Source code(tar.gz)
    Source code(zip)
  • v0.2.2(Apr 4, 2022)

    v0.2.2

    • Remove torchaudio
    • Add test data for integration tests
    • Bump version to 0.2.2

    Contributors: @CaibinSh @fgypas

    What's Changed

    • Remove torchaudio, add test data and bump version to 0.2.2 by @fgypas in https://github.com/Novartis/scAR/pull/15

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.1-beta...v0.2.2

    Source code(tar.gz)
    Source code(zip)
  • v0.2.1-beta(Apr 1, 2022)

    • fix a typo in scAR-gpu.yml
    • reorganise init.py files

    Contributor: @CaibinSh

    What's Changed

    • Develop by @CaibinSh in https://github.com/Novartis/scAR/pull/12

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.0-beta...v0.2.1-beta

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0-beta(Apr 1, 2022)

    • Support for training of the model with CPUs
    • Addition of two yaml files for CPU/GPU installation
    • Refactor of setup.py and structure of the package
    • Addition of tests with pytest
    • Addition of lint checks
    • Automate build with github actions (install package and run lint checks and pytest)
    • Update documentation
    • Version 0.2.0

    Co-authored-by: @CaibinSh @mr-nvs @Tobias-Ternent @fgypas

    What's Changed

    • 0.2.0-release by @fgypas in https://github.com/Novartis/scAR/pull/11

    Full Changelog: https://github.com/Novartis/scAR/commits/v0.2.0-beta

    Source code(tar.gz)
    Source code(zip)
AdaFocus (ICCV 2021) Adaptive Focus for Efficient Video Recognition

AdaFocus (ICCV 2021) This repo contains the official code and pre-trained models for AdaFocus. Adaptive Focus for Efficient Video Recognition Referenc

Rainforest Wang 115 Dec 21, 2022
A ssl analyzer which could analyzer target domain's certificate.

ssl_analyzer A ssl analyzer which could analyzer target domain's certificate. Analyze the domain name ssl certificate information according to the inp

vincent 17 Dec 12, 2022
Technical experimentations to beat the stock market using deep learning :chart_with_upwards_trend:

DeepStock Technical experimentations to beat the stock market using deep learning. Experimentations Deep Learning Stock Prediction with Daily News Hea

Keon 449 Dec 29, 2022
Algebraic effect handlers in Python

PyEffect: Algebraic effects in Python What IDK. Usage effects.handle(operation, handlers=None) effects.set_handler(effect, handler) Supported effects

Greg Werbin 5 Dec 27, 2021
Transformer in Computer Vision

Transformer-in-Vision A paper list of some recent Transformer-based CV works. If you find some ignored papers, please open issues or pull requests. **

506 Dec 26, 2022
Unrolled Generative Adversarial Networks

Unrolled Generative Adversarial Networks Luke Metz, Ben Poole, David Pfau, Jascha Sohl-Dickstein arxiv:1611.02163 This repo contains an example notebo

Ben Poole 292 Dec 06, 2022
Pytorch implementation of Feature Pyramid Network (FPN) for Object Detection

fpn.pytorch Pytorch implementation of Feature Pyramid Network (FPN) for Object Detection Introduction This project inherits the property of our pytorc

Jianwei Yang 912 Dec 21, 2022
Object-Centric Learning with Slot Attention

Slot Attention This is a re-implementation of "Object-Centric Learning with Slot Attention" in PyTorch (https://arxiv.org/abs/2006.15055). Requirement

Untitled AI 72 Jan 02, 2023
RAMA: Rapid algorithm for multicut problem

RAMA: Rapid algorithm for multicut problem Solves multicut (correlation clustering) problems orders of magnitude faster than CPU based solvers without

Paul Swoboda 60 Dec 13, 2022
PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Smooth ReLU in PyTorch Unofficial PyTorch reimplementation of the Smooth ReLU (SmeLU) activation function proposed in the paper Real World Large Scale

Christoph Reich 10 Jan 02, 2023
UniFormer - official implementation of UniFormer

UniFormer This repo is the official implementation of "Uniformer: Unified Transf

SenseTime X-Lab 573 Jan 04, 2023
CHERRY is a python library for predicting the interactions between viral and prokaryotic genomes

CHERRY is a python library for predicting the interactions between viral and prokaryotic genomes. CHERRY is based on a deep learning model, which consists of a graph convolutional encoder and a link

Kenneth Shang 12 Dec 15, 2022
Full Resolution Residual Networks for Semantic Image Segmentation

Full-Resolution Residual Networks (FRRN) This repository contains code to train and qualitatively evaluate Full-Resolution Residual Networks (FRRNs) a

Toby Pohlen 274 Oct 27, 2022
ML-based medical imaging using Azure

Disclaimer This code is provided for research and development use only. This code is not intended for use in clinical decision-making or for any other

Microsoft Azure 68 Dec 23, 2022
TorchGeo is a PyTorch domain library, similar to torchvision, that provides datasets, transforms, samplers, and pre-trained models specific to geospatial data.

TorchGeo is a PyTorch domain library, similar to torchvision, that provides datasets, transforms, samplers, and pre-trained models specific to geospatial data.

Microsoft 1.3k Dec 30, 2022
PyTorch implementation of Value Iteration Networks (VIN): Clean, Simple and Modular. Visualization in Visdom.

VIN: Value Iteration Networks This is an implementation of Value Iteration Networks (VIN) in PyTorch to reproduce the results.(TensorFlow version) Key

Xingdong Zuo 215 Dec 07, 2022
The code for "Deep Level Set for Box-supervised Instance Segmentation in Aerial Images".

Deep Levelset for Box-supervised Instance Segmentation in Aerial Images Wentong Li, Yijie Chen, Wenyu Liu, Jianke Zhu* Any questions or discussions ar

sunshine.lwt 112 Jan 05, 2023
deep learning model with only python and numpy with test accuracy 99 % on mnist dataset and different optimization choices

deep_nn_model_with_only_python_100%_test_accuracy deep learning model with only python and numpy with test accuracy 99 % on mnist dataset and differen

0 Aug 28, 2022
Official codebase used to develop Vision Transformer, MLP-Mixer, LiT and more.

Big Vision This codebase is designed for training large-scale vision models on Cloud TPU VMs. It is based on Jax/Flax libraries, and uses tf.data and

Google Research 701 Jan 03, 2023
Python Single Object Tracking Evaluation

pysot-toolkit The purpose of this repo is to provide evaluation API of Current Single Object Tracking Dataset, including VOT2016 VOT2018 VOT2018-LT OT

348 Dec 22, 2022