scAR (single-cell Ambient Remover) is a package for data denoising in single-cell omics.

Overview

scAR

scAR single-cell omics machine learning variational autoencoders denoising

scAR (single cell Ambient Remover) is a package for denoising multiple single cell omics data. It can be used for multiple tasks, such as, sgRNA assignment for scCRISPRseq, identity barcode assignment for cell indexing, protein denoising for CITE-seq, mRNA denoising for scRNAseq, and etc... It is built using probabilistic deep learning, illustrated as follows:

Table of Contents

Installation

Clone this repository,

$ git clone https://github.com/Novartis/scAR.git

Enter the cloned directory:

$ cd scAR

To install the dependencies, create a conda environment:

Please use scAR-gpu if you have an nvidia graphis card and the corresponging driver installed.

$ conda env create -f scAR-gpu.yml

or

Please use scAR-cpu if you don't have a graphis card availalble.

$ conda env create -f scAR-cpu.yml

To activate the scAR conda environment run:

$ conda activate scAR

Usage

There are two ways to run scAR.

  1. Use scAR API if you are Python users
>>> from scAR import model
>>> scarObj = model(adata.X.to_df(), empty_profile)
>>> scarObj.train()
>>> scarObj.inference()
>>> adata.layers["X_scAR_denoised"] = scarObj.native_counts
>>> adata.obsm["X_scAR_assignment"] = scarObj.feature_assignment  # feature assignment, e.g., sgRNAs, tags, and etc.. Only available in 'cropseq' mode

See the tutorials

  1. Run scAR from the command line
$ scar raw_count_matrix.pickle -t technology -e empty_profile.pickle -o output

raw_count_matrix.pickle, a pickle-formatted raw count matrix (MxN) with cells in rows and features in columns
empty_profile.pickle, a pickle-formatted feature frequencies (Nx1) in empty droplets
technology, a string, either 'scRNAseq' or 'CROPseq' or 'CITEseq'

Use scar --help command to see other optional arguments and parameters.

The output folder contains four (or five) files:

output
├── denoised_counts.pickle		# denoised count matrix
├── expected_noise_ratio.pickle	# estimated noise ratio
├── BayesFactor.pickle			# bayesian factor of ambient contamination
├── expected_native_freq.pickle	# estimated native frequencies
└── assignment.pickle			# feature assignment, e.g., sgRNAs, tags, and etc.. Gernerated under 'cropseq' mode

Dependencies

PyTorch 1.8 Python 3.8.6 torchvision 0.9.0 tqdm 4.62.3 scikit-learn 1.0.1

Resources

License

This project is licensed under the terms of License.
Copyright 2022 Novartis International AG.

Reference

If you use scAR in your research, please consider citing our manuscript,

@article {Sheng2022.01.14.476312,
	author = {Sheng, Caibin and Lopes, Rui and Li, Gang and Schuierer, Sven and Waldt, Annick and Cuttat, Rachel and Dimitrieva, Slavica and Kauffmann, Audrey and Durand, Eric and Galli, Giorgio G and Roma, Guglielmo and de Weck, Antoine},
	title = {Probabilistic modeling of ambient noise in single-cell omics data},
	elocation-id = {2022.01.14.476312},
	year = {2022},
	doi = {10.1101/2022.01.14.476312},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2022/01/14/2022.01.14.476312},
	eprint = {https://www.biorxiv.org/content/early/2022/01/14/2022.01.14.476312.full.pdf},
	journal = {bioRxiv}
}
Comments
  • Stochastic rounding to integers for downstream use in TotalVI/SCVI

    Stochastic rounding to integers for downstream use in TotalVI/SCVI

    Hi Caibin,

    I tried using scar's output as input for TotalVI/SCVI. As expected, those gave an error because the input is not integer anymore. I would suggest implementing stochastic rounding to integers as done in SoupX.

    Let me know if you're interested and I can find the time to implement it.

    Regards, Mikhael

    enhancement 
    opened by mdmanurung 9
  • BiocondaBot not triggered

    BiocondaBot not triggered

    Hi @fgypas , I made a new release v0.4.1 but bioconda somehow is not triggered upon the new release.

    In the new release, some codes related to building process have been refactored.

    • All information in setup.py (deleted) is integrated into setup.cfg.
    • An extra pyproject.toml file is added.

    I am wondering whether these affect the bioconda-recipes.

    Many thanks, Caibin

    opened by CaibinSh 7
  • New release

    New release

    Hi @fgypas ,

    I am making a new release. There are mainly three changes: 1) addition of a readthedocs; 2) code reformatting via black and pylint (pylint now can score >7, so I have increase the standard in the Action test from 0.5 to 6); 3) renaming 'scAR' to 'scar'.

    I have a couple of questions regarding whether these changes influence the bioconda recipe.

    • Will renaming package name (scAR) require modification in bioconda PR? All uppercase ('scAR') is changed to lowercase ('scar') in everywhere possible (inc. folder, environment, and etc.) But the repo name may stay as 'scAR' for a while, as renaming repo name requires permission from Nick.

    • Should we exclude the folder of datasets in the conda recipe? In addition, a folder, named 'datasets' contains >100 MBs data is added for the tutorial. Should we exclude it?

    question 
    opened by CaibinSh 3
  • Implementation in scvi-tools

    Implementation in scvi-tools

    Hi scAR team,

    I'm reaching out to gauge interest in having a mirror implementation in scvi-tools for scAR. Given the existing infrastructure in the scvi-tools repository, I was able to create a port of scAR quite easily as an external module. Of course, the implementation will link to this repository as the original and cites the paper in the docs. On top of that, the port would allow users of scvi-tools to use the pretrained scAR encoder for doublet detection using the solo model.

    Here's the pending pull request so you can check out what it would look like in the final implementation: https://github.com/scverse/scvi-tools/pull/1683

    Please let me know what you think!

    opened by ricomnl 2
  • Positive-valued denoising results for ADTs with raw 0 counts

    Positive-valued denoising results for ADTs with raw 0 counts

    Hi scar team!

    Thank you for developing this interesting package. I had a question about the resulting denoised values for CITE-seq experiments.

    I've noticed that some cells that originally have a 0 value for an ADT (as a raw count) will have a positive value (>0) for that ADT after the denoising procedure. Below, I show this case for the CD25 ADT in the 10xPBMC5k CITE-seq dataset (from the tutorial at https://scar-tutorials.readthedocs.io/en/latest/tutorials/scAR_tutorial_denoising_CITEseq.html).

    I'm a bit confused about how to best interpret these values and how they are occurring. Should these be set to 0 after the denoising procedure?

    Screen Shot 2022-05-25 at 1 16 37 AM question 
    opened by diegoalexespi 2
  • Sparsity values for mRNA decontamination?

    Sparsity values for mRNA decontamination?

    Hello,

    I was wondering what the recommendations for the sparsity value would be in denoising mRNA? Specifically if we don't know too much of the data besides UMI/nGenes in the cells etc.? I noticed its generally set at 1 for sgRNA decontamination, but what would the general recommended value be for mRNA?

    Thanks, Chang

    question 
    opened by cnk113 1
  • Number of training epochs + batch size

    Number of training epochs + batch size

    Dear scAR-Team,

    thank you for developing this package. I am currently exploring it and I would like to ask you

    1. how do you determine the number of epochs the user should use for feature_type = "mRNA"? In your tutorials you used 400 epochs and in your paper you mentioned that you fixed the epochs to 800. I applied it for various batch sizes (up to 1000) and noticed that the model is sensitive to it.

    2. I noticed that you use rather small batch-size - is scAR sensitive to the batch-size, it is just due to computational limitations or due to better perfromance?

    Thank you in advance!

    Best,

    question 
    opened by KalinNonchev 1
  • bump to version 0.3.2

    bump to version 0.3.2

    fix(*): changelog docs: adding docstring in documentation docs: adding Release notes in documentation docs: adding docstring in documentation test: adding semantic release refactor: further refactoring codes fix semantic release

    opened by CaibinSh 1
  • ask for permission of Webhooks

    ask for permission of Webhooks

    Hi @kliatsko ,

    We are currently refactoring and adding functionalities to scAR.

    Could you please grant the Webhooks permission for us to automate the documentation?

    Many thanks in advance. Best regards, Caibin on behalf of the scar team @fgypas @Tobias-Ternent @mr-nvs @AlexMTYZ.

    help wanted 
    opened by CaibinSh 1
  • New release

    New release

    • Additions of readthedocs
    • Code refactoring
    1. Renaming module names, e.g. changing "scAR" -> "scar"
    2. Renaming parameter names, e.g.

    changing "scRNAseq_tech" -> "feature_type" changing "model" -> "count_model" changing "scRNAseq_tech" -> "feature_type"

    • Black and Pylint re-formatting the code
    enhancement 
    opened by CaibinSh 1
  • Black github action

    Black github action

    Addition of black github action that runs on every push and every pull request. It shows in the stdout all the changes that need to be made (--diff), but returns exit code 0, even if errors are observed.

    opened by fgypas 1
Releases(v0.4.4)
  • v0.4.4(Aug 9, 2022)

    Documentation

    • Update dependency (03cf19e)
    • Update dependencies (9bd7f1c)
    • Update documentations (418996c)
    • Update dependencies (1bde351)
    • main: Add link to anndata and scanpy (8436e05)
    • main: Update dependencies (984df35)
    • main: Update documentation for .h5 file (2a309e0)
    • Add a link of binary installers (2faed3e)
    • Update documentations (e26a6e9)
    • Add competing methods (8564b2b)
    • scar: Add versionadded directives for parameter sparsity and round_to_int (33e35ca)
    • Update docs (a4da539)
    • Update introduction (a036b24)
    • Change readthedocs template (421e52f)
    • data_generator: Update docs (1f8f668)
    • data_generator: Re-style docs (afef9fb)
    • *: Re-style docs (2d550fa)

    Performance

    • main: Command line tool supports a new input: filtered_feature_bc_matrix.h5 (73bc13e)
    • setup: Add an error raise statement (f4fb1a8)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.3(Jun 15, 2022)

    Fix

    • setup: Fix a bug to allow sample reasonable numbers of droplets (ef6f7e4)
    • main: Fix a bug in main to set default NN number (794ff17)

    Documentation

    • main: Add scanpy as dependency (252a492)

    Performance

    • main: Set a separate batchsize_infer parameter for inference (8727f04)
    • setup: Add an option of random sampling droplets to speed up calculation (ce042dd)
    • setup: Enable manupulate large-scale emptydroplets (15f1840)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.2(Jun 7, 2022)

  • v0.4.1(May 19, 2022)

    What's Changed

    Feature

    • inference: add a round_to_int parameter to round the counts (float) for easy interpretation and better integration into other methods (#47) (902a2b9) (8694239)

    Build

    • setup: replace setup.py with setup.cfg and pyproject.toml (#51) (3dc999a)

    Chore

    Documentation

    • readthedocs: add scAR_logo image (#51) (c34f362)
    • tutorials: add ci=None to speed up plotting (#51) (902a2b9)

    Contributor

    @CaibinSh and @mdmanurung

    Full Changelog: https://github.com/Novartis/scar/compare/v0.4.0...v0.4.1

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(May 5, 2022)

  • v0.3.5(May 3, 2022)

  • v0.3.4(May 1, 2022)

  • v0.3.3(May 1, 2022)

  • v0.3.1(Apr 29, 2022)

  • v0.3.0(Apr 27, 2022)

    What's Changed

    Renaming module names, e.g. changing "scAR" -> "scar" Renaming parameter names, e.g.

    "scRNAseq_tech" -> "feature_type" "model" -> "count_model" "empty_profile" -> "ambient_profile" ...

    • Black and Pylint re-formatting the code
    • New release by @CaibinSh in https://github.com/Novartis/scAR/pull/26

    Contributor

    @CaibinSh @fgypas @mr-nvs @Tobias-Ternent

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.3...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.3(Apr 20, 2022)

    • Add integration test
    • Black formating
    • Bump version to 0.2.3

    Contributors: @fgypas , @mr-nvs and @CaibinSh

    What's Changed

    • Develop by @CaibinSh in https://github.com/Novartis/scAR/pull/19

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.2...v0.2.3

    Source code(tar.gz)
    Source code(zip)
  • v0.2.2(Apr 4, 2022)

    v0.2.2

    • Remove torchaudio
    • Add test data for integration tests
    • Bump version to 0.2.2

    Contributors: @CaibinSh @fgypas

    What's Changed

    • Remove torchaudio, add test data and bump version to 0.2.2 by @fgypas in https://github.com/Novartis/scAR/pull/15

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.1-beta...v0.2.2

    Source code(tar.gz)
    Source code(zip)
  • v0.2.1-beta(Apr 1, 2022)

    • fix a typo in scAR-gpu.yml
    • reorganise init.py files

    Contributor: @CaibinSh

    What's Changed

    • Develop by @CaibinSh in https://github.com/Novartis/scAR/pull/12

    Full Changelog: https://github.com/Novartis/scAR/compare/v0.2.0-beta...v0.2.1-beta

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0-beta(Apr 1, 2022)

    • Support for training of the model with CPUs
    • Addition of two yaml files for CPU/GPU installation
    • Refactor of setup.py and structure of the package
    • Addition of tests with pytest
    • Addition of lint checks
    • Automate build with github actions (install package and run lint checks and pytest)
    • Update documentation
    • Version 0.2.0

    Co-authored-by: @CaibinSh @mr-nvs @Tobias-Ternent @fgypas

    What's Changed

    • 0.2.0-release by @fgypas in https://github.com/Novartis/scAR/pull/11

    Full Changelog: https://github.com/Novartis/scAR/commits/v0.2.0-beta

    Source code(tar.gz)
    Source code(zip)
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style

Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style [NeurIPS 2021] Official code to reproduce the results and data p

Yash Sharma 27 Sep 19, 2022
Automatic Differentiation Multipole Moment Molecular Forcefield

Automatic Differentiation Multipole Moment Molecular Forcefield Performance notes On a single gpu, using waterbox_31ang.pdb example from MPIDplugin wh

4 Jan 07, 2022
Türkiye Canlı Mobese Görüntülerinde Profesyonel Nesne Takip Sistemi

Türkiye Mobese Görüntü Takip Türkiye Mobese görüntülerinde OPENCV ve Yolo ile takip sistemi Multiple Object Tracking System in Turkish Mobese with OPE

15 Dec 22, 2022
Pytorch implementation of set transformer

set_transformer Official PyTorch implementation of the paper Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks .

Juho Lee 410 Jan 06, 2023
Official implementation for ICDAR 2021 paper "Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer"

Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer Description Convert offline handwritten mathematical expressi

Wenqi Zhao 87 Dec 27, 2022
Deep Face Recognition in PyTorch

Face Recognition in PyTorch By Alexey Gruzdev and Vladislav Sovrasov Introduction A repository for different experimental Face Recognition models such

Alexey Gruzdev 141 Sep 11, 2022
Ağ tarayıcı.Gönderdiği paketler ile ağa bağlı olan cihazların IP adreslerini gösterir.

NetScanner.py Ağ tarayıcı.Gönderdiği paketler ile ağa bağlı olan cihazların IP adreslerini gösterir. Linux'da Kullanımı: git clone https://github.com/

4 Aug 23, 2021
[NeurIPS 2021] Towards Better Understanding of Training Certifiably Robust Models against Adversarial Examples | ⛰️⚠️

Towards Better Understanding of Training Certifiably Robust Models against Adversarial Examples This repository is the official implementation of "Tow

Sungyoon Lee 4 Jul 12, 2022
Revisiting Self-Training for Few-Shot Learning of Language Model.

SFLM This is the implementation of the paper Revisiting Self-Training for Few-Shot Learning of Language Model. SFLM is short for self-training for few

15 Nov 19, 2022
A simple consistency training framework for semi-supervised image semantic segmentation

PseudoSeg: Designing Pseudo Labels for Semantic Segmentation PseudoSeg is a simple consistency training framework for semi-supervised image semantic s

Google Interns 143 Dec 13, 2022
Multi-Task Learning as a Bargaining Game

Nash-MTL Official implementation of "Multi-Task Learning as a Bargaining Game". Setup environment conda create -n nashmtl python=3.9.7 conda activate

Aviv Navon 87 Dec 26, 2022
CMSC320 - Introduction to Data Science - Fall 2021

CMSC320 - Introduction to Data Science - Fall 2021 Instructors: Elias Jonatan Gonzalez and José Manuel Calderón Trilla Lectures: MW 3:30-4:45 & 5:00-6

Introduction to Data Science 6 Sep 12, 2022
Implementation of "DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing".

DeepOrder Implementation of DeepOrder for the paper "DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing". Project

6 Nov 07, 2022
FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes

FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes This repository contains the source code accompanying the paper: FlexConv: C

Robert-Jan Bruintjes 96 Dec 12, 2022
Official code for ICCV2021 paper "M3D-VTON: A Monocular-to-3D Virtual Try-on Network"

M3D-VTON: A Monocular-to-3D Virtual Try-On Network Official code for ICCV2021 paper "M3D-VTON: A Monocular-to-3D Virtual Try-on Network" Paper | Suppl

109 Dec 29, 2022
Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel order of RGB and BGR. Simple Channel Converter for ONNX.

scc4onnx Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel

Katsuya Hyodo 16 Dec 22, 2022
Real-time Joint Semantic Reasoning for Autonomous Driving

MultiNet MultiNet is able to jointly perform road segmentation, car detection and street classification. The model achieves real-time speed and state-

Marvin Teichmann 518 Dec 12, 2022
“Robust Lightweight Facial Expression Recognition Network with Label Distribution Training”, AAAI 2021.

EfficientFace Zengqun Zhao, Qingshan Liu, Feng Zhou. "Robust Lightweight Facial Expression Recognition Network with Label Distribution Training". AAAI

Zengqun Zhao 119 Jan 08, 2023
Language-Agnostic Website Embedding and Classification

Homepage2Vec Language-Agnostic Website Embedding and Classification based on Curlie labels https://arxiv.org/pdf/2201.03677.pdf Homepage2Vec is a pre-

25 Dec 27, 2022
Explore extreme compression for pre-trained language models

Code for paper "Exploring extreme parameter compression for pre-trained language models ICLR2022"

twinkle 16 Nov 14, 2022