Scarf’s basic workflow for scRNA-Seq

This workflow is meant to familiarize users with the Scarf API and how data is internally handled in Scarf. Please checkout the quick start guide if you are interested in the minimal steps required to run the analysis.

[1]:
%load_ext autotime
%config InlineBackend.figure_format = 'retina'

import scarf
scarf.__version__
[1]:
'0.7.8'
time: 1.53 s (started: 2021-08-22 17:55:15 +00:00)

Download the count matrix from 10x’s website using the fetch_dataset function. This is a convenience function that stores URLs of datasets that can be downloaded. The save_path parameter allows the data to be saved to a location of choice.

[2]:
scarf.fetch_dataset('tenx_5K_pbmc_rnaseq', save_path='scarf_datasets')
INFO: Download started...
--2021-08-22 17:55:22--  https://files.de-1.osf.io/v1/resources/zeupv/providers/osfstorage/609678235533b405a6e22fad
Resolving files.de-1.osf.io (files.de-1.osf.io)... 35.186.249.111
Connecting to files.de-1.osf.io (files.de-1.osf.io)|35.186.249.111|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://storage.googleapis.com/cos-osf-prod-files-de-1/17a47c2d3b323858ae9183e332fa31a01bf130eaa7ca6199ca9d4efe5217374f?response-content-disposition=attachment%3B%20filename%3D%22data.h5%22%3B%20filename%2A%3DUTF-8%27%27data.h5&GoogleAccessId=files-de-1%40cos-osf-prod.iam.gserviceaccount.com&Expires=1629654983&Signature=vMt234NEyL%2FmfMeLXSwPuZ1BVp4XeFEY4nkrYBCHhJ8Rama%2Bb51V%2FYvSnxcNdbZUMNnbYZcPEKJZpm3%2Bu5IRJoeWPEFmme%2FG9ffqu519osfnAq6o7veSWWxsxCrMLgpMpCCnchx0FWKeDAmGHqw%2Fj663E%2B2dFgwxEKYJ7HSRYc1Mk3JhATZgof1USrCdpoc6hHXvO04ouPzjUvCOdxAYOezy%2BPmEdDbe2kADmhIFgGHaT78FUtUNMEBOidRfELI0kN8czr9MKuSI6fDdjW%2FniGLyba5R0d52094jQa8v68VF0SA3%2BEXdpLVrcic356W3koC5pQ1rj9aDOZB66HzUaw%3D%3D [following]
--2021-08-22 17:55:23--  https://storage.googleapis.com/cos-osf-prod-files-de-1/17a47c2d3b323858ae9183e332fa31a01bf130eaa7ca6199ca9d4efe5217374f?response-content-disposition=attachment%3B%20filename%3D%22data.h5%22%3B%20filename%2A%3DUTF-8%27%27data.h5&GoogleAccessId=files-de-1%40cos-osf-prod.iam.gserviceaccount.com&Expires=1629654983&Signature=vMt234NEyL%2FmfMeLXSwPuZ1BVp4XeFEY4nkrYBCHhJ8Rama%2Bb51V%2FYvSnxcNdbZUMNnbYZcPEKJZpm3%2Bu5IRJoeWPEFmme%2FG9ffqu519osfnAq6o7veSWWxsxCrMLgpMpCCnchx0FWKeDAmGHqw%2Fj663E%2B2dFgwxEKYJ7HSRYc1Mk3JhATZgof1USrCdpoc6hHXvO04ouPzjUvCOdxAYOezy%2BPmEdDbe2kADmhIFgGHaT78FUtUNMEBOidRfELI0kN8czr9MKuSI6fDdjW%2FniGLyba5R0d52094jQa8v68VF0SA3%2BEXdpLVrcic356W3koC5pQ1rj9aDOZB66HzUaw%3D%3D
Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.4.48, 172.217.4.80, 172.217.4.208, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.4.48|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 18098007 (17M) [application/octet-stream]
Saving to: ‘scarf_datasets/tenx_5K_pbmc_rnaseq/data.h5’

     0K .......... .......... .......... .......... ..........  0% 1.77M 10s
    50K .......... .......... .......... .......... ..........  0% 2.69M 8s
   100K .......... .......... .......... .......... ..........  0% 5.27M 6s
   150K .......... .......... .......... .......... ..........  1% 6.74M 5s
   200K .......... .......... .......... .......... ..........  1% 7.29M 5s
   250K .......... .......... .......... .......... ..........  1% 12.9M 4s
   300K .......... .......... .......... .......... ..........  1% 10.3M 4s
   350K .......... .......... .......... .......... ..........  2% 23.8M 3s
   400K .......... .......... .......... .......... ..........  2% 14.5M 3s
   450K .......... .......... .......... .......... ..........  2% 20.2M 3s
   500K .......... .......... .......... .......... ..........  3% 23.3M 3s
   550K .......... .......... .......... .......... ..........  3% 11.4M 3s
   600K .......... .......... .......... .......... ..........  3% 56.9M 2s
   650K .......... .......... .......... .......... ..........  3% 45.0M 2s
   700K .......... .......... .......... .......... ..........  4% 23.3M 2s
   750K .......... .......... .......... .......... ..........  4% 34.7M 2s
   800K .......... .......... .......... .......... ..........  4% 26.7M 2s
   850K .......... .......... .......... .......... ..........  5% 41.4M 2s
   900K .......... .......... .......... .......... ..........  5% 32.5M 2s
   950K .......... .......... .......... .......... ..........  5% 43.3M 2s
  1000K .......... .......... .......... .......... ..........  5% 11.0M 2s
  1050K .......... .......... .......... .......... ..........  6%  213M 2s
  1100K .......... .......... .......... .......... ..........  6%  165M 2s
  1150K .......... .......... .......... .......... ..........  6%  134M 1s
  1200K .......... .......... .......... .......... ..........  7%  217M 1s
  1250K .......... .......... .......... .......... ..........  7% 53.1M 1s
  1300K .......... .......... .......... .......... ..........  7% 16.6M 1s
  1350K .......... .......... .......... .......... ..........  7% 7.01M 1s
  1400K .......... .......... .......... .......... ..........  8% 15.6M 1s
  1450K .......... .......... .......... .......... ..........  8% 14.7M 1s
  1500K .......... .......... .......... .......... ..........  8% 14.8M 1s
  1550K .......... .......... .......... .......... ..........  9% 11.8M 1s
  1600K .......... .......... .......... .......... ..........  9%  205M 1s
  1650K .......... .......... .......... .......... ..........  9%  210M 1s
  1700K .......... .......... .......... .......... ..........  9%  167M 1s
  1750K .......... .......... .......... .......... .......... 10%  184M 1s
  1800K .......... .......... .......... .......... .......... 10%  209M 1s
  1850K .......... .......... .......... .......... .......... 10%  192M 1s
  1900K .......... .......... .......... .......... .......... 11%  213M 1s
  1950K .......... .......... .......... .......... .......... 11%  180M 1s
  2000K .......... .......... .......... .......... .......... 11%  188M 1s
  2050K .......... .......... .......... .......... .......... 11%  215M 1s
  2100K .......... .......... .......... .......... .......... 12%  220M 1s
  2150K .......... .......... .......... .......... .......... 12%  174M 1s
  2200K .......... .......... .......... .......... .......... 12%  216M 1s
  2250K .......... .......... .......... .......... .......... 13%  209M 1s
  2300K .......... .......... .......... .......... .......... 13%  200M 1s
  2350K .......... .......... .......... .......... .......... 13%  169M 1s
  2400K .......... .......... .......... .......... .......... 13%  213M 1s
  2450K .......... .......... .......... .......... .......... 14%  206M 1s
  2500K .......... .......... .......... .......... .......... 14%  221M 1s
  2550K .......... .......... .......... .......... .......... 14% 4.38M 1s
  2600K .......... .......... .......... .......... .......... 14% 15.1M 1s
  2650K .......... .......... .......... .......... .......... 15% 20.3M 1s
  2700K .......... .......... .......... .......... .......... 15% 19.5M 1s
  2750K .......... .......... .......... .......... .......... 15%  122M 1s
  2800K .......... .......... .......... .......... .......... 16%  145M 1s
  2850K .......... .......... .......... .......... .......... 16%  150M 1s
  2900K .......... .......... .......... .......... .......... 16%  146M 1s
  2950K .......... .......... .......... .......... .......... 16%  129M 1s
  3000K .......... .......... .......... .......... .......... 17%  153M 1s
  3050K .......... .......... .......... .......... .......... 17%  144M 1s
  3100K .......... .......... .......... .......... .......... 17%  145M 1s
  3150K .......... .......... .......... .......... .......... 18%  131M 1s
  3200K .......... .......... .......... .......... .......... 18%  136M 1s
  3250K .......... .......... .......... .......... .......... 18%  192M 1s
  3300K .......... .......... .......... .......... .......... 18%  207M 1s
  3350K .......... .......... .......... .......... .......... 19%  194M 1s
  3400K .......... .......... .......... .......... .......... 19%  208M 1s
  3450K .......... .......... .......... .......... .......... 19%  208M 1s
  3500K .......... .......... .......... .......... .......... 20%  225M 1s
  3550K .......... .......... .......... .......... .......... 20%  176M 1s
  3600K .......... .......... .......... .......... .......... 20%  209M 1s
  3650K .......... .......... .......... .......... .......... 20%  223M 1s
  3700K .......... .......... .......... .......... .......... 21%  209M 1s
  3750K .......... .......... .......... .......... .......... 21%  196M 1s
  3800K .......... .......... .......... .......... .......... 21%  216M 1s
  3850K .......... .......... .......... .......... .......... 22%  211M 1s
  3900K .......... .......... .......... .......... .......... 22%  220M 1s
  3950K .......... .......... .......... .......... .......... 22%  184M 1s
  4000K .......... .......... .......... .......... .......... 22%  215M 1s
  4050K .......... .......... .......... .......... .......... 23% 2.73M 1s
  4100K .......... .......... .......... .......... .....
INFO: Download finished! File saved here: scarf_datasets/tenx_5K_pbmc_rnaseq/data.h5
INFO: Download started...
..... 23% 13.7M 1s
  4150K .......... .......... .......... .......... .......... 23% 5.81M 1s
  4200K .......... .......... .......... .......... .......... 24% 21.5M 1s
  4250K .......... .......... .......... .......... .......... 24% 41.7M 1s
  4300K .......... .......... .......... .......... .......... 24%  194M 1s
  4350K .......... .......... .......... .......... .......... 24%  237M 1s
  4400K .......... .......... .......... .......... .......... 25%  203M 1s
  4450K .......... .......... .......... .......... .......... 25%  240M 1s
  4500K .......... .......... .......... .......... .......... 25%  241M 1s
  4550K .......... .......... .......... .......... .......... 26%  211M 1s
  4600K .......... .......... .......... .......... .......... 26%  196M 1s
  4650K .......... .......... .......... .......... .......... 26%  232M 1s
  4700K .......... .......... .......... .......... .......... 26%  233M 1s
  4750K .......... .......... .......... .......... .......... 27%  178M 1s
  4800K .......... .......... .......... .......... .......... 27%  186M 1s
  4850K .......... .......... .......... .......... .......... 27%  228M 1s
  4900K .......... .......... .......... .......... .......... 28% 5.26M 1s
  4950K .......... .......... .......... .......... .......... 28% 17.1M 1s
  5000K .......... .......... .......... .......... .......... 28% 29.0M 1s
  5050K .......... .......... .......... .......... .......... 28% 34.4M 1s
  5100K .......... .......... .......... .......... .......... 29%  110M 1s
  5150K .......... .......... .......... .......... .......... 29%  229M 1s
  5200K .......... .......... .......... .......... .......... 29%  194M 1s
  5250K .......... .......... .......... .......... .......... 29%  212M 1s
  5300K .......... .......... .......... .......... .......... 30%  227M 0s
  5350K .......... .......... .......... .......... .......... 30%  153M 0s
  5400K .......... .......... .......... .......... .......... 30% 92.2M 0s
  5450K .......... .......... .......... .......... .......... 31%  169M 0s
  5500K .......... .......... .......... .......... .......... 31% 97.3M 0s
  5550K .......... .......... .......... .......... .......... 31% 99.3M 0s
  5600K .......... .......... .......... .......... .......... 31%  181M 0s
  5650K .......... .......... .......... .......... .......... 32%  131M 0s
  5700K .......... .......... .......... .......... .......... 32%  227M 0s
  5750K .......... .......... .......... .......... .......... 32%  239M 0s
  5800K .......... .......... .......... .......... .......... 33% 10.7M 0s
  5850K .......... .......... .......... .......... .......... 33%  226M 0s
  5900K .......... .......... .......... .......... .......... 33%  234M 0s
  5950K .......... .......... .......... .......... .......... 33%  233M 0s
  6000K .......... .......... .......... .......... .......... 34%  197M 0s
  6050K .......... .......... .......... .......... .......... 34%  241M 0s
  6100K .......... .......... .......... .......... .......... 34%  243M 0s
  6150K .......... .......... .......... .......... .......... 35%  240M 0s
  6200K .......... .......... .......... .......... .......... 35%  198M 0s
  6250K .......... .......... .......... .......... .......... 35%  213M 0s
  6300K .......... .......... .......... .......... .......... 35%  241M 0s
  6350K .......... .......... .......... .......... .......... 36%  215M 0s
  6400K .......... .......... .......... .......... .......... 36%  202M 0s
  6450K .......... .......... .......... .......... .......... 36%  225M 0s
  6500K .......... .......... .......... .......... .......... 37%  232M 0s
  6550K .......... .......... .......... .......... .......... 37%  236M 0s
  6600K .......... .......... .......... .......... .......... 37%  216M 0s
  6650K .......... .......... .......... .......... .......... 37%  219M 0s
  6700K .......... .......... .......... .......... .......... 38% 5.75M 0s
  6750K .......... .......... .......... .......... .......... 38% 14.7M 0s
  6800K .......... .......... .......... .......... .......... 38% 24.9M 0s
  6850K .......... .......... .......... .......... .......... 39% 7.42M 0s
  6900K .......... .......... .......... .......... .......... 39%  237M 0s
  6950K .......... .......... .......... .......... .......... 39%  228M 0s
  7000K .......... .......... .......... .......... .......... 39%  213M 0s
  7050K .......... .......... .......... .......... .......... 40%  237M 0s
  7100K .......... .......... .......... .......... .......... 40%  234M 0s
  7150K .......... .......... .......... .......... .......... 40%  236M 0s
  7200K .......... .......... .......... .......... .......... 41%  192M 0s
  7250K .......... .......... .......... .......... .......... 41%  220M 0s
  7300K .......... .......... .......... .......... .......... 41%  218M 0s
  7350K .......... .......... .......... .......... .......... 41%  241M 0s
  7400K .......... .......... .......... .......... .......... 42%  177M 0s
  7450K .......... .......... .......... .......... .......... 42%  231M 0s
  7500K .......... .......... .......... .......... .......... 42%  121M 0s
  7550K .......... .......... .......... .......... .......... 43%  101M 0s
  7600K .......... .......... .......... .......... .......... 43% 78.9M 0s
  7650K .......... .......... .......... .......... .......... 43%  171M 0s
  7700K .......... .......... .......... .......... .......... 43%  240M 0s
  7750K .......... .......... .......... .......... .......... 44%  237M 0s
  7800K .......... .......... .......... .......... .......... 44%  192M 0s
  7850K .......... .......... .......... .......... .......... 44%  239M 0s
  7900K .......... .......... .......... .......... .......... 44%  231M 0s
  7950K .......... .......... .......... .......... .......... 45%  239M 0s
  8000K .......... .......... .......... .......... .......... 45%  200M 0s
  8050K .......... .......... .......... .......... .......... 45%  224M 0s
  8100K .......... .......... .......... .......... .......... 46%  228M 0s
  8150K .......... .......... .......... .......... .......... 46%  208M 0s
  8200K .......... .......... .......... .......... .......... 46%  202M 0s
  8250K .......... .......... .......... .......... .......... 46%  179M 0s
  8300K .......... .......... .......... .......... .......... 47%  231M 0s
  8350K .......... .......... .......... .......... .......... 47%  226M 0s
  8400K .......... .......... .......... .......... .......... 47%  181M 0s
  8450K .......... .......... .......... .......... .......... 48%  229M 0s
  8500K .......... .......... .......... .......... .......... 48%  234M 0s
  8550K .......... .......... .......... .......... .......... 48%  228M 0s
  8600K .......... .......... .......... .......... .......... 48%  206M 0s
  8650K .......... .......... .......... .......... .......... 49%  228M 0s
  8700K .......... .......... .......... .......... .......... 49%  218M 0s
  8750K .......... .......... .......... .......... .......... 49%  179M 0s
  8800K .......... .......... .......... .......... .......... 50%  200M 0s
  8850K .......... .......... .......... .......... .......... 50%  226M 0s
  8900K .......... .......... .......... .......... .......... 50% 6.49M 0s
  8950K .......... .......... .......... .......... .......... 50%  107M 0s
  9000K .......... .......... .......... .......... .......... 51%  204M 0s
  9050K .......... .......... .......... .......... .......... 51%  220M 0s
  9100K .......... .......... .......... .......... .......... 51%  210M 0s
  9150K .......... .......... .......... .......... .......... 52%  176M 0s
  9200K .......... .......... .......... .......... .......... 52%  110M 0s
  9250K .......... .......... .......... .......... .......... 52% 28.2M 0s
  9300K .......... .......... .......... .......... .......... 52% 7.83M 0s
  9350K .......... .......... .......... .......... .......... 53% 6.98M 0s
  9400K .......... .......... .......... .......... .......... 53% 25.1M 0s
  9450K .......... .......... .......... .......... .......... 53% 27.1M 0s
  9500K .......... .......... .......... .......... .......... 54%  207M 0s
  9550K .......... .......... .......... .......... .......... 54%  131M 0s
  9600K .......... .......... .......... .......... .......... 54%  210M 0s
  9650K .......... .......... .......... .......... .......... 54%  219M 0s
  9700K .......... .......... .......... .......... .......... 55%  108M 0s
  9750K .......... .......... .......... .......... .......... 55% 72.7M 0s
  9800K .......... .......... .......... .......... .......... 55% 23.1M 0s
  9850K .......... .......... .......... .......... .......... 56% 5.11M 0s
  9900K .......... .......... .......... .......... .......... 56%  190M 0s
  9950K .......... .......... .......... .......... .......... 56%  157M 0s
 10000K .......... .......... .......... .......... .......... 56%  203M 0s
 10050K .......... .......... .......... .......... .......... 57%  200M 0s
 10100K .......... .......... .......... .......... .......... 57%  181M 0s
 10150K .......... .......... .......... .......... .......... 57%  173M 0s
 10200K .......... .......... .......... .......... .......... 57%  176M 0s
 10250K .......... .......... .......... .......... .......... 58%  201M 0s
 10300K .......... .......... .......... .......... .......... 58%  182M 0s
 10350K .......... .......... .......... .......... .......... 58%  162M 0s
 10400K .......... .......... .......... .......... .......... 59%  200M 0s
 10450K .......... .......... .......... .......... .......... 59%  183M 0s
 10500K .......... .......... .......... .......... .......... 59%  198M 0s
 10550K .......... .......... .......... .......... .......... 59%  177M 0s
 10600K .......... .......... .......... .......... .......... 60%  177M 0s
 10650K .......... .......... .......... .......... .......... 60%  174M 0s
 10700K .......... .......... .......... .......... .......... 60%  201M 0s
 10750K .......... .......... .......... .......... .......... 61%  156M 0s
 10800K .......... .......... .......... .......... .......... 61%  191M 0s
 10850K .......... .......... .......... .......... .......... 61%  197M 0s
 10900K .......... .......... .......... .......... .......... 61%  188M 0s
 10950K .......... .......... .......... .......... .......... 62%  173M 0s
 11000K .......... .......... .......... .......... .......... 62%  184M 0s
 11050K .......... .......... .......... .......... .......... 62%  187M 0s
 11100K .......... .......... .......... .......... .......... 63%  199M 0s
 11150K .......... .......... .......... .......... .......... 63%  160M 0s
 11200K .......... .......... .......... .......... .......... 63%  195M 0s
 11250K .......... .......... .......... .......... .......... 63%  192M 0s
 11300K .......... .......... .......... .......... .......... 64%  196M 0s
 11350K .......... .......... .......... .......... .......... 64% 4.81M 0s
 11400K .......... .......... .......... .......... .......... 64% 9.54M 0s
 11450K .......... .......... .......... .......... .......... 65% 23.3M 0s
 11500K .......... .......... .......... .......... .......... 65% 24.4M 0s
 11550K .......... .......... .......... .......... .......... 65% 21.7M 0s
 11600K .......... .......... .......... .......... .......... 65% 7.02M 0s
 11650K .......... .......... .......... .......... .......... 66%  223M 0s
 11700K .......... .......... .......... .......... .......... 66%  151M 0s
 11750K .......... .......... .......... .......... .......... 66%  196M 0s
 11800K .......... .......... .......... .......... .......... 67%  227M 0s
 11850K .......... .......... .......... .......... .......... 67%  223M 0s
 11900K .......... .......... .......... .......... .......... 67%  229M 0s
 11950K .......... .......... .......... .......... .......... 67%  192M 0s
 12000K .......... .......... .......... .......... .......... 68%  228M 0s
 12050K .......... .......... .......... .......... .......... 68%  228M 0s
 12100K .......... .......... .......... .......... .......... 68%  227M 0s
 12150K .......... .......... .......... .......... .......... 69%  207M 0s
 12200K .......... .......... .......... .......... .......... 69%  228M 0s
 12250K .......... .......... .......... .......... .......... 69%  227M 0s
 12300K .......... .......... .......... .......... .......... 69%  230M 0s
 12350K .......... .......... .......... .......... .......... 70%  137M 0s
 12400K .......... .......... .......... .......... .......... 70%  210M 0s
 12450K .......... .......... .......... .......... .......... 70%  230M 0s
 12500K .......... .......... .......... .......... .......... 71%  226M 0s
 12550K .......... .......... .......... .......... .......... 71%  205M 0s
 12600K .......... .......... .......... .......... .......... 71%  199M 0s
 12650K .......... .......... .......... .......... .......... 71%  227M 0s
 12700K .......... .......... .......... .......... .......... 72%  228M 0s
 12750K .......... .......... .......... .......... .......... 72%  193M 0s
 12800K .......... .......... .......... .......... .......... 72%  228M 0s
 12850K .......... .......... .......... .......... .......... 72%  228M 0s
 12900K .......... .......... .......... .......... .......... 73%  232M 0s
 12950K .......... .......... .......... .......... .......... 73%  204M 0s
 13000K .......... .......... .......... .......... .......... 73%  230M 0s
 13050K .......... .......... .......... .......... .......... 74%  227M 0s
 13100K .......... .......... .......... .......... .......... 74%  224M 0s
 13150K .......... .......... .......... .......... .......... 74%  191M 0s
 13200K .......... .......... .......... .......... .......... 74%  225M 0s
 13250K .......... .......... .......... .......... .......... 75%  216M 0s
 13300K .......... .......... .......... .......... .......... 75%  221M 0s
 13350K .......... .......... .......... .......... .......... 75%  203M 0s
 13400K .......... .......... .......... .......... .......... 76%  229M 0s
 13450K .......... .......... .......... .......... .......... 76%  227M 0s
 13500K .......... .......... .......... .......... .......... 76%  229M 0s
 13550K .......... .......... .......... .......... .......... 76%  191M 0s
 13600K .......... .......... .......... .......... .......... 77%  226M 0s
 13650K .......... .......... .......... .......... .......... 77%  222M 0s
 13700K .......... .......... .......... .......... .......... 77%  221M 0s
 13750K .......... .......... .......... .......... .......... 78% 7.92M 0s
 13800K .......... .......... .......... .......... .......... 78%  203M 0s
 13850K .......... .......... .......... .......... .......... 78% 90.0M 0s
 13900K .......... .......... .......... .......... .......... 78%  214M 0s
 13950K .......... .......... .......... .......... .......... 79%  131M 0s
 14000K .......... .......... .......... .......... .......... 79%  217M 0s
 14050K .......... .......... .......... .......... .......... 79%  207M 0s
 14100K .......... .......... .......... .......... .......... 80%  156M 0s
 14150K .......... .......... .......... .......... .......... 80% 12.3M 0s
 14200K .......... .......... .......... .......... .......... 80% 14.1M 0s
 14250K .......... .......... .......... .......... .......... 80% 34.5M 0s
 14300K .......... .......... .......... .......... .......... 81% 34.0M 0s
 14350K .......... .......... .......... .......... .......... 81% 30.5M 0s
 14400K .......... .......... .......... .......... .......... 81% 4.89M 0s
 14450K .......... .......... .......... .......... .......... 82%  139M 0s
 14500K .......... .......... .......... .......... .......... 82%  218M 0s
 14550K .......... .......... .......... .......... .......... 82%  103M 0s
 14600K .......... .......... .......... .......... .......... 82%  157M 0s
 14650K .......... .......... .......... .......... .......... 83%  211M 0s
 14700K .......... .......... .......... .......... .......... 83%  217M 0s
 14750K .......... .......... .......... .......... .......... 83%  136M 0s
 14800K .......... .......... .......... .......... .......... 84%  119M 0s
 14850K .......... .......... .......... .......... .......... 84%  211M 0s
 14900K .......... .......... .......... .......... .......... 84%  150M 0s
 14950K .......... .......... .......... .......... .......... 84%  198M 0s
 15000K .......... .......... .......... .......... .......... 85%  145M 0s
 15050K .......... .......... .......... .......... .......... 85%  203M 0s
 15100K .......... .......... .......... .......... .......... 85%  115M 0s
 15150K .......... .......... .......... .......... .......... 86%  134M 0s
 15200K .......... .......... .......... .......... .......... 86%  189M 0s
 15250K .......... .......... .......... .......... .......... 86%  139M 0s
 15300K .......... .......... .......... .......... .......... 86%  215M 0s
 15350K .......... .......... .......... .......... .......... 87%  107M 0s
 15400K .......... .......... .......... .......... .......... 87%  219M 0s
 15450K .......... .......... .......... .......... .......... 87%  140M 0s
 15500K .......... .......... .......... .......... .......... 87%  224M 0s
 15550K .......... .......... .......... .......... .......... 88%  126M 0s
 15600K .......... .......... .......... .......... .......... 88%  209M 0s
 15650K .......... .......... .......... .......... .......... 88%  216M 0s
 15700K .......... .......... .......... .......... .......... 89%  203M 0s
 15750K .......... .......... .......... .......... .......... 89%  197M 0s
 15800K .......... .......... .......... .......... .......... 89%  201M 0s
 15850K .......... .......... .......... .......... .......... 89%  222M 0s
 15900K .......... .......... .......... .......... .......... 90%  222M 0s
 15950K .......... .......... .......... .......... .......... 90%  173M 0s
 16000K .......... .......... .......... .......... .......... 90%  226M 0s
 16050K .......... .......... .......... .......... .......... 91% 11.5M 0s
 16100K .......... .......... .......... .......... .......... 91% 22.0M 0s
 16150K .......... .......... .......... .......... .......... 91% 20.6M 0s
 16200K .......... .......... .......... .......... .......... 91% 23.4M 0s
 16250K .......... .......... .......... .......... .......... 92% 76.1M 0s
 16300K .......... .......... .......... .......... .......... 92%  145M 0s
 16350K .......... .......... .......... .......... .......... 92%  134M 0s
 16400K .......... .......... .......... .......... .......... 93%  210M 0s
 16450K .......... .......... .......... .......... .......... 93%  197M 0s
 16500K .......... .......... .......... .......... .......... 93%  118M 0s
 16550K .......... .......... .......... .......... .......... 93%  137M 0s
 16600K .......... .......... .......... .......... .......... 94%  208M 0s
 16650K .......... .......... .......... .......... .......... 94%  152M 0s
 16700K .......... .......... .......... .......... .......... 94%  208M 0s
 16750K .......... .......... .......... .......... .......... 95% 5.86M 0s
 16800K .......... .......... .......... .......... .......... 95%  218M 0s
 16850K .......... .......... .......... .......... .......... 95%  213M 0s
 16900K .......... .......... .......... .......... .......... 95%  216M 0s
 16950K .......... .......... .......... .......... .......... 96%  197M 0s
 17000K .......... .......... .......... .......... .......... 96%  220M 0s
 17050K .......... .......... .......... .......... .......... 96%  211M 0s
 17100K .......... .......... .......... .......... .......... 97%  180M 0s
 17150K .......... .......... .......... .......... .......... 97%  205M 0s
 17200K .......... .......... .......... .......... .......... 97%  213M 0s
 17250K .......... .......... .......... .......... .......... 97%  215M 0s
 17300K .......... .......... .......... .......... .......... 98%  195M 0s
 17350K .......... .......... .......... .......... .......... 98%  209M 0s
 17400K .......... .......... .......... .......... .......... 98%  220M 0s
 17450K .......... .......... .......... .......... .......... 99%  202M 0s
 17500K .......... .......... .......... .......... .......... 99%  176M 0s
 17550K .......... .......... .......... .......... .......... 99%  204M 0s
 17600K .......... .......... .......... .......... .......... 99%  212M 0s
 17650K .......... .......... ...                             100%  216M=0.4s

2021-08-22 17:55:24 (41.8 MB/s) - ‘scarf_datasets/tenx_5K_pbmc_rnaseq/data.h5’ saved [18098007/18098007]

--2021-08-22 17:55:24--  https://files.de-1.osf.io/v1/resources/zeupv/providers/osfstorage/6096d5685533b405bfe1eeb3
Resolving files.de-1.osf.io (files.de-1.osf.io)... 35.186.249.111
Connecting to files.de-1.osf.io (files.de-1.osf.io)|35.186.249.111|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://storage.googleapis.com/cos-osf-prod-files-de-1/1ef8b465325d04f0b87ed06fe7813ec6698d1a9a45e3511e885ae45e732f9bb8?response-content-disposition=attachment%3B%20filename%3D%22data.zarr.tar.gz%22%3B%20filename%2A%3DUTF-8%27%27data.zarr.tar.gz&GoogleAccessId=files-de-1%40cos-osf-prod.iam.gserviceaccount.com&Expires=1629654985&Signature=TkUkKS6PQ0P%2F4HtCdmeTrFwOqFvom6yWak2CCSAxUThLY%2FKSnIwDSfxuGAMuzSh44%2BJkn4DmjttJoQKzHuhR%2Bsm%2B9q9Sbi%2BVCn55LBdgZtkHZeK58%2BHJFVv25xdxiaSFDuZpr4MZPSrawke846r%2Bgl8CBTZdr%2BXc8NCiYmIQh%2FgVnfFX%2BxxeEHirGHnqBxwqzPrlXDEnzxd6HRv31c6ICvbnAjWe%2Fa1vNFPPx68jzl%2BwAhcPIGJcEBzFHDX1JWaS%2FpaTxCSKOeFRxxbJQulw58DHcZD4dH3Qb%2BfZqz%2Bmiyvudy2slkOWFDRmEPuqe2JxdO186gtUkXfzV5h5Q9bU8w%3D%3D [following]
--2021-08-22 17:55:25--  https://storage.googleapis.com/cos-osf-prod-files-de-1/1ef8b465325d04f0b87ed06fe7813ec6698d1a9a45e3511e885ae45e732f9bb8?response-content-disposition=attachment%3B%20filename%3D%22data.zarr.tar.gz%22%3B%20filename%2A%3DUTF-8%27%27data.zarr.tar.gz&GoogleAccessId=files-de-1%40cos-osf-prod.iam.gserviceaccount.com&Expires=1629654985&Signature=TkUkKS6PQ0P%2F4HtCdmeTrFwOqFvom6yWak2CCSAxUThLY%2FKSnIwDSfxuGAMuzSh44%2BJkn4DmjttJoQKzHuhR%2Bsm%2B9q9Sbi%2BVCn55LBdgZtkHZeK58%2BHJFVv25xdxiaSFDuZpr4MZPSrawke846r%2Bgl8CBTZdr%2BXc8NCiYmIQh%2FgVnfFX%2BxxeEHirGHnqBxwqzPrlXDEnzxd6HRv31c6ICvbnAjWe%2Fa1vNFPPx68jzl%2BwAhcPIGJcEBzFHDX1JWaS%2FpaTxCSKOeFRxxbJQulw58DHcZD4dH3Qb%2BfZqz%2Bmiyvudy2slkOWFDRmEPuqe2JxdO186gtUkXfzV5h5Q9bU8w%3D%3D
Resolving storage.googleapis.com (storage.googleapis.com)... 172.217.4.208, 142.250.191.112, 142.250.191.144, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|172.217.4.208|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 31491651 (30M) [application/octet-stream]
Saving to: ‘scarf_datasets/tenx_5K_pbmc_rnaseq/data.zarr.tar.gz’

     0K .......... .......... .......... .......... ..........  0% 1.39M 22s
    50K .......... .......... .......... .......... ..........  0% 3.20M 15s
   100K .......... .......... .......... .......... ..........  0% 5.01M 12s
   150K .......... .......... .......... .......... ..........  0% 6.21M 10s
   200K .......... .......... .......... .......... ..........  0% 7.27M 9s
   250K .......... .......... .......... .......... ..........  0% 11.0M 8s
   300K .......... .......... .......... .......... ..........  1% 10.9M 7s
   350K .......... .......... .......... .......... ..........  1% 16.6M 7s
   400K .......... .......... .......... .......... ..........  1% 15.6M 6s
   450K .......... .......... .......... .......... ..........  1% 18.2M 6s
   500K .......... .......... .......... .......... ..........  1% 14.9M 5s
   550K .......... .......... .......... .......... ..........  1% 24.8M 5s
   600K .......... .......... .......... .......... ..........  2% 26.4M 5s
   650K .......... .......... .......... .......... ..........  2% 19.4M 4s
   700K .......... .......... .......... .......... ..........  2% 31.7M 4s
   750K .......... .......... .......... .......... ..........  2% 23.4M 4s
   800K .......... .......... .......... .......... ..........  2% 36.6M 4s
   850K .......... .......... .......... .......... ..........  2% 25.8M 4s
   900K .......... .......... .......... .......... ..........  3% 44.7M 3s
   950K .......... .......... .......... .......... ..........  3% 31.8M 3s
  1000K .......... .......... .......... .......... ..........  3% 26.9M 3s
  1050K .......... .......... .......... .......... ..........  3% 48.9M 3s
  1100K .......... .......... .......... .......... ..........  3% 87.5M 3s
  1150K .......... .......... .......... .......... ..........  3% 41.7M 3s
  1200K .......... .......... .......... .......... ..........  4% 39.8M 3s
  1250K .......... .......... .......... .......... ..........  4% 30.6M 3s
  1300K .......... .......... .......... .......... ..........  4% 78.4M 3s
  1350K .......... .......... .......... .......... ..........  4%  115M 3s
  1400K .......... .......... .......... .......... ..........  4% 35.7M 2s
  1450K .......... .......... .......... .......... ..........  4% 35.2M 2s
  1500K .......... .......... .......... .......... ..........  5%  209M 2s
  1550K .......... .......... .......... .......... ..........  5% 34.6M 2s
  1600K .......... .......... .......... .......... ..........  5% 84.3M 2s
  1650K .......... .......... .......... .......... ..........  5% 48.5M 2s
  1700K .......... .......... .......... .......... ..........  5%  190M 2s
  1750K .......... .......... .......... .......... ..........  5%  175M 2s
  1800K .......... .......... .......... .......... ..........  6% 66.1M 2s
  1850K .......... .......... .......... .......... ..........  6% 59.4M 2s
  1900K .......... .......... .......... .......... ..........  6% 51.0M 2s
  1950K .......... .......... .......... .......... ..........  6% 75.6M 2s
  2000K .......... .......... .......... .......... ..........  6%  209M 2s
  2050K .......... .......... .......... .......... ..........  6% 57.0M 2s
  2100K .......... .......... .......... .......... ..........  6% 81.6M 2s
  2150K .......... .......... .......... .......... ..........  7% 59.3M 2s
  2200K .......... .......... .......... .......... ..........  7%  205M 2s
  2250K .......... .......... .......... .......... ..........  7% 70.4M 2s
  2300K .......... .......... .......... .......... ..........  7% 64.0M 2s
  2350K .......... .......... .......... .......... ..........  7%  144M 2s
  2400K .......... .......... .......... .......... ..........  7% 62.5M 2s
  2450K .......... .......... .......... .......... ..........  8%  183M 2s
  2500K .......... .......... .......... .......... ..........  8%  136M 1s
  2550K .......... .......... .......... .......... ..........  8% 52.5M 1s
  2600K .......... .......... .......... .......... ..........  8% 82.9M 1s
  2650K .......... .......... .......... .......... ..........  8%  101M 1s
  2700K .......... .......... .......... .......... ..........  8%  123M 1s
  2750K .......... .......... .......... .......... ..........  9%  171M 1s
  2800K .......... .......... .......... .......... ..........  9% 51.7M 1s
  2850K .......... .......... .......... .......... ..........  9%  214M 1s
  2900K .......... .......... .......... .......... ..........  9% 89.5M 1s
  2950K .......... .......... .......... .......... ..........  9%  178M 1s
  3000K .......... .......... .......... .......... ..........  9% 62.8M 1s
  3050K .......... .......... .......... .......... .......... 10% 73.0M 1s
  3100K .......... .......... .......... .......... .......... 10% 44.5M 1s
  3150K .......... .......... .......... .......... .......... 10% 41.7M 1s
  3200K .......... .......... .......... .......... .......... 10%  181M 1s
  3250K .......... .......... .......... .......... .......... 10%  215M 1s
  3300K .......... .......... .......... .......... .......... 10% 10.4M 1s
  3350K .......... .......... .......... .......... .......... 11%  111M 1s
  3400K .......... .......... .......... .......... .......... 11%  209M 1s
  3450K .......... .......... .......... .......... .......... 11%  206M 1s
  3500K .......... .......... .......... .......... .......... 11%  213M 1s
  3550K .......... .......... .......... .......... .......... 11%  162M 1s
  3600K .......... .......... .......... .......... .......... 11%  206M 1s
  3650K .......... .......... .......... .......... .......... 12%  209M 1s
  3700K .......... .......... .......... .......... .......... 12%  206M 1s
  3750K .......... .......... .......... .......... .......... 12%  190M 1s
  3800K .......... .......... .......... .......... .......... 12%  199M 1s
  3850K .......... .......... .......... .......... .......... 12%  218M 1s
  3900K .......... .......... .......... .......... .......... 12%  187M 1s
  3950K .......... .......... .......... .......... .......... 13% 43.0M 1s
  4000K .......... .......... .......... .......... .......... 13%  167M 1s
  4050K .......... .......... .......... .......... .......... 13%  796K 1s
  4100K .......... .......... .......... .......... .......... 13% 9.16M 1s
  4150K .......... .......... .......... .......... .......... 13% 9.05M 1s
  4200K .......... .......... .......... .......... .......... 13%  180M 1s
  4250K .......... .......... .......... .......... .......... 13%  231M 1s
  4300K .......... .......... .......... .......... .......... 14%  215M 1s
  4350K .......... .......... .......... .......... .......... 14%  224M 1s
  4400K .......... .......... .......... .......... .......... 14%  179M 1s
  4450K .......... .......... .......... .......... .......... 14%  228M 1s
  4500K .......... .......... .......... .......... .......... 14%  226M 1s
  4550K .......... .......... .......... .......... .......... 14%  223M 1s
  4600K .......... .......... .......... .......... .......... 15%  212M 1s
  4650K .......... .......... .......... .......... .......... 15%  218M 1s
  4700K .......... .......... .......... .......... .......... 15%  221M 1s
  4750K .......... .......... .......... .......... .......... 15%  241M 1s
  4800K .......... .......... .......... .......... .......... 15%  185M 1s
  4850K .......... .......... .......... .......... .......... 15% 9.76M 1s
  4900K .......... .......... .......... .......... .......... 16% 17.7M 1s
  4950K .......... .......... .......... .......... .......... 16% 6.49M 1s
  5000K .......... .......... .......... .......... .......... 16% 16.6M 1s
  5050K .......... .......... .......... .......... .......... 16% 35.4M 1s
  5100K .......... .......... .......... .......... .......... 16%  201M 1s
  5150K .......... .......... .......... .......... .......... 16%  225M 1s
  5200K .......... .......... .......... .......... .......... 17%  174M 1s
  5250K .......... .......... .......... .......... .......... 17%  229M 1s
  5300K .......... .......... .......... .......... .......... 17%  228M 1s
  5350K .......... .......... .......... .......... .......... 17%  223M 1s
  5400K .......... .......... .......... .......... .......... 17%  211M 1s
  5450K .......... .......... .......... .......... .......... 17%  223M 1s
  5500K .......... .......... .......... .......... .......... 18%  224M 1s
  5550K .......... .......... .......... .......... .......... 18%  238M 1s
  5600K .......... .......... .......... .......... .......... 18%  171M 1s
  5650K .......... .......... .......... .......... .......... 18%  212M 1s
  5700K .......... .......... .......... .......... .......... 18%  201M 1s
  5750K .......... .......... .......... .......... .......... 18%  204M 1s
  5800K .......... .......... .......... .......... .......... 19%  191M 1s
  5850K .......... .......... .......... .......... .......... 19%  195M 1s
  5900K .......... .......... .......... .......... .......... 19%  205M 1s
  5950K .......... .......... .......... .......... .......... 19%  221M 1s
  6000K .......... .......... .......... .......... .......... 19%  183M 1s
  6050K .......... .......... .......... .......... .......... 19% 11.3M 1s
  6100K .......... .......... .......... .......... .......... 19%  238M 1s
  6150K .......... .......... .......... .......... .......... 20%  244M 1s
  6200K .......... .......... .......... .......... .......... 20%  204M 1s
  6250K .......... .......... .......... .......... .......... 20%  238M 1s
  6300K .......... .......... .......... .......... .......... 20%  234M 1s
  6350K .......... .......... .......... .......... .......... 20%  235M 1s
  6400K .......... .......... .......... .......... .......... 20%  198M 1s
  6450K .......... .......... .......... .......... .......... 21%  229M 1s
  6500K .......... .......... .......... .......... .......... 21%  242M 1s
  6550K .......... .......... .......... .......... .......... 21%  226M 1s
  6600K .......... .......... .......... .......... .......... 21%  219M 1s
  6650K .......... .......... .......... .......... .......... 21%  237M 1s
  6700K .......... .......... .......... .......... .......... 21%  221M 1s
  6750K .......... .......... .......... .......... .......... 22%  243M 1s
  6800K .......... .......... .......... .......... .......... 22%  200M 1s
  6850K .......... .......... .......... .......... .......... 22%  222M 1s
  6900K .......... .......... .......... .......... .......... 22%  243M 1s
  6950K .......... .......... .......... .......... .......... 22%  221M 1s
  7000K .......... .......... .......... .......... .......... 22%  203M 1s
  7050K .......... .......... .......... .......... .......... 23%  240M 1s
  7100K .......... .......... .......... .......... .......... 23%  227M 1s
  7150K .......... .......... .......... .......... .......... 23%  231M 1s
  7200K .......... .......... .......... .......... .......... 23% 3.97M 1s
  7250K .......... .......... .......... .......... .......... 23% 18.6M 1s
  7300K .......... .......... .......... .......... .......... 23% 87.7M 1s
  7350K .......... .......... .......... .......... .......... 24% 94.5M 1s
  7400K .......... .......... .......... .......... .......... 24%  144M 1s
  7450K .......... .......... .......... .......... .......... 24%  155M 1s
  7500K .......... .......... .......... .......... .......... 24%  158M 1s
  7550K .......... .......... .......... .......... .......... 24%  167M 1s
  7600K .......... .......... .......... .......... .......... 24%  134M 1s
  7650K .......... .......... .......... .......... .......... 25%  153M 1s
  7700K .......... .......... .......... .......... .......... 25%  168M 1s
  7750K .......... .......... .......... .......... .......... 25%  157M 1s
  7800K .......... .......... .......... .......... .......... 25%  139M 1s
  7850K .......... .......... .......... .......... .......... 25% 7.71M 1s
  7900K .......... .......... .......... .......... .......... 25% 30.4M 1s
  7950K .......... .......... .......... .......... .......... 26% 13.1M 1s
  8000K .......... .......... .......... .......... .......... 26%  190M 1s
  8050K .......... .......... .......... .......... .......... 26%  171M 1s
  8100K .......... .......... .......... .......... .......... 26%  176M 1s
  8150K .......... .......... .......... .......... .......... 26%  414K 1s
  8200K .......... .......... .......... .......... .......... 26% 6.24M 1s
  8250K .......... .......... .......... .......... .......... 26% 7.19M 1s
  8300K .......... .......... .......... .......... .......... 27% 34.2M 1s
  8350K .......... .......... .......... .......... .......... 27% 35.3M 1s
  8400K .......... .......... .......... .......... .......... 27% 71.2M 1s
  8450K .......... .......... .......... .......... .......... 27%  222M 1s
  8500K .......... .......... .......... .......... .......... 27% 61.5M 1s
  8550K .......... .......... .......... .......... .......... 27% 37.0M 1s
  8600K .......... .......... .......... .......... .......... 28%  128M 1s
  8650K .......... .......... .......... .......... .......... 28% 86.2M 1s
  8700K .......... .......... .......... .......... .......... 28%  169M 1s
  8750K .......... .......... .......... .......... .......... 28%  104M 1s
  8800K .......... .......... .......... .......... .......... 28%  117M 1s
  8850K .......... .......... .......... .......... .......... 28% 46.4M 1s
  8900K .......... .......... .......... .......... .......... 29% 6.46M 1s
  8950K .......... .......... .......... .......... .......... 29%  236M 1s
  9000K .......... .......... .......... .......... .......... 29%  212M 1s
  9050K .......... .......... .......... .......... .......... 29%  245M 1s
  9100K .......... .......... .......... .......... .......... 29%  238M 1s
  9150K .......... .......... .......... .......... .......... 29%  211M 1s
  9200K .......... .......... .......... .......... .......... 30%  201M 1s
  9250K .......... .......... .......... .......... .......... 30%  234M 1s
  9300K .......... .......... .......... .......... .......... 30%  214M 1s
  9350K .......... .......... .......... .......... .......... 30%  231M 1s
  9400K .......... .......... .......... .......... .......... 30%  214M 1s
  9450K .......... .......... .......... .......... .......... 30%  227M 1s
  9500K .......... .......... .......... .......... .......... 31%  240M 1s
  9550K .......... .......... .......... .......... .......... 31%  227M 1s
  9600K .......... .......... .......... .......... .......... 31%  193M 1s
  9650K .......... .......... .......... .......... .......... 31%  196M 1s
  9700K .......... .......... .......... .......... .......... 31%  227M 1s
  9750K .......... .......... .......... .......... .......... 31%  221M 1s
  9800K .......... .......... .......... .......... .......... 32%  195M 1s
  9850K .......... .......... .......... .......... .......... 32%  236M 1s
  9900K .......... .......... .......... .......... .......... 32%  227M 1s
  9950K .......... .......... .......... .......... .......... 32%  221M 1s
 10000K .......... .......... .......... .......... .......... 32%  132M 1s
 10050K .......... .......... .......... .......... .......... 32%  107M 1s
 10100K .......... .......... .......... .......... .......... 33%  215M 1s
 10150K .......... .......... .......... .......... .......... 33%  238M 1s
 10200K .......... .......... .......... .......... .......... 33%  141M 1s
 10250K .......... .......... .......... .......... .......... 33%  219M 1s
 10300K .......... .......... .......... .......... .......... 33%  244M 1s
 10350K .......... .......... .......... .......... .......... 33%  233M 1s
 10400K .......... .......... .......... .......... .......... 33%  191M 1s
 10450K .......... .......... .......... .......... .......... 34%  123M 1s
 10500K .......... .......... .......... .......... .......... 34%  221M 1s
 10550K .......... .......... .......... .......... .......... 34%  235M 1s
 10600K .......... .......... .......... .......... .......... 34%  219M 1s
 10650K .......... .......... .......... .......... .......... 34%  224M 1s
 10700K .......... .......... .......... .......... .......... 34%  238M 1s
 10750K .......... .......... .......... .......... .......... 35%  218M 1s
 10800K .......... .......... .......... .......... .......... 35%  130M 1s
 10850K .......... .......... .......... .......... .......... 35%  229M 1s
 10900K .......... .......... .......... .......... .......... 35%  221M 1s
 10950K .......... .......... .......... .......... .......... 35%  243M 1s
 11000K .......... .......... .......... .......... .......... 35% 8.81M 1s
 11050K .......... .......... .......... .......... .......... 36% 10.1M 1s
 11100K .......... .......... .......... .......... .......... 36% 15.7M 1s
 11150K .......... .......... .......... .......... .......... 36% 21.3M 1s
 11200K .......... .......... .......... .......... .......... 36%  165M 1s
 11250K .......... .......... .......... .......... .......... 36%  133M 1s
 11300K .......... .......... .......... .......... .......... 36%  229M 1s
 11350K .......... .......... .......... .......... .......... 37%  159M 1s
 11400K .......... .......... .......... .......... .......... 37%  192M 1s
 11450K .......... .......... .......... .......... .......... 37%  236M 1s
 11500K .......... .......... .......... .......... .......... 37%  230M 1s
 11550K .......... .......... .......... .......... .......... 37% 7.18M 1s
 11600K .......... .......... .......... .......... .......... 37% 25.0M 1s
 11650K .......... .......... .......... .......... .......... 38% 18.2M 1s
 11700K .......... .......... .......... .......... .......... 38%  112M 1s
 11750K .......... .......... .......... .......... .......... 38%  235M 1s
 11800K .......... .......... .......... .......... .......... 38%  149M 1s
 11850K .......... .......... .......... .......... .......... 38%  213M 1s
 11900K .......... .......... .......... .......... .......... 38%  235M 1s
 11950K .......... .......... .......... .......... .......... 39%  165M 1s
 12000K .......... .......... .......... .......... .......... 39%  183M 1s
 12050K .......... .......... .......... .......... .......... 39%  226M 1s
 12100K .......... .......... .......... .......... .......... 39%  109M 1s
 12150K .......... .......... .......... .......... .......... 39%  207M 1s
 12200K .......... .......... .......... .......... .......... 39%  142M 1s
 12250K .......... .......... .......... .......... .......... 39%  217M 1s
 12300K .......... .......... .......... .......... .......... 40%  181M 1s
 12350K .......... .......... .......... .......... .......... 40%  143M 1s
 12400K .......... .......... .......... .......... .......... 40%  213M 1s
 12450K .......... .......... .......... .......... .......... 40%  210M 1s
 12500K .......... .......... .......... .......... .......... 40% 5.69M 1s
 12550K .......... .......... .......... .......... .......... 40%  202M 1s
 12600K .......... .......... .......... .......... .......... 41%  121M 1s
 12650K .......... .......... .......... .......... .......... 41%  222M 1s
 12700K .......... .......... .......... .......... .......... 41%  144M 1s
 12750K .......... .......... .......... .......... .......... 41%  182M 1s
 12800K .......... .......... .......... .......... .......... 41%  225M 1s
 12850K .......... .......... .......... .......... .......... 41%  227M 1s
 12900K .......... .......... .......... .......... .......... 42%  212M 1s
 12950K .......... .......... .......... .......... .......... 42%  182M 1s
 13000K .......... .......... .......... .......... .......... 42%  222M 1s
 13050K .......... .......... .......... .......... .......... 42%  209M 1s
 13100K .......... .......... .......... .......... .......... 42%  111M 1s
 13150K .......... .......... .......... .......... .......... 42%  119M 1s
 13200K .......... .......... .......... .......... .......... 43%  216M 1s
 13250K .......... .......... .......... .......... .......... 43%  220M 1s
 13300K .......... .......... .......... .......... .......... 43%  216M 1s
 13350K .......... .......... .......... .......... .......... 43%  191M 1s
 13400K .......... .......... .......... .......... .......... 43%  227M 1s
 13450K .......... .......... .......... .......... .......... 43%  182M 1s
 13500K .......... .......... .......... .......... .......... 44%  168M 1s
 13550K .......... .......... .......... .......... .......... 44%  175M 1s
 13600K .......... .......... .......... .......... .......... 44% 91.3M 1s
 13650K .......... .......... .......... .......... .......... 44%  208M 1s
 13700K .......... .......... .......... .......... .......... 44%  213M 1s
 13750K .......... .......... .......... .......... .......... 44%  140M 1s
 13800K .......... .......... .......... .......... .......... 45%  221M 1s
 13850K .......... .......... .......... .......... .......... 45%  149M 1s
 13900K .......... .......... .......... .......... .......... 45%  199M 1s
 13950K .......... .......... .......... .......... .......... 45%  181M 1s
 14000K .......... .......... .......... .......... .......... 45%  115M 1s
 14050K .......... .......... .......... .......... .......... 45%  221M 1s
 14100K .......... .......... .......... .......... .......... 46%  144M 1s
 14150K .......... .......... .......... .......... .......... 46%  189M 1s
 14200K .......... .......... .......... .......... .......... 46%  208M 1s
 14250K .......... .......... .......... .......... .......... 46%  149M 1s
 14300K .......... .......... .......... .......... .......... 46% 7.51M 1s
 14350K .......... .......... .......... .......... .......... 46% 10.7M 1s
 14400K .......... .......... .......... .......... .......... 46% 17.2M 1s
 14450K .......... .......... .......... .......... .......... 47%  201M 1s
 14500K .......... .......... .......... .......... .......... 47%  222M 1s
 14550K .......... .......... .......... .......... .......... 47%  112M 1s
 14600K .......... .......... .......... .......... .......... 47%  218M 1s
 14650K .......... .......... .......... .......... .......... 47%  150M 1s
 14700K .......... .......... .......... .......... .......... 47%  216M 1s
 14750K .......... .......... .......... .......... .......... 48%  182M 1s
 14800K .......... .......... .......... .......... .......... 48%  111M 1s
 14850K .......... .......... .......... .......... .......... 48%  209M 1s
 14900K .......... .......... .......... .......... .......... 48%  139M 1s
 14950K .......... .......... .......... .......... .......... 48%  191M 1s
 15000K .......... .......... .......... .......... .......... 48%  224M 1s
 15050K .......... .......... .......... .......... .......... 49%  154M 1s
 15100K .......... .......... .......... .......... .......... 49%  202M 1s
 15150K .......... .......... .......... .......... .......... 49%  180M 1s
 15200K .......... .......... .......... .......... .......... 49%  103M 1s
 15250K .......... .......... .......... .......... .......... 49% 13.0M 1s
 15300K .......... .......... .......... .......... .......... 49% 10.9M 1s
 15350K .......... .......... .......... .......... .......... 50% 9.89M 1s
 15400K .......... .......... .......... .......... .......... 50% 34.7M 1s
 15450K .......... .......... .......... .......... .......... 50% 34.8M 1s
 15500K .......... .......... .......... .......... .......... 50% 34.7M 1s
 15550K .......... .......... .......... .......... .......... 50% 30.7M 1s
 15600K .......... .......... .......... .......... .......... 50%  198M 1s
 15650K .......... .......... .......... .......... .......... 51%  232M 1s
 15700K .......... .......... .......... .......... .......... 51%  204M 1s
 15750K .......... .......... .......... .......... .......... 51%  191M 1s
 15800K .......... .......... .......... .......... .......... 51%  226M 1s
 15850K .......... .......... .......... .......... .......... 51%  219M 1s
 15900K .......... .......... .......... .......... .......... 51%  203M 1s
 15950K .......... .......... .......... .......... .......... 52%  189M 1s
 16000K .......... .......... .......... .......... .......... 52%  200M 1s
 16050K .......... .......... .......... .......... .......... 52%  213M 0s
 16100K .......... .......... .......... .......... .......... 52%  218M 0s
 16150K .......... .......... .......... .......... .......... 52%  189M 0s
 16200K .......... .......... .......... .......... .......... 52%  214M 0s
 16250K .......... .......... .......... .......... .......... 53%  213M 0s
 16300K .......... .......... .......... .......... .......... 53%  211M 0s
 16350K .......... .......... .......... .......... .......... 53%  820K 1s
 16400K .......... .......... .......... .......... .......... 53% 40.7M 1s
 16450K .......... .......... .......... .......... .......... 53% 28.7M 1s
 16500K .......... .......... .......... .......... .......... 53% 53.3M 1s
 16550K .......... .......... .......... .......... .......... 53% 40.3M 1s
 16600K .......... .......... .......... .......... .......... 54% 28.3M 1s
 16650K .......... .......... .......... .......... .......... 54% 37.9M 1s
 16700K .......... .......... .......... .......... .......... 54% 62.4M 1s
 16750K .......... .......... .......... .......... .......... 54% 31.0M 1s
 16800K .......... .......... .......... .......... .......... 54% 32.9M 1s
 16850K .......... .......... .......... .......... .......... 54% 32.1M 1s
 16900K .......... .......... .......... .......... .......... 55% 66.4M 1s
 16950K .......... .......... .......... .......... .......... 55%  211M 1s
 17000K .......... .......... .......... .......... .......... 55%  170M 1s
 17050K .......... .......... .......... .......... .......... 55%  203M 0s
 17100K .......... .......... .......... .......... .......... 55%  220M 0s
 17150K .......... .......... .......... .......... .......... 55%  216M 0s
 17200K .......... .......... .......... .......... .......... 56%  192M 0s
 17250K .......... .......... .......... .......... .......... 56%  208M 0s
 17300K .......... .......... .......... .......... .......... 56%  240M 0s
 17350K .......... .......... .......... .......... .......... 56%  222M 0s
 17400K .......... .......... .......... .......... .......... 56%  197M 0s
 17450K .......... .......... .......... .......... .......... 56%  237M 0s
 17500K .......... .......... .......... .......... .......... 57%  221M 0s
 17550K .......... .......... .......... .......... .......... 57%  239M 0s
 17600K .......... .......... .......... .......... .......... 57%  193M 0s
 17650K .......... .......... .......... .......... .......... 57%  208M 0s
 17700K .......... .......... .......... .......... .......... 57%  246M 0s
 17750K .......... .......... .......... .......... .......... 57%  227M 0s
 17800K .......... .......... .......... .......... .......... 58%  191M 0s
 17850K .......... .......... .......... .......... .......... 58%  221M 0s
 17900K .......... .......... .......... .......... .......... 58%  235M 0s
 17950K .......... .......... .......... .......... .......... 58%  196M 0s
 18000K .......... .......... .......... .......... .......... 58% 10.2M 0s
 18050K .......... .......... .......... .......... .......... 58% 78.1M 0s
 18100K .......... .......... .......... .......... .......... 59% 20.0M 0s
 18150K .......... .......... .......... .......... .......... 59% 33.4M 0s
 18200K .......... .......... .......... .......... .......... 59% 31.4M 0s
 18250K .......... .......... .......... .......... .......... 59% 52.4M 0s
 18300K .......... .......... .......... .......... .......... 59% 48.0M 0s
 18350K .......... .......... .......... .......... .......... 59% 19.0M 0s
 18400K .......... .......... .......... .......... .......... 59% 26.0M 0s
 18450K .......... .......... .......... .......... .......... 60% 27.4M 0s
 18500K .......... .......... .......... .......... .......... 60% 30.0M 0s
 18550K .......... .......... .......... .......... .......... 60% 55.3M 0s
 18600K .......... .......... .......... .......... .......... 60%  175M 0s
 18650K .......... .......... .......... .......... .......... 60%  213M 0s
 18700K .......... .......... .......... .......... .......... 60%  233M 0s
 18750K .......... .......... .......... .......... .......... 61%  227M 0s
 18800K .......... .......... .......... .......... .......... 61%  178M 0s
 18850K .......... .......... .......... .......... .......... 61%  225M 0s
 18900K .......... .......... .......... .......... .......... 61%  225M 0s
 18950K .......... .......... .......... .......... .......... 61%  229M 0s
 19000K .......... .......... .......... .......... .......... 61%  195M 0s
 19050K .......... .......... .......... .......... .......... 62%  222M 0s
 19100K .......... .......... .......... .......... .......... 62%  228M 0s
 19150K .......... .......... .......... .......... .......... 62%  229M 0s
 19200K .......... .......... .......... .......... .......... 62%  187M 0s
 19250K .......... .......... .......... .......... .......... 62% 16.8M 0s
 19300K .......... .......... .......... .......... .......... 62% 53.5M 0s
 19350K .......... .......... .......... .......... .......... 63% 29.5M 0s
 19400K .......... .......... .......... .......... .......... 63% 25.5M 0s
 19450K .......... .......... .......... .......... .......... 63% 38.6M 0s
 19500K .......... .......... .......... .......... .......... 63% 24.8M 0s
 19550K .......... .......... .......... .......... .......... 63%  166M 0s
 19600K .......... .......... .......... .......... .......... 63% 53.9M 0s
 19650K .......... .......... .......... .......... .......... 64%  223M 0s
 19700K .......... .......... .......... .......... .......... 64%  210M 0s
 19750K .......... .......... .......... .......... .......... 64%  219M 0s
 19800K .......... .......... .......... .......... .......... 64%  201M 0s
 19850K .......... .......... .......... .......... .......... 64%  231M 0s
 19900K .......... .......... .......... .......... .......... 64%  244M 0s
 19950K .......... .......... .......... .......... .......... 65%  231M 0s
 20000K .......... .......... .......... .......... .......... 65%  186M 0s
 20050K .......... .......... .......... .......... .......... 65%  247M 0s
 20100K .......... .......... .......... .......... .......... 65%  227M 0s
 20150K .......... .......... .......... .......... .......... 65%  220M 0s
 20200K .......... .......... .......... .......... .......... 65%  213M 0s
 20250K .......... .......... .......... .......... .......... 66%  228M 0s
 20300K .......... .......... .......... .......... .......... 66%  227M 0s
 20350K .......... .......... .......... .......... .......... 66%  237M 0s
 20400K .......... .......... .......... .......... .......... 66%  187M 0s
 20450K .......... .......... .......... .......... .......... 66%  224M 0s
 20500K .......... .......... .......... .......... .......... 66%  238M 0s
 20550K .......... .......... .......... .......... .......... 66%  230M 0s
 20600K .......... .......... .......... .......... .......... 67% 79.3M 0s
 20650K .......... .......... .......... .......... .......... 67% 32.0M 0s
 20700K .......... .......... .......... .......... .......... 67% 84.4M 0s
 20750K .......... .......... .......... .......... .......... 67%  185M 0s
 20800K .......... .......... .......... .......... .......... 67%  194M 0s
 20850K .......... .......... .......... .......... .......... 67%  242M 0s
 20900K .......... .......... .......... .......... .......... 68%  192M 0s
 20950K .......... .......... .......... .......... .......... 68%  194M 0s
 21000K .......... .......... .......... .......... .......... 68% 80.1M 0s
 21050K .......... .......... .......... .......... .......... 68% 23.1M 0s
 21100K .......... .......... .......... .......... .......... 68% 32.5M 0s
 21150K .......... .......... .......... .......... .......... 68% 15.6M 0s
 21200K .......... .......... .......... .......... .......... 69% 26.0M 0s
 21250K .......... .......... .......... .......... .......... 69% 40.9M 0s
 21300K .......... .......... .......... .......... .......... 69%  201M 0s
 21350K .......... .......... .......... .......... .......... 69%  190M 0s
 21400K .......... .......... .......... .......... .......... 69%  206M 0s
 21450K .......... .......... .......... .......... .......... 69%  214M 0s
 21500K .......... .......... .......... .......... .......... 70%  213M 0s
 21550K .......... .......... .......... .......... .......... 70%  173M 0s
 21600K .......... .......... .......... .......... .......... 70%  203M 0s
 21650K .......... .......... .......... .......... .......... 70% 16.6M 0s
 21700K .......... .......... .......... .......... .......... 70% 24.4M 0s
 21750K .......... .......... .......... .......... .......... 70% 29.8M 0s
 21800K .......... .......... .......... .......... .......... 71% 89.7M 0s
 21850K .......... .......... .......... .......... .......... 71%  191M 0s
 21900K .......... .......... .......... .......... .......... 71%  214M 0s
 21950K .......... .......... .......... .......... .......... 71%  182M 0s
 22000K .......... .......... .......... .......... .......... 71%  181M 0s
 22050K .......... .......... .......... .......... .......... 71%  200M 0s
 22100K .......... .......... .......... .......... .......... 72%  216M 0s
 22150K .......... .......... .......... .......... .......... 72%  181M 0s
 22200K .......... .......... .......... .......... .......... 72%  218M 0s
 22250K .......... .......... .......... .......... .......... 72%  212M 0s
 22300K .......... .......... .......... .......... .......... 72%  218M 0s
 22350K .......... .......... .......... .......... .......... 72%  179M 0s
 22400K .......... .......... .......... .......... .......... 72%  210M 0s
 22450K .......... .......... .......... .......... .......... 73%  204M 0s
 22500K .......... .......... .......... .......... .......... 73%  210M 0s
 22550K .......... .......... .......... .......... .......... 73%  194M 0s
 22600K .......... .......... .......... .......... .......... 73%  219M 0s
 22650K .......... .......... .......... .......... .......... 73%  199M 0s
 22700K .......... .......... .......... .......... .......... 73%  219M 0s
 22750K .......... .......... .......... .......... .......... 74%  182M 0s
 22800K .......... .......... .......... .......... .......... 74%  187M 0s
 22850K .......... .......... .......... .......... .......... 74% 84.2M 0s
 22900K .......... .......... .......... .......... .......... 74% 89.7M 0s
 22950K .......... .......... .......... .......... .......... 74%  185M 0s
 23000K .......... .......... .......... .......... .......... 74%  213M 0s
 23050K .......... .......... .......... .......... .......... 75%  221M 0s
 23100K .......... .......... .......... .......... .......... 75%  197M 0s
 23150K .......... .......... .......... .......... .......... 75%  181M 0s
 23200K .......... .......... .......... .......... .......... 75%  220M 0s
 23250K .......... .......... .......... .......... .......... 75%  188M 0s
 23300K .......... .......... .......... .......... .......... 75%  216M 0s
 23350K .......... .......... .......... .......... .......... 76%  196M 0s
 23400K .......... .......... .......... .......... .......... 76%  209M 0s
 23450K .......... .......... .......... .......... .......... 76%  209M 0s
 23500K .......... .......... .......... .......... .......... 76% 51.3M 0s
 23550K .......... .......... .......... .......... .......... 76%  162M 0s
 23600K .......... .......... .......... .......... .......... 76% 83.8M 0s
 23650K .......... .......... .......... .......... .......... 77%  203M 0s
 23700K .......... .......... .......... .......... .......... 77% 32.6M 0s
 23750K .......... .......... .......... .......... .......... 77% 72.6M 0s
 23800K .......... .......... .......... .......... .......... 77%  139M 0s
 23850K .......... .......... .......... .......... .......... 77%  213M 0s
 23900K .......... .......... .......... .......... .......... 77% 76.3M 0s
 23950K .......... .......... .......... .......... .......... 78% 52.2M 0s
 24000K .......... .......... .......... .......... .......... 78%  106M 0s
 24050K .......... .......... .......... .......... .......... 78% 79.2M 0s
 24100K .......... .......... .......... .......... .......... 78%  203M 0s
 24150K .......... .......... .......... .......... .......... 78% 53.8M 0s
 24200K .......... .......... .......... .......... .......... 78%  107M 0s
 24250K .......... .......... .......... .......... .......... 79% 75.6M 0s
 24300K .......... .......... .......... .......... .......... 79%  216M 0s
 24350K .......... .......... .......... .......... .......... 79% 55.1M 0s
 24400K .......... .......... .......... .......... .......... 79%  109M 0s
 24450K .......... .......... .......... .......... .......... 79% 74.9M 0s
 24500K .......... .......... .......... .......... .......... 79% 62.5M 0s
 24550K .......... .......... ......
INFO: Download finished! File saved here: scarf_datasets/tenx_5K_pbmc_rnaseq/data.zarr.tar.gz
time: 10.2 s (started: 2021-08-22 17:55:17 +00:00)
.... .......... .......... 79%  222K 0s
 24600K .......... .......... .......... .......... .......... 80% 75.4M 0s
 24650K .......... .......... .......... .......... .......... 80% 93.0M 0s
 24700K .......... .......... .......... .......... .......... 80% 68.5M 0s
 24750K .......... .......... .......... .......... .......... 80% 86.6M 0s
 24800K .......... .......... .......... .......... .......... 80%  109M 0s
 24850K .......... .......... .......... .......... .......... 80% 65.8M 0s
 24900K .......... .......... .......... .......... .......... 81% 88.4M 0s
 24950K .......... .......... .......... .......... .......... 81% 75.6M 0s
 25000K .......... .......... .......... .......... .......... 81%  192M 0s
 25050K .......... .......... .......... .......... .......... 81% 70.3M 0s
 25100K .......... .......... .......... .......... .......... 81% 73.2M 0s
 25150K .......... .......... .......... .......... .......... 81% 73.7M 0s
 25200K .......... .......... .......... .......... .......... 82% 68.1M 0s
 25250K .......... .......... .......... .......... .......... 82%  189M 0s
 25300K .......... .......... .......... .......... .......... 82% 75.9M 0s
 25350K .......... .......... .......... .......... .......... 82% 47.4M 0s
 25400K .......... .......... .......... .......... .......... 82% 85.8M 0s
 25450K .......... .......... .......... .......... .......... 82%  230M 0s
 25500K .......... .......... .......... .......... .......... 83% 93.6M 0s
 25550K .......... .......... .......... .......... .......... 83% 78.5M 0s
 25600K .......... .......... .......... .......... .......... 83% 55.0M 0s
 25650K .......... .......... .......... .......... .......... 83% 95.3M 0s
 25700K .......... .......... .......... .......... .......... 83%  170M 0s
 25750K .......... .......... .......... .......... .......... 83% 70.6M 0s
 25800K .......... .......... .......... .......... .......... 84% 58.3M 0s
 25850K .......... .......... .......... .......... .......... 84%  105M 0s
 25900K .......... .......... .......... .......... .......... 84%  173M 0s
 25950K .......... .......... .......... .......... .......... 84% 72.4M 0s
 26000K .......... .......... .......... .......... .......... 84% 59.1M 0s
 26050K .......... .......... .......... .......... .......... 84% 48.2M 0s
 26100K .......... .......... .......... .......... .......... 85% 82.2M 0s
 26150K .......... .......... .......... .......... .......... 85%  218M 0s
 26200K .......... .......... .......... .......... .......... 85% 55.4M 0s
 26250K .......... .......... .......... .......... .......... 85%  214M 0s
 26300K .......... .......... .......... .......... .......... 85% 24.1M 0s
 26350K .......... .......... .......... .......... .......... 85%  197M 0s
 26400K .......... .......... .......... .......... .......... 86%  179M 0s
 26450K .......... .......... .......... .......... .......... 86%  227M 0s
 26500K .......... .......... .......... .......... .......... 86%  221M 0s
 26550K .......... .......... .......... .......... .......... 86% 53.6M 0s
 26600K .......... .......... .......... .......... .......... 86% 5.67M 0s
 26650K .......... .......... .......... .......... .......... 86%  218M 0s
 26700K .......... .......... .......... .......... .......... 86%  208M 0s
 26750K .......... .......... .......... .......... .......... 87%  220M 0s
 26800K .......... .......... .......... .......... .......... 87%  184M 0s
 26850K .......... .......... .......... .......... .......... 87%  223M 0s
 26900K .......... .......... .......... .......... .......... 87%  222M 0s
 26950K .......... .......... .......... .......... .......... 87%  241M 0s
 27000K .......... .......... .......... .......... .......... 87%  175M 0s
 27050K .......... .......... .......... .......... .......... 88% 7.64M 0s
 27100K .......... .......... .......... .......... .......... 88% 9.17M 0s
 27150K .......... .......... .......... .......... .......... 88% 11.3M 0s
 27200K .......... .......... .......... .......... .......... 88%  183M 0s
 27250K .......... .......... .......... .......... .......... 88%  211M 0s
 27300K .......... .......... .......... .......... .......... 88%  209M 0s
 27350K .......... .......... .......... .......... .......... 89%  229M 0s
 27400K .......... .......... .......... .......... .......... 89%  198M 0s
 27450K .......... .......... .......... .......... .......... 89%  211M 0s
 27500K .......... .......... .......... .......... .......... 89%  240M 0s
 27550K .......... .......... .......... .......... .......... 89%  226M 0s
 27600K .......... .......... .......... .......... .......... 89%  181M 0s
 27650K .......... .......... .......... .......... .......... 90%  212M 0s
 27700K .......... .......... .......... .......... .......... 90%  226M 0s
 27750K .......... .......... .......... .......... .......... 90%  207M 0s
 27800K .......... .......... .......... .......... .......... 90%  204M 0s
 27850K .......... .......... .......... .......... .......... 90%  238M 0s
 27900K .......... .......... .......... .......... .......... 90%  228M 0s
 27950K .......... .......... .......... .......... .......... 91%  222M 0s
 28000K .......... .......... .......... .......... .......... 91%  183M 0s
 28050K .......... .......... .......... .......... .......... 91% 8.18M 0s
 28100K .......... .......... .......... .......... .......... 91% 8.05M 0s
 28150K .......... .......... .......... .......... .......... 91% 20.2M 0s
 28200K .......... .......... .......... .......... .......... 91% 4.85M 0s
 28250K .......... .......... .......... .......... .......... 92% 18.7M 0s
 28300K .......... .......... .......... .......... .......... 92% 53.0M 0s
 28350K .......... .......... .......... .......... .......... 92%  175M 0s
 28400K .......... .......... .......... .......... .......... 92%  216M 0s
 28450K .......... .......... .......... .......... .......... 92%  227M 0s
 28500K .......... .......... .......... .......... .......... 92%  230M 0s
 28550K .......... .......... .......... .......... .......... 92%  202M 0s
 28600K .......... .......... .......... .......... .......... 93%  231M 0s
 28650K .......... .......... .......... .......... .......... 93%  222M 0s
 28700K .......... .......... .......... .......... .......... 93%  248M 0s
 28750K .......... .......... .......... .......... .......... 93%  250M 0s
 28800K .......... .......... .......... .......... .......... 93%  208M 0s
 28850K .......... .......... .......... .......... .......... 93%  248M 0s
 28900K .......... .......... .......... .......... .......... 94%  249M 0s
 28950K .......... .......... .......... .......... .......... 94%  251M 0s
 29000K .......... .......... .......... .......... .......... 94%  223M 0s
 29050K .......... .......... .......... .......... .......... 94%  249M 0s
 29100K .......... .......... .......... .......... .......... 94%  246M 0s
 29150K .......... .......... .......... .......... .......... 94%  238M 0s
 29200K .......... .......... .......... .......... .......... 95%  209M 0s
 29250K .......... .......... .......... .......... .......... 95%  246M 0s
 29300K .......... .......... .......... .......... .......... 95% 13.3M 0s
 29350K .......... .......... .......... .......... .......... 95% 4.37M 0s
 29400K .......... .......... .......... .......... .......... 95% 17.5M 0s
 29450K .......... .......... .......... .......... .......... 95% 20.4M 0s
 29500K .......... .......... .......... .......... .......... 96% 19.0M 0s
 29550K .......... .......... .......... .......... .......... 96% 49.2M 0s
 29600K .......... .......... .......... .......... .......... 96%  201M 0s
 29650K .......... .......... .......... .......... .......... 96%  242M 0s
 29700K .......... .......... .......... .......... .......... 96% 21.3M 0s
 29750K .......... .......... .......... .......... .......... 96%  241M 0s
 29800K .......... .......... .......... .......... .......... 97%  215M 0s
 29850K .......... .......... .......... .......... .......... 97%  247M 0s
 29900K .......... .......... .......... .......... .......... 97%  248M 0s
 29950K .......... .......... .......... .......... .......... 97%  244M 0s
 30000K .......... .......... .......... .......... .......... 97%  200M 0s
 30050K .......... .......... .......... .......... .......... 97%  239M 0s
 30100K .......... .......... .......... .......... .......... 98%  249M 0s
 30150K .......... .......... .......... .......... .......... 98% 16.5M 0s
 30200K .......... .......... .......... .......... .......... 98% 16.2M 0s
 30250K .......... .......... .......... .......... .......... 98% 41.3M 0s
 30300K .......... .......... .......... .......... .......... 98%  241M 0s
 30350K .......... .......... .......... .......... .......... 98%  234M 0s
 30400K .......... .......... .......... .......... .......... 99%  184M 0s
 30450K .......... .......... .......... .......... .......... 99%  199M 0s
 30500K .......... .......... .......... .......... .......... 99%  232M 0s
 30550K .......... .......... .......... .......... .......... 99%  225M 0s
 30600K .......... .......... .......... .......... .......... 99%  197M 0s
 30650K .......... .......... .......... .......... .......... 99%  237M 0s
 30700K .......... .......... .......... .......... .......... 99%  235M 0s
 30750K ...                                                   100% 64.2M=1.1s

2021-08-22 17:55:27 (28.2 MB/s) - ‘scarf_datasets/tenx_5K_pbmc_rnaseq/data.zarr.tar.gz’ saved [31491651/31491651]


1) Format conversion

The first step of the analysis workflow is to convert the file into the Zarr format that is supported by Scarf. We read in the data using CrH5Reader (stands for cellranger H5 reader). The reader object allows quick investigation of the file before the format is converted.

[3]:
reader = scarf.CrH5Reader('scarf_datasets/tenx_5K_pbmc_rnaseq/data.h5', 'rna')
time: 53.9 ms (started: 2021-08-22 17:55:27 +00:00)

We can quickly check the number of cells and features (genes as well as ADT features in this case) present in the file.

[4]:
reader.nCells, reader.nFeatures
[4]:
(5025, 33538)
time: 2.51 ms (started: 2021-08-22 17:55:27 +00:00)

Next we convert the data to the Zarr format that will later on be used by Scarf. For this we use Scarf’s CrToZarr class. This class will first quickly ascertain the type of data to be written and then create a Zarr format file for the data to be written into. CrToZarr takes two mandatory arguments. The first is the cellranger reader, and the other is the name of the output file.

NOTE: When we say zarr file, we actually mean zarr directory because, unlike HDF5, Zarr hierarchy is organized as a directory structure.

[5]:
writer = scarf.CrToZarr(reader, zarr_fn='scarf_datasets/tenx_5K_pbmc_rnaseq/data.zarr',
                        chunk_size=(2000, 1000))
writer.dump(batch_size=1000)
100%|██████████| 6/6 [00:06<00:00,  1.08s/it]
time: 6.68 s (started: 2021-08-22 17:55:27 +00:00)

The next step is to create a Scarf DataStore object. This object will be the primary way to interact with the data and all its constituent assays. When a Zarr file is loaded, Scarf checks if some per-cell statistics have been calculated. If not, then nFeatures (number of features per cell) and nCounts (total sum of feature counts per cell) are calculated. Scarf will also attempt to calculate the percent of mitochondrial and ribosomal content per cell. When we create a DataStore instance, we can decide to filter out low abundance genes with parameter min_features_per_cell. For example the value of 10 for min_features_per_cell below means that those genes that are present in less than 10 cells will be filtered out.

[6]:
ds = scarf.DataStore('scarf_datasets/tenx_5K_pbmc_rnaseq/data.zarr',
                     nthreads=4, min_features_per_cell=10)
INFO: Setting assay RNA to assay type: RNAassay
INFO: (RNA) Computing nCells and dropOuts
[########################################] | 100% Completed |  1.3s
INFO: (RNA) Computing nCounts
[########################################] | 100% Completed |  1.2s
WARNING: Minimum cell count (502) is lower than size factor multiplier (1000)
INFO: (RNA) Computing nFeatures
[########################################] | 100% Completed |  1.3s
INFO: Computing percentage of RNA_percentMito
[########################################] | 100% Completed |  1.0s
INFO: Computing percentage of RNA_percentRibo
[########################################] | 100% Completed |  1.0s
time: 6.55 s (started: 2021-08-22 17:55:34 +00:00)

2) Cell filtering

We can visualize the per-cell statistics in violin plots before we start filtering cells out.

[7]:
ds.plot_cells_dists()
../_images/vignettes_basic_tutorial_scRNAseq_14_0.png
time: 2.06 s (started: 2021-08-22 17:55:40 +00:00)

We can filter cells based on these cell attributes by providing upper and lower threshold values.

[8]:
ds.filter_cells(attrs=['RNA_nCounts', 'RNA_nFeatures', 'RNA_percentMito'],
                highs=[15000, 4000, 15],
                lows=[1000, 500, 0])
INFO: 597 cells flagged for filtering out using attribute RNA_nCounts
INFO: 461 cells flagged for filtering out using attribute RNA_nFeatures
INFO: 612 cells flagged for filtering out using attribute RNA_percentMito
time: 14.3 ms (started: 2021-08-22 17:55:42 +00:00)

Now we visualize the attributes again after filtering the values.

Note: the ‘I’ value given as the ``cell_key`` attribute signifies the column of the table that is set to ``False`` for cells that were filtered out or ``True`` for cells that are kept.

[9]:
ds.plot_cells_dists(cell_key='I', color='coral')
../_images/vignettes_basic_tutorial_scRNAseq_18_0.png
time: 1.2 s (started: 2021-08-22 17:55:42 +00:00)

The data stored under the ‘cellData’ level can easily be accessed using the cells attribute of the DataStore object.

[10]:
ds.cells.head()
[10]:
I ids names RNA_nCounts RNA_nFeatures RNA_percentMito RNA_percentRibo
0 True AAACCCAAGCGTATGG-1 AAACCCAAGCGTATGG-1 13537.0 3503.0 10.844353 16.783630
1 True AAACCCAGTCCTACAA-1 AAACCCAGTCCTACAA-1 12668.0 3381.0 5.975687 20.034733
2 False AAACCCATCACCTCAC-1 AAACCCATCACCTCAC-1 962.0 346.0 53.430353 2.494802
3 True AAACGCTAGGGCATGT-1 AAACGCTAGGGCATGT-1 5788.0 1799.0 10.919143 28.783690
4 True AAACGCTGTAGGTACG-1 AAACGCTGTAGGTACG-1 13186.0 2887.0 7.955407 35.750038
time: 34.3 ms (started: 2021-08-22 17:55:44 +00:00)

NOTE: We strongly discourage directly adding or removing the data from this table as Scarf will not be able to synchronize the changes to the disk. Instead use the methods of the cells attribute. Please refer to the insert, fetch, fetch_all, drop and update_key methods.


3) Feature selection

Similar to the cell table, Scarf also saves the feature level data that can be accessed as below:

[11]:
ds.RNA.feats.head()
[11]:
I ids names dropOuts nCells
0 False ENSG00000243485 MIR1302-2HG 5025 0
1 False ENSG00000237613 FAM138A 5025 0
2 False ENSG00000186092 OR4F5 5025 0
3 True ENSG00000238009 AL627309.1 4976 49
4 False ENSG00000239945 AL627309.3 5022 3
time: 23.1 ms (started: 2021-08-22 17:55:44 +00:00)

The feature selection step is performed on the normalized data. The default normalization method for RNAassay-type data is library-size normalization, wherein the count values are divided by the sum of total values for a cell. These values are then multiplied by a scalar factor. The default value of this scalar factor is 1000. However, if the total counts in a cell are less than this value, then on multiplication with this scalar factor the values will be ‘scaled up’ (which is not a desired behaviour). In the filtering step above, we set the low threshold for RNA_nCounts at 1000, and hence it is safe to use 1000 as a scalar factor. The scalar factor can be set by modifying the sf attribute of the assay. Let’s print the default value of sf

[12]:
ds.RNA.sf
[12]:
1000
time: 3.22 ms (started: 2021-08-22 17:55:44 +00:00)

Now the next step is to identify the highly variable genes in the dataset (for the RNA assay). This can be done using the mark_hvgs method of the assay. The parameters govern the min/max variance (corrected) and mean expression threshold for calling genes highly variable.

The variance is corrected by first dividing genes into bins based on their mean expression values. Genes with minimum variance is selected from each bin and a Lowess curve is fitted to the mean-variance trend of these genes. mark_hvgs will by default run on the default assay.

A plot is produced, that for each gene shows the corrected variance on the y-axis and the non-zero mean (means from cells where the gene had a non-zero value) on the x-axis. The genes are colored in two gradients which indicate the number of cells where the gene was expressed. The colors are yellow to dark red for HVGs, and blue to green for non-HVGs.

The mark_hvgs function has a parameter cell_key that dictates which cells to use to identify the HVGs. The default value of this parameter is I, which means it will use all the cells that were not filtered out.

[13]:
ds.mark_hvgs(min_cells=20, top_n=500, min_mean=-3, max_mean=2, max_var=6)
INFO: (RNA) Computing nCells
[########################################] | 100% Completed |  2.0s
INFO: (RNA) Computing normed_tot
[########################################] | 100% Completed |  2.1s
INFO: (RNA) Computing sigmas
[########################################] | 100% Completed |  2.2s
INFO: 497 genes marked as HVGs
../_images/vignettes_basic_tutorial_scRNAseq_27_1.png
time: 9.84 s (started: 2021-08-22 17:55:44 +00:00)

As a result of running mark_hvgs, the feature table now has an extra column **I__hvgs** which contains a True value for genes marked HVGs. The naming rule in Scarf dictates that cells used to identify HVGs are prepended to the column name (with a double underscore delimiter). Since we did not provide any cell_key parameter the default value was used, i.e. the filtered cells. This resulted in I becoming the prefix.

[14]:
ds.RNA.feats.head()
[14]:
I ids names I__hvgs dropOuts nCells stats_I_avg stats_I_c_var__200__0.1 stats_I_normed_n stats_I_normed_tot stats_I_nz_mean stats_I_sigmas
0 False ENSG00000243485 MIR1302-2HG False 5025 0 NaN NaN NaN NaN NaN NaN
1 False ENSG00000237613 FAM138A False 5025 0 NaN NaN NaN NaN NaN NaN
2 False ENSG00000186092 OR4F5 False 5025 0 NaN NaN NaN NaN NaN NaN
3 True ENSG00000238009 AL627309.1 False 4976 49 0.000914 1.555434 33.0 4.593283 0.13919 0.000196
4 False ENSG00000239945 AL627309.3 False 5022 3 NaN NaN NaN NaN NaN NaN
time: 42.1 ms (started: 2021-08-22 17:55:54 +00:00)

4) Graph creation

Creating a neighbourhood graph of cells is the most critical step in any Scarf workflow. This step internally involves multiple substeps:

  • data normalization for selected features

  • linear dimensionality reduction using PCA

  • creating an approximate nearest neighbour graph index (using the HNSWlib library)

  • querying cells against this index to identify nearest neighbours for each cell

  • edge weight computation using the compute_membership_strengths function from the UMAP package

  • fitting MiniBatch Kmeans (The kmeans centers are used later, for UMAP initialization)

make_graph method is responsible for graph construction. It method takes a mandatory parameter: feat_key. This should be a column in the feature metadata table that indicates which genes to use to create the graph. Since, we have already identified the hvgs in the step above, we use those genes. Note that we do not need to write *I__hvgs* but just hvgs as the value of the parameter. We also supply values for two very important parameters here: k (number of nearest neighbours to be queried for each cell) and dims (number of PCA dimensions to use for graph construction). n_centroids parameter controls number of clusters to create for the data using Kmeans algorithm. We perform a more accurate clustering of data in the later steps.

[15]:
ds.make_graph(feat_key='hvgs', k=11, dims=15, n_centroids=100)
INFO: No value provided for parameter `log_transform`. Will use default value: True
INFO: No value provided for parameter `renormalize_subset`. Will use default value: True
INFO: No value provided for parameter `pca_cell_key`. Will use default value: I
INFO: Using PCA for dimension reduction
INFO: No value provided for parameter `ann_metric`. Will use default value: l2
INFO: No value provided for parameter `ann_efc`. Will use default value: min(100, max(k * 3, 50))
INFO: No value provided for parameter `ann_ef`. Will use default value: min(100, max(k * 3, 50))
INFO: No value provided for parameter `ann_m`. Will use default value: 48
INFO: No value provided for parameter `rand_state`. Will use default value: 4466
INFO: No value provided for parameter `local_connectivity`. Will use default value: 1.0
INFO: No value provided for parameter `bandwidth`. Will use default value: 1.5
INFO: Normalizing with feature subset
[########################################] | 100% Completed |  1.0s
Writing data to normed__I__hvgs/data: 100%|██████████| 3/3 [00:01<00:00,  1.73it/s]
INFO: Calculating mean of norm. data
[########################################] | 100% Completed |  0.1s
INFO: Calculating std. dev. of norm. data
[                                        ] | 0% Completed |  0.0s

[########################################] | 100% Completed |  0.1s
Fitting PCA: 100%|██████████| 2/2 [00:00<00:00,  2.45it/s]
Fitting ANN: 100%|██████████| 2/2 [00:00<00:00,  4.10it/s]
Fitting kmeans: 100%|██████████| 2/2 [00:00<00:00,  3.10it/s]
Estimating seed partitions: 100%|██████████| 2/2 [00:00<00:00,  5.26it/s]
INFO: Saving loadings to RNA/normed__I__hvgs/reduction__pca__15__I

INFO: Saving ANN index to RNA/normed__I__hvgs/reduction__pca__15__I/ann__l2__50__50__48__4466
INFO: Saving kmeans clusters to RNA/normed__I__hvgs/reduction__pca__15__I/kmeans__100__4466
Saving KNN graph: 100%|██████████| 2/2 [00:00<00:00,  4.87it/s]
INFO: ANN recall: 99.92%

Smoothening KNN distances: 100%|██████████| 1/1 [00:04<00:00,  4.04s/it]
time: 12.2 s (started: 2021-08-22 17:55:54 +00:00)


5) Low dimensional embedding and clustering

Next we run UMAP on the graph calculated above. Here we will not provide which cell key or feature key to be used, because we want the UMAP to run on all the cells that were not filtered out and with the feature key used to calculate the latest graph. We can provide the parameter values for the UMAP algorithm here.

[16]:
ds.run_umap(fit_n_epochs=250, spread=5, min_dist=1, parallel=True)
/home/docs/checkouts/readthedocs.org/user_builds/scarf/envs/0.7.8/lib/python3.8/site-packages/umap/umap_.py:1330: RuntimeWarning: divide by zero encountered in power
  return 1.0 / (1.0 + a * x ** (2 * b))
        completed  0  /  250 epochs
        completed  25  /  250 epochs
        completed  50  /  250 epochs
        completed  75  /  250 epochs
        completed  100  /  250 epochs
        completed  125  /  250 epochs
        completed  150  /  250 epochs
        completed  175  /  250 epochs
        completed  200  /  250 epochs
        completed  225  /  250 epochs
        completed  0  /  100 epochs
        completed  10  /  100 epochs
        completed  20  /  100 epochs
        completed  30  /  100 epochs
        completed  40  /  100 epochs
        completed  50  /  100 epochs
        completed  60  /  100 epochs
        completed  70  /  100 epochs
        completed  80  /  100 epochs
        completed  90  /  100 epochs
time: 21.9 s (started: 2021-08-22 17:56:06 +00:00)

The UMAP results are saved in the cell metadata table as seen below in columns: RNA_UMAP1 and RNA_UMAP2

[17]:
ds.cells.head()
[17]:
I ids names RNA_UMAP1 RNA_UMAP2 RNA_nCounts RNA_nFeatures RNA_percentMito RNA_percentRibo
0 True AAACCCAAGCGTATGG-1 AAACCCAAGCGTATGG-1 8.368184 18.407541 13537.0 3503.0 10.844353 16.783630
1 True AAACCCAGTCCTACAA-1 AAACCCAGTCCTACAA-1 20.305178 30.461788 12668.0 3381.0 5.975687 20.034733
2 False AAACCCATCACCTCAC-1 AAACCCATCACCTCAC-1 NaN NaN 962.0 346.0 53.430353 2.494802
3 True AAACGCTAGGGCATGT-1 AAACGCTAGGGCATGT-1 -16.047424 32.967999 5788.0 1799.0 10.919143 28.783690
4 True AAACGCTGTAGGTACG-1 AAACGCTGTAGGTACG-1 -9.697846 10.873372 13186.0 2887.0 7.955407 35.750038
time: 32.7 ms (started: 2021-08-22 17:56:28 +00:00)

plot_layout is a versatile method to create a scatter plot using Scarf. Here we can plot the UMAP coordinates of all the non-filtered out cells.

[18]:
ds.plot_layout(layout_key='RNA_UMAP')
../_images/vignettes_basic_tutorial_scRNAseq_37_0.png
time: 390 ms (started: 2021-08-22 17:56:28 +00:00)

plot_layout can be used to easily visualize data from any column of the cell metadata table. Next, we visualize the number of genes expressed in each cell.

[19]:
ds.plot_layout(layout_key='RNA_UMAP', color_by='RNA_nCounts', cmap='coolwarm')
../_images/vignettes_basic_tutorial_scRNAseq_39_0.png
time: 1.21 s (started: 2021-08-22 17:56:28 +00:00)

There has been a lot of discussion over the choice of non-linear dimensionality reduction for single-cell data. tSNE was initially considered an excellent solution, but has gradually lost out to UMAP because the magnitude of relations between the clusters cannot easily be discerned in a tSNE plot. Scarf contains an implementation of tSNE that runs directly on the graph structure of cells. So, essentially the same data that was used to create the UMAP and clustering is used.

[20]:
ds.run_tsne(alpha=10, box_h=1, early_iter=250, max_iter=500, parallel=True)
Saving KNN matrix in MTX format: 100%|██████████| 4/4 [00:00<00:00, 20.30it/s]
ERROR: SG-tSNE failed, possibly due to missing libraries or file permissions. SG-tSNE currently fails on readthedocs
time: 276 ms (started: 2021-08-22 17:56:29 +00:00)
sgtsne: error while loading shared libraries: libmetis.so.5: cannot open shared object file: No such file or directory

NOTE: The tSNE implementation is currently not supported on Windows.

[21]:
# Here we run plot_layout under exception catching because if you are not on Linux then the `run_tnse` would have failed.
try:
    ds.plot_layout(layout_key='RNA_tSNE')
except KeyError:
    print ("'RNA_tSNE1' not found in MetaData")
'RNA_tSNE1' not found in MetaData
time: 3.93 ms (started: 2021-08-22 17:56:30 +00:00)

6) Cell clustering

Identifying clusters of cells is one of the central tenets of single cell approaches. Scarf includes two graph clustering methods and any (or even both) can be used on the dataset. The methods start with the same graph as the UMAP algorithm above to minimize the disparity between the UMAP and clustering results. The two clustering methods are:

  • Paris: This is the default clustering algorithm.

  • Leiden: Leiden is a widely used graph clustering algorithm in single-cell genomics.

Paris is the default algorithm in Scarf due to its ability to highlight cluster relationships. Paris is a hierarchical graph clustering algorithm that is based on node pair sampling. Paris creates a dendrogram of cells which can then be cut to obtain desired number of clusters. The advantage of using Paris, especially in the larger datasets, is that once the dendrogram has been created one can change the desired number of clusters with minimal computation overhead.

[22]:
# We start with Leiden clustering
ds.run_leiden_clustering(resolution=0.5)
time: 159 ms (started: 2021-08-22 17:56:30 +00:00)

We can visualize the results using the plot_layout method again. Here we plot both UMAP and colour cells based on the their cluster identity, as obtained using Leiden clustering.

[23]:
ds.plot_layout(layout_key='RNA_UMAP', color_by='RNA_leiden_cluster')
../_images/vignettes_basic_tutorial_scRNAseq_48_0.png
time: 812 ms (started: 2021-08-22 17:56:30 +00:00)

The results of the clustering algorithm are saved in the cell metadata table. In this case, they have been saved under the column name RNA_leiden_cluster.

[24]:
ds.cells.head()
[24]:
I ids names RNA_UMAP1 RNA_UMAP2 RNA_leiden_cluster RNA_nCounts RNA_nFeatures RNA_percentMito RNA_percentRibo
0 True AAACCCAAGCGTATGG-1 AAACCCAAGCGTATGG-1 8.368184 18.407541 2 13537.0 3503.0 10.844353 16.783630
1 True AAACCCAGTCCTACAA-1 AAACCCAGTCCTACAA-1 20.305178 30.461788 2 12668.0 3381.0 5.975687 20.034733
2 False AAACCCATCACCTCAC-1 AAACCCATCACCTCAC-1 NaN NaN -1 962.0 346.0 53.430353 2.494802
3 True AAACGCTAGGGCATGT-1 AAACGCTAGGGCATGT-1 -16.047424 32.967999 7 5788.0 1799.0 10.919143 28.783690
4 True AAACGCTGTAGGTACG-1 AAACGCTGTAGGTACG-1 -9.697846 10.873372 1 13186.0 2887.0 7.955407 35.750038
time: 38 ms (started: 2021-08-22 17:56:31 +00:00)

We can export the Leiden cluster information into a pandas dataframe. Setting key to I makes sure that we do not include the data for cells that were filtered out.

[25]:
leiden_clusters = ds.cells.to_pandas_dataframe(['RNA_leiden_cluster'], key='I')
leiden_clusters.head()
[25]:
RNA_leiden_cluster
0 2
1 2
3 7
4 1
6 5
time: 10.5 ms (started: 2021-08-22 17:56:31 +00:00)

Now we run Paris clustering algorithm using run_clustering method. Paris clustering requires only one parameter: n_clusters, which determines the number of clusters to create. Here we set the number of clusters same as that were obtained using Leiden clustering.leiden_clusters.nunique()

[26]:
ds.run_clustering(n_clusters=leiden_clusters.nunique()[0])
time: 505 ms (started: 2021-08-22 17:56:31 +00:00)

Visualizing Paris clusters

[27]:
ds.plot_layout(layout_key='RNA_UMAP', color_by='RNA_cluster')
../_images/vignettes_basic_tutorial_scRNAseq_56_0.png
time: 868 ms (started: 2021-08-22 17:56:31 +00:00)

Discerning similarity between clusters can be difficult from visual inspection alone, especially for tSNE plots. plot_cluster_tree function plots the relationship between clusters as a binary tree. This tree is simply a condensation of the dendrogram obtained using Paris clustering.

[28]:
ds.plot_cluster_tree(cluster_key='RNA_cluster', width=1)
Constructing graph from dendrogram: 100%|██████████| 3939/3939 [00:00<00:00, 58036.08it/s]
Identifying the top node for cluster: 100%|██████████| 13/13 [00:00<00:00, 365.01it/s]
../_images/vignettes_basic_tutorial_scRNAseq_58_1.png
time: 993 ms (started: 2021-08-22 17:56:32 +00:00)

The tree is free form (i.e the position of clusters doesn’t convey any meaning) but allows inspection of cluster similarity based on branching pattern. The sizes of clusters indicate the number of cells present in each cluster. The tree starts from the root node (black dot with no incoming edges).


7) Marker gene identification

Now we can identify the genes that are differentially expressed between the clusters using the run_marker_search method. The method to identify the differentially expressed genes in Scarf is optimized to obtain quick results. We have not compared the sensitivity of our method to other differential expression detecting methods. We expect specialized methods to be more sensitive and accurate to varying degrees. Our method is designed to quickly obtain key marker genes for populations from a large dataset. For each gene individually, following steps are carried out:

  • Expression values are converted to ranks (dense format) across cells.

  • A mean of ranks is calculated for each group of cells

  • The mean value for each group is divided by the sum of mean values to obtain the ‘specificity score’

  • The gene is saved as a marker gene if it’s specificity score is higher than a given threshold.

This method does not perform any statistical test of significance and uses ‘specificity score’ as a measure of importance of each gene for a cluster.

[29]:
ds.run_marker_search(group_key='RNA_cluster', threshold=0.25)
Finding markers: 100%|██████████| 272/272 [00:26<00:00, 10.24it/s]
time: 26.9 s (started: 2021-08-22 17:56:33 +00:00)

Using the plot_marker_heatmap method, we can also plot a heatmap with the top marker genes from each cluster. The method will calculate the mean expression value for each gene from each cluster.

[30]:
ds.plot_marker_heatmap(group_key='RNA_cluster', topn=5, figsize=(5, 9))
../_images/vignettes_basic_tutorial_scRNAseq_63_0.png
time: 3.11 s (started: 2021-08-22 17:57:00 +00:00)

The markers list for specific clusters can be obtained like this:

[31]:
ds.get_markers(group_key='RNA_cluster', group_id='1')
[31]:
score names
ids
ENSG00000121552 0.949142 CSTA
ENSG00000170458 0.947154 CD14
ENSG00000106565 0.945043 TMEM176B
ENSG00000197249 0.945023 SERPINA1
ENSG00000116701 0.943234 NCF2
... ... ...
ENSG00000144802 0.251001 NFKBIZ
ENSG00000163661 0.250964 PTX3
ENSG00000100605 0.250697 ITPK1
ENSG00000182095 0.250591 TNRC18
ENSG00000163131 0.250493 CTSS

655 rows × 2 columns

time: 116 ms (started: 2021-08-22 17:57:03 +00:00)

We can directly visualize the expression values for a gene of interest. It is usually a good idea to visually confirm the gene expression pattern across the cells atleast this way.

[32]:
ds.plot_layout(layout_key='RNA_UMAP', color_by='CD14')
../_images/vignettes_basic_tutorial_scRNAseq_67_0.png
time: 1.41 s (started: 2021-08-22 17:57:03 +00:00)

That is all for this vignette.