The GCTx format and cmap{Py, R, M, J} packages: resources for optimized storage and integrated traversal of annotated dense matrices.

Bioinformatics
Authors
Keywords
Abstract

MOTIVATION: Facilitated by technological improvements, pharmacologic and genetic perturbational datasets have grown in recent years to include millions of experiments. Sharing and publicly distributing these diverse data creates many opportunities for discovery, but in recent years the unprecedented size of data generated and its complex associated metadata have also created data storage and integration challenges.

RESULTS: We present the GCTx file format and a suite of open-source packages for the efficient storage, serialization and analysis of dense two-dimensional matrices. We have extensively used the format in the Connectivity Map to assemble and share massive datasets currently comprising 1.3 million experiments, and we anticipate that the format's generalizability, paired with code libraries that we provide, will lower barriers for integrated cross-assay analysis and algorithm development.

AVAILABILITY AND IMPLEMENTATION: Software packages (available in Python, R, Matlab and Java) are freely available at https://github.com/cmap. Additional instructions, tutorials and datasets are available at clue.io/code.

SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Year of Publication
2019
Journal
Bioinformatics
Volume
35
Issue
8
Pages
1427-1429
Date Published
2019 04 15
ISSN
1367-4811
DOI
10.1093/bioinformatics/bty784
PubMed ID
30203022
PubMed Central ID
PMC6477971
Links
Grant list
U01 HG008699 / HG / NHGRI NIH HHS / United States
U54 HL127366 / HL / NHLBI NIH HHS / United States