tools/sage.csv+: we got Sage's project description as csv, convert to json

At the same time documenting our current requirements from a graph
(pretty basic).

Adds tools/sage_csv_to_sage_json.py

2 files changed, 244 insertions, 0 deletions

diff --git a/tools/sage.csv b/tools/sage.csv
new file mode 100644
index 00000000..01ee8017
--- /dev/null
+++ b/tools/sage.csv
@@ -0,0 +1,166 @@
+id,name,dependent on
+1,Kick-off meeting,
+2,Definition of project plan,
+3,Definition of roles and responsibilities,
+4,Agreement on Data sharing modalities,
+5,,
+6,Aim 1 - Screen drugs against merlin-deficient VS and M cells,
+7,Final list of compounds ready,
+8,Assemble 20 to 40 drugs panel,
+9,Drugs in hands,8
+10,QC check on purchased drugs,9
+11,MTA negotiation with companies to secure drug supply,
+12,"Assemble, test growth rates and choose hum Upfront meningioma lines for M screening set ",
+13,Assemble pairs of lines for VS screening set (total 8 lines) Upfront,
+14,Optimize assay conditions,
+15,Primary Screening of 20 to 40 drugs against both M and VS,"14,11,9"
+16,Analysis of data from primary screening,15
+17,Upload of primary screening data into Synapse and report,16
+18,Secondary screening (independent Human M and VS lines) of top drugs,17
+19,Analysis of secondary screening data from secondary screening and interpretation,18
+20,Upload of data into Synapse and reporting - Identification report of active single agents,19
+21,Primary screening of drug combination,"17,58"
+22,Upload of primary screening of combination data into report Synapse and reporting,21
+23,Secondary screening of drug combination,"57,21"
+24,Upload of secondary screening data into Synapse and report - reporting,23
+25,Selection of drug(s)/Combination(s) for invivo screening,23
+26,Analysis of Human vs Mouse cell lines and selection of winner cell line for NCATS screen,"18,58,60,62"
+27,Selection of 'winner' cell line,"26,64"
+28,NCATS screening ,27
+29,NCATS hit evaluation,28
+30,Upload of data into Synapse and reporting,29
+31,NCATS hits in primary (or secondary?) screening panels (?),30
+32,Upload of data into Synapse and reporting,31
+33,,
+34,,
+35,Aim 2 - Collect and Integrate molecular System data to define targets and improve models,
+36,Exome/RNA analysis of human NF2-associated tumor at Novartis,"13,12"
+37,First deposition of data into the Cancer Cell Line Encyclopedia,36
+38,Report on Novartis's activities at project's meetings,36
+39,Report on Novartis's activities at project's meetings,36
+40,Report on Novartis's activities at project's meetings,36
+41,Report on Novartis's activities at project's meetings,36
+42,Report on Novartis's activities at project's meetings,36
+43,,
+44,Availability of data from DoD program at Gusella Lab on Exome scanning and RNAseq of Schwannomas and Meningiomas and upload on Synapse,
+45,Upload of consortium members pre-existing data into Synapse,
+46,Upload of publicly available data into the Synapse,"44, 45"
+47,Transcriptome profile of untreated cell lines (baseline analysis),"12,13"
+48,Upload Cell lines baseline characterization only,"13, 47"
+49,Transcriptome profile of screening lines control vs treated/responders,17
+50,Upload Control vs treated/responsers,49
+51,Transcriptome profile of screening lines control vs treated/responders of drug combination,"49, 24"
+52,Upload Control vs combination's responders,51
+53,Report on transcriptome profiling,52
+54,Jumping Library analysis of structural genomic changes in screening set lines,"12,13"
+55,Report on Jumping Library analysis,54
+56,TC call to decide first 6 cell lines to analyse,
+57,Kinome analysis of cell lines for baseline characterization ,"56,12,13"
+58,Upload of data into Synapse and reporting,57
+59,Kinome analysis of single agent drug responding cell lines,17
+60,Upload of data of single agent drug responding cell lines into Synapse,59
+61,Kinome analysis of combination of drugs responding cell lines,24
+62,Upload of data of combination of drugs responding cell lines into Synapse,61
+63,Report of both single agent and combination responders cell lines,"59, 62"
+64,Analysis of integrated system data,"44,37,46,51"
+65,Report on analysis of integrated system data - Generate report a list of proposed new targets,"64,68,70"
+66,"Target (genes, pathways) validation by shRNA",58
+67,Report on ongoing activities,66
+68,Final Report - ID of new targets for NF2,66
+69,Screening of drugs against validated targets,68
+70,Upload of data into Synapse and reporting,69
+71,Develop CRISP/Cas modified NF2 mutant AC and SC lines,58
+72,Upload of data into Synapse and reporting,71
+73,Develop CRISPR/Cas modified lines bearing cooperating mutations,72
+74,Upload of data into Synapse and reporting,73
+75,Test CRISPR-Cas lines with active drug(s)/combination(s),22
+76,Upload of data into Synapse and reporting,75
+77,RNAseq of genetically mod lines,"65, 72"
+78,Upload of data into Synapse and reporting,77
+79,Creation of a new set of genetically-accurate screening panel for NF2,"75, 74"
+80,Kinome analysis of invivo tumors,90
+81,Upload of data into Synapse and reporting,80
+82,RNAseq analysis of invivo tumors,90
+83,Upload of data into Synapse and reporting,82
+84,,
+85,Aim 3 - Screen drugs invivo in genetically engineered mouse (VS) and xenograft (M) models,
+86,Breeding of PeriostinCre; nf2flox/flox mice,
+87,Report on first year activities,86
+88,Evaluation of single agent responders for animal studies,"50, 60"
+89,Test drug(s)/combination(s) in PeriostinCre; nf2flox/flox mice,"20,88,86"
+90,First invivo drug testing (treatment),89
+91,First invivo drug testing (statistics),90
+92,Upload of data into Synapse and reporting,91
+93,Evaluation of single agent responders for animal studies,"52, 62"
+94,First invivo combination drug testing (treatment),93
+95,First invivo combination drug testing (statistics),94
+96,Upload of data into Synapse and reporting,95
+97,Last invivo drug testing slot (treatment),
+98,Last invivo drug testing (statistics),"89, 97"
+99,Upload of data into Synapse and reporting,98
+100,Colony breeding and stereotactic transplantation studies,
+101,Report on first year activities,100
+102,Evaluation of single agent responsers for animal studies,88
+103,Test drug(s)/combination(s) in BenMen1 xenograft model,"100, 20, 10"
+104,First invivo drug testing,103
+105,Upload of data into Synapse and reporting,104
+106,Evaluation of single agent responders for animal studies,93
+107,First invivo drug testing,106
+108,Upload of data into Synapse and reporting,107
+109,Last invivo drug testing slot,103
+110,Upload of data into Synapse and reporting,109
+111,Prioritize drug(s) for NF2 clinical trials,"80, 82"
+112,Supply human tumors,
+113,Report on first year activities,112
+114,Report on second year activities,113
+115,Final report,112
+116,Supply human tumors to Novartis for transcriptome and exome analysis and generation of transplantable human,
+117,Analysis of human tumors from patients,
+118,Report on first year activities,116
+119,Report on second year activities,118
+120,Neuropathological assessment of mouse model tumors,89
+121,Upload of data into Synapse and Final Reporting,120
+122,Generation of transplantable human xenograft,116
+123,Model available to the community (???),122
+124,,
+125,Aim 4 - Identify at least 1 drug to move forward for a clinical trial,
+126,Evaluation and Design of potential clinical trial(s),111
+127,Clinical trial start,126
+128,,
+129,"Aim 5 - Data Sharing, Dissemination plan and management",
+130,Setting up of Synapse platform (website,
+131,Training of researchers for data input,130
+132,"Data deposition, integration and sharing",131
+133,Report on 1y activities,
+134,Report on 2y activities,133
+135,Final report,132
+136,,
+137,Project Group calls (bi-weekly),
+138,Group call: decision on drug panel,
+139,Group call: decision on first cell lines to investigate for kinome and transcriptome profile,
+140,Group call: pre-existing data on drugs or cell lines to be uploaded in Synapse,
+141,,
+142,,
+143,,
+144,Project Management,
+145,Synodos Meeting (kick-off),
+146,General management,145
+147,Synodos Meeting(NF conference),146
+148,Synodos Meeting (Ext Review committee),146
+149,First Year report,148
+150,Synodos Meeting(NF conference),146
+151,Synodos Meeting (Ext Review committee),146
+152,Second Year report,151
+153,Synodos Meeting(NF conference),146
+154,Synodos Meeting (Ext Review committee),146
+155,Final report,"154,146"
+156,,
+157,,
+158,,
+159,,
+160,,
+161,,
+162,,
+163,,
+164,,
+165,,
\ No newline at end of file
diff --git a/tools/sage_csv_to_sage_json.py b/tools/sage_csv_to_sage_json.py
new file mode 100755
index 00000000..b89670ab
--- /dev/null
+++ b/tools/sage_csv_to_sage_json.py
@@ -0,0 +1,78 @@
+#!/bin/env python
+
+"""
+Convert sage.csv to a loadable file. Here is our current 'file format':
+
+json dictionary with two keys:
+'nodes' is a list of dictionaries, each containing
+'state': 'perm' (should not be required, try without it)
+'id': must be unique
+'name': display only (will be used as id if it is missing, true that?)
+'type': for gannt should be 'deliverable'
+'start': a date for 'deliverable' typed nodes
+'end': a date for 'deliverable' typed nodes
+'status': one of 'done', 'current', 'waiting' for 'deliverable' type
+
+'links is a list of dictionaries with mandatory keys:
+'name'
+'source': id (string)
+'target' id (string)
+"""
+import json
+import csv
+import random
+from datetime import datetime, timedelta
+from time import mktime
+
+class MyEncoder(json.JSONEncoder):
+
+    def default(self, obj):
+        if isinstance(obj, datetime):
+            return int(mktime(obj.timetuple()))
+        return json.JSONEncoder.default(self, obj)
+
+def write_json(output_filename, nodes, links):
+    seen_ids = set()
+    assert(type(nodes) == list)
+    assert(type(links) == list)
+    for node in nodes:
+        assert(type(node) == dict)
+        assert('id' in node)
+        assert(node['id'] not in seen_ids)
+        seen_ids.add(node['id'])
+        assert('name' in node)
+    for link in links:
+        assert(type(link) == dict)
+        assert('name' in link)
+        assert('source' in link)
+        assert('target' in link)
+    with open(output_filename, 'w+') as fd:
+
+        json.dump({'nodes': nodes, 'links':links}, fd,
+                  cls=MyEncoder)
+
+def read_sage_csv():
+    week = timedelta(days=7)
+    day = timedelta(days=1)
+    all_start = datetime(year=2018, month=1, day=1)
+    lines = list(csv.reader(open('sage.csv')))
+    assert(['id', 'name', 'dependent on'] == lines[0])
+    del lines[0]
+    nodes = [{'id': num, 'name': desc,
+              'type': 'deliverable',
+              'start': all_start + week * i,
+              'end': all_start + week * i + day * (((i * 33) % 17) + 1),
+              'status': random.choice(['unknown', 'done', 'current', 'waiting']),
+              } for i, (num, desc, _) in enumerate(lines)]
+    links = []
+    for num, desc, deps in lines:
+        deps = deps.split(',')
+        if len(deps) == 0:
+            continue
+        for dep in deps:
+            links.append({'source': num, 'target': dep, 'name': 'depends on'})
+    return nodes, links
+
+if __name__ == '__main__':
+    nodes, links = read_sage_csv()
+    write_json('sage.json', nodes, links)