Download as PDF, PPTX
Uploaded byKeiichiro Ono
Cytoscapeの現状とCyberinfrastructure
2015年1月9日に産総研で行われたセミナーの資料です。現時点におけるCytoscapeプロジェクトの現状と、今後のライフサイエンス向けソフトウェアの開発についてのビジョンです。 http://www.cytoscape.org/index.html
![[DL Hacks]BERT: Pre-training of Deep Bidirectional Transformers for Language ...](https://cdn.slidesharecdn.com/ss_thumbnails/20181129suzukibert1-181204064830-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DL輪読会]StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators](https://cdn.slidesharecdn.com/ss_thumbnails/stylegan-nada-210813013304-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DDBJing31] 軽量仮想環境を用いてNGSデータの解析再現性を担保する](https://cdn.slidesharecdn.com/ss_thumbnails/31ddbjingohta-150626022623-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
Cytoscapeの現状とCyberinfrastructure
- 2. Cytoscapeの現状とCyberinfrastructure 再現可能な大規模オミクスデータ解析ワークフローに対応したツール群を目指して 大野 圭一朗 UC, SanDiego Trey Ideker Lab Cytoscape Core Team 産総研セミナー 1/9/2015
- 3.
- 4. - 大野 圭一朗(Keiichiro Ono) - カリフォルニア大学 サンディエゴ校医学部 - Trey Ideker Lab - Software Engineer / Research Associate - National Resource for Network Biology (NRNB) - nrnb.org
- 5. Keiichiro Ono Background Bioinformatics Computer Science Work Research Bioinformaticsworkflow Visualization pipeline Data Visualization Networks Other Biological Data Integration Molecular Interactions Pathways Annotations Software Development Cytoscape NeXO Cyberinfrastructure All kinds of small tools Like Art Kandinsky Mondrian Music Electronica Techno Minimal Detroit Jazz Sci-fi Movie Novel Life US San Diego San Francisco Bay Area Los Angeles Orange County Japan Gifu Tokyo
- 6.
- 7. Keiic Work Research Bioinformatics workflow Visualization pipeline Data Visualization Networks OtherBiological Data Integration Molecular Interactions Pathways Annotations Software Development Cytoscape NeXO Cyberinfrastructure All kinds of small tools
- 8. Keiichiro Ono Background Bioinformatics Computer Science Work Research Bioinformaticsworkflow Visualization pipeline Data Visualization Networks Other Biological Data Integration Molecular Interactions Pathways Annotations Software Development Cytoscape NeXO Cyberinfrastructure All kinds of small tools Like Art Kandinsky Mondrian Music Electronica Techno Minimal Detroit Jazz Sci-fi Movie Novel Life US San Diego San Francisco Bay Area Los Angeles Orange County Japan Gifu Tokyo
- 9.
- 10.
- 11.
- 12.
- 13. - Cytoscapeプロジェクトの現状 - 最新リリース:v3.2.0 - web関連テクノロジーへの取り組み: Cytoscape.js - 自動化されたワークフローで Cytoscapeを利用する: cyREST
- 14.
- 15.
- 16. 未来 Service-oriented workflow Cytoscape-as-a-service Docker Reproducible Science WebTechnologies NIH The Commons
- 17.
- 18. Cytoscape: Open Source Platformfor Network Analysis and Visualization
- 19. Cytoscapeとは？ - 生物学的ネットワーク解析と可視化のた めのオープンソースプラットフォーム - 北米とヨーロッパの研究機関による共同 プロジェクト -生物学分野ではデファクトスタンダード の一つ - Appと呼ばれるソフトウェアにより拡張 可能 - 200以上のAppがApp Storeで公開中
- 20. Open Platform forLarge-Scale Network Analysis and Visualization Human Interactome data from BioGRID visualized by Cytoscape
- 21.
- 22.
- 25. App Store DailyDownload Plot
- 28.
- 29.
- 30. Cytoscape Ecosystem - Cytoscape3.2.0 - デスクトップ版 - Cytoscape.js - JavaScript版ライブラリ - Cytoscape 2.8.3 - レガシー版
- 31.
- 33.
- 34. Cytoscape 3.2 - 新機能 -Chart Editor - Export as Web Application
- 35.
- 36.
- 37. Chart Editor - Visualizemultiple data points to a single view - Time series data - Multiple GO terms - Chart types: Bar, Box, Pie, Heat Map, Ring - Part of standard Visual Style Editor - Everything will be saved into session files
- 38.
- 39. Export as WebApplication Cytoscapeで作成した可視化を Webアプリケーションとして書き出す
- 40. Export as WebApplication Cytoscapeで作成した可視化 をシンプルなWebアプリケー ションとして書き出す機能
- 44. ここまでのまとめ - Cytoscape 3.2.0 -最新のデスクトップ版リリース - Cytoscape.js - ネットワーク可視化のWebアプリ ケーションを構築するための部品 - 両者は完全に独立したプロジェクトだ が、データレベルでの連携をさせる機 能を追加中
- 45.
- 46.
- 47. User Type I -いわゆる「ベンチバイオロ ジスト」 - Excelを多用 - ドメインの専門家でデータ を生み出す人々 - しかし解析と可視化はま だ…と言う場合も多い
- 48. User Type II -バイオインフォマティシャン - Python + SciPy/NumPy, R + Bioconductor, MATLABといっ たもの日常的に利用する - 必要に応じてライブラリも書く - 大規模な計算機リソースも多用 する - 「手作業は悪！」
- 49. どちらのユーザーも重要！ - Type I:“Bench Biologists” - Domain experts - Data producers - Type II: Computational Biologists - Experts of large-scale data analysis - Especially important for genome-scale data analysis Cytoscapeにはこちらに フォーカスした機能が少ない…
- 50. User Type II -バイオインフォマティシャン - Python + SciPy /NumPy, R + Bioconductor, MATLABといっ たもの日常的に利用する - 必要に応じてコードも書く - 大規模な計算機リソースも多 用する - 「手作業は悪！」
- 51. Requests from TypeII Users - I have 200 networks in my session and I need to create one PDF per view. How can I do it with Cytoscape? - I need to use igraph for network analysis, but its visualization feature is limited. I want to use Cytoscape as an external visualization engine for R. - Usually I use IPython Notebook to record my work. How can I integrate Cytoscape into my workflow? - I want to generate Style for each time point and create small multiples of networks.
- 52.
- 53. What is cyREST? -ほぼ全ての言語から利用できるRESTfulなAPI モジュール - つまり、CytoscapeのレイアウトやVisual Mappingと言った基本機能やデータ構造に好 みのプログラミング言語からアクセス可能 REST
- 54.
- 55. Mapping Cytoscape APIto HTTP Methods Create Read Update Delete Cytoscape Operations POST GET PUT DELETE HTTP Methods
- 56. Get full networkwith unique ID 52 as JSON GET http://localhost:1234/v1/networks/52
- 57.
- 58.
- 59.
- 60. Demo: Cytoscape Controlled from IPythonNotebook REST http://bit.ly/1wcKXVV
- 61.
- 62.
- 63.
- 64.
- 65.
- 67. 2005 - Cytoscape 2.2:Javaで書かれたデスク トップアプリケーション - GoogleがGoogle Mapベータ版をリリース - 「JavaScriptの再発見」
- 68.
- 70. 2015 - Cytoscape 3.2.0:モジュール化されたJavaアプ リケーション - 多くのクライアントソフトウェアがウェブアプ リケーション化 - JavaScript everywhere - 緩やかなデスクトップアプリケーションの死 - サービスの集合としてのアプリケーション
- 71.
- 72. ソフトウェア開発スタイルの変化 - An applicationis a collection of smaller services - JavaScript is a first-class citizen in the world of programming languages - Design application with cloud services in mind
- 73.
- 74. In the modernera, software is commonly delivered as a service: called web apps, or software-as-a-service. The twelve-factor app is a methodology for building software-as-a-service apps that: • Use declarative formats for setup automation, to minimize time and cost for new developers joining the project • Have a clean contract with the underlying operating system, offering maximum portability between execution environments • Are suitable for deployment on modern cloud platforms, obviating the need for servers and systems administration • Minimize divergence between development and production, enabling continuous deployment for maximum agility • And can scale up without significant changes to tooling, architecture, or development practices.
- 75.
- 77. This MANIFESTO counters currenttrends in bioinformatics where institutes and companies are creating monolithic software solutions aimed mostly at end-users.
- 78. –THE SMALL TOOLSMANIFESTO FOR BIOINFORMATICS “Every single tool should do the smallest possible task really well”
- 79.
- 80.
- 81. Bioinformatics Open SourceConference (BOSC)
- 82.
- 83. @New York Times@FacebookHQ in Boston
- 84.
- 85. データ解析ツールの傾向 - Python isbecoming the standard language for “Data Scientists” - Python itself is a very slow language, but is a perfect glue - Lots of tools are made by scientists (e.g. Anaconda by Continuum) - They do understand current problems in modern scientific computing, and trying to solve them
- 86. (by Peter Wang@PyData Sermon Valley 2014)
- 87.
- 88.
- 89. - Visualization needsvaries, especially for complex data sets like the one from life science domain - For that purpose, Java is not the best language to implement applications - Even large-scale data visualization applications are moving to the web browsers - Canvas (Cytoscape.js), WebGL (Three.js), SVG (D3.js) - Most of the talented hackers are working on the web browsers, i.e., JavaScript
- 90.
- 91.
- 92. 科学系計算機環境における課題 - No morefree lunch - Even if you buy expensive machines, you cannot get free performance gain anymore. You have to design your code for massively distributed environment. (From Scale-up to Scale-out) - Complex Data Analysis Pipeline - Needs for complex, customized data visualization - Reproducibility - パイプラインの構築そのものが複雑で再現性の確保が困難
- 93.
- 94. Universe of Toolsfor Bioinformatics !
- 95. この変化の生物学者にとっての意味 - “Omics-Scale" DataAnalysis - デスクトップマシンを超えた計算機リソースの必要性 - 複数のソフトウェアを接続することによる解析 ➡ 「ポータブル」で再現可能なワークフローを構築する作 業が複雑になっているため、ソフトウェア技術者との協 業も重要度を増している
- 96.
- 97.
- 98.
- 99. 私の職場の場合 - PI - 研究者 -学生 - ソフトウェア開発者
- 100.
- 101.
- 102. 実装 - 院生の作ったプロトタイプ - Cytoscape用のApp -コマンドライン版 → 同じことの繰り返しが含まれる！
- 103.
- 104. Cytoscape as aCollection of Services
- 105.
- 106.
- 107. Srivas, Rohith etal. “Assembling Global Maps of Cellular Function through Integrative Analysis of Physical and Genetic Networks.” Nature Protocols 6.9 (2011): 1308–1323. PMC. Web. 1 Dec. 2014.
- 108. Core algorithm 1 asPython Java Implementation of Algorithms Cytoscape 2.x Plugin Biological Problem Cytoscape 3.x App Core algorithm 2 as Python Core algorithm n as Python PanGIA Service (Implement in Python again…?) by Sourav by Greg, Rohith by Greg, Rothith and Cytoscape Team by David History of PanGIA Application
- 109.
- 110.
- 111.
- 112. NeXO Web - TermEnrichment Analysis - From list of genes, perform hypergeometric test over set of machine-generated ontology (NeXO) terms and display terms with p-values - It is independent from all other parts of NeXO Web application
- 113. Term Enrichment Service APIby Flask Python Core SciPy NumPy Overview of NeXO Term Enrichment Service NeXO Web RESTful API
- 114. Term Enrichment Service APIby Flask Python Core SciPy NumPy Overview of NeXO Term Enrichment Service NeXO Web RESTful API
- 115. Option 1: Asa Cytoscape App - Re-implement this algorithm as a Cytoscape App (Java Application) - Pros: - Easy to install - Cons: - A lot of work… - Should be written in Java - Does not scale-out!
- 116. Option 2: Asa Service - Wrap existing applications and deploy to platform of users’ choice: - Laptops, private servers, and commercial cloud services (AWS/Google Computing Cloud, etc.) - Pros: - Scales-out - Client-independent - Workflow-friendly - Cons: - Need to adopt to the new way of software design - Relatively more complex deployment
- 117. –THE SMALL TOOLSMANIFESTO FOR BIOINFORMATICS “Every single tool should do the smallest possible task really well”
- 118. これからのツール - Best practice:アプリケーションを小さなサービス の集合としてデザインする - 文字通り、各サービスが独立したアプリケーショ ンとしても動作するよう心がける - デスクトップでもクラスタでも動作するように
- 119. cat genes.txt |uniq | wc -l
- 120. fetch | process| visualize
- 121.
- 122.
- 123.
- 124.
- 125.
- 126.
- 127.
- 128. Software Distribution Problem -“It-worked-on-my-machine” syndrome - This is a serious problem especially when you want to share your workflow with collaborators.
- 131. What is Docker? -Container to run applications in an isolated environment - Application = Layer of images - Sharable Environments - Environments as code
- 132.
- 133. Docker Hub - Sharingenvironments as code! - Dockerfile - Definition of your container - Example: http://bit.ly/15N23P8
- 134. How Docker-nized World Works? Container1: IPython + SciPy Stack Your Machine Container 2: R + Bioconductor Container 3: Neo4j
- 135. Your Machine Container 1: IPython+ SciPy Stack Container 2: R + Bioconductor Container 3: Neo4j
- 136. Your Machine Container 1: IPython+ SciPy Stack Container 2: R + Bioconductor Container 3: Neo4j
- 137.
- 138.
- 139.
- 140.
- 141.
- 142. Cytoscape Cyberinfrastructure (CI) -今後リリースする（システム）生物学向け サービスの総称 - ネットワークデータ解析 - 可視化 （レイアウトなど） - データ共有
- 143. Cytoscape Cyberinfrastructure (CI) -小さな再利用可能なモジュールとしてソフ トウェアをリリース - コマンドラインツール - RESTful APIでラップしたサービス - Cytoscape App
- 144. Interactive Data Analysis Environments In-HouseDatabases External Computing Resources - Graph Layout - Statistical Analysis - Data Pre-processing RStudio - NumPy - SciPy - Pandas - NetworkX IPython Notebook File / Code Hosting ServicesPublic Data Repository PSICQUIC Services EBI RDF Platform Other Bioinformatics Web Applications / Services - igraph - rCurl Command Line Tools > sed > awk > grep > curl Web Browsers Data Repository & Collaboration Service Data Bus (Internet) Your Workstation Cytoscape App Store Cytoscape Desktop Apps Core REST
- 145. Cytoscape Cyberinfrastructure (CI) -実際のリリースはこれから - 優先度の高いものから実装 - テストケースとしてHotNetや自動レイ アウトのEC2上でのサービス化等
- 146.
- 147.
- 148. We (the NIH)Are Working On, But As Yet Do Not Have Good Answers To: 1. Today, how much are we actually spending on data and software related activities? 2. How much should we be spending to achieve the maximum benefit to biomedical science relative to what we spend in other areas? Biomedical Research as an Open Digital Enterprise by Philip E. Bourne Ph.D. Associate Director for Data Science (NIH)
- 149. Reproducibility ! Most ofthe 27 Institutes and Centers of the NIH are currently reviewing the ability to reproduce research they are funding ! The NIH recently convened a meeting with publishers to discuss the issue – a set of guiding principles arose Biomedical Research as an Open Digital Enterprise by Philip E. Bourne Ph.D. Associate Director for Data Science (NIH)
- 150. NIH The Commons (Definitionby Dr. Bourne) • Is Not: • A database • Confined to one physical location • A new large infrastructure • Owned by any one group • Is: • A conceptual framework • Analogous to the Internet • A collaboratory • A few shared rules • All research objects have unique identifiers • All research objects have limited provenance
- 151.
- 152. バイオインフォ向けOSS - 日本発のものは残念ながら少ない - なぜ？ -文化の違い - 予算配分の違い
- 153. 「継続は力なり」 - Cytoscapeはこの分野では成功したツールの一つ - なぜ？ -特別に新しいことはしていない！ - しかし長年に渡り地道な作業を続けている - Funding Agencyからのインフラとしての認識
- 154. 「マイクロバリア」を取り除く - 人は些細な事で諦めます - 簡単にセットアップできるように -インストーラーの準備 - Dockerfileの公開 - R: install.packages()ですぐに使えるようにする - Python: pipで一発インストール出来るようにする
- 155.
- 156.
- 157.
- 158.