The Power of Graphs to Analyze Biological Data
•
0 likes•5,009 views
This talk will illustrate the power and flexibility of Graph Databases and Neo4j specifically to help in the overall analysis of biological data sets. Davy will show how to build a visual exploration environment that helps researchers at identifying clusters within various biological data sets, including gene expression and mutation prevalence data. Additionally, he will demo BRAIN (Bio Relations and Intelligence Network), a powerful data exploration platform that combines various scientific data sources (including Pubmed, Swissprot and Drugbank). It uses Neo4J under the cover to both store and enable powerful querying capabilities that provide key insights and deductions.
Report
Share
Report
Share
Download to read offline
Recommended
Is one enough? Data warehousing for biomedical research
The document discusses challenges in storing and managing real-world biomedical data from multiple sources for analysis. It describes three different data warehouse case studies used at Novartis - Avalon, MAGMA, and the Entity Warehouse. The Entity Warehouse takes a novel approach of modeling data as entities that can be linked together, with results stored in tables by type. It is designed to integrate both internal and external data while allowing broad access. However, the document concludes that no single warehouse fits all needs, and multiple solutions may be required to fully enable data analysis.
Is that a scientific report or just some cool pictures from the lab? Reproduc...
Requirements for reproducibility in computational chemistry publications include making available the data, code or algorithms, and results from the study. Authors should provide all data necessary to understand and assess their conclusions. Source code or detailed algorithm descriptions should also be included to allow independent reproduction of the work. Finally, publications must contain the actual results from applying the method rather than just describing results. Adopting these standards of transparency helps ensure others can evaluate and build upon published research claims.
From Queries to Algorithms to Advanced ML: 3 Pharmaceutical Graph Use Cases
Dr. Alexander Jarasch - German Centre of Diabetes Research
Neo4j Health Care & Life Sciences Workshop 2021
Machine learning in the life sciences with knime
This document discusses using machine learning and the KNIME platform to build predictive models for problems in the life sciences using molecular data. It provides an example of building a random forest model to predict biological activity of molecules using molecular fingerprints as features. The model achieves high accuracy but predicts inactivity for almost all molecules due to class imbalance in the data. To address this, the document suggests adjusting the decision boundary of the model by setting it at the point on the ROC curve that retrieves most actives without including too many inactives. In summary, it presents an example of applying machine learning to predict biological activity from molecular data and discusses techniques for handling class imbalance.
Analyzing Perturbed Co-Expression Networks in Cancer Using a Graph Database
This document summarizes analyzing perturbed co-expression networks in cancer using a graph database. It describes building co-expression networks from The Cancer Genome Atlas RNA-seq data for several cancer types and tissues. Key network analysis methods applied include PageRank to identify important genes, Louvain clustering to identify communities of co-expressed genes, and finding shortest paths. The analysis provides insights into differences between normal and tumor networks and identifies gene communities related to processes like immune response and DNA replication.
Big biomedical data is a lie
Big biomedical data is often not large enough for advanced machine learning techniques. Platforms for sharing data are problematic as each operates independently with different data formats and tools. Computation power alone cannot solve these issues. The document discusses solutions like allowing larger datasets to be assembled using standards, developing interoperable "meta-platforms", and choosing machine learning approaches wisely based on the problem and dataset. It provides examples of projects that have successfully combined datasets and used techniques like Spark clustering to better leverage available data.
VariantSpark a library for genomics by Lynn Langit
VariantSpark is a library for scalable genomic analysis that can process large genomic datasets containing millions of variants and thousands of samples. It uses machine learning techniques like k-means clustering and random forests for unsupervised and supervised analysis. VariantSpark can analyze whole genome datasets faster than other methods and scale to process 100% of genomic data. It also integrates with cloud platforms like AWS and Databricks for easy access and demo of its capabilities through Jupyter notebooks.
Building an informatics solution to sustain AI-guided cell profiling with hig...
This document discusses building an informatics solution to support AI-guided cell profiling using high-content microscopy imaging. The research group aims to accelerate drug discovery using AI and intelligent experimental design. Their goal is to build pipelines of containerized services and workflows to continuously process imaging data, train models, and design new experiments in an automated and scalable manner. This will enable data-driven science through predictive modeling and closing the loop between data generation, analysis, and experimentation.
Recommended
Is one enough? Data warehousing for biomedical research
The document discusses challenges in storing and managing real-world biomedical data from multiple sources for analysis. It describes three different data warehouse case studies used at Novartis - Avalon, MAGMA, and the Entity Warehouse. The Entity Warehouse takes a novel approach of modeling data as entities that can be linked together, with results stored in tables by type. It is designed to integrate both internal and external data while allowing broad access. However, the document concludes that no single warehouse fits all needs, and multiple solutions may be required to fully enable data analysis.
Is that a scientific report or just some cool pictures from the lab? Reproduc...
Requirements for reproducibility in computational chemistry publications include making available the data, code or algorithms, and results from the study. Authors should provide all data necessary to understand and assess their conclusions. Source code or detailed algorithm descriptions should also be included to allow independent reproduction of the work. Finally, publications must contain the actual results from applying the method rather than just describing results. Adopting these standards of transparency helps ensure others can evaluate and build upon published research claims.
From Queries to Algorithms to Advanced ML: 3 Pharmaceutical Graph Use Cases
Dr. Alexander Jarasch - German Centre of Diabetes Research
Neo4j Health Care & Life Sciences Workshop 2021
Machine learning in the life sciences with knime
This document discusses using machine learning and the KNIME platform to build predictive models for problems in the life sciences using molecular data. It provides an example of building a random forest model to predict biological activity of molecules using molecular fingerprints as features. The model achieves high accuracy but predicts inactivity for almost all molecules due to class imbalance in the data. To address this, the document suggests adjusting the decision boundary of the model by setting it at the point on the ROC curve that retrieves most actives without including too many inactives. In summary, it presents an example of applying machine learning to predict biological activity from molecular data and discusses techniques for handling class imbalance.
Analyzing Perturbed Co-Expression Networks in Cancer Using a Graph Database
This document summarizes analyzing perturbed co-expression networks in cancer using a graph database. It describes building co-expression networks from The Cancer Genome Atlas RNA-seq data for several cancer types and tissues. Key network analysis methods applied include PageRank to identify important genes, Louvain clustering to identify communities of co-expressed genes, and finding shortest paths. The analysis provides insights into differences between normal and tumor networks and identifies gene communities related to processes like immune response and DNA replication.
Big biomedical data is a lie
Big biomedical data is often not large enough for advanced machine learning techniques. Platforms for sharing data are problematic as each operates independently with different data formats and tools. Computation power alone cannot solve these issues. The document discusses solutions like allowing larger datasets to be assembled using standards, developing interoperable "meta-platforms", and choosing machine learning approaches wisely based on the problem and dataset. It provides examples of projects that have successfully combined datasets and used techniques like Spark clustering to better leverage available data.
VariantSpark a library for genomics by Lynn Langit
VariantSpark is a library for scalable genomic analysis that can process large genomic datasets containing millions of variants and thousands of samples. It uses machine learning techniques like k-means clustering and random forests for unsupervised and supervised analysis. VariantSpark can analyze whole genome datasets faster than other methods and scale to process 100% of genomic data. It also integrates with cloud platforms like AWS and Databricks for easy access and demo of its capabilities through Jupyter notebooks.
Building an informatics solution to sustain AI-guided cell profiling with hig...
This document discusses building an informatics solution to support AI-guided cell profiling using high-content microscopy imaging. The research group aims to accelerate drug discovery using AI and intelligent experimental design. Their goal is to build pipelines of containerized services and workflows to continuously process imaging data, train models, and design new experiments in an automated and scalable manner. This will enable data-driven science through predictive modeling and closing the loop between data generation, analysis, and experimentation.
Introduction to Graph Databases @ SAI
This document discusses big data and graph databases. It begins by introducing the author and defining big data in terms of volume, variety and velocity. It then discusses how graph databases can help tackle problems related to big data by being schema-less, allowing for index-free queries, and enabling traversal-based queries. Several use cases for graph databases are provided like recommendations, access control and social networks. The document concludes by discussing potential applications of graph databases for longitudinal patient data, innovation networking and various domains.
DNA analysis on your laptop: Spot the differences
Presentation/workshop at Tech Tuesday. Organized by Permanent Beta and Seats2meet, Utrecht. https://www.meetup.com/PermanentBeta/events/234410344/
D02-NextGenSeq-MOLGENIS
Large scale NGS pipelines using the MOLGENIS platform: processing the Genome of the Netherlands (Morris Swertz)
2015 genome-center
This document discusses the challenges and opportunities biology faces with increasing data generation. It outlines four key points:
1) Research approaches for analyzing infinite genomic data streams, such as digital normalization which compresses data while retaining information.
2) The need for usable software and decentralized infrastructure to perform real-time, streaming data analysis.
3) The importance of open science and reproducibility given most researchers cannot replicate their own computational analyses.
4) The lack of data analysis training in biology and efforts at UC Davis to address this through workshops and community building.
20170426 - Deep Learning Applications in Genomics - Vancouver - Simon Fraser ...
This document discusses Google's capabilities for handling large genomic and biomedical data sets. It describes how Google uses technologies like Google Cloud, BigQuery, Dataflow and TensorFlow to process, store and analyze massive volumes of genomic and medical data. Google's systems can handle hundreds of terabytes to petabytes of data and enable fast querying and machine learning on these data sets. The document also provides examples of how Google is applying these capabilities to challenges in genomics, healthcare and precision medicine.
2016 davis-plantbio
The document discusses the challenges and opportunities that will arise from the exponential growth of biological data in the coming years. It outlines four key areas: 1) Research approaches will need to effectively analyze infinite amounts of data. 2) Software and decentralized infrastructure will be needed to process the data. 3) Open science and reproducible research practices are important for data-driven biology. 4) Training the next generation of biologists in data analysis skills will be a major challenge. The document advocates for open source tools, reproducible research methods, and expanded training programs to help biology take advantage of the coming data deluge.
Swertz bosc2010 molgenis
The document describes MOLGENIS, an open-source software system that allows users to define data models and generate full-featured web applications and databases from those models. Key features include a graphical user interface, database integration, support for common data formats, and the ability to rapidly develop applications by editing simple domain-specific models. The system has been applied to build several genomic and biomedical databases.
Cloud Accelerated Genomics by Allen Day of Google
Abstract:
In this session we will explore how Google's Cloud services (CloudML, Vision, Genomics API) can be used to process genomic and phenotypic data and solve problems in healthcare and agriculture.
Bio:
Allen Day is a Science Advocate with Google Cloud. He's a professional software developer and storyteller with expertise in computational biology, statistics, and distributed computing. Prior to joining Google in Seattle, Allen was based in Singapore as Chief Scientist at MapR, a Silicon Valley BigData platform company.
20170428 - Look to Precision Agriculture to Bootstrap Precision Medicine - Cu...
The document discusses applications of deep learning concepts and techniques to problems in genomics and precision agriculture. It describes how deep neural networks can be used for tasks like calling genetic variants from DNA sequencing data more accurately, enabling marker-assisted breeding in crops by identifying desirable genetic variants, and integrating diverse data sources like images and sensor data for optimization in precision agriculture. The document also discusses opportunities and challenges for applying these approaches to problems in the cannabis industry.
Accelerate pharmaceutical r&d with mongo db
This document provides a summary of a presentation on using MongoDB and big data technologies to accelerate pharmaceutical research and development at AstraZeneca. The presentation discusses:
- AstraZeneca's focus on using next generation sequencing and big data to predict drug effectiveness and identify new drug targets
- Pilot projects using MongoDB to store and query unstructured genomic data at scale, which proved the technology's ability to enable researchers more quickly
- A vision for an experiment management system to integrate various data sources and processing pipelines using big data technologies
Accelerate Pharmaceutical R&D with Big Data and MongoDB
This document provides a summary of a presentation on using MongoDB and big data technologies to accelerate pharmaceutical research and development at AstraZeneca. The presentation discusses:
- AstraZeneca's focus on using next generation sequencing and big data to predict drug effectiveness and find associations between gene sequences and drug responses.
- Pilot projects using MongoDB to store and query unstructured genomic and clinical trial data at scale in a flexible document format.
- How these pilots helped prove the value of NoSQL databases for enabling faster exploration and analysis of large, complex datasets by researchers.
- Future visions for using experimental management systems and big data analytics to integrate multiple data types and power predictive analytics across AstraZeneca's drug development pipelines
Facilitating Data Curation: a Solution Developed in the Toxicology Domain
Christophe Debruyne, Jonathan Riggio, Emma Gustafson, Declan O'Sullivan, Mathieu Vinken, Tamara Vanhaecke, Olga De Troyer.
Presented at the 2020 IEEE 14th International Conference on Semantic Computing, San Diego, California, 3-5 February 2020
Toxicology aims to understand the adverse effects of
chemical compounds or physical agents on living organisms. For
chemicals, much information regarding safety testing of cosmetic
ingredients is now scattered in a plethora of safety evaluation
reports. Toxicologists in our university intend to collect this
information into a single repository. Their current approach uses
spreadsheets, does not scale well, and makes data curation and
querying cumbersome. Semantic technologies (e.g., RDF, OWL,
and Linked Data principles) would be more appropriate for
this purpose. However, this technology is not very accessible to
toxicologists without extensive training. In this paper, we report
on a tool that supports subject matter experts in the construction
of an RDF–based knowledge base for the toxicology domain. The
tool is using the jigsaw metaphor for guiding the subject matter
experts. We demonstrate that the jigsaw metaphor is a viable
option for generating RDF. Future work includes investigating
appropriate methods and tools for knowledge evolution and data
analysis.
20170406 Genomics@Google - KeyGene - Wageningen
Genomics use cases for Agriculture using DeepLearning (DeepVariant) and Genomics API from Google Cloud
2016 bergen-sars
This document outlines a 12-step program for biology to adapt to the era of data-intensive science. It summarizes the author's background and research interests. It then discusses the rapid growth of biological data from techniques like DNA sequencing. It introduces the concept of digital normalization as a way to efficiently process large transcriptome datasets. Finally, it outlines some proposed steps for the field, including investing in computational training, a focus on biological questions, and moving to continuous data updating models.
3D culture in phenotypic screening : advantages, process changes and new tech...
It was a real pleasure to be in the « High-Content and Phenotypic Screening » meeting in Cambridge. We were invited by our partner Molecular Devices to give a talk during the "User meeting Molecular Devices" about our vision of 3D culture in phenotypic screening and the impact of new technologies. We also presented a poster about "Neurotoxicity assay on 2D and 3D culture using High Content Screening technology".
20170402 Crop Innovation and Business - Amsterdam
This document discusses applying machine learning and artificial intelligence techniques like deep neural networks to problems in genomics and agriculture. It provides examples of using Google Cloud platforms and services for storing and analyzing large genomic datasets, as well as developing models for tasks like variant calling from sequencing data and marker-assisted breeding. The document advocates that Google is well-positioned to handle massive volumes of genomic and agricultural data and help advance the application of AI in these domains.
2013 10-30-sbc361-reproducible designsandsustainablesoftware
This document provides guidance on organizing computational biology projects. It recommends:
1. Organizing related projects together under a common root directory, with each project in its own subdirectory.
2. Using a logical top-level structure within each project with chronological organization at lower levels. This allows tracking experiments over time.
3. Including directories for data, source code, results, and documentation. Results subdirectories should be named with dates to allow sorting experiments chronologically.
Taking detailed, dated notes in lab notebooks or Markdown files integrated with analysis code allows fully documenting projects and easily repeating or extending prior work.
VariantSpark: applying Spark-based machine learning methods to genomic inform...
Genomic information is increasingly used in medical practice giving rise to the need for efficient analysis methodology able to cope with thousands of individuals and millions of variants. Here we introduce VariantSpark, which utilizes Hadoop/Spark along with its machine learning library, MLlib, providing the means of parallelisation for population-scale bioinformatics tasks. VariantSpark is the interface to the standard variant format (VCF), offers seamless genome-wide sampling of variants and provides a pipeline for visualising results.
To demonstrate the capabilities of VariantSpark, we clustered more than 3,000 individuals with 80 Million variants each to determine the population structure in the dataset. VariantSpark is 80% faster than the Spark-based genome clustering approach, ADAM, the comparable implementation using Hadoop/Mahout, as well as Admixture, a commonly used tool for determining individual ancestries. It is over 90% faster than traditional implementations using R and Python. These benefits of speed, resource consumption and scalability enables VariantSpark to open up the usage of advanced, efficient machine learning algorithms to genomic data.
The package is written in Scala and available at https://github.com/BauerLab/VariantSpark.
Humanizing bioinformatics
In this talk, I explain the need for basic visualization know-how in bioinformatics.
Building an Internet of Genomics
This presentation describes the work of the Global Alliance for Genomics and Health, and its members, to develop standards and technologies to make genomics and clinical data more findable, accessible, and useful.
MongoDB Analytics
This document discusses using MongoDB for chemical similarity analysis. It describes storing compound fingerprints in MongoDB collections and querying them to find similar compounds. The approach involves building a fingerprint index, mapping over compounds to calculate similarities, and reducing the results to find the closest matches. This allows evaluating chemical similarity for millions of compounds faster than pairwise comparisons.
Coalition cocktail - hack the elections
Trying to predict the possible political coalitions for the recent Belgian elections based upon social interaction data.
More Related Content
Similar to The Power of Graphs to Analyze Biological Data
Introduction to Graph Databases @ SAI
This document discusses big data and graph databases. It begins by introducing the author and defining big data in terms of volume, variety and velocity. It then discusses how graph databases can help tackle problems related to big data by being schema-less, allowing for index-free queries, and enabling traversal-based queries. Several use cases for graph databases are provided like recommendations, access control and social networks. The document concludes by discussing potential applications of graph databases for longitudinal patient data, innovation networking and various domains.
DNA analysis on your laptop: Spot the differences
Presentation/workshop at Tech Tuesday. Organized by Permanent Beta and Seats2meet, Utrecht. https://www.meetup.com/PermanentBeta/events/234410344/
D02-NextGenSeq-MOLGENIS
Large scale NGS pipelines using the MOLGENIS platform: processing the Genome of the Netherlands (Morris Swertz)
2015 genome-center
This document discusses the challenges and opportunities biology faces with increasing data generation. It outlines four key points:
1) Research approaches for analyzing infinite genomic data streams, such as digital normalization which compresses data while retaining information.
2) The need for usable software and decentralized infrastructure to perform real-time, streaming data analysis.
3) The importance of open science and reproducibility given most researchers cannot replicate their own computational analyses.
4) The lack of data analysis training in biology and efforts at UC Davis to address this through workshops and community building.
20170426 - Deep Learning Applications in Genomics - Vancouver - Simon Fraser ...
This document discusses Google's capabilities for handling large genomic and biomedical data sets. It describes how Google uses technologies like Google Cloud, BigQuery, Dataflow and TensorFlow to process, store and analyze massive volumes of genomic and medical data. Google's systems can handle hundreds of terabytes to petabytes of data and enable fast querying and machine learning on these data sets. The document also provides examples of how Google is applying these capabilities to challenges in genomics, healthcare and precision medicine.
2016 davis-plantbio
The document discusses the challenges and opportunities that will arise from the exponential growth of biological data in the coming years. It outlines four key areas: 1) Research approaches will need to effectively analyze infinite amounts of data. 2) Software and decentralized infrastructure will be needed to process the data. 3) Open science and reproducible research practices are important for data-driven biology. 4) Training the next generation of biologists in data analysis skills will be a major challenge. The document advocates for open source tools, reproducible research methods, and expanded training programs to help biology take advantage of the coming data deluge.
Swertz bosc2010 molgenis
The document describes MOLGENIS, an open-source software system that allows users to define data models and generate full-featured web applications and databases from those models. Key features include a graphical user interface, database integration, support for common data formats, and the ability to rapidly develop applications by editing simple domain-specific models. The system has been applied to build several genomic and biomedical databases.
Cloud Accelerated Genomics by Allen Day of Google
Abstract:
In this session we will explore how Google's Cloud services (CloudML, Vision, Genomics API) can be used to process genomic and phenotypic data and solve problems in healthcare and agriculture.
Bio:
Allen Day is a Science Advocate with Google Cloud. He's a professional software developer and storyteller with expertise in computational biology, statistics, and distributed computing. Prior to joining Google in Seattle, Allen was based in Singapore as Chief Scientist at MapR, a Silicon Valley BigData platform company.
20170428 - Look to Precision Agriculture to Bootstrap Precision Medicine - Cu...
The document discusses applications of deep learning concepts and techniques to problems in genomics and precision agriculture. It describes how deep neural networks can be used for tasks like calling genetic variants from DNA sequencing data more accurately, enabling marker-assisted breeding in crops by identifying desirable genetic variants, and integrating diverse data sources like images and sensor data for optimization in precision agriculture. The document also discusses opportunities and challenges for applying these approaches to problems in the cannabis industry.
Accelerate pharmaceutical r&d with mongo db
This document provides a summary of a presentation on using MongoDB and big data technologies to accelerate pharmaceutical research and development at AstraZeneca. The presentation discusses:
- AstraZeneca's focus on using next generation sequencing and big data to predict drug effectiveness and identify new drug targets
- Pilot projects using MongoDB to store and query unstructured genomic data at scale, which proved the technology's ability to enable researchers more quickly
- A vision for an experiment management system to integrate various data sources and processing pipelines using big data technologies
Accelerate Pharmaceutical R&D with Big Data and MongoDB
This document provides a summary of a presentation on using MongoDB and big data technologies to accelerate pharmaceutical research and development at AstraZeneca. The presentation discusses:
- AstraZeneca's focus on using next generation sequencing and big data to predict drug effectiveness and find associations between gene sequences and drug responses.
- Pilot projects using MongoDB to store and query unstructured genomic and clinical trial data at scale in a flexible document format.
- How these pilots helped prove the value of NoSQL databases for enabling faster exploration and analysis of large, complex datasets by researchers.
- Future visions for using experimental management systems and big data analytics to integrate multiple data types and power predictive analytics across AstraZeneca's drug development pipelines
Facilitating Data Curation: a Solution Developed in the Toxicology Domain
Christophe Debruyne, Jonathan Riggio, Emma Gustafson, Declan O'Sullivan, Mathieu Vinken, Tamara Vanhaecke, Olga De Troyer.
Presented at the 2020 IEEE 14th International Conference on Semantic Computing, San Diego, California, 3-5 February 2020
Toxicology aims to understand the adverse effects of
chemical compounds or physical agents on living organisms. For
chemicals, much information regarding safety testing of cosmetic
ingredients is now scattered in a plethora of safety evaluation
reports. Toxicologists in our university intend to collect this
information into a single repository. Their current approach uses
spreadsheets, does not scale well, and makes data curation and
querying cumbersome. Semantic technologies (e.g., RDF, OWL,
and Linked Data principles) would be more appropriate for
this purpose. However, this technology is not very accessible to
toxicologists without extensive training. In this paper, we report
on a tool that supports subject matter experts in the construction
of an RDF–based knowledge base for the toxicology domain. The
tool is using the jigsaw metaphor for guiding the subject matter
experts. We demonstrate that the jigsaw metaphor is a viable
option for generating RDF. Future work includes investigating
appropriate methods and tools for knowledge evolution and data
analysis.
20170406 Genomics@Google - KeyGene - Wageningen
Genomics use cases for Agriculture using DeepLearning (DeepVariant) and Genomics API from Google Cloud
2016 bergen-sars
This document outlines a 12-step program for biology to adapt to the era of data-intensive science. It summarizes the author's background and research interests. It then discusses the rapid growth of biological data from techniques like DNA sequencing. It introduces the concept of digital normalization as a way to efficiently process large transcriptome datasets. Finally, it outlines some proposed steps for the field, including investing in computational training, a focus on biological questions, and moving to continuous data updating models.
3D culture in phenotypic screening : advantages, process changes and new tech...
It was a real pleasure to be in the « High-Content and Phenotypic Screening » meeting in Cambridge. We were invited by our partner Molecular Devices to give a talk during the "User meeting Molecular Devices" about our vision of 3D culture in phenotypic screening and the impact of new technologies. We also presented a poster about "Neurotoxicity assay on 2D and 3D culture using High Content Screening technology".
20170402 Crop Innovation and Business - Amsterdam
This document discusses applying machine learning and artificial intelligence techniques like deep neural networks to problems in genomics and agriculture. It provides examples of using Google Cloud platforms and services for storing and analyzing large genomic datasets, as well as developing models for tasks like variant calling from sequencing data and marker-assisted breeding. The document advocates that Google is well-positioned to handle massive volumes of genomic and agricultural data and help advance the application of AI in these domains.
2013 10-30-sbc361-reproducible designsandsustainablesoftware
This document provides guidance on organizing computational biology projects. It recommends:
1. Organizing related projects together under a common root directory, with each project in its own subdirectory.
2. Using a logical top-level structure within each project with chronological organization at lower levels. This allows tracking experiments over time.
3. Including directories for data, source code, results, and documentation. Results subdirectories should be named with dates to allow sorting experiments chronologically.
Taking detailed, dated notes in lab notebooks or Markdown files integrated with analysis code allows fully documenting projects and easily repeating or extending prior work.
VariantSpark: applying Spark-based machine learning methods to genomic inform...
Genomic information is increasingly used in medical practice giving rise to the need for efficient analysis methodology able to cope with thousands of individuals and millions of variants. Here we introduce VariantSpark, which utilizes Hadoop/Spark along with its machine learning library, MLlib, providing the means of parallelisation for population-scale bioinformatics tasks. VariantSpark is the interface to the standard variant format (VCF), offers seamless genome-wide sampling of variants and provides a pipeline for visualising results.
To demonstrate the capabilities of VariantSpark, we clustered more than 3,000 individuals with 80 Million variants each to determine the population structure in the dataset. VariantSpark is 80% faster than the Spark-based genome clustering approach, ADAM, the comparable implementation using Hadoop/Mahout, as well as Admixture, a commonly used tool for determining individual ancestries. It is over 90% faster than traditional implementations using R and Python. These benefits of speed, resource consumption and scalability enables VariantSpark to open up the usage of advanced, efficient machine learning algorithms to genomic data.
The package is written in Scala and available at https://github.com/BauerLab/VariantSpark.
Humanizing bioinformatics
In this talk, I explain the need for basic visualization know-how in bioinformatics.
Building an Internet of Genomics
This presentation describes the work of the Global Alliance for Genomics and Health, and its members, to develop standards and technologies to make genomics and clinical data more findable, accessible, and useful.
Similar to The Power of Graphs to Analyze Biological Data (20)
20170426 - Deep Learning Applications in Genomics - Vancouver - Simon Fraser ...
20170426 - Deep Learning Applications in Genomics - Vancouver - Simon Fraser ...
20170428 - Look to Precision Agriculture to Bootstrap Precision Medicine - Cu...
20170428 - Look to Precision Agriculture to Bootstrap Precision Medicine - Cu...
Accelerate Pharmaceutical R&D with Big Data and MongoDB
Accelerate Pharmaceutical R&D with Big Data and MongoDB
Facilitating Data Curation: a Solution Developed in the Toxicology Domain
Facilitating Data Curation: a Solution Developed in the Toxicology Domain
3D culture in phenotypic screening : advantages, process changes and new tech...
3D culture in phenotypic screening : advantages, process changes and new tech...
2013 10-30-sbc361-reproducible designsandsustainablesoftware
2013 10-30-sbc361-reproducible designsandsustainablesoftware
VariantSpark: applying Spark-based machine learning methods to genomic inform...
VariantSpark: applying Spark-based machine learning methods to genomic inform...
More from datablend
MongoDB Analytics
This document discusses using MongoDB for chemical similarity analysis. It describes storing compound fingerprints in MongoDB collections and querying them to find similar compounds. The approach involves building a fingerprint index, mapping over compounds to calculate similarities, and reducing the results to find the closest matches. This allows evaluating chemical similarity for millions of compounds faster than pairwise comparisons.
Coalition cocktail - hack the elections
Trying to predict the possible political coalitions for the recent Belgian elections based upon social interaction data.
FluxGraph @ GraphDevRoom
The document describes FluxGraph, a time-aware graph implementation built on top of Datomic. FluxGraph makes graphs time-aware by allowing traversal of the graph through time, time-scoped iteration over vertices and edges, and temporal comparison of graphs. It implements the Blueprints graph API for compatibility and stores graph data and changes in Datomic for versioning and time-aware querying capabilities.
The power of graphs to analyze biological data
This document discusses using graphs to analyze biological datasets. It provides examples of using MongoDB to store gene expression data and performing Pearson correlation calculations to create a co-expression graph in Neo4j. It also discusses a prototype called FluxGraph that adds time-awareness to graphs, allowing traversal of graph states through time and comparison of temporal graphs. Potential use cases discussed include longitudinal analysis of patient data over many years.
FluxGraph: a time-machine for your graphs
The document describes FluxGraph, a time-aware graph library built on top of Datomic. FluxGraph allows users to travel through different versions of a graph over time, iterate over vertices and edges within a specific time scope, and compare graphs at different points in time. It implements the Blueprints graph interface to provide time-aware graph capabilities while retaining compatibility with existing graph tools and libraries.
8 things I like about Datomic
Introduction to Datomic that highlights its core feature set.
Intorudction to FluxGraph, a temporal Graph Database implemented on top of Datomic.
More from datablend (6)
Recently uploaded
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Communications Mining Series - Zero to Hero - Session 1
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
UiPath Test Automation using UiPath Test Suite series, part 5
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Artificial Intelligence for XMLDevelopment
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Securing your Kubernetes cluster_ a step-by-step guide to success !
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
“I’m still / I’m still / Chaining from the Block”
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
DevOps and Testing slides at DASA Connect
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Epistemic Interaction - tuning interfaces to provide information for AI support
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
Recently uploaded (20)
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
The Power of Graphs to Analyze Biological Data
- 1. GraphConnect
- 2. the power of graphs to analyze biological data
- 3. about me who am i ... ➡ big data architect @ datablend - continuum provide big data and nosql consultancy • 5 years of hands-on expertise in the pharma/biotech sector • Davy Suvee @DSUVEE
- 4. big data in pharma massive data scalable number crunching platform complex data visual insights-driven platform full genome sequencing biological networks graphs!!
- 5. big data in pharma (2 specific use cases) outlier detection platform neo4j, mongodb/cassandra and gephi euretos - brain neo4j, mongodb, solr and prefuse
- 6. gene expression clustering ➡ oncology data set: ★ 4.800 samples ★ 27.000 genes ➡ Question: ★ for a particular subset of samples, which genes are co-expressed?
- 7. storing gene expressions (mongodb) { "_id" : { "$oid" : "4f1fb64a1695629dd9d916e3"} , "sample_name" : "122551hp133a21.cel" , "genomics_id" : 122551 , "sample_id" : 343981 , "donor_id" : 143981 , "sample_type" : "Tissue" , "sample_site" : "Ascending colon" , "pathology_category" : "MALIGNANT" , "pathology_morphology" : "Adenocarcinoma" , "pathology_type" : "Primary malignant neoplasm of colon" , "primary_site" : "Colon" , "expressions" : [ { "gene" : "X1_at" , "expression" : 5.54217719084415} , { "gene" : "X10_at" , "expression" : 3.92335121981739} , { "gene" : "X100_at" , "expression" : 7.81638155662255} , { "gene" : "X1000_at" , "expression" : 5.44318512260619} , … ] }
- 8. correlating samples (mongodb/map-reduce) x pearson correlation y 43 99 21 65 25 79 42 75 57 87 59 81 0,52
- 9. co-expression graph (neo4j) 122551 correlat ed 6 create an edge between both nodes 8 value : 0, ➡ create a node for each sample ➡ if correlation between two samples >= 0.8 122553 122552
- 11. euretos - brain ➡ pubmed: 23 million biomedical articles 1300 new ones added every day • google-like search interface • ➡ reading an article ... • malaria is transferred by mosquitoes
- 13. euretos - brain ooooooh crap ...
- 14. euretos - brain ➡ nanopub (nanopub.org) • the smallest unit of publishable information ➡ assertion • subject: malaria • predicate: transferred by • object: mosquito ➡ provenance • how this came to be (meta-data)
- 15. euretos - brain ➡ unfortunately, malaria is encoded in various ways ... db1 db2 db3 malaria P22384 AQ879 malaria
- 16. euretos - brain malaria transferred by mosquito
- 17. euretos - brain ➡ brain (http://www.euretos.com/brain) exploration and analysis platform • millions of concepts/triples/nanopubs • pubmed, uniprot, omim, pubchem, ... • ➡ architectural stack • • • meta-data is stored in mongodb graph in neo4j swing interface connecting to rest endpoints
- 18. brain
- 19. brain
- 20. brain
- 21. brain
- 22. brain
- 23. brain
- 24. brain
- 25. brain
- 26. Questions?
- 27. datablend - continuum Follow us E-MAIL twitter.com/data_blend www.datablend.be info@datablend.be www.datablend.be info@datablend.be 0499/05.00.89