The document describes FluxGraph, a time-aware graph library built on top of Datomic. FluxGraph allows users to travel through different versions of a graph over time, iterate over vertices and edges within a specific time scope, and compare graphs at different points in time. It implements the Blueprints graph interface to provide time-aware graph capabilities while retaining compatibility with existing graph tools and libraries.
Managing Genetic Ancestry at Scale with Neo4j and Kafka - StampedeCon 2015StampedeCon
At the StampedeCon 2015 Big Data Conference: The global Monsanto R&D pipeline produces millions of new plant populations every year; each which contributes to a dataset of genetic ancestry spanning several decades. Historically the constraints of modeling and processing this data within an RDBMS has made drawing inferences from this dataset complex and computationally infeasible at large scale. Fortunately, the genetic history of any plant population forms a naturally occurring directed acyclic graph, a property that has allowed us to utilize graph theory to re-imagine how ancestral lineage data is modeled, stored, and queried.
In this talk we present our solutions to these problems, as realized using a graph-based approach within Neo4j. We will discuss our learnings around using Neo4j in a production setting that includes transactional and high-throughput computation, including how we transitioned from recursive JOIN queries to using Cypher and the Neo4j traversal framework to take full advantage of index-free adjacency. Our approach to polyglot persistence will be discussed via our use of a distributed commit log, Apache Kafka, to feed our graph store from sources of live transactional data. Finally, we will touch upon how we are using these technologies to annotate our genetic ancestry dataset with molecular genomics data in order to build an pipeline-scale genotype imputation platform with core algorithms built using Apache Spark.
HOBBIT's versioning benchmark at Graph-TA.
(This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 688227.)
Managing Genetic Ancestry at Scale with Neo4j and Kafka - StampedeCon 2015StampedeCon
At the StampedeCon 2015 Big Data Conference: The global Monsanto R&D pipeline produces millions of new plant populations every year; each which contributes to a dataset of genetic ancestry spanning several decades. Historically the constraints of modeling and processing this data within an RDBMS has made drawing inferences from this dataset complex and computationally infeasible at large scale. Fortunately, the genetic history of any plant population forms a naturally occurring directed acyclic graph, a property that has allowed us to utilize graph theory to re-imagine how ancestral lineage data is modeled, stored, and queried.
In this talk we present our solutions to these problems, as realized using a graph-based approach within Neo4j. We will discuss our learnings around using Neo4j in a production setting that includes transactional and high-throughput computation, including how we transitioned from recursive JOIN queries to using Cypher and the Neo4j traversal framework to take full advantage of index-free adjacency. Our approach to polyglot persistence will be discussed via our use of a distributed commit log, Apache Kafka, to feed our graph store from sources of live transactional data. Finally, we will touch upon how we are using these technologies to annotate our genetic ancestry dataset with molecular genomics data in order to build an pipeline-scale genotype imputation platform with core algorithms built using Apache Spark.
HOBBIT's versioning benchmark at Graph-TA.
(This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 688227.)
Graph databases in computational bioloby: case of neo4j and TitanDBAndrei KUCHARAVY
Code used for demos is available from: https://github.com/chiffa/neo4jDemo repositry
Code used for IO over the reactome is available from: https://github.com/chiffa/PolyPharma
Visual Exploration of Clinical and Genomic Data for Patient StratificationNils Gehlenborg
Talk presented at the Simons Foundation Biotech Symposium "Complex Data Visualization: Approach and Application" (12 September 2014)
http://www.simonsfoundation.org/event/complex-data-visualization-approach-and-application/
In this talk I describe how we integrated a sophisticated computational framework directly into the StratomeX visualization technique to enable rapid exploration of tens of thousands of stratifications in cancer genomics data, creating a unique and powerful tool for the identification and characterization of tumor subtypes. The tool can handle a wide range of genomic and clinical data types for cohorts with hundreds of patients. StratomeX also provides direct access to comprehensive data sets generated by The Cancer Genome Atlas Firehose analysis pipeline.
http://stratomex.caleydo.org
An introduction to the STINGER dynamic graph structure and analysis package. Shows the motivation for STINGER, what has been done with it, and how you can use it. More at http://cc.gatech.edu/stinger
Patient-Generated Data for Cancer Treatment and ManagementTommy Snitz
Research poster created by myself and Matthew Villarreal while we were students of The University of Texas at Austin's Health Informatics and Health IT Program.
Looks into the benefits and challenges of using patient-generated data in cancer treatment and management
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...Emad Shash
Tumor conferences are multidisciplinary meetings at which the
management of cancer patients is discussed. They have been
an integral part of oncology services and are regarded
as an essential component of quality control and continuing
medical education. There are data to suggest that the tumor conference enhances patient care. Many studies of effectiveness have been conducted. Reported benefits include improved patient management and treatment. In this presentation, I'll try to assess the role of the multidisciplinary tumor conference in patient management in gynecologic oncology services.
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transforming Data into Meaningful Information to Support Improved Patient Care"
Presentation "Transforming Data into Meaningful Information to Support Improved Patient Care"
Learning Objectives:
∙ Understand the information imperative for Cancer Care Ontario (CCO), one of the largest provincial health organizations in Canada, as it supports
population-based care co-ordination and administration for 3 clinical domains in the province of Ontario: cancer care, renal care, and access to
care
∙ Learn how the organization built the Informatics Centre of Excellence to better enable the acquisition, management, reporting, and analysis of one
of the broadest and richest data sets in the country
∙ Discuss concrete examples of how CCO has used leading-edge analytic techniques to drive health system performance.
Vickie Welch
Director, Informatics Centre of Excellence
Cancer Care Ontario
Hakim Lakhani
Director, Reporting and Analytics, Informatics Centre of Excellence
Cancer Care Ontario
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Graph databases in computational bioloby: case of neo4j and TitanDBAndrei KUCHARAVY
Code used for demos is available from: https://github.com/chiffa/neo4jDemo repositry
Code used for IO over the reactome is available from: https://github.com/chiffa/PolyPharma
Visual Exploration of Clinical and Genomic Data for Patient StratificationNils Gehlenborg
Talk presented at the Simons Foundation Biotech Symposium "Complex Data Visualization: Approach and Application" (12 September 2014)
http://www.simonsfoundation.org/event/complex-data-visualization-approach-and-application/
In this talk I describe how we integrated a sophisticated computational framework directly into the StratomeX visualization technique to enable rapid exploration of tens of thousands of stratifications in cancer genomics data, creating a unique and powerful tool for the identification and characterization of tumor subtypes. The tool can handle a wide range of genomic and clinical data types for cohorts with hundreds of patients. StratomeX also provides direct access to comprehensive data sets generated by The Cancer Genome Atlas Firehose analysis pipeline.
http://stratomex.caleydo.org
An introduction to the STINGER dynamic graph structure and analysis package. Shows the motivation for STINGER, what has been done with it, and how you can use it. More at http://cc.gatech.edu/stinger
Patient-Generated Data for Cancer Treatment and ManagementTommy Snitz
Research poster created by myself and Matthew Villarreal while we were students of The University of Texas at Austin's Health Informatics and Health IT Program.
Looks into the benefits and challenges of using patient-generated data in cancer treatment and management
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...Emad Shash
Tumor conferences are multidisciplinary meetings at which the
management of cancer patients is discussed. They have been
an integral part of oncology services and are regarded
as an essential component of quality control and continuing
medical education. There are data to suggest that the tumor conference enhances patient care. Many studies of effectiveness have been conducted. Reported benefits include improved patient management and treatment. In this presentation, I'll try to assess the role of the multidisciplinary tumor conference in patient management in gynecologic oncology services.
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transforming Data into Meaningful Information to Support Improved Patient Care"
Presentation "Transforming Data into Meaningful Information to Support Improved Patient Care"
Learning Objectives:
∙ Understand the information imperative for Cancer Care Ontario (CCO), one of the largest provincial health organizations in Canada, as it supports
population-based care co-ordination and administration for 3 clinical domains in the province of Ontario: cancer care, renal care, and access to
care
∙ Learn how the organization built the Informatics Centre of Excellence to better enable the acquisition, management, reporting, and analysis of one
of the broadest and richest data sets in the country
∙ Discuss concrete examples of how CCO has used leading-edge analytic techniques to drive health system performance.
Vickie Welch
Director, Informatics Centre of Excellence
Cancer Care Ontario
Hakim Lakhani
Director, Reporting and Analytics, Informatics Centre of Excellence
Cancer Care Ontario
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
2. about me
who am i ...
➡ working as an it lead / software architect @ janssen pharmaceutica
• dealing with big scientific data sets
• hands-on expertise in big data and NoSQL technologies
➡ founder of datablend
• provide big data and NoSQL consultancy
Davy Suvee • share practical knowledge and big data use cases via blog
@DSUVEE
5. graphs and time ...
➡ graphs are continuously changing ...
➡ graphs and time ...
★ neo-versioning by david montag 1
2
★ representing time dependent graphs in neo4j by the isi foundation
★ modeling a multilevel index in neo4j by peter neubauer 3
1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
6. graphs and time ...
➡ graphs are continuously changing ...
➡ graphs and time ...
★ neo-versioning by david montag 1
2
★ representing time dependent graphs in neo4j by the isi foundation
★ modeling a multilevel index in neo4j by peter neubauer 3
copy and relink semantics
๏ graph size
๏ object identity
๏ mixing data-model and time-model
1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
8. FluxGraph ...
➡ towards a time-aware graph ...
➡ implement a blueprints-compatible graph on top of Datomic
9. FluxGraph ...
➡ towards a time-aware graph ...
➡ implement a blueprints-compatible graph on top of Datomic
➡ make FluxGraph fully time-aware
★ travel your graph through time
★ time-scoped iteration of vertices and edges
★ temporal graph comparison
11. travel through time
FluxGraph fg = new FluxGraph();
Davy
Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
12. travel through time
FluxGraph fg = new FluxGraph();
Davy
Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
Peter
Vertex peter = ...
13. travel through time
FluxGraph fg = new FluxGraph();
Davy
Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
Peter
Vertex peter = ...
Vertex michael = ...
Michael
14. travel through time
FluxGraph fg = new FluxGraph();
Davy
kn
ow
Vertex davy = fg.addVertex();
s
davy.setProperty(“name”,”Davy”);
Peter
Vertex peter = ...
Vertex michael = ...
Edge e1 = Michael
fg.addEdge(davy, peter,“knows”);
15. travel through time
Davy
Date checkpoint = new Date();
kn
ow
s
Peter
Michael
16. travel through time
David
Date checkpoint = new Date();
kn
ow
s
davy.setProperty(“name”,”David”); Peter
Michael
17. travel through time
David
Date checkpoint = new Date();
kn
ow
s
davy.setProperty(“name”,”David”); Peter
kn
Edge e2 =
ow
fg.addEdge(davy, michael,“knows”);
s
Michael
28. time-scoped iteration
t1 t2 t3 tcurrrent
change change change
Davy Davy’ Davy’’ Davy’’’
➡ how to find the version of the vertex you are interested in?
36. time-scoped iteration
➡ When does an element change?
➡ vertex:
★ setting or removing a property
★ add or remove it from an edge
★ being removed
37. time-scoped iteration
➡ When does an element change?
➡ vertex: ➡ edge:
★ setting or removing a property ★ setting or removing a property
★ add or remove it from an edge ★ being removed
★ being removed
38. time-scoped iteration
➡ When does an element change?
➡ vertex: ➡ edge:
★ setting or removing a property ★ setting or removing a property
★ add or remove it from an edge ★ being removed
★ being removed
➡ ... and each element is time-scoped!
44. use case: longitudinal patient data
t1 t2 t3 t4 t5
smoking smoking death
patient patient patient patient patient
cancer cancer
45. use case: longitudinal patient data
➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
46. use case: longitudinal patient data
➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
➡ example analysis:
★ if a male patient is no longer smoking in 2005
★ what are the chances of getting lung cancer in 2010, comparing
patients that smoked before 2005
patients that never smoked
47. use case: longitudinal patient data
➡ get all male non-smokers in 2005
fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
48. use case: longitudinal patient data
➡ get all male non-smokers in 2005
fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
Iterator<Vertex> males =
fg.getVertices("gender", "male").iterator()
49. use case: longitudinal patient data
➡ get all male non-smokers in 2005
fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
Iterator<Vertex> males =
fg.getVertices("gender", "male").iterator()
while (males.hasNext()) {
Vertex p2005 = males.next();
boolean smoking2005 =
p2005.getEdges(OUT,"smokingStatus").iterator().hasNext();
}
50. use case: longitudinal patient data
➡ which patients were smoking before 2005?
boolean smokingBefore2005 =
((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() {
public TimeAwareElement filter(TimeAwareVertex element) {
return element.getEdges(OUT, "smokingStatus").iterator().hasNext()
? element : null;
}
}).iterator().hasNext();
51. use case: longitudinal patient data
➡ which patients have cancer in 2010
working set of smokers
Graph g =
fg.difference(smokerws,
time2010.toDate(),
time2005.toDate());
52. use case: longitudinal patient data
➡ which patients have cancer in 2010
working set of smokers
Graph g =
fg.difference(smokerws,
time2010.toDate(),
time2005.toDate());
➡ extract the patients that have an edge to the cancer node
56. gephi plugin for blueprints!
1
➡ available on github
http://github.com/datablend/gephi-blueprints-plugin
➡ Support for neo4j, orientdb, dex, rexter, ...
1. Kudos to Timmy Storms (@timmystorms)