Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Visual Programming
Environments for
Science and Business
MITCH MILLER
SCIENTIFIC THINKING
CODE CAMP 2015
SEPTEMBER 19, 2015
Disclaimer
 This talk represents my opinion and personal experience using 2 fine
software systems developed by third part...
Overview
 Introduction: first demo
 What is a ‘visual programming environment’
 The two systems we’ll look at today
 W...
Demo 1: set-up
 Task: produce report of all compounds registered during January
Visual Programming: informal
definition
 Drag functional components onto canvas to create program
 Configure most compon...
Component types
 File I/O
 Read/write text files
 Read/write MS Office documents
 XML
 JSON
 PDF
 Database access
...
Component types (continued)
 Web service consumption
 Domain-specific processing
 Chemical structure I/O
 Chemical str...
Component types (continued)
 Visualization
 Graphing
 Statistical calculations
 Scripting
 Tip: aim for brief scripts...
KNIME
 Originally a production of the University of Konstanz, Germany 2004
 Currently produced by KNIME.com AG, a compan...
KNIME (continued)
 Java based
 Written in Java
 Scripted, extensible in Java
 URL: https://www.knime.org/
Pipeline Pilot
 Developed and sold by BIOVIA, San Diego, CA
 Originally developed by Scitegic, San Diego in 1999
 Desig...
KNIME Terminology
 Components are called “Nodes”
 Programs are “Workflows”
 Reusable sets of Nodes are “Metanodes”
 Gr...
Pipeline Pilot Terminology
 Components are called “Components”
 Programs are “Protocols”
 Reusable sets of Components a...
Different systems shown today
serve different populations
 KNIME can be used ad hoc on the desktop of a power user. It is...
Programs can be deployed outside
the development client
 Give users a URL to access your program
 Users of BIOVIA Electr...
Server Features
 User access configuration
 Shared data sources
 Automatic jobs
 Etc.
Second demo
 Exploration of data set using KNIME and Pipeline Pilot
 Data set comes from National Cancer Institute (NCI)...
Additional demos
 Pipeline Pilot Web Port sample
Suggestions for getting started
 Download the KNIME software(knime.org)
 Install on your computer
 Look at the sample w...
Types of applications
 Reporting
 Data set comparisons
 ETL
 Data Analysis
References
 Scholarly article on KNIME and Pipeline Pilot
 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3414708/
 www.k...
Who is your speaker?
 Mitch Miller, Ph.D. in Chemistry and 20+ years of IT experience
 Independent consultant: Scientifi...
Upcoming SlideShare
Loading in …5
×

Code camp 2015 visual programming mm

424 views

Published on

Talk delivered at Vermont Code Camp 2015

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Code camp 2015 visual programming mm

  1. 1. Visual Programming Environments for Science and Business MITCH MILLER SCIENTIFIC THINKING CODE CAMP 2015 SEPTEMBER 19, 2015
  2. 2. Disclaimer  This talk represents my opinion and personal experience using 2 fine software systems developed by third parties  The software systems shown are very complex and have hundreds of components. I have only worked with a small number.  Every task shown today can be accomplished in multiple ways. I’m only showing of those ways.
  3. 3. Overview  Introduction: first demo  What is a ‘visual programming environment’  The two systems we’ll look at today  What are these systems capable of?  Second set demos (in-depth)
  4. 4. Demo 1: set-up  Task: produce report of all compounds registered during January
  5. 5. Visual Programming: informal definition  Drag functional components onto canvas to create program  Configure most components by setting parameters  Connect components to route data from one to another  Run and observe data traveling down the lines
  6. 6. Component types  File I/O  Read/write text files  Read/write MS Office documents  XML  JSON  PDF  Database access  Connect  Query  Update
  7. 7. Component types (continued)  Web service consumption  Domain-specific processing  Chemical structure I/O  Chemical structure processing and analysis  Sequence processing  Extensibility  Add your own libraries for more sophisticated processing
  8. 8. Component types (continued)  Visualization  Graphing  Statistical calculations  Scripting  Tip: aim for brief scripts  Data transformation  If/else processing  Filtering  Column selection  And many more…
  9. 9. KNIME  Originally a production of the University of Konstanz, Germany 2004  Currently produced by KNIME.com AG, a company in Zurich, Switzerland  KNIME stands for KoNstanz Information MinEr  Pronounced “Nighm”  A general purpose data analytics platform  Free version available for download  For-sale version available with added extensions
  10. 10. KNIME (continued)  Java based  Written in Java  Scripted, extensible in Java  URL: https://www.knime.org/
  11. 11. Pipeline Pilot  Developed and sold by BIOVIA, San Diego, CA  Originally developed by Scitegic, San Diego in 1999  Designed for scientists to “rapidly create, test and publish scientific services that automate the process of accessing, analyzing and reporting scientific data” (http://accelrys.com/products/collaborative-science/biovia- pipeline-pilot/)  Client-server system  Commercial product  Extensible using .NET and Java  Scripted using an original language, ‘PilotScript’
  12. 12. KNIME Terminology  Components are called “Nodes”  Programs are “Workflows”  Reusable sets of Nodes are “Metanodes”  Groups of related Nodes are “Extensions”
  13. 13. Pipeline Pilot Terminology  Components are called “Components”  Programs are “Protocols”  Reusable sets of Components are “Subprotocols”  Groups of related Components are “Packages”  Different protocols can be combined  One protocol provides initial UI –including a Web form  A second protocol handles form data processing (‘work protocol’)
  14. 14. Different systems shown today serve different populations  KNIME can be used ad hoc on the desktop of a power user. It is also used by companies in a variety of industries  Pipeline Pilot is geared towards scientists and is part of an enterprise system and requires a server installation
  15. 15. Programs can be deployed outside the development client  Give users a URL to access your program  Users of BIOVIA Electronic Lab Notebook and other software can access Pipeline Pilot protocols outside the Pipeline Pilot UI  Users access a Web application that shows them the data they’re looking for in a purpose-built user interface  The application does not look like the system with which it was built  For-sale version of KNIME Server provides similar functionality
  16. 16. Server Features  User access configuration  Shared data sources  Automatic jobs  Etc.
  17. 17. Second demo  Exploration of data set using KNIME and Pipeline Pilot  Data set comes from National Cancer Institute (NCI)’s Developmental Therapeutics Program (DTP)  Results of laboratory tests for activity against 60 types of human cancer cell lines  Data freely available: https://dtp.cancer.gov/discovery_development/nci-60/default.htm
  18. 18. Additional demos  Pipeline Pilot Web Port sample
  19. 19. Suggestions for getting started  Download the KNIME software(knime.org)  Install on your computer  Look at the sample workflows  Start simple; build up
  20. 20. Types of applications  Reporting  Data set comparisons  ETL  Data Analysis
  21. 21. References  Scholarly article on KNIME and Pipeline Pilot  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3414708/  www.knime.org  https://www.youtube.com/user/KNIMETV  http://accelrys.com/products/collaborative-science/biovia- pipeline-pilot/  https://dtp.cancer.gov/
  22. 22. Who is your speaker?  Mitch Miller, Ph.D. in Chemistry and 20+ years of IT experience  Independent consultant: Scientific Thinking, LLC  mitch.miller@thinkscience.us  Some recent projects  Ongoing custodian of one chemical database implementation for ChemIDplus project within the National Library of Medicine  Upgraded 10-year-old Java Servlet lab workflow application to latest version of JDK, Internet Explorer 11 and implemented enhancements  Windows service to handle communication between 2 legacy applications  Import wizard for chemical array designer  Merged a set of chemical databases and harmonized data

×