This document describes ChemData, an open-source tool for interactive analysis and visualization of large chemical datasets. ChemData stores chemical data in a MongoDB database and uses VTK for visualization. It allows users to perform queries, similarity searching, and multidimensional analysis through features like scatter plots, histograms, parallel coordinates, and k-means clustering. ChemData also integrates with other open chemistry tools like Avogadro and represents molecular structures using the ChemicalJSON format. Future directions include integration with additional databases and computational job results.

1. Exploring Large Chemical
Data Sets
Interactive Analysis and Visualization
Kyle Lutz and Marcus D. Hanwell
August 21, 2012
Skolnik Symposium

2. Overview
● An open-source, cross-platform
cheminformatics tool
● A general-purpose tool for chemical data
exploration and analysis
● Interactive, editable and queryable
database of chemical data on the desktop
● Part of the Open Chemistry application
suite (Avogadro and MoleQueue)
● Leverages several open-source projects:
Qt, VTK, Chemkit, Open Babel, MongoDB

3. Architecture
● Native, cross-platform C++ application built with Qt
● Stores chemical data in a NoSQL MongoDB database
● Uses VTK for 2D and 3D data set visualization

4. Main Window

5. Molecule Details

6. Queries
Supports different
queries:
● Name
● Formula
● InChI
● InChIKey
● Structure and
Substructure

7. Similarity Searching

8. Charts and Plots
Scatter Plot Histogram of logP
of Polar Surface Area (TPSA)
against Volume (VABC)

9. Multidimensional Analysis
● Provide tools for viewing and analyzing large
amounts of data with multiple dimensions
○ Scatter Plot Matrix
○ Parallel Coordinates
○ K-Means Clustering
● Interactive charts supporting selection
● Easy to add new chemical descriptors

10. Scatter Plot Matrix
Polar Surface Area vs. logP vs. Mass vs. Rotatable Bonds vs Volume

11. Parallel Coordinates
Polar Surface Area vs. logP vs. Mass vs. Rotatable Bonds vs Volume

12. K-Means Clustering
● ~30 numeric molecular descriptors
● 1D, 2D, and 3D visualization
● Selection and extraction of molecules from clusters

13. Similarity Visualization
● Similarity Clustering
● Calculated from fingerprint similarity or structural
similarity

14. Similarity Visualization
60%
30%
45%

15. ChemicalJSON
Example: ethane.cjson
● JSON (JavaScript Object Notation) is
a "lightweight data-interchange
format"
● Store molecular structure, geometry,
identifiers and descriptors all as a
single JSON object
● Benefits:
○ More compact than XML/CML
○ Native language of MongoDB and
JSON-RPC
○ Easily converted to a binary
representation (BSON)
Specification avaialble at: http://wiki.openchemistry.org/Chemical_JSON

16. ChemicalJSON in MongoDB
● Nearly identical to what is stored in a file
○ A few extra fields stored
■ 2D diagram (as PNG)
■ Heavy atom count (for substructure searching)
■ Binary fingerprints (for similarity searching)
■ InChIKey for indexing and as a unique key
■ Mongo's OID ("_id") field
● Trivial to write out to a .cjson file:
db.molecules.find({"name" : "ethanol"},
{"diagram" : 0,
"heavyAtomCount" : 0,
"fp2_fingerprint" : 0,
"_id" : 0})

17. Open Chemistry with ParaViewWeb
● Uses ParaView's client-server architecture
● Interactive 3D rendering
● Runs in any modern web browser
URL: http://paraviewweb.kitware.com/OpenChemistry/

18. Open Chemistry with ParaViewWeb
ChemData

19. RPC / Avogadro Integration
● Uses JSON-RPC to communicate with other
applications (most notably Avogadro)
● Visualize data directly from the database
● Uses ChemicalJSON to represent molecular
structures and transfer molecular information

20. Future Directions
● Direct integration with 3rd party databases
(PubChem, PDB, ...)
● Broader support for storing and analyzing
computational job results
○ Linked with molecular structures
○ Direct from CML or converted/parsed
● Plugins to facilitate extension
○ Descriptors
○ Visualization
○ Chemical file input/output
● Scaling studies, working with multiple data
servers and terabytes of data

21. Comments/Questions?
Home Page
http://wiki.openchemistry.org/ChemData
Source Code
https://github.com/OpenChemistry/chemdata
ParaViewWeb Demo
http://paraviewweb.kitware.com/OpenChemistry