Take network/pathway-like images related to ES OR (i)PSCs (MeSH tagged) from the database at NCBI and create dynamic networks from them.

1. Project Name
Image to Network
September 2014 – December 2014
Akrita Agarwal

2. Problem Statement
• Take network/pathway-like images related to
ES OR (i)PSCs (MeSH tagged) from the
database at NCBI and create dynamic
networks from them.
• Identify the text, nodes and arrows in the
system to depict the flow of the network.

3. Proposed Plan-of-Action
• Read images into the C++ code and detect text
areas to identify nodes.
• Isolate the text area and process the text using
Tesseract – a Google API
– Train tesseract for current data for better
accuracy.
– Test tesseract with current data
– Seek semantic help using meta-maps to improve
text recognition

4. Proposed Plan-of-Action
• Process the nodes and store them in a graph.
• Detect arrowheads – arrows ( -> ) and
inhibitors ( -|)
• Introduce Links in graph by detecting the
orientation (direction) of arrowheads.
• Create GPML files
• Process GPML files (maybe using Cytoscape)

5. Detect text areas to identify nodes
• Used OpenCV in C++ to process images.
• Found image contours to detect text areas in
the image. Here the focus is just on detecting
text area, not identifying words.
• Once the contours are detected, we draw
rectangles around them.
• Images in the following slides.

6. Test images

12. Isolate text area
Process text using Tesseract
• Tesseract takes a text area as input.
• Generates output in the form of a text file
containing identified text.
• Created a graph with each text area as a node.
• The left top and right bottom coordinates of
each node (rectangle) are its properties.
• Stored graph in a csv file.
• Images in the following slides.

13. Test Images

16. Detect arrowheads – arrows ( -> ) and
inhibitors ( -|)
Approach 1 – Polynomial
• Arrows can be treated as a polynomial with 6
or 7 edges -
• The irregularities in the arrows led to a bad
approximation on arrow-detection.
• Smoothening filters improved the results.
• Low accuracy

17. Approach 2 – Cascade Filters
• Cascade filters use brute force object training to
find objects in images.
• In our approach I used 600 positive images
(arrows) and 1000 negative images (text, images,
diagrams, other non-arrow images) to train the
classifier.
• The results are highly over-fitted and require
further parameter tuning for better results.
• Needs more work.
• Good approach but parameters vary with images,
thus not very reliable.
• Image in the following slide.

19. Current issues
• All algorithms give good results for all the
images. Might work well on some, not very
well on others.
• Text detection can be improved by training
Tesseract more.
• Arrow detection is still not giving good results.

20. Resources
• The code is available on Github - here
• Images can be found on NCBI - here
• Google Tesseract OCR - here
• Cascade classifiers can be tricky to work with.
A good tutorial is - here
• Node-OpenCV - here