Kno.e.sis Approach to Impactful Research & Training for Exceptional Careers

KNO.E.SIS APPROACH TO IMPACTFUL RESEARCH
& TRAINING FOR EXCEPTIONAL CAREERS
Put Knoesis Banner
UGA, 10 Dec 2016
Amit Sheth
LexisNexis Ohio Eminent Scholar & Exec. Director,
The Ohio Center of Excellence in Knowledge-enabled Computing (Kno.e.sis)
Wright State, USA

What is Kno.e.sis about: Visions and Themes
●Computing for Human Experience
●Semantic, Cognitive and Perceptual Computing
●Physical-Cyber-Social Big & Smart Data
●BioHealth: esp. personalized digital health/connected
health and medicine

More about Kno.e.sis
We innovate and employ semantic web, machine learning, NLP,
IR, data mining, network science and cloud computing
techniques.
Our highly interdisciplinary research impact health and clinical
applications, biomedical and translational research,
epidemiology, cognitive science, social good, policy,
development, etc.
● ~ 60 funded researchers (BS/MS/PhD students, postdocs/scholars), a
couple of SW engineers, and 3 staff,
● ~15 faculty members.
● Our main cloud infrastructure has 864 CPUs, 17TB main, 18TB SSD
and 435TB disk storage.

How we measure success
Kno.e.sis is highly successful in its primary mission of exceptional
student outcomes: they have exceptional publication and real-
world impact (http://j.mp/kimpact), and our PhDs compete with
their counterparts from top 10 schools for initial jobs in research
universities, top industry research labs and highly competitive
companies, while a few have been successful entrepreneurs.
This is enabled by vision to keep ahead of the competition,
interdisciplinary faculty, world-class computing and physical
infrastructure, exceptional funding, extensive collaborations with
academia and industry.

33%
35%
32%
DoD & Industry
• Metabolomics &
Proteomics
• Medical Info Decisions
• Human Detection
on Synthetic FMV
• Sensor & Information
• Material Genomics
• Cardiology Semantic
Analysis
NIH
• kHealth - Asthma
• eDrug Trends
• Depression on Social Media
• Drug Abuse Early Warning
NSF
• Harassment on Social Media
• Citizen & Physical Sensing
• Twitris - Collective Intelligence
• Aerial Surveillance
• Visual Experience
• Web Robot Traffic
Kno.e.sis’ research in World Wide Web ranks Wright State University among the top 10 organizations in the
world based on 10-yr impact. Its total budget for currently active projects is $13+ million, with $6+ million for new
projects starting after July 2015. The significant majority of funds are highly competitive federal grants. World-class
research is complemented by exceptional student outcomes and commercialization with local economic impact.
As an Ohio COE on Bio Health
Innovation, Kno.e.sis conducts
research leading to building
intelligent systems for clinical,
biomedical, policy, and
epidemiological applications.
Example clinical/healthcare
applications include major diseases
such as asthma, depression,
cardiology, dementia and GI.
This is complemented by social and
development challenges such as
marijuana legalization policy,
harassment on social media, gender-
based violence, and disaster
coordination.
60+ Funded
Students
• 40 PhD
• 16 MS
• 5 BS
Kno.e.sis: Ohio Center of Excellence in Knowledge-enabled Computing

Tanvi
Banerjee
Robert
Carlson
Katherine
Cauley
Keke
Chen
Raminta Daniulaityte
Guozhu
Dong
Derek
Doran
John Flach
Michael
Raymer
T.K. Prasad
Nick
Reo
Valerie
Shalin
Amit
Sheth
Shaojun
Wang
Thomas
Wischgoll
Funded students: > 60
• 30+ PhD
• 25+ MS
• growing number of BS
*
*
Kno.e.sis Faculty: practically all have active funds, multidisciplinary collaborations

Success defined by outcomes and impact
A key reason for Kno.e.sis' success is its unique work culture
involving teamwork to solve complex problems. Practically all our
work involves real world challenges, real-world data, interdisciplinary
collaborators, path-breaking research to solve the real-world
challenges, real-world deployments, real world use, and measurable
real world impact. These are demonstrable in diverse forms such as
policy impact (e.g., FDA advisory), operational real-time use of our
tools to save lives during disasters, open source data and tools, and
licensing/tech transfer and commercialization of our technology
used to launch a local startup.

J. McCarthy M. Weiser D. Engelbart J. C. R. Licklider
Computing for Human Experience:
Toward a symbiotic partnership between machines and people
htttp://j.mp/k-che
http://wiki.knoesis.org/index.php/Computing_For_Human_Experience

In conscious activities like reading, the human
brain can process approximately 50 bits of
information per second. Contrast that with the
astounding bandwidth of your senses, which
process around 11 million bits of information every
second.
… and do it efficiently and at scale
What if we could automate this interpretation of data?

SSN
Ontology
2 Interpreted data
(deductive)
[in OWL]
e.g., threshold
1 Annotated Data
[in RDF]
e.g., label
0 Raw Data
[in TEXT]
e.g., number
3 Interpreted data
(abductive)
[in OWL]
e.g., diagnosis
Intelligo
“150”
Systolic blood pressure of 150 mmHg
Elevated
Blood
Pressure
Hyperthyroidism
……
Levels of Abstraction

● Social Media Big Data – Twitris, eDrugTrends
● Sensor/IoT Big Data – CityPulse, kHealth
● Healthcare Big Data – kHealth, EMR, Prediction
● Biomedical Big Data – Biomarker from NextGen
Sequencing and Proteomics, SCOONER
● Big and Smart Data Certificate
Kno.e.sis private cloud: 864 CPU cores, 18TB RAM, 17TB SSD, 435TB disk
Let’s talk Big Data @ Kno.e.sis

kHealth - Asthma
Principal Investigators: Amit P. Sheth
Co-Investigators: Krishnaprasad Thirunarayan , Maninder Kalra
Other Faculty: Tanvi Banerjee
Students: Utkarshini Jaimini, ….
Ohio Center of Excellence in Knowledge-Enabled Computing
Grant Number: 1 R01 HD087132-01
Project Title: KHealth: Semantic Multisensory Mobile Approach
to Personalized Asthma Care
Timeline: 07/01/2016 – 06/30/2019
Award Amount: $938,725

Sensordrone
(Carbon monoxide,
temperature, humidity)
Node Sensor
(exhaled Nitric Oxide)
Sensors
Android Device
(w/ kHealth App)
Total cost: ~ $500
Along with two sensors in the kit, the application uses a variety of population
level signals from the web:
Pollen level Air Quality Temperature & Humidity
kHealth Kit for the application for asthma management

What can we do to avoid asthma episode?
15
Real-time health signals from personal level (e.g., Wheezometer, NO in breath, accelerometer,
microphone), public health (e.g., CDC, Hospital EMR), and population level (e.g., pollen level,
CO2) arriving continuously in fine grained samples potentially with missing information and
uneven sampling frequencies.
Variety Volume
VeracityVelocity
Value
What risk factors influence asthma control?
What is the contribution of each risk factor?
How is my Asthma control?
Should I take additional medication today?
How can I reduce my asthma attacks at home?
semantics
Understanding relationships between
health signals and asthma attacks
for providing actionable information
WHY Big Data to Smart Data: Asthma example

Personal level
Signals
Public level
Signals
Population level
Signals
Domain
Knowledge
Risk Model
Events from
Social Streams
Take medication before
going to work.
Avoid going out in the
evening due to high pollen
levels.
Contact doctor.
Analysis
Personalized Actionable
Information
Data Acquisition &
aggregation
kHealth: Health Signal Processing Architecture

Population Level
Personal
Wheeze – Yes
Do you have tightness of chest? –Yes
ObservationsPhysical-Cyber-Social System Health Signal Extraction Health Signal Understanding
<Wheezing=Yes, time, location>
<ChectTightness=Yes, time, location>
<PollenLevel=Medium, time, location>
<Pollution=Yes, time, location>
<Activity=High, time, location>
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
RiskCategory
<PollenLevel, ChectTightness, Pollution,
Activity, Wheezing, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
.
.
.
Expert
Knowledge
Background
Knowledge
tweet reporting pollution level
and asthma attacks
Acceleration readings from
on-phone sensors
Sensor and personal
observations
Signals from personal, personal
spaces, and community spaces
Risk Category assigned by
doctors
Qualify
Quantify
Enrich
Outdoor pollen and pollution
Public Health
Well Controlled - continue
Not Well Controlled – contact nurse
Poor Controlled – contact doctor
Patient Health Score (Diagnostic): Details

Context-Aware Harassment
Detection on Social Media
Principal Investigators: Prof. Amit P. Sheth
Co-Investigators: Valerie Shalin, Krishnaprasad Thirunarayan
Other Faculty: Debra Steele-Johnson, Dr. Jack L. Dustin
PhD Students: Lu Chen, Wenbo Wang, Monireh Ebrahimi, Kathleen Renee Wylds
MS Students: Pranav Karan, Rajeshwari Kandakatla
Collaboration with Beavercreek High School
 NSF Award#: CNS 1513721
 TWC SBE: Medium: Context-Aware Harassment Detection on Social Media
 Timeline: 01 Sep. 2015 - 31 Aug. 2018
 Award Amount: $925,104 + $16,000 (REU)

Social and Physical Sensing Enabled Decision Support for
Disaster Management and Response
Principal Investigators: Prof. Amit P. Sheth, Prof. Srinivasan Parthasarathy (OSU)
Co-Principal Investigators: Densheng Liu (OSU), Ethan Kubatko (OSU), Valerie Shalin,
Krishnaprasad Thirunarayan
PhD Students: Sarasi Lalithsena, Pavan Kapanipathi, Hussein Olimat
MS Student: Siva Kumar
Postdoctoral Researcher: Tanvi Banerjee
 NSF Award#: EAR 1520870
 Hazards SEES: Social and Physical Sensing Enabled Decision Support for
Disaster Management and Response
 Timeline: 01 Jul. 2015 - 31 Jul. 2019
 Award Amount: $1,975,000 (WSU: $787,500)

kHealth - Dementia
Principal Investigators: Tanvi Banerjee
Mentors: Amit Sheth, Larry Lawhorne
Students: ….
Grant Number: 1K01LM012439-01
Project Title: Managing Dementia through Multisensory Smart
Phone Approach to Support Aging in Place
Timeline: 09/01/2016 – 08/30/2019
Award Amount: $509,909

eDrug Trends
Principal Investigators: Prof. Amit P. Sheth, Prof. Raminta Daniulaityte
Co-Investigators: Robert Carlson, Krishnaprasad Thirunarayan, Ramzi Nahhas,
Silvia Martins (Columbia), Edward W. Boyer (U. Mass.)
PhD Students: Farahnaz Golroo, Sanjaya Wijeratne, Lu Chen, Adarsh Alex
MS Student: Adarsh Alex
Postdoctoral Researcher: Francois Lamy
Software Engineer: Gary Smith
 NIH Award#: 5 R01 DA039454-02
 Trending: Social media analysis to monitor cannabis and synthetic
cannabinoid use
 Timeline: 15 Sep. 2014 - 14 Sep. 2018
 Award Amount: $1,689,019 + $162,505

Modeling Social Behavior for
Healthcare Utilization in Depression
Principal Investigators: Prof. Amit P. Sheth, Prof. Jyotishman Pathak (Cornell)
Co-Investigators: Krishnaprasad Thirunarayan, Tanvi Banerjee, William V. Bobo (Mayo Clinic),
Nilay D Shah (Mayo Clinic), Lila J Rutten (Mayo Clinic), Jennifer B McCormick (Mayo Clinic),
Gyorgy Simon (Mayo Clinic)
Other Faculty: Debra Steele-Johnson, Jack Dustin
PhD Students: Ashutosh Jadhav, Amir Hossein Yazdavar, Hussein Al-Olimat
Master Student: Surendra Marupudi
Visiting Scholar: SoonJye Kho
 NIH Award#: 1 R01 MH105384-01A1
 Modeling Social Behavior for Healthcare Utilization in Depression
 Timeline: 1 Jul. 2015 - 30 Jun. 2019
 Award Amount: $1,934,525 (WSU: $505,600)

PFI: AIR-TT: Market Driven Innovations and Scaling up of
Twitris - A System for
Collective Social Intelligence
PI: Amit Sheth
Kno.e.sis - Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University

Overview: Scale Up and Commercialization Progress
During PFI-AIR Project
● Trial license signed during first half of NSF-PFI-Air project leading to
launching of Cognovi Labs in February 2016 near the university,
exclusive license negotiation in final stages.
● Activities of Cognovi Labs
● Initial team completed.
● Ongoing: business model development,
market segment validation,
more customer trials and
seeking additional investment.
Prof. Amit Sheth and his students.
Inventor
Erica Waite James Mainard Flavio Lobato
CFO CEO
Chairman
Rodolfo Bellesi John D’Orazio Dr. Robert
Lee
Advisor Advisor Advisor
Cognovi Labs is led by an experienced
launch team using proven processes for
technology commercialization and
corporate growth.

Twitris Technology: Real-time, Actionable Insights from
Social-media
Spatio-Temporal-Thematic
● Provides thematic context through analysis of place and time.
People-Content-Network
● Analyzes influential users and identifies who is being listened to.
Sentiment-Emotion-Intent
● Extracts and assigns structured sentiment and emotion scoring from
unstructured content to understand motivation, feelings, opinion and
intent.Key Differentiators:
● Comprehensive (above)
● Semantic Processing: use of public and proprietary knowledge.
● Real-time processing: used in live blogging of election debate;
coordination during disasters.
● Scalable: deployed on a large cloud (864 CPUs, 17 TB main, 435 TB disk).

Snapshot of Some Real-world Applications/Trials
Domains: Branding, Disaster Coordination, Social Movements, Election, Development, Epidemiology,..

Sample Media Coverage of Twitris Technology/Startup
● Web App Analyzes Tweets in Real Time for a Record of Historic Events [Twitris'
monitoring/analysis of major events incl. Occupy Wall Street, Egypt revolution 2011, Haiti Earthquake
2010, Iran Elections 2009.]
● Using Crisis Mapping to Aid Uttarakhand, The Hindu, 2013. [Data we collected was used by
Google Crisis Maps that helped crisis response planners]
● Digital soldiers emerge heroes in Kashmir flood rescue, Hindustan Times, 2014 [Discusses
rescue coordination effort using the Twitris technology by digital volunteers].
● The Twitris Sentiment Analysis Tool by Cognovi Labs Predicted the Brexi hours Earlier Than
Polls, TechCrunch, June 29, 2016.
● Cognovi Labs will Study the Presidential Debates Tonight with Their Emotional Sentiment
Tool, TechCrunch, Sept 26, 2016.
Details at: http://knoesis.org/amit/media. Other related coverage appears in:

Example TechCrunch Coverage of Twitris & Cognovi Labs
TechCrunch is the most viewed technology web site.

● NMR-Based Urinary Metabolomics in Rats Exposed to Burn Pit Emissions and Respirable Sand,
$241K, Reo, Raymer
● PFI: AIR-TT: Market-driven Innovations and Scaling up of Twitris - A System for Collective Social
Intelligence; 200K, Sheth, Mackay
● CRII: CSR: Towards Understanding and Mitigating the Impact of Web Robot Traffic on Web
Systems; 174K, Doran
● Medical Information Decision Assistance and Support; 25K, Prasad, Sheth
● Choose Ohio First: Growing the STEMM Pipeline in the Dayton Region FY2016/FY2017; Raymer
● Westwood Partnership to Prevent Juvenile Repeat Violent Offenders; $200K, Sheth, Doran,
Dustin
● Semantic Web-based Data Exchange and Interoperability for OEM-Supplier Collaboration; 89K,
Prasad, Sheth
● NIDA National Early Warning System Network (iN3): An Innovative Approach; 299K, Carlson,
Sheth, Boyer, Daniulaityte, Nahas
● CUTE: Instructional Laboratories for Cloud Computing Education; 200K, Chen, Wang, Mateti
● SemMat: Federated Semantic Services Platform for Materials Science and Engineering; 315K,
Sheth, Prasad, Srinivasan
● Materials Database Knowledge Discovery and Data Mining; 190K, Sheth, Prasad, Srinivasan
* Grants with Kno.e.sis faculty as
coPI or investigator not included
Additional Funded Projects (when Kno.e.sis faculty is a PI/jointPI*)

● Exceptional first jobs: George Mason U., CWRU, IBM Research, LinkedIn,
Twitter, Amazon; 10 of Raymer and Sheth’s advisee are/were faculty
● Exceptional starting salaries: e.g., $150-250K for a PhD; six figures for M.S.
● Exceptional internships: IBM Research Almaden/Watson, Samsung
Research, Bosch R&D, GE Research, Google, Mayo, NLM, QCRI, etc.
● Major Awards: 2015-2016 George Thomas Post-Graduate Fellowship, Eric &
Wendy Schmidt Data Science for Social Good Fellowship, USAID and
ICT4Peace Fellowship, ITU Telecom World Young Innovators, etc.
● Exceptional publication impact: 11 out of Sheth’s 20 PhDs have 1,000+
citations each (3 over 5,000); Average citation for Sheth’s first 18 PhDs:
1,700+
● Exceptional professional services: Invited talks; PC membership of 10-20
conferences, usually involving top conferences in their fields
● 50% of the Sheth’s PhDs have filed at least 2 patents; 1 former PhD has filed
30+ patents.
Student Success: Our most important performance measure

Showcase: http://knoesis.org/hemant

Showcase: http://knoesis.org/researchers/cory

From NSF Program Manager of recently completed project that lead to outcomes
(# and quality of PhDs/MS/UG; publications, participation/use in real world
disaster events, extensive national and international media coverage,....).
● “ Thank you for your wonderfully informative final report for your collaborative
project.
● I am glad to see your research results in developing an effective Twitris
platform for use by PIOs that utilizes social media for coordination of efforts for
an emergency response.
● I appreciate your efforts in broadening the impacts of your project via
collaboration with emergency personnel and including them in the final testing
of Twitris. It is commendable that your tool has been successfully used in past
year’s major disaster events.
● I am pleased that your work has been reported in publications that will
hopefully lead to an even broader adoption of your techniques and tools.
Accomplishments and accolades (a sample)

● Among top universities in the world in World Wide Web (cf: 10-yr impact, Microsoft
Academic Search: among top 10 – 2013 data)
● Among the largest academic groups in the US in Semantic Web + Social/Sensor
Webs, Mobile/Cloud/Cognitive Computing, Big Data, IoT, Health/Clinical &
Biomedicine Applications
● Exceptional student success: internships and jobs at top salary (IBM
Watson/Research, MSR, Amazon, CISCO, Oracle, Yahoo!, Samsung, research
universities, NLM, startups )
● 100 researchers including 15 World Class faculty (> 3K citations/faculty avg) and >
40 funded PhD students
● Extensive research for largely multidisciplinary projects; world class resources;
industry sponsorships/collaborations (Google, IBM, …)

We are World Class….(last available data, but not updated recently by MAS)

Created Twitris, a Commercial Grade software, which had significant NSF
and AFRL research funds, followed by NSF I-CORP and NSF-PFI-AIR
funding; currently VC/entrepreneurs are evaluating for potential
licensing/start up
One recent patent awarded, two filed
Many more patents filed by companies where Kno.e.sis students intern
ezDI has funded Sponsored Research for the fifth year in the row and now
has major successful products
A local entrepreneur has just signed Sponsored Research that is essentially
incubating his company in Kno.e.sis
On going SBIR/STTR funding and participating; faculty (Raymer) led
company,....
Economic & IP Development

Extensive Major Media Coverage

Kno.e.sis is the shining example of
world-class research and
student outcomes.
Information…Meaning….Actions….Impact
http://knoesis.org
http://facebook.com/Kno.e.sis

Kno.e.sis Approach to Impactful Research & Training for Exceptional Careers

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Kno.e.sis Approach to Impactful Research & Training for Exceptional Careers

Similar to Kno.e.sis Approach to Impactful Research & Training for Exceptional Careers (20)

Recently uploaded

Recently uploaded (20)

Kno.e.sis Approach to Impactful Research & Training for Exceptional Careers