Unlocking livestock development potential through science, influence and capacity
development ILRI APM, Addis Ababa, 15-17 May 2013
Efficient and effective data management for ILRI research
projects: A holistic approach
This document is licensed for use under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported Licence May 2013
Karanja, Titus; Quiros, Carlos
Introduction
Data are expensive to collect, manage and store. It is worthless putting a lot of effort into making sure that the methods used for obtaining the
data are of high scientific relevance if the same stringent quality standards are not maintained throughout the project and CRP life-cycle. Thus,
the whole data management process, including archiving and making it accessible, needs to be planned at the start of the research activity.
The Present…
• ILRI works in many regions and many CRPs
• ILRI’s data management in research activities has been rather ad-hoc i.e. it happens ‘on the go’
• Data are decentralized across the institute, sat in separate databases in different locations (Database server, HPC, own computers)
• No standard data management standards/protocols across research activities
The “Very Near” Future…
• Centralized data for CRP 3.7 and ‘key indicators’, systematic approach to the location of other databases i.e. not on own computers!
• Standard Data Management protocols/standards e.g. software, etc.
• Accessibility of data across the institute/regions/partners e.g. via online data portals
• Enhanced capacity in new and emerging data management technologies
• Getting data out in the public domain e.g. open access
Current Data Collection
Entry Platforms
• Direct data collection and
entry via Android(ODK),
CSPro(Netbooks & PDAs),
SurveyBe(Netbooks)
• Paper questionnaires + PC
data entry
• SMS-data collection (for
M&E)
• Web-based data entry (CGP-
Tanzania)
META
Template
Indicator information
(Name, data type, lookup source,
input type, etc.)
Data analysis
Sets of queries
Metadata
(About this template, example
of analysis, etc.)
Project
Metadata
(About this project)
Templates
...
CSPro, Access
Excel, Web, ...
MySQL Database
Common storage
Data entry
applications
Data processing
scripts
Generator
Software
Data entry plugins
...
Read
Create
Create
Read data
Store
Use
Stage 1…the start
• Standard data collection
technologies
• Standard indicators
templates
• New and open
technologies
Stage 2…data collection, relay, cleaning
and analysis
• Near or Real-time relay of data to
server
• Real-time data quality assurance
checks
• Automated Data cleaning
• Data Audit trail
Stage 3…storage, access
and reporting
• Many data access
methods
• Sufficient server
storage
• Continued data back-
ups
Proposed Data Management Framework