Presented by Steve Clement, senior Java developer, at Liferay Symposium 2013. Water For People, an international nonprofit organization dedicated to supporting improved access to water and sanitation systems and services in developing countries, was looking for a way to track and hone the progress it is making toward solving the water and sanitation crisis via its programs. EffectiveUI was enlisted to create this new platform – called Re-Imagine Reporting – and selected Liferay as the backbone. This session, led by Steve Clement, Sr. Java Developer at EffectiveUI, will demonstrate how his team innovated on Liferay to build a new and transparent way for Water For People to manage data, improve outcomes, and prove its efficacy to stakeholders. Topics will include data import via Documents and Media all the way through to the final visualization using AJAX IPC calls and structured content, focusing on how they leveraged Liferay to roll up the data from the lowest geographical level up to a global view.

1. Liferay and Water For People
From Data to Information
Steve Clement
Sr. Java Developer at EffectiveUI, Inc.
#WaterForPeopleLR

2. Liferay Symposium 2013
Agenda
•
•
•
•
•
•
Introduction
About EffectiveUI
Water For People: Re-Imagine Reporting
What we did: DEMO
How we did it
Q&A
#WaterForPeopleLR

3. Liferay Symposium 2013
 World’s most mature and award winning user-focused
technology firm
 Silver Partner with Liferay
 Founded April, 2005, ~100 full-time employees
 Denver (headquarters)
 Rochester, NY
 Manhattan
 Seattle
 San Francisco
#WaterForPeopleLR

4. Liferay Symposium 2013
The Client
Water For People brings together local entrepreneurs,
civil society, governments, and communities to establish
creative, collaborative solutions that allow people to
build and maintain their own reliable safe water systems.
#WaterForPeopleLR

5. Liferay Symposium 2013
The Challenge
Provide a transparent reporting platform to show not only
the financial and water/sanitation service data, but also the
narrative story for the locales served by Water For People
#WaterForPeopleLR

6. Liferay Symposium 2013
Why Liferay?
• Industry leader
• Support for a mix of content publishing and custom
application
• Wealth of out-of-box tools
• Open source
• Shared vision of making a better world
#WaterForPeopleLR

7. Liferay Symposium 2013
Data begins at the source
Monitoring:
Data collected into
a system known as
FLOW
#WaterForPeopleLR

8. Liferay Symposium 2013
Data goes from FLOW to Excel
#WaterForPeopleLR

9. Liferay Symposium 2013
Additional Data
•
•
•
•
Financial
Community surveys
Readiness Surveys
Narrative content
…more spreadsheets
#WaterForPeopleLR

10. Liferay Symposium 2013
But what does it all mean?
#WaterForPeopleLR

11. Liferay Symposium 2013
Requirements
• Location Hierarchy: roll up the data
• Get spreadsheet data into application
• Visualize data meaningfully
Start with the ending…
#WaterForPeopleLR

12. Liferay Symposium 2013
DEMO
Water For People: Re-Imagine Reporting
#WaterForPeopleLR

13. Liferay Symposium 2013
End Demo
Let’s talk about data…
#WaterForPeopleLR

14. Liferay Symposium 2013
Location Hierarchy
• Four levels: district, country, region, global
• Data is tied to district level
• Data needs to roll up to higher levels
#WaterForPeopleLR

15. Liferay Symposium 2013
Warning!
It’s about to get technical…
#WaterForPeopleLR

16. Liferay Symposium 2013
Location Hierarchy: Data Model
District
Region
region_id PK
Name
latitude
longitude
…
etc.
Country
country_id PK
region_id FK
country_name
latitude
longitude
…
etc.
district_id PK
country_id FK
district_name
latitude
longitude
…
etc.
DistrictScore
Possible solution: latitude and longitude can
help plot data points on map.
#WaterForPeopleLR
district_score_id PK
district_id FK
num_water_points
year
…
etc.

17. Liferay Symposium 2013
But wait…
Navigation = Location Hierarchy!
Control Panel’s Site Pages:
Child page navigation perfectly matches our
location hierarchy!
#WaterForPeopleLR

18. Liferay Symposium 2013
Navigation as Data
Model Liferay’s Layout table to provide us with our
We can use
foreign key. The relationship of child to parent is already
provided.
Layout
DistrictScore
plid PK
parentLayoutId
name
friendlyURL
…
etc.
district_id PK
district_plid FK
num_water_points
year
…
etc.
#WaterForPeopleLR

19. Liferay Symposium 2013
Service Builder
We use
districtLayoutPlid as a
foreign key
Finder methods include all
districts for a year
#WaterForPeopleLR

20. Liferay Symposium 2013
From here to there: how do we do this?
Database
• Register a dedicated data folder in Documents & Media
• Let the admins associate the spreadsheets with
parsers for that data type
• Register parsers with parser data types
• Extract data from spreadsheet and insert into Db using
classes generated by Service Builder
#WaterForPeopleLR

21. Liferay Symposium 2013
Register Data Folder ID
Get Folder ID
1. URL includes …&_20_folderId=10531
2. Or SQL:
SELECT folderId FROM lportal.dlfolder where name = 'Data';
Then, in portal-ext.properties:
data.folder.id=10531
#WaterForPeopleLR

22. Liferay Symposium 2013
Get Folder Id Where Spreadsheets Live
Spring MVC Controller
Add FileEntry objects and data type codes to model
#WaterForPeopleLR

23. Liferay Symposium 2013
Display dataTypes and FileEntries
Create drop down of parser data types
Iterate to display in table
#WaterForPeopleLR

24. Liferay Symposium 2013
Admins select a file and associates with a parser type
#WaterForPeopleLR

25. Liferay Symposium 2013
Spring for map of parsers: dependency injection
In Spring Controller, inject parser map…
#WaterForPeopleLR

26. Liferay Symposium 2013
Extract Data From Spreadsheet
Apache POI
Java API for Microsoft Documents
Get the POI Workbook from Liferay’s FileEntry object
#WaterForPeopleLR

27. Liferay Symposium 2013
Pull out data of
spreadsheet
•
•
Column index from
cell
Appropriate data type
from cell
#WaterForPeopleLR

28. Liferay Symposium 2013
Service Builder has created our model and utility
classes to interact with the database
We look up the DistrictScore by unique
compound key. If we don’t have it, we
create it. Otherwise, we update the
values.
#WaterForPeopleLR

29. Liferay Symposium 2013
Show the data rolled up in the pin
What are we really asking for?
Answer:
Two scores keyed to a
location name and
rendered in a specific
location on the map
#WaterForPeopleLR

30. Liferay Symposium 2013
Multiple Web Content
portlets need to get the
data for the map pins
#WaterForPeopleLR

31. Liferay Symposium 2013
From Database to Pin
• Structured Content / Template
• System Portlet for Ajax call
• Render data using Highcharts
#WaterForPeopleLR

32. Liferay Symposium 2013
Structured Content
Central America
(174, 103)
Need to put in X and Y
coordinates of map point
Possible default values if data
not in system but still known
#WaterForPeopleLR

33. Liferay Symposium 2013
Template for structured content
Call JavaScript function
with structured content
data
#WaterForPeopleLR

34. Liferay Symposium 2013
JavaScript calls
for “efScores”
resource and
expects JSON
response called
“scoreMap”
#WaterForPeopleLR

35. Liferay Symposium 2013
Create a system portlet for serving resources called from
theme
liferay-portlet.xml
#WaterForPeopleLR

36. Liferay Symposium 2013
In the call for the “efScores” resource, we get the current
layout (i.e., the page we are on) and pass it and the year
parameter into a service class
Spring MVC
Controller used to
serve JSON
resource
Current page
“scoreMap” returned
in JSON view
#WaterForPeopleLR

37. Liferay Symposium 2013
So how do we get the rollup?
Liferay’s Layout API!
#WaterForPeopleLR

38. Liferay Symposium 2013
We can start with the current layout plid and walk down to all
the district-level plids using the API:
Pseudo code:
IF
layout.hasChildren()
THEN
List<Layout> children = layout.getChildren()
//Go through all children until we get to lowest level
ELSE
Add layout.getPlid() to collection of district-level plids
#WaterForPeopleLR

39. Liferay Symposium 2013
Service Builder Finder Methods
#WaterForPeopleLR

40. Liferay Symposium 2013
Average the District Scores and
group by the Layout names that are
one level beneath the current layout
“Guatemala”
• Service: 65
• Sustainability: 28
#WaterForPeopleLR
Example:
Central America
• Guatemala
• Honduras
• Nicaragua

41. Liferay Symposium 2013
Get the scores from scoreMap[location name]
If we don’t have a
score defined in
the content
structure (-1), get
the score from the
map.
Send to function for creating pins
#WaterForPeopleLR

42. Liferay Symposium 2013
Create HTML
element containers
for the pins and then
call Highcharts API
#WaterForPeopleLR

43. Liferay Symposium 2013
Rich charting and graphing tool
with JavaScript API
#WaterForPeopleLR

44. Liferay Symposium 2013
Successes
• Rapid development
• Complex data rendered into easily understood visual
information
• Out-of-box tools for CMS and DMS helped guide
solution
• Liferay Theming gave consistent look and feel
• Customizing existing portlets gave us exactly what we
needed
• Portlets as page composition: breaking up
development tasks into manageable units
#WaterForPeopleLR

45. Liferay Symposium 2013
Looking forward
• Show trends
• Establish as a platform for other non-profits
• Continue innovation based on Water for
People’s rich data and Liferay’s capabilities
• Improve lives
#WaterForPeopleLR

46. Liferay Symposium 2013
Thank you!
Questions: #WaterForPeopleLR
Resource
URL
EffectiveUI
www.effectiveui.com
Water For People
www.waterforpeople.org
Water For People: Re-imagine
Reporting
reporting.waterforpeople.org
Spring MVC and Liferay
www.effectiveui.com/blog/2013/09/17/l
iferay-and-spring-mvc
Apache POI - the Java API for
Microsoft Documents
poi.apache.org
Highcharts JS
www.highcharts.com/
#WaterForPeopleLR