Wajam is a social search and advertising company that processes large amounts of data daily. The document discusses implementing an agile project management process for big data projects at Wajam. It proposes tracking tasks with custom markers that include a name and priority level. Tasks would move through backlog, development, testing, evaluation, and deployment stages with adaptations made to storage and compatibility at each stage. This process is intended to provide continuous tracking and improvement of big data projects.

1. Great Minds Search Alike™By Jean-Louis Quéguiner
Senior Big data Developer at Wajam
Agile Project Management
in a Big Data context

2. Great Minds Search Alike™
Wajam is a leader in social search and social
advertising technology.
Wajam gives you recommendations from
friends on Google, Bing, eBay, Amazon,
Wikipedia, TripAdvisor and more.
What is Wajam?

3. © Wajam 2015
1. Context
2. Architecture
3. Proposed process
CONTENT

4. © Wajam 2015
● 350 GB of logs every day
● 7-10 Millions active users
● 20 Millions of ads every day
● More than 200 Specific events
● Around 1 to 2 events deployed every week
CONTEXT

5. © Wajam 2015
ARCHITECTURE

6. © Wajam 2015
PROPOSED PROCESS
Backlog Sync Dev Test Eval
Size
Adapt
storage
Backward
Compatibilty
Merge &
Reload
Done
Sync :
1- Define with BI their needed output
2- Define with Dev our needed input
Eval Size:
1- Will it fit in queue?
2- Will it fit in HDFS
Adapt Storage :
1- file structure
2- adapt MySQL columns / index/ tables
Backward Compatibilty:
If output structure changed &
If output(t) is used as input for t+1
then adaptation is needed

7. © Wajam 2015
PROPOSED MARKER
NAME +
PRIORITY
TASK NAME +
DESCRIPTION
ADD MOUSE ON
HOVER AD EVENT
ALEX 1

8. © Wajam 2015
PROPOSED PROCESS
Backlog Sync Dev Test Eval
Size
Adapt
storage
Backward
Compatibilty
Merge &
Reload
Done
TODO 1
TODO 2
TODO 3
A 1 A 2
A 3
J 1J 2
J 3

9. © Wajam 2015
CONTINUOUS IMPROVEMENT & CONCLUSIONS
● Re-evaluate priorities every morning
● Re-communicate priorities every morning
● Custom Steps
● Parallel task tracking
● Help focus
● Easy to track
● More detailed and flexible than classic scrum approach

10. THANK YOU
© Wajam 2015