The document discusses designing data preparation tools that can support users with different technical proficiencies, from non-technical users to expert users. It proposes using both visual "transform cards" and a script IDE mode to bridge the needs of different users. The tool would use progressive disclosure of scripting capabilities to ease non-technical users into more technical functions. A demo of the tool discussed implementing transform cards and ways to improve predictive data transformations through feedback.

1. Alon Bartur Joseph McKenneyJingshu Chen
Principal Product Manager User Experience Designer Senior Software Engineer
Hadoop Summit
June 9th, 2015
From Beginner to Expert:
Data Wrangling For All

3. Data is Messy

4. Data is Messy

5. DATA PRODUCT
Simplicity
DATA SOURCE
Complexity
People are the Bottleneck

6. Predictive Transformation

7. 2005 2007 2009 2011 2013 2015
(Abridged) History of Hadoop
YARN

8. 2013 2014 2015
Recently Launched Hadoop Applications

9. ““
LAUREN, DATA SCIENTIST
Preparing data for modeling or predictive analytics is painful, manual
and not repeatable across the organization. We need a more efficient
and reproducible way of preparing data for analysis.
““ I receive constant requests from the business for access to different data. I
need a faster way to produce the results they’re looking for in a
form that they can understand.
BEN, DATA PLATFORM ADMIN
Our Original Personas

13. /(2013-12-[0-9]+)/

14. Abstractions Can Be Useful

15. Abstractions Can Be Useful
[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:.[a-z0-9!#$
%&'*+/=?^_`{|}~-]+)*@
(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?.)+[a-
z0-9](?:[a-z0-9-]*[a-z0-9])?

16. Abstractions Can Be Useful
[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:.[a-z0-9!#$
%&'*+/=?^_`{|}~-]+)*@
(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?.)+[a-
z0-9](?:[a-z0-9-]*[a-z0-9])?
{email}

17. ““
LAUREN, DATA SCIENTIST
Preparing data for modeling or predictive analytics is painful, manual
and not repeatable across the organization. We need a more efficient
and reproducible way of preparing data for analysis.
““ I receive constant requests from the business for access to different data. I
need a secure way of pointing them directly to the raw data so they can
explore and prepare it themselves.
BEN, DATA PLATFORM ADMIN
Today’s Personas

18. ““
LAUREN, DATA SCIENTIST
Preparing data for modeling or predictive analytics is painful, manual
and not repeatable across the organization. We need a more efficient
and reproducible way of preparing data for analysis.
““ I receive constant requests from the business for access to different data. I
need a secure way of pointing them directly to the raw data so they can
explore and prepare it themselves.
BEN, DATA PLATFORM ADMIN
““
JOHN, DATA ANALYST
I want direct access to the raw data so I can actually see the content of
different datasets to define my analytic requirements.
Today’s Personas

19. LevelofAbstraction
Technical Proficiency

20. LevelofAbstraction
Technical Proficiency
MapReduce
Pig, Hive
/(2013-12-[0-9]+)/

21. LevelofAbstraction
Technical Proficiency
MapReduce
Pig, Hive
?
/(2013-12-[0-9]+)/

22. Supporting Multiple Personas is Hard

23. Supporting Multiple Personas is Hard

24. Supporting Multiple Personas is Hard

25. Supporting Multiple Personas is Hard

26. So why try?

27. So why try?
…to speak a common language

28. Jingshu Chen
User Experience Designer
Designing for Multiple Personas

29. Start the design process from the extremes

30. Start the design process from the extremes

31. Non
Technical
Start the design process from the extremes

32. Non
Technical
Super
Technical
Start the design process from the extremes

33. Non
Technical
Super
Technical
Start the design process from the extremes
?

34. Non
Technical
Super
Technical
Start the design process from the extremes

35. Non
Technical
Super
Technical
Start the design process from the extremes

36. Non
Technical
Super
Technical
Start the design process from the extremes

37. Start the design process from the extremes
Non
Technical
Super
Technical

38. Start the design process from the extremes
Non
Technical
Super
Technical

39. Start the design process from the extremes
Non
Technical
Super
Technical

40. Start the design process from the extremes
Non
Technical
Super
Technical

41. Start the design process from the extremes
Non
Technical
Super
Technical

42. Non Technical Super Technical

43. Non Technical Super Technical

44. Non Technical Super Technical
>

45. Non Technical Super Technical
>

46. Non Technical Super Technical
>
I have to learn how to script?
The technical guys do the scripting
“
”

47. Non Technical Super Technical
>
I have to learn how to script?
The technical guys do the scripting
“
”
Why would I learn a new product?
I could code this in two minutes
“
”

48. Non Technical Super Technical
transform data

49. Non Technical Super Technical
How might we empower users to
in a way that matches their mental model?
transform data

50. Non Technical Super Technical
Visual Transform Cards Script IDE Mode

51. Visual Transform Cards

52. Visual Transform Cards

53. Visual Transform Cards

54. Non Technical Super Technical
Visual Transform Cards Script IDE Mode

55. Script IDE Mode

56. Script IDE Mode

57. Script IDE Mode

58. Script IDE Mode

59. Script IDE Mode

60. Create consistent and seamless experience
Non Technical Super Technical

61. Create consistent and seamless experience
Non Technical Super Technical
Visual Transform Cards Script IDE Mode

62. Create consistent and seamless experience
Non Technical Super Technical
Visual Transform Cards Script IDE Mode

63. Create consistent and seamless experience
Non Technical Super Technical
Visual Transform Cards Script IDE Mode

64. Create consistent and seamless experience
Transform Builder
Progressive Script Disclosure
…
Predictive TransformationNon Technical Super Technical
Visual Transform Cards Script IDE Mode

65. Transform Builder

66. Transform Builder

67. Transform Builder

68. Progressive Script Disclosure

69. Progressive Script Disclosure
Mini

70. Progressive Script Disclosure
Mini Full

71. Progressive Script Disclosure
Mini Full IDE

72. Progressive Script Disclosure
Mini Full IDE

73. LevelofAbstraction
Technical Proficiency

74. LevelofAbstraction
Technical Proficiency
MapReduce
Pig, Hive
/(2013-12-[0-9]+)/

75. LevelofAbstraction
Technical Proficiency
MapReduce
Pig, Hive
/(2013-12-[0-9]+)/

76. LevelofAbstraction
Technical Proficiency
MapReduce
Pig, Hive
/(2013-12-[0-9]+)/

77. Building Flexible Interfaces
Joseph McKenney
Senior Software Engineer

78. LevelofAbstraction
Technical Proficiency
MapReduce
Pig, Hive
/(2013-12-[0-9]+)/

79. LevelofAbstraction
Technical Proficiency
MapReduce
Pig, Hive
/(2013-12-[0-9]+)/
Predictive
Transformation

80. DEMO

81. Implementing Cards

82. Implementing Cards

83. Interact
Predict
Preview
Improving Predictive Transformation

84. Improving Selections
Adjusting Selections Negative Selections Selection Management

85. Giving Feedback

86. Where do we go from here?

87. Offices in SF & South Bay
We’re Hiring!
Stop by booth P04

88. Sign up for a free trial at trifacta.com/trial
Questions?