Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

21st Century Frogman Lessons Learned H4D Stanford 2017

78,769 views

Published on

mission model, mission model canvas, customer development, Hacking for Defense, lean startup, stanford, startup, steve blank, Pete Newell, Joe Felter, minimum viable product

Published in: Education
  • Be the first to comment

  • Be the first to like this

21st Century Frogman Lessons Learned H4D Stanford 2017

  1. 21st Century Frogman Data Driven Solutions for Navy SEALs FINALINITIAL 1 Problem: Training injuries lead to BUD/S inefficiency and the loss of quality candidates from the program Solution: Predictive algorithm + dashboard to display safety risk Problem: Instructors waste time recording and entering data during training Solution: App that streamlines real-time data entry
  2. The Team And our fantastic Sponsor and Mentors Andrew Sparks BS Aero ‘03 MBA ‘18 Subject Matter Expert Support Commander DeCoster Lieutenant Prybella Sponsor Paul Dawes Mentor Lisa Wallace Industry Mentor Colonel Burns Military Mentor 2 Priyanka Sekhar BS CS ‘17 MS CS ‘18 Machine Learning Kelsey Schroeder BS MCS ‘17 MS MS&E ‘17 Strategy Gerardo Rendón BS SYMSYS ‘17 MS MS&E ‘18 Design / Data Mining
  3. Customer Interview Breakdown 3 66 Active Duty Military 31 Military Civilians 28 Industry Experts 125 Total Customer Interviews
  4. The Emotional Rollercoaster Highlights Pre - Coordination Challenges 1 - Spring Break Work 2 - Database Disaster 3 - Problems Everywhere 4 - Trip #1 to NSWCEN 5 - Instructor Surveys 6 - Begin MVP 7 - Product Mission Fit! 8 - Betrayal 9 - Development 10 - Trip #2 to NSWCEN, deliver prototype to Sponsor 4 Team Emotion Flow Over 10 Weeks
  5. Database and Dataflow Challenges 9 Databases in use at NSWCEN 5
  6. - NSW employee when asked about current tools for data analysis “CTRL-F” Team assessment: They are CTRL-Fing this entire thing up 6
  7. NSWCEN Visit #1
  8. Discovery of Our True Beneficiary Initial belief about our Primary / Secondary Beneficiary Leadership Instructor Student Medical N6 KM HP 12 1 2
  9. Injury underreporting….viewed as a sign of Strength Overall, 50% would view the classmate as Tough/Dedicated 1 3 Tough / Dedicated Stupid Same Weak Tough / Dedicated Tough / Dedicated
  10. - Instructor when asked if he would change training due to heightened risk of injury “No.” 1 4
  11. Discovery of Our True Beneficiary Initial belief about our Primary / Secondary Beneficiary Leadership Instructor Student Medical N6 KM HP 12XXPivot Discovery of the True Primary / Secondary Beneficiary Leadership Instructor Student Medical N6 KM HP 12 1 5
  12. Instructors’ Greatest Pain Points 1 6
  13. YES! If I could capture observations while in the field using an app… would greatly enhance grading and cadre to student debrief/counseling. - 1st Phase Instructor Fact. A high pain point for me, because I need my instructors commenting on as much as they observed as possible. The easier that is, the more they capture. - 1st Phase Officer 1 7
  14. MVPs Evolve…...Feature-Specific Learnings - More refined User Interface - Ability to select recommended comments or enter free text - Ability to search by student or evolution - Can select multiple students - Removed positive / negative comment feature to reduce clicks - Added audio record feature 1 8
  15. MVPs Evolve…...Feature-Specific Learnings 1 9 Add comment Report injury
  16. Internal Readiness Level Prototype low-fidelity Minimum Viable Product 1 2 3 4 5 6 7 8 9 2 0
  17. Deployment Rollercoaster Product Mission Fit!!! Let’s code!! Supporter or Saboteur?? People still use that?? 2 1 We’ll build a web app!
  18. Deployment roadmap 2 2 Open source and make available to CLD contract winner 5/19 - Coding Begins 6/2 - NSWCEN Visit #2 Proof of Concept Evaluation Summer 2017 - Contract Bids Solicited Contractors Prep Bids Fall 2017 - Contract Awarded
  19. -Reduced data entry burden: Less time spent entering data during/post evolution -Ease of use: User friendly U/I, intuitive note-taking process -More optimal utilization of instructor expertise: Instructors spend more time training students and less time entering data, which. 21st Century Frogman: Mission Model Canvas - UI/UX Design - Backend database design - Integration with current systems and standards - Earn customer buy-in - Gather data during alpha release to enable V2 design - Produce technical specs and deliver learnings to leadership - Continued sponsorship by NSWCEN CSO (CAPT CD), OPS (CDR DD), POC (LT CP) - NSW N6 Comms/IT - NSW N3 Database Managers - CLD Manager H4D Team: 1) Successful U/I proof of concept with placeholder student data 2) App provides groundwork for contracted solution Primary Beneficiary - 1st Phase BUD/S Instructors: 1) Reduced data entry time during an evolution by 50% 2) Reduced post evolution data entry / synthesizing time by 75% - Alpha Version of app used by instructors. Valuable feedback is gleaned. - Company that wins new CLD contract uses our app code and integrate into their solution - Push to other military centers with similar needs Fixed: - Software design & engineering; software products - Travel to NSWCEN (paid for by NSWCEN) Variable: - Installation and integration; ongoing support - Software experience in conjunction with tools (e.g., Balsamiq, Thunkable) - Credibility from reputation of NSW teammate to aid customer willingness to try product - Guidance and expertise from mentors in industry and military applications - Need support from NSWCEN (allow implementation) and customer interviews. - BUD/S instructors who have had previously negative experiences with GideonSoft app Beneficiaries Mission AchievementMission Budget/Costs Buy-In/Support Deployment Value Proposition Key Activities Key Resources Key Partners - Primary: 1st Phase Instructors - difficult to enter data, stagnant data, student injuries, lost students - Secondary: NSWCEN / BTC Operations Officer - stagnant data, lost students, student injuries, awareness of “factory floor” - Other: Students - injuries, instructors 2 3
  20. Mission Achievement H4D Team: 1) Successful U/I proof of concept with placeholder student data 2) App provides groundwork for contracted solution Primary Beneficiary - 1st Phase BUD/S Instructors: 1) Reduced data entry time during an evolution by 50% 2) Reduced post evolution data entry / synthesizing time by 75% 2 4
  21. Beneficiary Mission Achievement 2 5 20.7 (20 secs per unique entry, 2 minutes to upload, 30 sec to edit 10% of comments) hours saved per class 94% time reduction per comment Increase comment frequency Generate clean data for analysis (beyond CTRL-F) 2 5Higher quality of instruction 9:00 per comment 0:30 per comment
  22. San Francisco Police Academy 250 trainees/year 8 months of training/student No digital note-taking mechanism Beyond Naval Special Warfare Center 2 6 Other applications: Military training centers (Army Special Forces) Amateur/professional sports teams (Palo Alto High School)
  23. Key Learnings 1) Wet and sandy: not a comfortable combination 2) Customer discovery: essential to problem understanding 3) Sponsor support: goes a long way 4) Security/Legal: can be last-minute blockers 5) Defense: large opportunity for impact 2 7
  24. SPECIAL THANKS TO Dana DeCoster Chris Prybella Jay Hennessy Christian Dunbar Scott McCallum Russell Black Phil Richey Brandon Barker Kevin Mangini Celeste Machado Anonymous Brownshirt Steve Blank Joe Felter Pete Newell Steve Weinstein Melisa Tokmak Isaac Matthews Darren Hau Jared Dunnmon Lisa Wallace Paul Dawes Curtis Burns 2 8
  25. Appendix 2 9
  26. 21st Century Frogman: Value Proposition Canvas Customer Jobs Train Navy SEALs Primary Beneficiary: 1st Phase Instructors Gains Pains Gain Creators Pain Relievers - Reduce manual tasks; automate tagging of target activities Products & Services Free text data entry app for 1st Phase BUD/S evolutions - Less time wasted entering redundant data - More streamlined data-entry experience during training evolutions - Having to enter all data twice (field and then office) - No way to pre-populate comments on paper - Reduce data entry burden for instructors through automation 3 0
  27. 21st Century Frogman: Value Proposition Canvas Products & Services Free text data entry app for 1st Phase BUD/S evolutions - Reduce data entry burden for instructors through automation Customer Jobs Build and deliver high quality, high character SEALs to the community- Feeling of not knowing what is going on down on factory floor - Attrition of “hard” guys - Sub-optimal utilization of instructor expertise Secondary Beneficiary: NSWCEN / BTC Operations Officer Gains Pains Gain Creators Pain Relievers - Instructors are happy - Instructors spend less time on redundant tasks (organizational efficiency) - Instructors spend more time monitoring students, which could improve safety and reduce injuries 3 1
  28. 21st Century Frogman: Value Proposition Canvas Products & Services Free text data entry app for 1st Phase BUD/S evolutions - Easier Data entry for instructors Customer Jobs Survive Training so they can hunt bad guys- knees, shoulder, back - Instructor scrutiny - cold water - sand in crevices - illness, chafing - sleep deprivation Other Beneficiary: Students Gains Pains Gain Creators Pain Relievers - Happier Instructors TBA 3 2
  29. Instructors (BUDS 1st phase) day in the life 3 3 6:00 am - 6:30 am: Get breakfast 6:30 am - 7:00 am: Brief students on what evolution will consist of 7:00 am – 12:00 pm: Take out the boats and do some runs in the ocean with students. Come back, have students do a PT session, time them and take notes on their performance. Then teach SEAL skills class. 12:00 pm – 12:45 pm: Lunch 1:00 pm – 3:00 pm: Timed evolution for students; take notes on performance. 3:00 pm – 3:30 pm: Input student performance data on spreadsheets 3:30 pm – 5:00 pm: Oversee student pool exercises 5:30 pm: Finish inputting last minute notes, make final arrangements, leave work Stay until 8:30 for dinner and another evolution (about two or three times a week).
  30. Customer Archetype Leadership Instructor Student Medical N6 KM HP Position CDRE / XO OPS / CMC First phase BUD/S BUD/S Candidate Corpsman, MD Comms, IT Knowledge Management Contracted Company Age - M/F Rank 36-50 Male CAPT / CDR / Master Chief 25-35, Male, Petty Officer, Chief 18-30 Male Seaman, Petty Officer 25-40 M/F HM1, HM2, LT-CDR 20-40 M/F Petty Officer, Chief, CIV 25-50 M/F Contracted Civilian 30-50 M/F Civilian Motivation Safely & efficiently produce elite Navy SEALs Hold standard high, safe training, supervise evolutions Finish BUD/S, serve country, be a part of an elite team Treat injuries effectively, ensure safety Avoid IT violations, keep comms working Serve country, steady 9-5 job, keep boss happy Manage NSWCEN data, get next contract Pains Stagnant Data High Attrition Oversight Inputting data, unreliable software Instructors, cold water, sand, injuries Redundant data entry, unreliability IT violations, restrictive IT security Tasks that are outside their scope of work Competitors, contracting, restrictive IT Gains User friendly dashboard User friendly, automatic updates Injury prevention Reduced data entry, less hurt students New tech that is allowed 3 4
  31. User Flow (1st Phase Instructors) 3 5 Record comments during evolution on notepad or Notes app CLD After evolution, type comments into Excel spreadsheet Email spreadsheet to CLD contractor (HP) HP inputs data into CLD database Record comments during evolution on notepad or Notes app After evolution, type comments directly into CLD CLD WhatwethoughtReality Integration with CLD is a must
  32. ~80% (79.2%) of comments fall in the top 20% of categories (by frequency of use) 28% of comments fall under “Other - Negative” Pareto reigns supreme 3 6 Behavior category # % Other - Negative 369 28 Demonstrated less than 100 percent effort 133 10 High Performer 115 8.7 Malicious Boat Ducking 99 7.5 Physically weak 94 7.1 Other - Positive 65 4.9 Poor Performance Counseling 64 4.9 Displayed unprofessional behavior 39 3.0 Exemplary performance 32 2.4 Consistently demonstrated leadership 31 2.3 TOTAL 1041 79%
  33. Emerging Vision Step 1: MVP Team Frogman Builds and deploys Step 2: Integrate with analytics to provide attrition stats, medical drops, real time analytics on desktop and mobile Step 3: Wearables for collecting performance data in real time. Direct input to analytics dashboard. 3 7
  34. Emerging vision 3 8Analytics Admin data schedules, rosters Performance data free-text comments, run/swim times, incident reports
  35. Emerging vision 3 9 V1 V2 V3 Admin data Daily training schedule Manual Manual Auto Phase roster Manual Auto Auto Performance data Free-text comments Auto Auto Auto Incident/injury reports Auto Auto Auto Run/swim times Manual Manual Auto Analytics Comment analysis No Yes Yes Attrition statistics No Yes Yes Injury pinpointing No Yes Yes
  36. 4 0 Emerging Vision STEP 1 STEP 2 STEP 3 Feature Summary GET: Free-text comments, formatted to compatible spreadsheet, roster manually inputted/edited KEEP: prefilled comments, prioritization, analytics GROW: Attrition stats. 1st Phase current stats dashboard, roster auto- populated 2b: Comment analytics (identify top guys, NLP characterization) 3a: Graph of performance vs. character, fine-tuned stats, real-time injuries, student geo-tracking 3b: centralized schedule, preloaded into app Instructor Value Prop -Less time spent entering data during/post evolution -User friendly U/I -Automatic roster updates -Quick reference student performance trends -Attendance/times recorded automatically and instantly -No more “lost” students Leadership Value Prop -Instructors spend more time training students and less time entering data, which safety & injuries (hypothesis) -Trends, comparison to historical averages -”Factory floor” awareness dashboard, “red flag” alerts -Analytics and awareness -Be able to see 2x2 matrix graph of performance and character real-time Student database integration SMS integration, $ (RFID)
  37. UI-specific learnings • Selection of multiple students • Suggested comments (positive and negative) • Injury reporting option • Search bar for students in addition to list • Alphabetical ordering instead of ordering by groups/boat crews • Select evolution first • Big buttons with text • Voice feature (not shown in current mockup) 4 1
  38. Current mockup Test here: https://h4d.bubbleapps.io/version-test 4 2
  39. Time saved calculation 20 seconds/per comment, but 0 seconds per duplicate (20 seconds) * (52% unique comments) = 10.4 seconds 30 seconds to edit a comment, done to about 10% (30 seconds) * (10% of comments) = 3 seconds 2 minutes to upload, will upload ~10 comments at a time (2 minutes) / (10 comments) = 12 seconds Time spent per comment: 25.4 seconds (round to 30) Time saved per comment: 9:00 - 0:30 = 8:30 1315 comments/9 classes, so assume 146 comments/class Time saved per class = (8.5 minutes) * (146 comments) = 20.7 hours 4 3
  40. 4 4
  41. 4 5
  42. 4 6
  43. 4 7
  44. 4 8
  45. 4 9

×