Bridging the Gap Between Real Time/Offline and AI/ML Capabilities in Modern Serverless Apps (MOB310) - AWS re:Invent 2018

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Bridging the Gap Between Real Time/Offline and AI/ML Capabilities in Modern Serverless Apps Brice Pellé Enterprise Support Lead Amazon Web Services M O B 3 1 0 Ed Lima Solutions Architect Amazon Web Services

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda • The mobile/web app challenges in 2018 • Addressing the difficult basics • Solving the challenges • Boosting your application with intelligence

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The mobile/web app challenge in 2018 • In 2018, these problems are still difficult to solve • But they should be the basic requirements • Making an app stand out is becoming even harder • Millions of apps available across various application stores

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. To solve the challenge, developers need to • Iterate quickly • Focus on development, not servers • Provide the basics at a low cost of development • Offer innovative ways for users to interact with their apps • Make applications smarter, without a PhD in data sciences

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The modern mobile/web app must Solve the security challenge • AuthN and AuthZ by default Solve the data challenge • Network friendly (less calls / more data / just right) • Instantaneous, with real-time feedback Solve the screen and mobility challenge • Responsive & progressive • Always available, even offline Solve the differentiation challenge • Innovative & new ways to interact • How can your app stand out? Oh and … no servers!

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Iterating quickly with AWS Amplify • CLI & toolchain for the client • Client library & Javascript framework support • Supported services • Database • API • Lambda functions • Authentication • Analytics • Hosting • Storage > npm install aws-amplify > npm install –g @aws-amplify/cli

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Solve the security challenge Identity is mission critical for applications Three key areas to address • Authentication • Authorization • User management Implementing auth infrastructure is hard!

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Secure, always! • Leverage Amazon Cognito • Built with best security practices • Iterate quickly with AWS Amplify • With toolchain • Provision serverless backend resources • Use Auth module to easily interact • Wrap app in authenticator to require sign-in • Uses customizable pre-built UI components amplify import awsmobile from './aws-exports' import Amplify, { Auth } from 'aws-amplify' import { withAuthenticator } from 'aws- amplify-react’ ... Amplify.configure(awsmobile) ... class App extends Component { async componentDidMount() { session = await Auth.currentSession() this.setState({ session }) } } export default withAuthenticator(App) © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS AppSync Managed serverless GraphQL service Connect to data sources in your account Add data sync, real-time, and offline capabilities for any data source or API GraphQL facade for any AWS service Conflict detection and resolution in the cloud Enterprise security features: IAM, Amazon Cognito, OIDC, API keys

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a serverless messaging app with NoSQL 1..1 1..n 1..1 1..n 1..n Relations between NoSQL Tables User MessageConversation User Conversation

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Schemas with AWS Amplify > amplify add api type Conversation @model { id: ID! name: String! createdAt: String messages: [Message] @connection(name: "ConvoMsgs") } type Message @searchable @model { id: ID! content: String file: S3Object messageConversationId: ID conversation: Conversation @connection(name: "ConvoMsgs") } • Define and deploy DynamoDB tables based on schema • Creates AWS AppSync API & resolvers • Supports custom transformers > amplify api gql-compile; amplify api push type Mutation @custom { createMessage(input: CustomCreateMessageInput!): Message @custom(subscription: true, watchedFields: ["messageConversationId"]) } © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. GraphQL transformer – mix and match data sources searchMessages mutations queries listMessages createMessage type Message @searchable @model { id: ID! content: String file: S3Object messageConversationId: ID conversation: Conversation @connection( name: "ConvoMsgs") }

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Application overview • Data presentation split between components • Design independent of data source • Components wrapped in graphql HOC InputBar ConversationBar SearchBar const SearchResultListWithData = graphql(search, { skip: props => !props.term, options: props => ({ variables: { userTerm: `.*${props.term}.*`, msgFilter: buildMsgFilter(props.term, props.conversations) }, fetchPolicy: 'cache-and-network' }) })(SearchResultList)

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS AppSync with AWS Amplify Four easy steps to configure AWS AppSync 1. Import modules 2. Configure client 3. Advanced settings a. Complex data handling b. Union and interface handling 4. Wrap app in HOCs a. Make client available to provider b. Load app data from storage c. Launch app when data ready d. Require authentication import AWSAppSyncClient from 'aws-appsync' import { Rehydrated } from 'aws-appsync-react’ import awsmobile from './aws-exports' const client = new AWSAppSyncClient({ url: awsmobile.aws_appsync_graphqlEndpoint, region: awsmobile.aws_appsync_region, auth: { type: awsmobile.aws_appsync_authenticationType, jwtToken: async () =>(await Auth.currentSession()) .getIdToken().getJwtToken() }, complexObjectsCredentials: () => Auth.currentCredentials(), cacheOptions: { fragmentMatcher } }) const WithProvider = () => ( <ApolloProvider client={client}> <Rehydrated> <App /> </Rehydrated> </ApolloProvider> ) export default withAuthenticator(WithProvider) a b c d a b © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Network friendly— Batch requests • Wrap component with graphql • Data fetch when component is mounted but skipped if necessary • Cache lookup for quick user feedback • Use fetchMore to fetch additional data Messenger const MessengerWithData = graphql(getConvoMessages, { skip: props => !props.conversation, options: props => ({ variables: {convoId: props.conversation}, fetchPolicy: 'cache-and-network' }) })(Messenger) export default Messenger export { MessengerWithData } © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Instantaneous—Using subscriptions Subscriptions are • Granular • Real-time • In line notifications Initiated on component mount for each opened conversation Canceled on unmount or conversation change componentDidMount() { const { subscribeToMore, conversation: { id: convoId }, userId } = this.props this.unsubscribe = subscribeToMore({ document: onCreateMessage, variables: { messageConvoId: convoId }, updateQuery: (previous, data) => {...} }) } componentWillUnmount() { this.unsubscribe() } MessagePane © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. subscription onCreateMessage { createMessage {…} } Websocket URL and connection payload Secure Websocket connection (wss://) Subscriptions—Handshake process AppSync Data Sources

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Solve the screen challenge: Responsive Using a responsive design allows for “write once, run anywhere” Several well-known frameworks available

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Solve the mobility challenge: Always available With AWS AppSync SDK • Configured by default • Use Rehydrated component to rehydrate cache from storage • Use: fetchPolicy to control cache and network usage • AWS AppSync SDK maintains an outbox of pending actions • If no connection, then store action • On reconnection, send action • Development: Use redux tools

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Offline data rendering • Provide optimistic response • Useful when offline or slow network • Required when using offline capabilities! • Provide all fields present in the returned selection set mutation({ variables, optimisticResponse: { createMessage: {...} }, update: (proxy, data) => { const QUERY = { query: getConvoMessages, variables: { convoId } } const prev = proxy.readQuery(QUERY) const data = {...} proxy.writeQuery({ ...QUERY, data }) } }) MessagePane © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Platform services Application services Frameworks & interfaces Caffe2 CNTK Apache MXNet PyTorch TensorFlow Torch Keras Gluon AWS Deep Learning AMIs Amazon SageMaker Amazon Rekognition Amazon Transcribe Amazon Translate Amazon Polly Amazon Comprehend Amazon Lex Amazon Mechanical Turk The AWS Machine Learning stack

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. PLATFORM SERVICES Application services FRAMEWORKS & INTERFACES Caffe2 CNTK Apache MXNet PyTorch TensorFlow Torch Keras Gluon AWS Deep Learning AMIs Amazon SageMaker Amazon Rekognition Amazon Transcribe Amazon Translate Amazon Polly Amazon Comprehend Amazon Lex Amazon Mechanical Turk The AWS Machine Learning stack

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Common language use cases Information bots Education Accessibility Knowledge management Voice of customer applications Customer service/ call centers Enterprise digital assistant Semantic search Captioning workflows LocalizationPersonalization

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Translate Natural and fluent language translation Real-time translation Batch analysis Automatic language recognition Low cost

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Polly Turn text into lifelike speech using deep learning Wide selection of voices and languages Synchronize speech Fine-grained control Unlimited replay

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Comprehend Storm World Series Australia Stock market Washington Health Crisis Machine learning Library of news articles Amazon Comprehend Discover insights and relationships in text using deep learning

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Comprehend Discover insights and relationships in text using deep learning Entities Key phrases Language Sentiment Amazon Comprehend

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. A m a z o n . c o m , I n c . i s l o c a t e d i n S e a t t l e , W A a n d w a s f o u n d e d J u l y 5 t h , 1 9 9 4 b y J e f f B e z o s . O u r c u s t o m e r s l o v e b u y i n g e v e r y t h i n g f r o m b o o k s t o b l e n d e r s a t g r e a t p r i c e s N a m e d e n t i t i e s A m a z o n . c o m : O r g a n i z a t i o n S e a t t l e , W A : L o c a t i o n J u l y 5 t h , 1 9 9 4 : D a t e J e f f B e z o s : P e r s o n K e y p h r a s e s O u r c u s t o m e r s b o o k s b l e n d e r s g r e a t p r i c e s S e n t i m e n t P o s i t i v e L a n g u a g e E n g l i s h Text analysis

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Lex Building conversational interfaces into any application using voice and text Integrated development in the AWS console Trigger Lambda functions Multi-step conversations One-click deployment Enterprise connectors Fully managed

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Rekognition Image Detect objects, scenes, and faces, extract text and identify unsafe content in images Object and scene detection Facial analysis Face recognition Text in imageUnsafe image detection Celebrity recognition Face comparison

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. An AI component import React from "react"; import { graphql } from "react-apollo"; import rekognition from "../../images/rekognition.png"; import detectLabels from "../../graphql/queries/detectLabels"; class Labels extends React.Component { render() { const { data: { loading, error, detectLabels } } = this.props; if (loading) { return (<JSX…>); } else if (error) { const err = JSON.stringify(error.message); return ( return (<JSX…>); } else { const response = JSON.parse(detectLabels.response); return (<JSX…>); } } export default graphql(detectLabels, { skip: props => !props.path, options: props => ({ variables: { bucket: props.bucket, key: props.path }, fetchPolicy: "cache-and-network" }) })(Labels);

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS AppSync as GraphQL facade for AI services import gql from "graphql-tag"; export default gql` query invokeBot($bot: String, $text: String) { invokeBot(bot: $bot, text: $text) { response } } `; { "version" : "2017-02-28", "operation": "Invoke", "payload": { "field": "lex", "arguments": { "bot": "$context.arguments.bot", "sender": "$context.identity.sub", "text": "$context.arguments.text" } } } exports.handler = (event, context, callback) => { switch (event.field) { case "lex": const lexparams = { botAlias: "$LATEST", botName: event.arguments.bot, inputText: event.arguments.text, userId: event.arguments.sender }; lex.postText(lexparams, function(err, data) { if (err) { console.error("Error JSON: ", JSON.stringify(err, null, 2)); callback(err); } else { let result = { bot: event.arguments.bot, text: event.arguments.text, response: data.message }; callback(null, result); } }); break;

75. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The modern mobile/web app must Solve the security challenge • AWS Amplify and Amazon Cognito Solve the data challenge • AWS AppSync and GraphQL Solve the screen and mobility challenge • AWS AppSync + PWA built-in offline capabilities Solve the differentiation challenge • Great user (+ developer) experience (real-time, pagination, toolchain) • Add Intelligence to your app with AWS AI/ML Oh and … no servers!

77. Thank you! © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Brice Pellé Enterprise Support Lead Amazon Web Services Ed Lima Solutions Architect Amazon Web Services @ednergizer@BicePelle