• Save
appengine ja night #24 Google Cloud Endpoints and BigQuery (English)
Upcoming SlideShare
Loading in...5
×
 

appengine ja night #24 Google Cloud Endpoints and BigQuery (English)

on

  • 1,118 views

This is the slide of Google Cloud Endpoints and BigQuery.

This is the slide of Google Cloud Endpoints and BigQuery.
I wrote it for "appengine ja night # 24".

Statistics

Views

Total Views
1,118
Views on SlideShare
1,112
Embed Views
6

Actions

Likes
1
Downloads
0
Comments
0

1 Embed 6

https://twitter.com 6

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

appengine ja night #24 Google Cloud Endpoints and BigQuery (English) appengine ja night #24 Google Cloud Endpoints and BigQuery (English) Presentation Transcript

  • Google Cloud Endpoints & BigQuery appengine ja night #24Ryo Yamasaki @vierjp 1
  • Self Introduction Ryo Yamasaki •  Blog •  http://blog.vier.jp/ •  Twitter •  @vierjp •  Work •  I mainly develop server side applications by Java. •  I was a technical leader for the smart phone service provider company in the previous job. •  I used the products of Google Cloud Platform, such as App Engine, Cloud Storage and BigQuery. •  Publishing Android apps •  WiFi Tunes Sync •  WiFi Tunes Sync Pro 2
  • Google Cloud Endpoints Overview 3
  • Google Cloud Endpoints In one sentence…Cloud Endpoints make it possible to use AppEngine more easily as the backend of clientapplications. 4
  • Google Cloud Endpoints You don’t need to write codes for communicating with servers anymore. Range  which  Cloud  Endpoints  supports   クライアント サーバー側 Get  values   Set  GET  or   which   Send   Receive   POST   required  to   request request Parameters   execute  API Process   Server  Side   API using  the   Client  Library request   Framework parameters Convert   Parse  json   Process   objects  to   and   using  the   Receive   json  format   Mapping   results   response and     values  to   obtained return  json   your  objects   string   5
  • Google Cloud Endpoints You can generate client libraries for Android, iOS, and JavaScript. App  Engine   Cloud   Endpoints Android   iOS   JavaScript   6
  • Google Cloud Endpoints Authentication support with OAuth 2.0 7
  • Google Cloud Endpoints You can use all of the services and features available in App Engine. Cloud   Endpoints   AppEngine   Google  Cloud   Datastore   Url  Fetch   Mail   Task  Queues   Storage   8
  • Google Cloud Endpoints APIs that was created with Endpoints can be execute from the APIs Explorer •  Comply with “Google APIs Discovery Service” •  You can execute APIs with OAuth2 authentication Execute  View List  View 9
  • Google Cloud Endpoints The Development Process 10
  • Google Cloud Endpoints Requirements (Java) •  Eclipse •  AppEngine SDK 1.7.5 or newer •  Google Plugin for Eclipse (newer) 11
  • Google Cloud Endpoints The Development Process 1.  Create your Eclipse Project 2.  Write your backend app, and add annotations 3.  Test your API using the APIs Explorer 4.  Generating the client library 5.  Write your client app, using the client library 12
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App If  you  make  Android   If  you  do  not  make   clients Android  clients 2.Right-­‐click  on  the   New-­‐>Project-­‐>   1.Create  an  Android   Android  project  and   Google  -­‐> project select  "Generate   “Web  ApplicaWon   AppEngine  Backend". Project” 13
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App @Api(name = "appData", version = "v1", description = "AppData API description v1")public class AppDataV1Endpoint { Process which acquires one data@ApiMethod(name = "data.get", path = "data/get/{id}”) specified by ID and returns it.public AppDataV1 get(String id) throws ServiceException { @Named("id") String id) throws ServiceException { try { // Get Entity Key key = KeyFactory.createKey("AppData", id); DatastoreService ds = DatastoreServiceFactory.getDatastoreService(); Entity entity = ds.get(key); // Generate a DTO to return, refill the value of Entity. Point AppDataV1 result = entity2Dto(entity); - Annotations: @Api, @ApiMethod, @Named return result; - Exception is thrown to notify an error to a client. } } catch (EntityNotFoundException e) { // Error handling in case the data corresponding to ID is not found. throw new NotFoundException("id:" + id + " is not found."); } catch (Exception e) { // Error handling in case an unexpected error occurs. throw new InternalServerErrorException(e); } } 14
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  Annotation offered by Endpoints•  @Api •  Defined for the class •  Declare API (ex. appData API v1) •  If you define multiple versions of the same API, create multi classes.•  @ApiMethod •  Defined for the method •  Declare an individual function (ex. ”data.get”)•  @Named •  Defined for the method argument •  If you specify @Nullable, @Named is required.•  @Nullable •  Defined for the method argument •  Specify if you want to allow Null 15
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  @Api @Api(name = "appData", version = "v1", description = "AppData API description v1”) ・JavaScript gapi.client.appData.data.get({id:$scope.id}).execute(function(response,rawResponseJson) { // callback }); ・Android AppData.Builder builder = new AppData.Builder(AndroidHttp.newCompatibleTransport(), new GsonFactory(), null); AppData appDataApi = CloudEndpointUtils.updateBuilder(builder).build(); AppDataV1 response = appDataApi.data().get(id).execute(); return response; 16
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  @ApiMethod @ApiMethod(name = "data.get", path = "data/get/{id}”) @ApiMethod(name = "data.put", path = "data/put”) ・JavaScript gapi.client.appData.data.get({id:$scope.id}).execute(function(response,rawResponseJson) { // callback }); ・Android AppData.Builder builder = new AppData.Builder(AndroidHttp.newCompatibleTransport(), new GsonFactory(), null); AppData appDataApi = CloudEndpointUtils.updateBuilder(builder).build(); AppDataV1 response = appDataApi.data().get(id).execute(); return response; 17
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App Exception classes that can be used for notification of errors BadRequestExcepWon   • For  input  errors  (h[p  status  code:  400) UnauthorizedExcepWon   • For  authenWcaWon  errors  (h[p  status  code:  401) ForbiddenExcepWon   • For  permission  errors  (h[p  status  code:  403) NotFoundExcepWon   • For  resource  not  found  errors  (h[p  status  code:  404) InternalServerErrorExcepWon   • For  system  errors  (h[p  status  code:  500) ServiceUnavailableExcepWon   • For  service  unavailable  errors  (h[p  status  code:  503) 18
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  How to create your exception class •  Extends “com.google.api.server.spi.ServiceException” •  Sample Code public class ConflictException extends ServiceException{ public ConflictException(String message){ super(409, message); } } 19
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  Point •  API name (value of name attribute of @API) should start with lower case. •  Can’t return a literal as the return value. •  Cant return Entity type of App Engine as the return value. •  By using CollectionResponse, paging can be implemented easily. •  The following types can be obtained if defined as the method argument. •  javax.servlet.http.HttpServletRequest •  javax.servlet.ServletContext 20
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  Tips: Unit testing for server side with JUnit •  You can write unit test codes as well as the other test codes of AppEngine app general. •  Simplified by AppEngineTestCase of Slim3. 21
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  Testing using the APIs Explorer •  Input items are automatically reflected on form. •  Required items are displayed in red. •  The contents of a request can be referred to. •  The contents of a response can be referred to. •  You can execute APIs with OAuth2 authentication 22
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  Use Google Plugin for Eclipse•  From the command line sh  appengine-­‐java-­‐sdk-­‐1.7.7/bin/endpoints.sh  get-­‐java-­‐client-­‐lib    jp.vier.sample.endpoints.sample.v1.SampleEndpoint 23
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  Point •  If the code is incorrect, an error occurs in the process of generating client libraries. •  Error informations are displayed on "Error Log" view. (Notice: It isnt a "Problems" view.) 24
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  JavaScript: Initialize (with AngularJS)<script type="text/javascript">var ENDPOINTS_ROOT = https://[your_app_id].appspot.com/_ah/api;function init() { // Initialization process for using the API of Endpoints. // Specify the API name that is defined by @API on the server side to the first argument. gapi.client.load(’appData, v1, function() { // initialize AngularJS angular.bootstrap(document,[ui.bootstrap,my.filter]); }, ENDPOINTS_ROOT);}</script><script src="https://apis.google.com/js/client.js?onload=init"></script> 25
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  JavaScript: Execute Method (with AngularJS)$scope.getAppData=function() { // Execute API gapi.client.appData.data.get({id:$scope.id}).execute(function(response) { // To process the response received $scope.appData = response; $scope.$apply(); }); ・By calling the function of JavaScript, you can get the results from the API on the server.・Calling "apply" is needed with AngularJS. 26
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  JavaScript: Error Handling (with AngularJS)$scope.getAppData=function() { // Execute API gapi.client.appData.data.get({id:$scope.id}).execute(function(response,rawResponseJson) { // for dev server if(response===false){ var rawResponse = JSON.parse(rawResponseJson); // Display an error dialog $scope.openErrorModal(rawResponse[0].error.message); $scope.$apply(); // for production }else if(response.error && response.error.data && response.error.data.length > 0){ // Display an error dialog $scope.openErrorModal(response.error.data[0].message); $scope.$apply(); // success ・You can get the http status code and error message. }else{ ・Errors behavior is different between the dev server and production environment. (Issue 9171) $scope.appData = response; ・"rawResponseJson" is normally not necessary, but It is needed to error $scope.$apply(); handling on dev server. } });} 27
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  Android: Execute MethodAppData.Builder builder = new AppData.Builder(AndroidHttp.newCompatibleTransport(),new GsonFactory(), null);AppData appDataApi = CloudEndpointUtils.updateBuilder(builder).build();AppDataV1 response = appDataApi.data().get(id).execute();・By calling the method of Client Library, you can get the results from the API on theserver.・It will be executed using the AsyncTask normally. 28
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  Android: Error Handling@Override  protected  AppDataV1  doInBackground(Void...  unused)  {          try  {                  AppData.Builder  builder  =  new  AppData.Builder(AndroidH[p.newCompaWbleTransport(),  new  GsonFactory(),  null);                  AppData  appDataApi  =  CloudEndpointUWls.updateBuilder(builder).build();                  AppDataV1  response  =  appDataApi.data().get(id).execute();                  return  response;          }  catch  (final  GoogleJsonResponseExcepWon  e)  {                  handler.post(new  Runnable()  {                          @Override                          public  void  run()  {                                  //  Get  the  error  message  that  was  passed  to  the  excepWon  instance  on  the  server  side.                                      String  resMessage  =  e.getDetails().getMessage();                                  Toast.makeText(context,  "Error:  "  +  resMessage,  Toast.LENGTH_LONG).show();                                  return;   ・You can catch the error returned from the server as                        }   GoogleJsonResponseException.                });   ・You can get the http status code and error message.                return  null;          }  catch  (IOExcepWon  e)  {   ・IOException should be treated as a communication                      //  CommunicaWon  error error. handler.post(new  Runnable()  {                          @Override                          public  void  run()  {                                  Toast.makeText(context,  “A  communicaWon  error  has  occurred.  Please  check  the  network  status.",  Toast.LENGTH_LONG).show();                                  return;                          }                  });                  return  null;          }  } 29
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App •  Point •  By calling the function or method of Client Library, you can get the results from the API on the server. •  How to use the client library is almost same as that of the Google API. •  Therefore, when you are in trouble, searching how to use the Google API might be a good idea. 30
  • Google Cloud Endpoints Create  Eclipse   Write  your   Test  using  the   GeneraWng   Write  your   Project Backend  App APIs  Explorer   Client  Libraries Client  App Tips:How to connect to the dev server from Android.•  Use “CloudEndpointUtils” on Android. •  It is generated by the “Generate AppEngine Backend” •  How to use: Show ”Android: Execute Method” •  Set true to CloudEndpointUtils#LOCAL_ANDROID_RUN.•  Change the settings of your dev server. •  Debug Configurations -> “Arguments” Tab -> Program arguments •  Add “ --address=0.0.0.0” 31
  • Google Cloud Endpoints OAuth 2.0 authentication 32
  • Google Cloud Endpoints OAuth2 : Create Client ID APIs Console -> API Access For JavaScript For Android 33
  • Google Cloud Endpoints OAuth2: Implementation of Backend /** ClientId for Web */private static final String CLIENT_ID_WEB = "10*********09-7ufji**********k2a91.apps.googleusercontent.com";/** ClientId for Android */private static final String CLIENT_ID_ANDROID = "10*********09-o4e8d**********gt2ts3.apps.googleusercontent.com";/** Audience for Android */private static final String AUDIENCE_ANDROID = CLIENT_ID_WEB; - Definition of Client ID is required for using the OAuth2 authentication. - When it is accessed using the token that is obtained by authentication in@ApiMethod( the ClientID that is not defined, the user argument will be null. name = "data.put", (In this case the request is treated as not authenticated) - "Audience" is required if you are supporting Android clients. path = "data/put", clientIds = {CLIENT_ID_WEB, CLIENT_ID_ANDROID }, audiences = AUDIENCE_ANDROID)public AppDataV2 put(AppDataV2 appDataV2, User user) throws ServiceException { // Authentication check - When you define the argument of com.google.appengine.api.users.User type, if (user == null) { user information is set in the variable in the case of throw new UnauthorizedException(“Login is required. "); OAuth2 authenticated. } - An authentication state can be judged by whether user is null. // Registration of data - If authentication is required and user is null, throw} UnauthorizedException. 34
  • Google Cloud Endpoints OAuth2: Implementation of JavaScript var  mainCtrl  =    funcWon($scope)  {        var  CLIENT_ID  =  10*********09-­‐7uti**********k2a91.apps.googleusercontent.com;      var  SCOPES  =  h[ps://www.googleapis.com/auth/userinfo.email’;      //  OAuthライブラリをloadして自動で認証を試みる   - Use Client Id for Web    gapi.client.load(oauth2,  v2,  funcWon(response)  {   - At minimum, you should request access                $scope.signin(true,  $scope.userAuthed);   to the userinfo.email scope.    });      //  OAuth認証関数   - Execute the authentication process    $scope.signin=funcWon(mode)  {   -­‐  Specify  the  "immediate:true"  for  automaWcally                  gapi.auth.authorize({client_id:  CLIENT_ID,   authenWcate.                          scope:  SCOPES,  immediate:  mode,   -­‐  Specify  the  "immediate:false"  for  explicitly                          response_type:  token  id_token},   authenWcate.                          $scope.userAuthed);        };   - Get a Token and user informations in the callback      //  認証後にユーザー情報を取得するcallback関数   which is called after the authentication process.    $scope.userAuthed  =  funcWon()  {                  var  request  =  gapi.client.oauth2.userinfo.get().execute(funcWon(response)  {                          if  (!response.code)  {                                  var  token  =  gapi.auth.getToken();                                  token.access_token  =  token.id_token;   -­‐  For  sign  out  ,  call  gapi.auth.setToken(null);                                  gapi.auth.setToken(token);                                     -­‐  How  to  sign  in  explicitly                                    //  User情報を取得 $scope.user=response;   <a  ng-­‐click="signin(false);">  Sign  In  </a>                                  //  反映させる $scope.$apply();                          }                  });          }  }     35
  • Google Cloud Endpoints OAuth2: Implementation of Android (Manage Credential) private  staWc  final  String  CLIENT_ID  =  "10*********09-­‐7uti**********k2a91.apps.googleusercontent.com";     Define the clientId specified as the Audience on the server side.private  GoogleAccountCredenWal  credenWal;     Define a Credential@Override  protected  void  onCreate(Bundle  savedInstanceState)  {          super.onCreate(savedInstanceState);          setContentView(R.layout.acWvity_main);            //  Create  a  credenWal credenWal  =  GoogleAccountCredenWal.usingAudience(this,  "server:client_id:"  +  CLIENT_ID);  }    private  void  setAccountName(String  accountName)  {          //  set  account  name  to  credenWal        credenWal.setSelectedAccountName(accountName);  }       36
  • Google Cloud Endpoints OAuth2: Implementation of Android (Choose Account) //  Display  Account  Picker private  void  chooseAccount()  {          startAcWvityForResult(credenWal.newChooseAccountIntent(),  REQUEST_ACCOUNT_PICKER);  }    @Override  protected  void  onAcWvityResult(int  requestCode,  int  resultCode,  Intent  data)  {          super.onAcWvityResult(requestCode,  resultCode,  data);          switch  (requestCode)  {          //  Get  the  selecWon  result  of  Account  Picker case  REQUEST_ACCOUNT_PICKER:                  if  (data  !=  null  &&  data.getExtras()  !=  null)  {                          Bundle  bundle  =  data.getExtras();                          String  accountName  =  bundle.getString(AccountManager.KEY_ACCOUNT_NAME);                          if  (accountName  !=  null)  {                                  //  Set  account  name  to  credenWal setAccountName(accountName);                          }                  }                  break;          }  }   37
  • Google Cloud Endpoints OAuth2: Implementation of Android (Execute Method) AppData.Builder  builder  =  new  AppData.Builder(                AndroidH[p.newCompaWbleTransport(),                new  GsonFactory(),                  credenWal);  AppData  appDataApi  =  CloudEndpointUWls.updateBuilder(builder).build();   To execute API using authentication information, credential is specified as an argument. If authentication information is not required, you specify null. 38
  • Google Cloud Endpoints OAuth2: Two kinds of token Id_token access_token Used  by  official  sample  code Used  by  API  Explorer  (*1)  Documented Not  Documented  (*2)  You  can  get    only    a  NickName  and  an  Email   You  can  get    the  UserId,  the  Email,  NickName,  and  Address    from  variable  of  User  Type.  (*3) AuthDomain    from  variable  of  User  Type. Perhaps,  on  the  server  side,  it  is  not  possible  to   On  the  server  side,  it  is  possible  to  exercise  the  exercise  the  privileges  of  authenWcated  users. privileges  of  authenWcated  users.  (*4) AutomaWcally  check  the  validity  of  ClientID  that   (same  as  on  the  le|) was  used  to  generate  the  Token. 1.  Regarding authentication, Behavior is different in the API Explorer and JS-Android   implementation written in the document.2.  It is not clear whether the way of using access token will be supported continuously. Because it is not written in the document.3.  It may be fixed as a bug because the status of issue has been accepted.4.  It seems that there is no structure to get a token or a credential as a function of Endpoints. You need to get access token directly from HttpServletRequest#getHeader. 39
  • Google Cloud Endpoints Demo of JavaScript & Android •  APIs on backend •  Get messages •  Post a message (require authentication)•  Client •  List messges •  Post a message •  OAuth2 authntication 40
  • Google Cloud Endpoints Useful links •  Official documentation •  http://goo.gl/6wp7R•  Official Sample (tictactoe) •  Android - http://goo.gl/xjX1B •  Backend & JS - http://goo.gl/V8MEV•  API Explorer •  https://developers.google.com/apis-explorer/#s/•  Google APIs Discovery Service •  https://developers.google.com/discovery/?hl=ja•  APIs Console •  https://code.google.com/apis/console/ 41
  • BigQuery Demo of the new features •  Small JOIN &Big JOIN•  GROUP BY &GROUP EACH BY 42
  • BigQuery Useful links•  BigQuery Official documentation •  https://developers.google.com/bigquery/•  BigQuery Query Reference (JOIN) •  http://goo.gl/LKEZq•  BigQuery Query Reference (GROUP BY) •  http://goo.gl/EI2aK•  Google Developers Blog (BigQuery gets big new features to make data analysis even easier) •  http://goo.gl/Unwmq 43
  • Google Cloud Endpoints & BigQuery Thanks for reading! 44