Simon @ MiCloud
We are...
前情提要
很久很久以前...
時代不同,⾯面對資料的態度也不同...
● 存得起來的,叫做Storage (儲存)!
● 看得到的,叫做Data (資料)!
● 看得懂的,叫做Information (資訊)!
● ⽤用得出來的,才能夠叫做Intelligent
(智慧)
...
資料...就是應該被儲存...
● POS data
● Log data
● User behavior
● Data Warehouse
● Transactions
貌似正常的⼈人
看似無意義的log
接下來,我們要開幹了!
Tools
Req?
Web
App
Service
絕世武功秘笈…
我們需要的是
● ⼀一個存取的⽅方式
● 越簡單越好...
● 輕量...
● 快速...
天下武功,唯快不破...
Google BigQuery - 秘笈
Node.js - 武功
select repository language from
372211 259329 223470
合體!!!
絕世武功秘笈
But...⼈人⽣生最厲害就是這個BUT!
Hello Google! Login Required :)
WTF… =_=
Node.js & Oauth & Google
Hello Google! What do u want =_=
Authenticate with
Oauth2.0
Tools
Auth
Req?
Web
App
Service
$ npm search oauth
google-api-utility模組開發歷程
透過初始化設定之後,
即可以直接進⾏行api呼叫動作
!
1. 設定檔抽離
2. 結合request模組進⾏行API呼叫
google-api-utility module
基本資訊
● https://github.com/peihsinsu/google-api-utility
安裝
● npm install google-api-utility
操作
● ...
Google Oauth2
Web Server Application Service Account
Request token
Authorization
code
Exchange code
for token
Token respon...
Service Account
Configures
Authenticate can be cache
and auto refresh
Others… Request… 

Request… Request…
Create and
sign...
免不了的⿇麻煩....
Service Owner在Google的設定
Prepare Authentications
$ openssl pkcs12 -in privatekey.p12 -out privatekey.pem -nocerts
$ openssl rsa -in privatekey.pem -out key.pem
Generate ke...
var auth = require('google-api-utility')

auth.init({

scope: 'https://www.googleapis.com/auth/bigquery https://
www.googl...
使⽤用範例 - 呼叫BigQuery
var request = auth.request;
var bqurl = 'https://www.googleapis.com/bigquery/v2/projects/%s/
datasets';...
Tools
Auth
Req?
API
Web
App
Service
Google API Explore HTTPS://developers.google.com/apis-explorer/
Google API Explore - Query
Google API Explore - Auth
Operation Scope
Google API Explore - Response
Tools
Auth
Req?
API
SDK Web
App
Service
Idea...
● bigquery.init({...configurations...})
● bigquery.[what].[do](...)
- bigquery.dataset.list(....)
- bigquery.table...
bigquery module
基本資訊
● https://github.com/peihsinsu/bigquery
安裝
● npm install bigquery
操作
● bigquery.init(config)
● bigque...
重新包裝 - bigquery模組
var bq = require('bigquery')

, prjId = 'your-bigquery-project-id';
bq.init({

client_secret: '/path/to/...
Source Code...
var util = require('util')
, auth = require('google-api-utility')
, request = auth.request
, _ = require('u...
NOT ONLY Node.js Love JavaScript…..
The fancy integrate with Sheet
More and more chart….
Charting in Apps Script
Google provided node.js tool
Operation with googleapis
var googleapis = require('googleapis');
!
var jwt = new googleapis.auth.JWT(
'429100748......hv3...
Operation with googleapis
jwt.authorize(function(err, tokens) {
googleapis.discover('bigquery', 'v2').execute(function(e,c...
Full Code
HTTP://OpenNodes.arecord.us
OR
Sample
Copy
Run!
http://micloud.tw
http://about.me/peihsinsu
JsConf 2014 - Google BigQuery API Node.js實作記錄
JsConf 2014 - Google BigQuery API Node.js實作記錄
JsConf 2014 - Google BigQuery API Node.js實作記錄
JsConf 2014 - Google BigQuery API Node.js實作記錄
Upcoming SlideShare
Loading in …5
×

JsConf 2014 - Google BigQuery API Node.js實作記錄

538
-1

Published on

發表於2014年度JsConf.CN,提供給想要實作Google API服務的Node.js開發人員參考

Published in: Technology, News & Politics
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
538
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

JsConf 2014 - Google BigQuery API Node.js實作記錄

  1. 1. Simon @ MiCloud
  2. 2. We are...
  3. 3. 前情提要 很久很久以前...
  4. 4. 時代不同,⾯面對資料的態度也不同... ● 存得起來的,叫做Storage (儲存)! ● 看得到的,叫做Data (資料)! ● 看得懂的,叫做Information (資訊)! ● ⽤用得出來的,才能夠叫做Intelligent (智慧) 和沛科技 CEO & Founder - 翟本喬
  5. 5. 資料...就是應該被儲存... ● POS data ● Log data ● User behavior ● Data Warehouse ● Transactions
  6. 6. 貌似正常的⼈人
  7. 7. 看似無意義的log
  8. 8. 接下來,我們要開幹了!
  9. 9. Tools Req? Web App Service
  10. 10. 絕世武功秘笈…
  11. 11. 我們需要的是 ● ⼀一個存取的⽅方式 ● 越簡單越好... ● 輕量... ● 快速... 天下武功,唯快不破...
  12. 12. Google BigQuery - 秘笈
  13. 13. Node.js - 武功 select repository language from 372211 259329 223470
  14. 14. 合體!!! 絕世武功秘笈
  15. 15. But...⼈人⽣生最厲害就是這個BUT! Hello Google! Login Required :) WTF… =_=
  16. 16. Node.js & Oauth & Google Hello Google! What do u want =_= Authenticate with Oauth2.0
  17. 17. Tools Auth Req? Web App Service
  18. 18. $ npm search oauth
  19. 19. google-api-utility模組開發歷程 透過初始化設定之後, 即可以直接進⾏行api呼叫動作 ! 1. 設定檔抽離 2. 結合request模組進⾏行API呼叫
  20. 20. google-api-utility module 基本資訊 ● https://github.com/peihsinsu/google-api-utility 安裝 ● npm install google-api-utility 操作 ● apiutil.init(config) ● apiutil.request(options, callback)
  21. 21. Google Oauth2 Web Server Application Service Account Request token Authorization code Exchange code for token Token response Use token to call Google API Your App Google Servers User login ! & consent Use r Create and sign JWT Use JWT to request token Token response Use token to call Google API Server App Google Servers
  22. 22. Service Account Configures Authenticate can be cache and auto refresh Others… Request… 
 Request… Request… Create and sign JWT Use JWT to request token Token response Use token to call Google API Server App Google Servers
  23. 23. 免不了的⿇麻煩....
  24. 24. Service Owner在Google的設定
  25. 25. Prepare Authentications
  26. 26. $ openssl pkcs12 -in privatekey.p12 -out privatekey.pem -nocerts $ openssl rsa -in privatekey.pem -out key.pem Generate key.pem $ openssl pkcs12 -in privatekey.p12 -nodes -nocerts > key.pem OR
  27. 27. var auth = require('google-api-utility')
 auth.init({
 scope: 'https://www.googleapis.com/auth/bigquery https:// www.googleapis.com/auth/cloud-platform',
 client_secret: '/path-to-client_secret.json',
 key_pem: '/path-to-key.pem'
 }); 使⽤用範例 - 初始化 此處需要綁定所欲呼叫的API 相關授權之Scope位置 設定client_secret.json 與相關pem檔案位置,供jwt 運算使⽤用
  28. 28. 使⽤用範例 - 呼叫BigQuery var request = auth.request; var bqurl = 'https://www.googleapis.com/bigquery/v2/projects/%s/ datasets';request({
 url: util.format(bqurl, project),
 method: 'GET'
 }, function(err, req, doc){
 // implements
 }); 同原request模組操作⽅方式 結合原request模組之 function,供api呼叫使⽤用
  29. 29. Tools Auth Req? API Web App Service
  30. 30. Google API Explore HTTPS://developers.google.com/apis-explorer/
  31. 31. Google API Explore - Query
  32. 32. Google API Explore - Auth Operation Scope
  33. 33. Google API Explore - Response
  34. 34. Tools Auth Req? API SDK Web App Service
  35. 35. Idea... ● bigquery.init({...configurations...}) ● bigquery.[what].[do](...) - bigquery.dataset.list(....) - bigquery.table.load(..., callback)
  36. 36. bigquery module 基本資訊 ● https://github.com/peihsinsu/bigquery 安裝 ● npm install bigquery 操作 ● bigquery.init(config) ● bigquery.[category].[operation](options, callback)
  37. 37. 重新包裝 - bigquery模組 var bq = require('bigquery')
 , prjId = 'your-bigquery-project-id'; bq.init({
 client_secret: '/path/to/client_secret.json',
 key_pem: '/path-to-key.pem'
 }); bq.dataset.list(prjId, function(e,r,d){
 if(e) console.log(e);
 console.log(JSON.stringify(d));
 }); bigquery模組可參考:https://github.com/peihsinsu/bigquery 透過bq呼叫某個操作之下的function
  38. 38. Source Code... var util = require('util') , auth = require('google-api-utility') , request = auth.request , _ = require('underscore') exports.init = auth.init; exports.job = { token: '', listds : function(project, cb){ var bqurl = 'https://www.googleapis.com/bigquery/v2/projects/%s/datasets'; request({ url: util.format(bqurl, project), method: 'GET' }, cb?cb:auth.commonCb); }, … (skip) } 封裝相同類別的api在⼀一起! ex: job相關的放在job物件中
  39. 39. NOT ONLY Node.js Love JavaScript…..
  40. 40. The fancy integrate with Sheet
  41. 41. More and more chart….
  42. 42. Charting in Apps Script
  43. 43. Google provided node.js tool
  44. 44. Operation with googleapis var googleapis = require('googleapis'); ! var jwt = new googleapis.auth.JWT( '429100748......hv3@developer.gserviceaccount.com', '/path/to/key.pem', null, [ 'https://www.googleapis.com/auth/bigquery', 'https://www.googleapis.com/auth/cloud-platform' ]);
  45. 45. Operation with googleapis jwt.authorize(function(err, tokens) { googleapis.discover('bigquery', 'v2').execute(function(e,client) { if(e) console.log(e); else client.bigquery.datasets.list(param).withAuthClient(jwt) .execute( function(err, response) { if(err) console.log(err); console.log(JSON.stringify(response)); }); }); });
  46. 46. Full Code
  47. 47. HTTP://OpenNodes.arecord.us OR Sample Copy Run!
  48. 48. http://micloud.tw http://about.me/peihsinsu
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×