The document discusses Node.js and Google Cloud Storage. It covers topics like using OAuth2 to authenticate with JSON Web Tokens and service accounts, uploading files via simple, multipart, and resumable upload methods, and managing file metadata, access control lists, versions, and directories without a true folder structure in Cloud Storage. The author reflects on lessons learned like ensuring proper permissions when accessing buckets and the value of sharing knowledge gained from experimenting with Google services.

1. Node.js 與 Google
Cloud Storage
多麼痛的領悟

2. 關於我
• Ian Wu
• 瘋⼈人院院⻑⾧長
• 頑⽪皮⼯工坊 Backend Engineer
• http://blog.ianwu.tw/about-me/

3. Why google cloud storage
• 內建 CDN
• Google Cloud Storage behaves essentially like
a Content Delivery Network (CDN) with no work on
your part because publicly readable objects are, by
default, cached in the Google Cloud Storage network.
• try try see
• try 到死
• 有 USD 500 的 credit

4. OAuth2
• JWT (JSON Web Token)
• Google Cloud console
• credential: service account
• covert p12 > pem
Authentication - Google Cloud Storage — Google Cloud Platform
https://cloud.google.com/storage/docs/authentication#service_accounts

5. OAuth2
• Get token
• payload
{
iss: '460520686343-k6tfn73sentmh0ss5nu67kniorbcta8n@developer.gserviceaccount.com',
scope: 'https://www.googleapis.com/auth/devstorage.full_control',
aud: 'https://accounts.google.com/o/oauth2/token',
exp: 1418280623,
iat: 1418280563
}
• jwt sign
// sign with RSA SHA256
var cert = fs.readFileSync('google_cloud_key.pem'); // get private key
var claim = jwt.sign(payload, cert, {
Get Google JWT token.
https://gist.github.com/onlinemad/28341a343ecde186a410
algorithm: 'RS256'
});

6. OAuth2
• token
{
access_token: 'ya29.2QA9sZg_YtCTGJf1d6Vzxr_4ypioiaIdHJBmgxq6b1HsJuAPODCHnCvt',
token_type: 'Bearer',
expires_in: 3600
}
• 使⽤用 token
headers: { Authorization: 'Bearer ' + token.access_token }

7. Upload URI
• Upload URI, for media upload requests
• upload/storage/v1/b/bucket/o
• Metadata URI, for metadata-only requests:
• storage/v1/b/bucket/o
• APIs Explorer currently supports metadata
requests only.

8. Upload method
• simple
• 就 post 上傳檔案
• multipart（推薦使⽤用）
• 可以連 metadata ⼀一起上傳
• request 某⼀一個版本以上才有⽀支援
• resumable
• 沒⽤用過
• node-youtube-resumable-upload
https://github.com/grayleonard/node-youtube-resumable-upload

9. multipart
• request
var url = 'https://www.googleapis.com/upload/storage/v1/b/yourbucket/o?' +
qs.stringify(querystring);
request.post({
preambleCRLF: true,
postambleCRLF: true,
url: url,
multipart: [
{ 'Content-Type': 'application/json', body: JSON.stringify(metadata) },
{ body: __newFile }
],
headers: { Authorization: 'Bearer ' + token.access_token }
});

10. multipart
• body
{
cacheControl: 'public, max-age=604800',
acl: [{
entity: 'allUsers',
role: 'READER'
}, {
entity: 'project-owners-692227494718',
role: 'OWNER'
}]
}
• query string
• 不能跟 Request body ⼀一起⽤用

11. Directory structure
• ⼀一切都是平的
• 跟 s3 ⼀一樣
• 所以沒有建⽴立 folder 這件事情
• name = foo/bar.jpg;

12. Directory structure
• simple
• /o?name=foo%2Fbar.jpg
• multipart
• body.name = foo/bar.jpg

13. Access URL
• Standard（推薦）
• storage.googleapis.com/<bucket>/<object>
• <bucket>.storage.googleapis.com/<object>
• CNAME
• travel-maps.example.com CNAME c.storage.googleapis.com
• no ssl
• Cookie-based Authentication
• 沒⽤用過

14. Versioning
• 預設是關掉的
➜ ~ gsutil versioning get gs://onlinemad-versioning
gs://onlinemad-versioning: Suspended
➜ ~ gsutil versioning set on gs://onlinemad-versioning
Enabling versioning for gs://onlinemad-versioning/...
➜ ~
• qs + generation
{
"kind": "storage#object",
"id": "onlinemad-dev/uploaded.jpg/1418291876469000",
"selfLink": "https://www.googleapis.com/storage/v1/b/onlinemad-dev/o/uploaded.jpg",
"name": "uploaded.jpg",
"bucket": "onlinemad-dev",
"generation": "1418291876469000",
"metageneration": "1",
"contentType": "image/jpeg",
"updated": “2014-12-11T09:57:56.468Z”,
}

15. ACL
[
{
"entity": "project-owners-460520686343",
"projectTeam": {
"projectNumber": "460520686343",
"team": "owners"
},
"role": "OWNER"
},
{
"entity": "project-editors-460520686343",
"projectTeam": {
"projectNumber": "460520686343",
"team": "editors"
},
"role": "OWNER"
},
{
"entity": "project-viewers-460520686343",
"projectTeam": {
"projectNumber": "460520686343",
"team": "viewers"
},
"role": "READER"
},
{
"entity": "user-00b4903a9745459d3abf193213c0f30d5dea50ee7e3e318007a7edfaecb646e5",
"entityId": "00b4903a9745459d3abf193213c0f30d5dea50ee7e3e318007a7edfaecb646e5",
"role": "OWNER"
}
]

16. ACL
• 我需要 public read
• 所以request.post({
preambleCRLF: true,
postambleCRLF: true,
url: url,
multipart: [{
'Content-Type': 'application/json',
body: JSON.stringify({
name: 'acl_multipart_upload_public_read.jpg',
acl: [{
entity: 'allUsers',
role: 'READER'
}]
})
}, {
body: data
}],
headers: {
Authorization: 'Bearer ' + token.access_token
}
})

17. ACL
➜ ~ gsutil acl get gs://onlinemad-dev/
acl_simple_upload_public_read.jpg
AccessDeniedException: Access denied. Please ensure you
have OWNER permission on gs://onlinemad-dev/
acl_simple_upload_public_read.jpg.

19. 這是 feature 不是 bug
這是 feature 不是 bug
這是 feature 不是 bug

20. ACL

22. ACL
request.post({
preambleCRLF: true,
postambleCRLF: true,
url: url,
multipart: [{
'Content-Type': 'application/json',
body: JSON.stringify({
name: 'acl_multipart_upload_public_read_add_owner.jpg',
acl: [{
entity: 'allUsers',
role: 'READER'
}, {
entity: 'project-owners-460520686343',
role: 'OWNER'
}]
})
}, {
body: data
}],
headers: {
Authorization: 'Bearer ' + token.access_token
}
})

23. 我的領悟

24. 「還沒有⼈人分享 Google Service 時，
請勿輕易嘗試」
– Ian Wu

25. 「當你試了 Google Service 時，
請來分享」
– Ian Wu

26. 謝謝⼤大家