API Design Patterns Part 3 Fundamentals

1. Part 3 Fundamentals
8. Partial Updates and Retrievals
Rick Hwang
2022/04/20
1
● Why we might want to update or retrieve only
specific pieces of a resource
○ 鄉民都只對特殊資訊感興趣
● How best to communicate the fields of interest
to the API service
○ 鄉民都只對自己有利的感興趣
● How to handle individual items in complex
fields, such as maps and interfaces
● Whether to support addressing individual items
in a repeated field (e.g., arrays)
● Defining default values and handling implicit
field masks
● How to deal with invalid field specifications

2. 2
● Why we might want to update or retrieve only
specific pieces of a resource
○ 鄉民都只對特殊資訊感興趣
● How best to communicate the fields of interest
to the API service
○ 鄉民都只對自己有利的感興趣
● How to handle individual items in complex
fields, such as maps and interfaces
● Whether to support addressing individual items
in a repeated field (e.g., arrays)
● Defining default values and handling implicit
field masks
● How to deal with invalid field specifications

3. 8.1.1 Partial retrieval
○ 取的整個 Resource 是必要的？
○ 如果 Resource 的資料結構有數百的 fields or
subfields？
○ 需要針對特殊的情境，限制取得的 fields，例如：
○ 很小的 IoT 設備 - 硬體資源限制
○ PII - 資安政策 (Member Service)
○ 不是所有 Client 都需要取得全部內容
8.1 Motivation
8.1.2 Partial update
● 能否針對個別的 fields 修改？
● 確保資料更新後的一致性
● 標準方法不夠用的時候怎麼辦？
3

4. 4
aws ec2 describe-instances

5. 5
Protocol Data Structure
Web Services HTTP/1.1 SOAP / XML
REST HTTP/1.1 JSON
gRPC HTTP/2 Protobuf
補
充

6. Note: User1 與 User2 針對同一個 Object，循序的改動不一樣的屬性。
6

7. 7
1. Resource 新增
description field，也提供
新的 client
2. 舊的 client 持續在使用
=> 想像是 HTTP Client
or SDK 那種
3. 沒有考慮 back
compatible 就會出現
data loss
1
2
3

8. 1. 循序性的寫入，需要一致性檢查 (consistency checks)、或者 locking
2. versioning and compatiablity, see chatper 24
a. support new features
b. fix bugs
c. to add new fields to resources while still considering the new version backward compatible
核心問題
8

9. 1. Enabling partial retrievals => GET
2. Enabling partial updates => PATCH
8.2 Overview - Goals
9
a field mask is just a collection of strings, but these
strings represent a list of fields that we’re interested in
on a given resource.

10. The Field Mask
● Enabling partial retrievals and
pr
10

11. 11

12. 12

13. 8.3 Implementation
How we transport the field mask without causing any significant
disruption to the standard requests that defined in chapter 7.
13

14. 8.2.1 Transport
two constraints:
1. GET: no body to the request permitted
(and many HTTP servers will strip it out if
one is provided)
2. PATCH: resource-oriented design
dictates that the body of a PATCH
request must be the resource
representation being updated
two potential places:
1. headers
2. query strings: how the repeated query
string parameters are interpreted will
depend on the HTTP server in use
14

15. 15
GET with query string, body, header PATCH with query string, body, header

16. 16
HTTP Methods Query String Body HTTP Header
GET v v (?) v
PATCH v v v
Node.JS (express)
HTTP Methods Query String Body HTTP Header
GET v ?? v
PATCH v
ASP.NET (Kestrel)

17. 17
ASP.NET (C#) ?fieldMask=a&fieldMask=b string

18. have to augment the request
messages for both the standard
update method and standard get
method.
18

19. Does this idea of partial updates
and retrievals extend inside nested
structures?
Or does it only apply at the very
top level, effectively treating
resources as completely flat
structures?
8.3.2 Maps and nested interface
19

20. How to represent field:
maxMessageCount
● JSON Path: $.loggingConfig.maxMEssageCount
20

21. 1. Separate parts of a field specification
must use a dot character (.) as a
separator.
2. All fields of a nested message may be
referred to using an asterisk character
(*).
3. Map keys should always be strings.
4. All parts of a field specification that
can’t be represented as an unquoted
string literal must be quoted using
backtick characters (`).
5. A literal backtick character may be
escaped by using two backtick
characters (``)
Rules for field mask to specify nested fields.
21

22. JSON Path
XPath (for XML)
https://github.com/json-path/JsonPath
https://zh.wikipedia.org/zh-tw/XPath
→ XSTL (時代的眼淚)
22
補
充

23. 23

24. 8.3.3 Repeated fields
● prefix of "administrators.*." as a way of
saying “For each administrator
● retrieve only the name of an administrator,
use a field mask value of
"administrators.*.name"
24

25. 8.3.4 Default values
the goal of a default value is to do “the right thing” for the user.
GET: standard get method, the default is almost always the complete list of fields
available on a resource.
標準 GET 會取得所有的 Fields
There is an exception to this guideline. In cases where a resource has fields
which, for whatever reason, would cause a fundamentally worse user experience
for API consumers, these fields should be removed from the default of the field
mask being left unset.
如果有一些 Fields 會導致 API 使用者的使用體驗不好，這些 fields 應該從 field mask 的預設中 被移除
使用者有興趣的時候，自行透過 fieldMask 指定 Fields 取得。
25

26. GET /api/chatrooms?fieldMask=participants
{
"1": {
"id": 1,
"title": "My Chat!",
"description": "喇低賽",
"participants": [] ⇐ 裡面有 10k 個鄉民，要處理很久
},
"2": {
"id": 2,
"title": "Your Chat!",
"description": "低賽喇",
"participants": [] ⇐ 裡面有 100k 個鄉民，要處理很久
}
}
Design: w/o participants in default
GET /api/chatrooms <Default>
{
"1": {
"id": 1,
"title": "My Chat!",
"description": "喇低賽",
},
"2": {
"id": 2,
"title": "Your Chat!",
"description": "低賽喇",
}
}
26
w/o participants

27. 預設 = 製造了淺規則？
● 文件要明確說明，預設 fieldMask 的行為
● 即使有文件，還是很容易誤解。
● 不想讓使用者猜測哪些 fields 是預設的？
● 使用 asterick (“*”) 表示返回所有的 fields
● 取得資源使用預設 fieldMask 是用，但是更
新資源，Default FieldMask 就沒啥用
GET /api/chatrooms?fieldMask=*
{
"1": {
"id": 1,
"title": "My Chat!",
"description": "喇低賽",
"participants": [] ⇐ 裡面有 10k 個鄉民，要處理很久
}
}
GET /api/chatrooms?fieldMask=id,title
{
"1": {
"id": 1,
"title": "My Chat!",
}
}
27
個
人
觀
點
DesignA: 特定欄位的正向表列？
Design B: 全部欄位的正向表列？

28. 預設 = Policy (政策 / 慣例 / 文化 / 潛規則)
● 衝突來源
● 怪異
● 高雄式左轉
● 習慣遲到
● 公司的淺規則
● 默契
● 共識
● 守法
● 提前到、準時
● 91APP 的政策
28
個
人
觀
點

29. 預設
● 專業領域 (Domain Know How)
● 企業文化
● 鄉民文化
● 潛在共識
● 生態系
○ .NET method name starts with uppercase
○ Java method name starts with lowercase
29
個
人
觀
點

30. The goal of a default value is to do “the right thing” for the user.
啥叫做正確的事情？ ← 這句話不夠具體
● 給不同使用者，最必要的資訊。
○ Chatroom 的參與者，需要怎樣的聊天室資訊？
○ Chatroom 的管理者，需要怎樣的聊天室資訊？
● 91APP 的專業判斷 => Domain Expert 必須思考的 => PO + Team
○ 對於 會員 (消費者) 而言，訂單 (Order) 要呈現什麼資訊？
○ 對於 91APP 的客戶 而言，訂單 (Order) 要呈現什麼資訊？
○ 對於 91APP 營管 而言，訂單 (Order) 要呈現什麼資訊？
○ 對於 91APP 合作夥伴 而言，訂單 (Order) 要呈現什麼資訊？
30
個
人
觀
點

31. What does an unset field mask mean on a standard update method (using an
HTTP PATCH method)?
8.3.5 Implicit field masks
31

32. 32

33. What does an unset field mask mean on a standard update method (using an
HTTP PATCH method)?
Most commonly, HTTP PATCH indicates an intent to update the resource with only
the data provided in the body of the request.
8.3.5 Implicit field masks
33

34. 8.3.5 Implicit field masks
34
PATCH/chatRooms/1
{"description":"Newdescription"}
// 只更新 id=1 的 description

35. ● dynamic data structure => object 型別
○ ch5 提到的是 static data structure
● settings.test 有三種值：
○ “STRING_VALUE”
○ null
○ 不存在
● 如果要刪除 settings.test 要怎麼做？
○ 不是 null
○ “undefined” 不是 JSON 資料型別
註：“undefined” 是語言層級的型別。
8.3.6 Updating dynamic data structures
35

36. 36

37. 8.3.6 Updating dynamic data structures
● settings.test 有三種值：
○ “STRING_VALUE”
○ null
○ 不存在 => undefined 的意思
● 如果要刪除 settings.test 要怎麼做？
○ 不是 null
● “undefined” 不是 JSON 資料型別，是 JS
語言層級的型別。
○ python 可能是 None
37

38. 透過 FieldMask
38

39. 針對 “dynamic data structures” 的理解 Q_Q
39
個
人
觀
點
v1 v2

40. 8.3.7 Invalid fields=> 參閱 CH24 Versioning and compatibility
40
個
人
觀
點
=>
ch24
v1 v2

41. ● the goal is still quite limited:
a. to minimize unnecessary data transfer and
b. allow fine-grained modification of API resources.
● consider field masks and partial retrievals in particular as SQL-like querying tools to fetch specific
data
8.4 Trade-offs
41

42. 這章講的概念，是不是很像某個東西的自幹版？
42

43. Bo 先生：啥時候才有 GraphQL 的 Design Guideline 啊？
43

44. 8.4.1 Universal support
44
● NOT at all a requirement that every API must support partial retrieval.
● more an issue of concurrency than resource size and complexity

45. 是否需要刻個 Libraries 實踐這些通用規則？
C# 開發者表示：這還要問？
Node.JS 開發者表示：我也要
Java 開發者表示：我也要
PHP 開發者表示：我。。。
45
個
人
觀
點

46. 8.4.2 Alternative implementations
● JSON Patch (RFC6902), (npm)
● JSON Merge Patch (RFC7396), (npm)
● 哪裡有類似的概念：K8s 實作
○ kubectl apply -f xxx.yaml
46

47. 1. Partial retrieval is particularly important in cases where resources are large or
clients consuming resource data have limited hardware. => POS
2. Partial updates are critical for fine-grained updates without worrying about conflicts.
3. Field masks, which support ways to address fields, nested fields in interfaces, and
map keys, should be used to indicate the fields that should be retrieved or updated.
4. Field masks should NOT provide a mechanism to address items in array fields by
their position or index in that field.
5. By default, field masks should assume a value of everything for partial retrievals and
an implicit field mask (8.3.5) for partial updates.
6. If fields are invalid, they should be treated as though they do exist but have a value
of undefined.
Summary
47