The Magic of LINE 購物 Testing
Service QA Engineer
winter.hung@linecorp.com
cheng.guo@linecorp.com
What is LINE 購物
上線時間:2018年1月18日
月均訪客數:900萬人
回購率:40%
轉換率:12%
平台電商夥伴:20家
品牌電商夥伴:370家
中小企業夥伴 : 1300 家
跨境電商夥伴: 1 家
可搜尋商品數:約2600萬件
What is LINE 購物
How to test in LINE 購物?
UI tests
API tests
Unit tests
Automation Testing Pyramid
- Most of the test cases were unit tests
- API tests access all the services using REST
- UI tests are always expensive to maintain
How to test in LINE 購物?
Measure data is reliable
Analyse cost of doing any test result
Gathering acceptance criteria
Improve practical solutions
Control lean principles
How to test in LINE 購物?
Measure data is reliable
Sitemap.xml APIParser
How to test in LINE 購物?
Analyse cost and response time
How to test in LINE 購物?
Gathering acceptance criteria
1 Test 30 times
2
3
Response time < 500 ms
JSON Schema Pass
How to test in LINE 購物?
Improve practical solutions
Continuous
Testing
(24HR)
monitor
(30min)
LINE
Notify
Health
Check
(every minute)
How to test in LINE 購物?
Control lean principles tools
How to test API in LINE 購物?
Front-End
Just-API GraphiQL
Back-End
PyRestTest Swagger
Just-API
What is Just-API?
Just-API is a specification based framework to test GraphQL APIs. It
takes API suite specification from YAML files.
There are several tools and companies available that offer nice UI to test
APIs, but none of them really handle complex API automation flows. So why
not a framework that allows users to test complex API scenarios with
minimal coding.
What is GraphQL?
GraphQL is a query language for your API, and a server-side runtime for
executing queries by using a type system you define for your data.
GraphiQL is an in-browser tool for writing, validating, and testing GraphQL queries.
GraphQL server operates on a single URL/endpoint, usually /graphql, and all
GraphQL requests for a given service should be directed at this endpoint.
GraphQL Query
Type getLocation(ip: String!){
Type country{
iso_code: String
}
}
}
{
getLocation(ip: "8.8.8.8") {
country{
iso_code
}
}
}
{
"data": {
"getLocation": {
"country": {
"iso_code": "US"
}
}
}
}
Schema Query Server Response
Just-API Test
meta:
name: GraphQL location service
configuration:
host: api.graphloc.com
scheme: https
specs:
- name: Get Location of a given ip address
request:
method: post
path: /graphql
headers:
- name: content-type
value: application/json
payload:
body:
type: json
content:
query: >
{
getLocation(ip: "8.8.8.8") {
country {
iso_code
}
}
}
Create a YAML
response:
status_code: 200
json_data:
- path: $.data.getLocation.country.iso_code
value: US
1
2
3
4
5
6
7
8
GraphQL/Just-api
$ ./node_modules/.bin/just-api
1
2
3
PythonRestTest
What is Pyresttest?
• A REST testing and API microbenchmark tool
• Tests are defined in basic YAML or JSON config files, no code needed
• Minimal dependencies (pycurl, pyyaml, optionally future)
• Returns exit codes on failure
• Logic is written and extensible in Python
REST API Query
https://jsonplaceholder.typicode.com/todos/1
{ "userId": 1, "id": 1, "title": "delectus aut autem", "completed": false }
Query
Server Response
Pyresttest
- config:
- testset: "Test"
- test:
- name: "GET todos 1"
- url: "/todos/1"
- expected_status: [200]
- validators:
- json_schema: {schema: {file:
'smoke.schema'}}
Create a YAML Create a Schema
{
"properties": {
"userId": {
"type": "integer"
},
"id": {
"type": "integer"
},
"title": {
"type": "string"
},
"completed": {
"type": "boolean"
}
}
}
1
2
3
4
Pyresttest
$ python resttest.py https://jsonplaceholder.typicode.com examples/smoke.yaml
Test Group :: Default :: NO Of Test Cases: 1
Test Name: GET todos 1 : SUCCEEDED
Tests PASSED: 1; Tests FAILED: 0; Tests EXECUTED: 1
Test Group Default SUCCEEDED: : 1/1 Tests Passed!
Total Tests PASSED: 1; Total Tests FAILED: 0; Total Tests EXECUTED: 1
Magic
Measure A GA In Cypress
Cypress
• Run	in	the	browser,	faster
• Easy	to	debug,	using	CLI	or	Chrome	Devtools
Cypress - Example
Fixture
Load a fixed set of data located in a file.
Route
Spy
UnLogin Login TOS
Flow
Stub UI GA
Login
UnLogin
PageView
Event
VerifyGA Example
Login
Stub TOS
GA
Spy the GA
Cypress – GA Util
Cypress – Run in CLI
Thank you!
Service QA Engineer
winter.hung@linecorp.com
cheng.guo@linecorp.com

The Magic of LINE 購物 Testing

  • 1.
    The Magic ofLINE 購物 Testing Service QA Engineer winter.hung@linecorp.com cheng.guo@linecorp.com
  • 2.
    What is LINE購物 上線時間:2018年1月18日 月均訪客數:900萬人 回購率:40% 轉換率:12% 平台電商夥伴:20家 品牌電商夥伴:370家 中小企業夥伴 : 1300 家 跨境電商夥伴: 1 家 可搜尋商品數:約2600萬件
  • 3.
  • 4.
    How to testin LINE 購物? UI tests API tests Unit tests Automation Testing Pyramid - Most of the test cases were unit tests - API tests access all the services using REST - UI tests are always expensive to maintain
  • 5.
    How to testin LINE 購物? Measure data is reliable Analyse cost of doing any test result Gathering acceptance criteria Improve practical solutions Control lean principles
  • 6.
    How to testin LINE 購物? Measure data is reliable Sitemap.xml APIParser
  • 7.
    How to testin LINE 購物? Analyse cost and response time
  • 8.
    How to testin LINE 購物? Gathering acceptance criteria 1 Test 30 times 2 3 Response time < 500 ms JSON Schema Pass
  • 9.
    How to testin LINE 購物? Improve practical solutions Continuous Testing (24HR) monitor (30min) LINE Notify Health Check (every minute)
  • 10.
    How to testin LINE 購物? Control lean principles tools
  • 11.
    How to testAPI in LINE 購物? Front-End Just-API GraphiQL Back-End PyRestTest Swagger
  • 12.
  • 13.
    What is Just-API? Just-APIis a specification based framework to test GraphQL APIs. It takes API suite specification from YAML files. There are several tools and companies available that offer nice UI to test APIs, but none of them really handle complex API automation flows. So why not a framework that allows users to test complex API scenarios with minimal coding.
  • 14.
    What is GraphQL? GraphQLis a query language for your API, and a server-side runtime for executing queries by using a type system you define for your data. GraphiQL is an in-browser tool for writing, validating, and testing GraphQL queries. GraphQL server operates on a single URL/endpoint, usually /graphql, and all GraphQL requests for a given service should be directed at this endpoint.
  • 15.
    GraphQL Query Type getLocation(ip:String!){ Type country{ iso_code: String } } } { getLocation(ip: "8.8.8.8") { country{ iso_code } } } { "data": { "getLocation": { "country": { "iso_code": "US" } } } } Schema Query Server Response
  • 16.
    Just-API Test meta: name: GraphQLlocation service configuration: host: api.graphloc.com scheme: https specs: - name: Get Location of a given ip address request: method: post path: /graphql headers: - name: content-type value: application/json payload: body: type: json content: query: > { getLocation(ip: "8.8.8.8") { country { iso_code } } } Create a YAML response: status_code: 200 json_data: - path: $.data.getLocation.country.iso_code value: US 1 2 3 4 5 6 7 8
  • 17.
  • 18.
  • 19.
    What is Pyresttest? •A REST testing and API microbenchmark tool • Tests are defined in basic YAML or JSON config files, no code needed • Minimal dependencies (pycurl, pyyaml, optionally future) • Returns exit codes on failure • Logic is written and extensible in Python
  • 20.
    REST API Query https://jsonplaceholder.typicode.com/todos/1 {"userId": 1, "id": 1, "title": "delectus aut autem", "completed": false } Query Server Response
  • 21.
    Pyresttest - config: - testset:"Test" - test: - name: "GET todos 1" - url: "/todos/1" - expected_status: [200] - validators: - json_schema: {schema: {file: 'smoke.schema'}} Create a YAML Create a Schema { "properties": { "userId": { "type": "integer" }, "id": { "type": "integer" }, "title": { "type": "string" }, "completed": { "type": "boolean" } } } 1 2 3 4
  • 22.
    Pyresttest $ python resttest.pyhttps://jsonplaceholder.typicode.com examples/smoke.yaml Test Group :: Default :: NO Of Test Cases: 1 Test Name: GET todos 1 : SUCCEEDED Tests PASSED: 1; Tests FAILED: 0; Tests EXECUTED: 1 Test Group Default SUCCEEDED: : 1/1 Tests Passed! Total Tests PASSED: 1; Total Tests FAILED: 0; Total Tests EXECUTED: 1
  • 23.
  • 24.
  • 25.
  • 26.
    Fixture Load a fixedset of data located in a file.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 38.
  • 40.
    Thank you! Service QAEngineer winter.hung@linecorp.com cheng.guo@linecorp.com