Slide URL: https://omohayui.github.io/gtuggirls35/ Event URL: https://gtuggirls.connpass.com/event/67808/

1. BigQuery Data Studio
GTUG Girls #35
Created by Yui Takeuchi ( )@omohayui
1

2. 2 . 1

3. Yui Takeuchi
So wareEngineer@DeNA
FrontEnd (js,html5,sass)
ServerSide (Golang,Perl,PHP)
Infra (GCP ,Linux ,MySQL,Oracle)
2 . 2

4. Important
2 . 3

5. PR
2 . 4

6. BigQuery
1PB 10
" "
Google Cloud Platform
3

7. ▼
https://cloudplatform-jp.googleblog.com/2016/05/bigquery-dataproc.html
4 . 1

8. http://www.publickey1.jp/blog/12/bigquery_1.html
4 . 2

9. 1
↓
4 . 3

10. 4 . 4

11. root server
leaf servers
4 . 5

12. https://cloud.google.com/bigquery/what-is-bigquery
5 . 1

13. GCP
5 . 2

14. BigQuery
10GB
1TB ( 1TB $5)
6

15. SLECT *
dryrun
https://cloud.google.com/bigquery/pricing?hl=ja#free
7

16. Google DataStudio
Google DataStudio
8

17. Google
Adwords
BigQuery
CloudSQL
Google
Google
etc...
9

18. Beta
10

19. NPB( )
(
11

20. MLB
MLB
12

21. The Lahman Baseball Database
csv
(2016 ) csv
http://www.seanlahman.com/baseball-archive/statistics/
2016 - comma-delimited version
13

22. 14

23. GCP
YES ▶
NO ▼
15 . 1

24. GCP
15 . 2

25. GCP
Analytics for MLB
ID analytics-for-mlb
※ ID
analytics-for-mlb-{name}
https://console.cloud.google.com/project
16

26. BigQuery API
※
https://console.cloud.google.com/flows/enableapi?apiid=bigquery
17

27. BigQuery
BigQuery
18

28. Dataset
BigQuery
https://bigquery.cloud.google.com/queries/analytics-for-mlb
19

29. Create New Dataset
Dataset ID mlb_data
Data location US
Data expiration Never
20

30. Table
or Create New Table
21

31. 1. baseballdatabank-2017.1.zip
2. core CSV
Master.csv -
Batting.csv -
Pitching.csv -
Teams.csv -
22

32. 23

33. Source Data Create from Source
Location File upload
Choose file [ ]
File format CSV
Table name
Schema Automatically detect
Header rows to skip 1
※ CSV 1 Load
Create Table
24

34. Tips
2
Repeat job Select Previous Job
Choose File Table Name
25

35. Recent Jobs
Load mlb_data dataset
Table
Master
26

36. Table Schema
STRING INTEGER Automatically detect
field

37. 27

38. SQL
Legacy SQL
SQL
Standard SQL
Standard SQL
28

39. Legacy SQL
Master Query Table
SELECT fileld
Tips Schema field
playerID, nameFirst, nameLast field
RUN QUERY
29

40. Query
SELECT playerID, nameFirst, nameLast
FROM [analytics-for-mlb:mlb_data.Master] LIMIT 1000
30

41. Standard SQL
1. Show options
2. SQL Dialect Use Legacy SQL
Tips SQL "#standardSQL" OK
3. Table
31

42. Query
#StandardSQL
SELECT playerID, nameFirst, nameLast
FROM `analytics-for-mlb.mlb_data.Master` LIMIT 1000
32

43. Batting Table 2010-2016
COMPOSE QUERY Query
https://cloud.google.com/bigquery/query-reference?
hl=ja
33

44. Query
#standardSQL
SELECT yearID, lgID, teamID, SUM(HR) as totalHR
FROM `analytics-for-mlb.mlb_data.Batting`
WHERE yearID BETWEEN 2010 AND 2016
GROUP BY yearID, lgID, teamID
ORDER BY yearID, lgID, totalHR
34

45. 2010-2016 K/BB
10
Pitching, Master
K/BB ÷
1
3.5
K/BB
35

46. Query
#standardSQL
-- K/BB
SELECT
yearID,
CONCAT(nameFirst, " ", nameLast) AS name, -- 選手指名
CASE WHEN BB = 0 THEN 0 ELSE SO/BB END AS kbb -- 0 割
FROM `analytics-for-mlb.mlb_data.Pitching` AS p
LEFT OUTER JOIN(
SELECT
playerID, nameFirst, nameLast
FROM `analytics-for-mlb.mlb_data.Master`
) AS m
ON p.playerID = m.playerID
WHERE
yearID BETWEEN 2010 AND 2016
AND CASE WHEN BB = 0 THEN 0 ELSE SO/BB END >= 10
36

47. CSV
Download as CSV
37

48. 2008-2016 30
Batting, Master, Team
Master
Team
38

49. Query
Download as CSV
#standardSQL
SELECT
bm.yearID AS year, t.lgID AS league, t.divID AS division, t.name AS
FROM(
(
SELECT
yearID, teamID, SUM(H) AS totalHit -- 合計 数
FROM `analytics-for-mlb.mlb_data.Batting` as b
JOIN(
-- Master 選手 生年月日 取得
SELECT
birthYear, playerID
FROM `analytics-for-mlb.mlb_data.Master`
) AS m
ON b.playerID = m.playerID
WHERE
i 歳以下
39

50. View
Query View
Save View
Table ID total_hit_u30

51. 40

52. Google Data Studio
or
https://datastudio.google.com/
41

53. or
42

54. TotalHits U30 Group By Team
> BIgQuery >
> Analytics for MLB > mlb_data
> total_hit_u30 View
BigQuery
43 . 1

55. 43 . 2

56. 44

57. https://datastudio.google.com/navigation/datasources
45

58. or
TotalHits U30 By Team
https://datastudio.google.com/navigation/reporting
46

59. TotalHits U30 Group By Team
ok

60. 47

61. 30
TotalHits U30 Group By Team
year
team
totalHitU30
totalHitU30,
48 . 1

62. 48 . 2

63. TotalHits U30 Group By Team
year
team
rank
rank,
49 . 1

64. 49 . 2

65. >
league division
50

66. league
51

67. division
52

68. 10
( )
2 1
( )
53

69. AL ×
2009 2012 1 , 2 , 1 , 1
2015
2011 2014
2011
54 . 1

70. 54 . 2

71. MLB
( )
NPB
1 ( )
MLB
55

72. BigQuery
-
Google DataStudio
Google
56