This document contains an agenda for a presentation on BigQuery. The agenda includes sections on why big data is important, getting started with BigQuery including enabling APIs and creating projects, loading sample data and running queries, and integrating BigQuery with other tools through its API. It also provides examples of different types of queries that can be run on BigQuery including simple aggregates, complex processing, nested selects, and joins.

1. Baltimore

2. BIGQUERY
Code Labs
DevFest DC - 2017

3. SELECT tweet.@dineshMD,email.dinesh@acarin.net from Contacts
Dinesh Ganesan

4. Why do we need Big Data tools?
BigQuery - Show the power
Getting Started
Let’s Play - Analyzing USA Census Data within few seconds.
Loading data into BigQuery
Talking to BigQuery
Q&A
4
Agenda?

5. Why Big Data?
5
3. Why “Big Data” Is a Big Deal - Harvard Magazine
Enabled Biologist to compute the genome sequence within 24 hours rather waiting for a full year.
- Reducing Cost & Time
Enabled NASA to provide Climate Modeling as Service(MERRA) using data since 1979 ( few hundred peta bytes )
- Better Living and New business opportunities.
2012 – 100 Billion Google Searches every month. Leverages 30 trillion URLs for data collection
- Optimal Knowledge from huge data size.
2. MERRA: MODERN ERA-RETROSPECTIVE ANALYSIS FOR RESEARCH AND APPLICATIONS
1. Google Search - Wikipedia

6. Getting Started - GCloud Registration
1. https://console.cloud.google.com
1
2 3
6

7. Getting Started - Project Creation
1
3
7
2
4
Refresh the page if you don’t see the project name

8. Getting Started - Enable API
8
5

9. Getting Started - Lab
6 7 8
9

10. Getting Started - Lab
9
10

11. Query Editor
11
Query Editor
UDF Editor

12. Query Execution
select year, SUM(record_weight) as Births from
publicdata:samples.natality where (mother_race = 18 and
father_race = 18) group by year order by year;
Asian Indian Kids born
The race of the child. One of the following numbers:
1 - White
2 - Black
3 - American Indian
4 - Chinese
5 - Japanese
6 - Hawaiian
7 - Filipino
9 - Unknown/Other
18 - Asian Indian
28 - Korean
39 - Samoan
48 - Vietnamese

13. SOURCE DATA Formats
13
CSV
JSON
Avro
Cloud datastore backups

14. SOURCE DATA Formats
14
http://www.ssa.gov/OACT/babynames/names.zip
https://goo.gl/qZW9sx

15. Loading data - CSV Direct Upload
15
1
2
3

16. Loading data - CSV Direct Upload
16
4

17. Loading data - CSV Direct Upload
17
4
name:string,gender:string,count:integer

18. Loading Data - GCP Storage
18
gs://ssa-babynames/names/yob2013.txt
How about loading multiple files into a single
table?

19. Running Queries
19
Standard SQL

20. Simple Aggregates
20
Select COUNT(GoogleDevs),MAX(GoogleDevs),
STDDEV(GoogleDevs) FROM …

21. Complex Processing
21
Select … FROM … where
REGEXP_MATCH(url,”Acarin$”) AND user
CONTAINS ‘Mr.Foo’

22. Nested Select
22
Select count(*) from ( Select devGurus ….) Group
BY Name

23. Small JOIN
23
Select tbl_school.student_id from tbl_school JOIN
tbl_classroom ON tbl_classroom.student_id =
tbl_shool.student_id

24. Talking to BigQuery

25. Accessing data through API
25
https://developers.google.com/api-client-library/

26. Validating API
26
https://developers.google.com/apis-explorer/#p/bigquery/v2/

27. Validating API
27

28. Integrating with Google Sheet
28
https://developers.google.com/apps-script/advanced/bigquery
https://goo.gl/evaRRC
1 2

29. Q&A
T H A N K Y O U
SELECT @dineshMD from Contact.tweet