Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1© Cloudera, Inc. All rights reserved. 1© Cloudera, Inc. All rights reserved.
Cloudera
(@d1ce_)
2© Cloudera, Inc. All rights reserved.
- (@d1ce_)
- Cloudera
- Hadoop
-
Who am I?
3© Cloudera, Inc. All rights reserved.
- Cloudera
- Cloudera
-
-
Agenda
4© Cloudera, Inc. All rights reserved.
- Cloudera
- Cloudera
-
-
Agenda
5© Cloudera, Inc. All rights reserved.
Cloudera
6© Cloudera, Inc. All rights reserved.
Cloudera
7© Cloudera, Inc. All rights reserved.
-
-
: The Economist
https://www.economist.com/news/briefing/2172163
4-how-it-shapin...
8© Cloudera, Inc. All rights reserved.
- 2008 4
- 2017 5
- 1,600
- 28
-
-
- 3,000
- Cloudera University 45,000
9© Cloudera, Inc. All rights reserved.
-
-
-
Cloudera
10© Cloudera, Inc. All rights reserved.
-
-
-
Cloudera
11© Cloudera, Inc. All rights reserved.
/ SE / SE
12© Cloudera, Inc. All rights reserved.
-
- / SE
-
SE
13© Cloudera, Inc. All rights reserved.
-
-
-
-
15© Cloudera, Inc. All rights reserved.
- Cloudera
- Cloudera
-
-
Agenda
16© Cloudera, Inc. All rights reserved.
Cloudera
17© Cloudera, Inc. All rights reserved.
Cloudera
- 2011 5
- 2018 1 ~300
APJ
(Local + Global)
(Local + Global)
(Local + Glo...
18© Cloudera, Inc. All rights reserved.
Cloudera
APJ
(Local + Global)
(Local + Global)
(Local + Global)
EMEA
US
19© Cloudera, Inc. All rights reserved.
Cloudera
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
20© Cloudera, Inc. All rights reserved.
Cloudera
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
21© Cloudera, Inc. All rights reserved.
Cloudera
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
...
22© Cloudera, Inc. All rights reserved.
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
23© Cloudera, Inc. All rights reserved.
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
-
-
-
-
24© Cloudera, Inc. All rights reserved.
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
25© Cloudera, Inc. All rights reserved.
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
Internal ...
26© Cloudera, Inc. All rights reserved.
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
Internal ...
27© Cloudera, Inc. All rights reserved.
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
Internal ...
28© Cloudera, Inc. All rights reserved.
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
29© Cloudera, Inc. All rights reserved.
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
Support Q...
30© Cloudera, Inc. All rights reserved.
Internal Systems
Engineering
Support
Quality
Support Planning
& Programs
31© Cloudera, Inc. All rights reserved.
- Cloudera
- Cloudera
-
-
Agenda
32© Cloudera, Inc. All rights reserved.
-
-
-
-
-
-
-
33© Cloudera, Inc. All rights reserved.
-
-
It is a capital mistake to theorize
before one has data. Insensibly one
begins...
34© Cloudera, Inc. All rights reserved.
35© Cloudera, Inc. All rights reserved.
36© Cloudera, Inc. All rights reserved.
37© Cloudera, Inc. All rights reserved.
38© Cloudera, Inc. All rights reserved.
- Cloudera
-
-
-
- OS
39© Cloudera, Inc. All rights reserved.
-
SE
40© Cloudera, Inc. All rights reserved.
-
-
- OS
Cluster Validation
41© Cloudera, Inc. All rights reserved.
Cloudera
JVM
Linux
Disk, CPU, Mem
JVM
Linux
Disk, CPU, Mem
JVM
Linux
Disk, CPU, Me...
42© Cloudera, Inc. All rights reserved.
: Kudu:
HDFS
: HBase :
ZooKeeper
: Sentry Kerberos/AD/LDAP
Tableau
SAS
YARN
MapRed...
43© Cloudera, Inc. All rights reserved.
: Kudu:
HDFS
: HBase :
ZooKeeper
: Sentry Kerberos/AD/LDAP
Tableau
SAS
YARN
MapRed...
44© Cloudera, Inc. All rights reserved.
-
-
-
-
-
-
-
45© Cloudera, Inc. All rights reserved.
- UNIX
- cat logs |
grep “^2018-02-03” | # 2018/2/3
grep “Finished memstore flush ...
46© Cloudera, Inc. All rights reserved.
- UNIX
- cat logs |
grep “^2018-02-01” | # 2018/2/1
grep “Finished memstore flush ...
47© Cloudera, Inc. All rights reserved.
- : Pandas
50© Cloudera, Inc. All rights reserved.
- Cloudera
- Cloudera
-
-
Agenda
51© Cloudera, Inc. All rights reserved.
-
-
-
-
-
52© Cloudera, Inc. All rights reserved.
-
- LVM sar mount, df
-
- lsblk/blkid/dmsetup
-
53© Cloudera, Inc. All rights reserved.
Thank you
daisuke@cloudera.com
Upcoming SlideShare
Loading in …5
×
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

6

Share

Download to read offline

Cloudera のサポートエンジニアリング #supennight

Download to read offline

サポートエンジニア Night vol.2 で話した内容です https://techplay.jp/event/651515

Related Books

Free with a 30 day trial from Scribd

See all

Cloudera のサポートエンジニアリング #supennight

  1. 1. 1© Cloudera, Inc. All rights reserved. 1© Cloudera, Inc. All rights reserved. Cloudera (@d1ce_)
  2. 2. 2© Cloudera, Inc. All rights reserved. - (@d1ce_) - Cloudera - Hadoop - Who am I?
  3. 3. 3© Cloudera, Inc. All rights reserved. - Cloudera - Cloudera - - Agenda
  4. 4. 4© Cloudera, Inc. All rights reserved. - Cloudera - Cloudera - - Agenda
  5. 5. 5© Cloudera, Inc. All rights reserved. Cloudera
  6. 6. 6© Cloudera, Inc. All rights reserved. Cloudera
  7. 7. 7© Cloudera, Inc. All rights reserved. - - : The Economist https://www.economist.com/news/briefing/2172163 4-how-it-shaping-up-data-giving-rise-new-economy https://www.economist.com/news/leaders/2172165 6-data-economy-demands-new-approach-antitrust- rules-worlds-most-valuable-resource
  8. 8. 8© Cloudera, Inc. All rights reserved. - 2008 4 - 2017 5 - 1,600 - 28 - - - 3,000 - Cloudera University 45,000
  9. 9. 9© Cloudera, Inc. All rights reserved. - - - Cloudera
  10. 10. 10© Cloudera, Inc. All rights reserved. - - - Cloudera
  11. 11. 11© Cloudera, Inc. All rights reserved. / SE / SE
  12. 12. 12© Cloudera, Inc. All rights reserved. - - / SE - SE
  13. 13. 13© Cloudera, Inc. All rights reserved. - - - -
  14. 14. 15© Cloudera, Inc. All rights reserved. - Cloudera - Cloudera - - Agenda
  15. 15. 16© Cloudera, Inc. All rights reserved. Cloudera
  16. 16. 17© Cloudera, Inc. All rights reserved. Cloudera - 2011 5 - 2018 1 ~300 APJ (Local + Global) (Local + Global) (Local + Global) EMEA US
  17. 17. 18© Cloudera, Inc. All rights reserved. Cloudera APJ (Local + Global) (Local + Global) (Local + Global) EMEA US
  18. 18. 19© Cloudera, Inc. All rights reserved. Cloudera Internal Systems Engineering Support Quality Support Planning & Programs
  19. 19. 20© Cloudera, Inc. All rights reserved. Cloudera Internal Systems Engineering Support Quality Support Planning & Programs
  20. 20. 21© Cloudera, Inc. All rights reserved. Cloudera Internal Systems Engineering Support Quality Support Planning & Programs - - 30 4 Storage Access & Integration Data Processing Enterprise ZooKeeper, HDFS HBase, Accumulo Kudu, Director HCatalog, Hive, Impala, Oozie, Parquet, Avro, Sentry, Navigator Optimizer, Sqoop CDSW, Flume, Kafka, Mahout, Spark, MapReduce, YARN, Pig, Search BDR, CDSW, CM, Hue, Security, Encryption
  21. 21. 22© Cloudera, Inc. All rights reserved. Internal Systems Engineering Support Quality Support Planning & Programs
  22. 22. 23© Cloudera, Inc. All rights reserved. Internal Systems Engineering Support Quality Support Planning & Programs - - - -
  23. 23. 24© Cloudera, Inc. All rights reserved. Internal Systems Engineering Support Quality Support Planning & Programs
  24. 24. 25© Cloudera, Inc. All rights reserved. Internal Systems Engineering Support Quality Support Planning & Programs Internal Systems Engineering - Hadoop - OpenStack -
  25. 25. 26© Cloudera, Inc. All rights reserved. Internal Systems Engineering Support Quality Support Planning & Programs Internal Systems Engineering - Hadoop - EDH Openstack (COTT) - (SSD)
  26. 26. 27© Cloudera, Inc. All rights reserved. Internal Systems Engineering Support Quality Support Planning & Programs Internal Systems Engineering - Hadoop - EDH Openstack (COTT) - (SSD) - - -
  27. 27. 28© Cloudera, Inc. All rights reserved. Internal Systems Engineering Support Quality Support Planning & Programs
  28. 28. 29© Cloudera, Inc. All rights reserved. Internal Systems Engineering Support Quality Support Planning & Programs Support Quality - Support Planning & Programs -
  29. 29. 30© Cloudera, Inc. All rights reserved. Internal Systems Engineering Support Quality Support Planning & Programs
  30. 30. 31© Cloudera, Inc. All rights reserved. - Cloudera - Cloudera - - Agenda
  31. 31. 32© Cloudera, Inc. All rights reserved. - - - - - - -
  32. 32. 33© Cloudera, Inc. All rights reserved. - - It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts. Sherlock Holmes in “A Scandal in Bohemia” by Sir Arthur Conan Doyle via ”Systems Performance” written by Brendan Gregg
  33. 33. 34© Cloudera, Inc. All rights reserved.
  34. 34. 35© Cloudera, Inc. All rights reserved.
  35. 35. 36© Cloudera, Inc. All rights reserved.
  36. 36. 37© Cloudera, Inc. All rights reserved.
  37. 37. 38© Cloudera, Inc. All rights reserved. - Cloudera - - - - OS
  38. 38. 39© Cloudera, Inc. All rights reserved. - SE
  39. 39. 40© Cloudera, Inc. All rights reserved. - - - OS Cluster Validation
  40. 40. 41© Cloudera, Inc. All rights reserved. Cloudera JVM Linux Disk, CPU, Mem JVM Linux Disk, CPU, Mem JVM Linux Disk, CPU, Mem : Kudu: HDFS : HBase : ZooKeeper : Sentry Kerberos/AD/LDAP Tableau SAS YARN MapReduce Spark : Impala : Solr /API : Hive, Pig Docker : CDSW : Hue Ingest : Flume, Kafka S3/ADLS Import/ Export : Sqoop
  41. 41. 42© Cloudera, Inc. All rights reserved. : Kudu: HDFS : HBase : ZooKeeper : Sentry Kerberos/AD/LDAP Tableau SAS YARN MapReduce Spark : Impala : Solr /API : Hive, Pig : CDSW : Hue S3/ADLS JVM Linux Disk, CPU, Mem JVM Linux Disk, CPU, Mem JVM Linux Disk, CPU, Mem Docker Ingest : Flume, Kafka Import/ Export : Sqoop
  42. 42. 43© Cloudera, Inc. All rights reserved. : Kudu: HDFS : HBase : ZooKeeper : Sentry Kerberos/AD/LDAP Tableau SAS YARN MapReduce Spark : Impala : Solr /API : Hive, Pig : CDSW : Hue S3/ADLS JVM Linux Disk, CPU, Mem JVM Linux Disk, CPU, Mem JVM Linux Disk, CPU, Mem Docker Ingest : Flume, Kafka Import/ Export : Sqoop
  43. 43. 44© Cloudera, Inc. All rights reserved. - - - - - - -
  44. 44. 45© Cloudera, Inc. All rights reserved. - UNIX - cat logs | grep “^2018-02-03” | # 2018/2/3 grep “Finished memstore flush of” | cut -d: -f 1,2 | # sed -e ‘s/.$//’ | # 10 uniq -c #
  45. 45. 46© Cloudera, Inc. All rights reserved. - UNIX - cat logs | grep “^2018-02-01” | # 2018/2/1 grep “Finished memstore flush of” | cut -d: -f 1,2 | # sed -e ‘s/.$//’ | # 10 uniq -c #
  46. 46. 47© Cloudera, Inc. All rights reserved. - : Pandas
  47. 47. 50© Cloudera, Inc. All rights reserved. - Cloudera - Cloudera - - Agenda
  48. 48. 51© Cloudera, Inc. All rights reserved. - - - - -
  49. 49. 52© Cloudera, Inc. All rights reserved. - - LVM sar mount, df - - lsblk/blkid/dmsetup -
  50. 50. 53© Cloudera, Inc. All rights reserved. Thank you daisuke@cloudera.com
  • higebu

    Apr. 12, 2018
  • HajimeOsako

    Feb. 23, 2018
  • wchevreuil

    Feb. 8, 2018
  • LingeshwaranRadhakri

    Feb. 8, 2018
  • objectx

    Feb. 7, 2018
  • AkiraMiki

    Feb. 7, 2018

サポートエンジニア Night vol.2 で話した内容です https://techplay.jp/event/651515

Views

Total views

3,186

On Slideshare

0

From embeds

0

Number of embeds

1,333

Actions

Downloads

11

Shares

0

Comments

0

Likes

6

×