The document outlines a big data analysis setup using Hadoop technology, detailing configurations for various components such as HDFS, Hive, Zookeeper, Flume, and Sqoop, along with their respective versions. It mentions the use of Ubuntu operating systems for deployment, along with other tools like Spring for web applications. Additionally, a GitHub repository link for collecting RSS data is provided.

1. 2017 BIGDATA
BIGDATA ANALYSIS WITH HADOOP

2. INDEX03
04
01
02
05

3. 01
RSS
Hive
3

4. 01
HDFS BROWSER
5 RSS
HDFS
,
4

5. 01
HIVEQL
5

6. 01
Spring
‘Sqoop’
MySQL
6

7. 01
,
7

8. 01 8
0 E
Hadoop01(Ubuntu14.04) Hadoop02(Ubuntu14.04) Hadoop03(Ubuntu14.04) Hadoop04(Ubuntu14.04)
2
3C3
2
3C3
0C3 F 3
3C3
D 0 E2
3C3
C E 3
E 0 E
1 C 3C3
0A Web(Ubuntu 16.04)
0 1
F0
D C

9. 02
19
.
.
HDFS
.
.
9

10. 03
‣ Zookeeper HA
‣ 5 RSS , , ,
‣ RSS Flume HDFS
‣ Hive 5
‣ 1 Hive
‣ Sqoop MySQL
‣ Spring ,
10

11. 04
▸ Ubuntu 14.04( ) * 4, Ubuntu 16.04(Web)
▸ Hadoop 2.7.2
▸ Hive 2.1.1
▸ Zookeeper 3.4.10
▸ Flume 1.7
▸ Sqoop 1.4.6
▸ MySQL 5.6
▸ Spring Boot 1.3
▸ Spring Web, Spring Security, Spring JDBC
‣ 2017 04 ~ 2017 05 09
11

12. 05
‣ 5 Java ROME Library RSS
‣ Hadoop HDFS , Zookeeper HA
‣ Hive
‣ ,
‣ HCatalog Sqoop
‣ RSS Flume MySQL
‣ Crontab Hive , , Sqoop
‣ Spring RESTful
12

13. HTTPS://GITHUB.COM/JINH574/
JAVA-COLLECTRSSDATA
GIT ADDRESS