Security for
Big Data Systems
如何做好大数据的系统安全
By Steve Mushero
May, 2015
Build & Manage Servers Optimize & Manage Servers Manage Cloud Servers Copyright © 2015 ChinaNetCloud
Running the World’s Internet Servers www.ChinaNetCloud.com
We have lots of data
我们有很多数据
Running the World’s Internet Servers www.ChinaNetCloud.com
We get data from everything
数据无处不在
Running the World’s Internet Servers www.ChinaNetCloud.com
From Every Part of Life
融入生活的每一部分
Running the World’s Internet Servers www.ChinaNetCloud.com
How to protect it ?
如何保护数据
Running the World’s Internet Servers www.ChinaNetCloud.com
How to protect it ? Like Gold !
如何像保护黄金那样去保护它
Running the World’s Internet Servers www.ChinaNetCloud.com
Protecting Gold – Safes
保护黄金-保险箱
Running the World’s Internet Servers www.ChinaNetCloud.com
Protecting Gold – Vaults
保护黄金-地下保险库
Running the World’s Internet Servers www.ChinaNetCloud.com
Protecting Gold – Banks
保护黄金-银行
Running the World’s Internet Servers www.ChinaNetCloud.com
Protecting Gold – Forts
保护黄金-地堡
Running the World’s Internet Servers www.ChinaNetCloud.com
Those risks were physical
这些风险都是物理的
Running the World’s Internet Servers www.ChinaNetCloud.com
Today’s Risks are Digital
当今的风险是电子化的
Running the World’s Internet Servers www.ChinaNetCloud.com
Gold Thieves Arrive by Car with Guns
偷黄金的人是持枪驾车
Running the World’s Internet Servers www.ChinaNetCloud.com
Data Thieves Arrive by Cable
偷数据的人是接数据线
Running the World’s Internet Servers www.ChinaNetCloud.com
From Anywhere
从任何地方
Running the World’s Internet Servers www.ChinaNetCloud.com
By Anyone
任何人
Running the World’s Internet Servers www.ChinaNetCloud.com
Data Risks ?
数据风险是什么?
Running the World’s Internet Servers www.ChinaNetCloud.com
Data Risks ?
数据风险是什么?
Running the World’s Internet Servers www.ChinaNetCloud.com
Risk Types – Stolen Data
风险类型-偷数据
Running the World’s Internet Servers www.ChinaNetCloud.com
Risk Types – Tampered Data
风险类型-篡改数据
Running the World’s Internet Servers www.ChinaNetCloud.com
Risk Types – Privacy
风险类型-隐私侵犯
Running the World’s Internet Servers www.ChinaNetCloud.com
Risk Areas – Collecting Data
风险类型-收集数据
Running the World’s Internet Servers www.ChinaNetCloud.com
Risk Areas – Processing Data
风险类型-处理数据
Running the World’s Internet Servers www.ChinaNetCloud.com
Risk Areas – Storing Data
风险类型-存储数据
Running the World’s Internet Servers www.ChinaNetCloud.com
Risk Paths – Outsiders (Hackers)
风险路径-外部(黑客)
Running the World’s Internet Servers www.ChinaNetCloud.com
Risk Paths – Insiders (Employees)
风险路径-内部(员工)
Running the World’s Internet Servers www.ChinaNetCloud.com
Risk Paths – Sys Admin (Privileged
Users)
风险路径-系统管理(特权用户)
Running the World’s Internet Servers www.ChinaNetCloud.com
What to do ?
怎么办
Running the World’s Internet Servers www.ChinaNetCloud.com
Securing Data – Difficult & Frustrating
安全加固数据 – 又难又麻烦
Running the World’s Internet Servers www.ChinaNetCloud.com
How to Secure it ?
如何来办
Running the World’s Internet Servers www.ChinaNetCloud.com
How to Balance Security vs. Usability ?
如何平衡数据安全和可用性
Usability – 可用性
Features - 特点
Performance - 性能
Convenience - 便捷
Security
安全
Running the World’s Internet Servers www.ChinaNetCloud.com
Every part should be good
要做好每一方面
Running the World’s Internet Servers www.ChinaNetCloud.com
Every part should be good
要做好每一方面
Weakest Link
最薄弱的环节
Running the World’s Internet Servers www.ChinaNetCloud.com
4 Security Zones
4大安全区域
Gathering
收集收据
Processing Data
处理数据
Storing Data
存储数据
Infrastructure
底层设施
Running the World’s Internet Servers www.ChinaNetCloud.com
4 Security Zones
4大安全区域
Gathering
收集数据
Processing Data
Storing Data
Infrastructure
Running the World’s Internet Servers www.ChinaNetCloud.com
Gathering & Ingesting Data
收集和摄取数据
• Secure gathering 安全收集
• Personal Identifying Info (PII)
个人身份信息
• Anonymisation
匿名
Running the World’s Internet Servers www.ChinaNetCloud.com
Data migration/ETL junctions
数据迁移/ETL结合
• Secure Systems 安全加固系统
• Process Validation处理验证
Running the World’s Internet Servers www.ChinaNetCloud.com
4 Security Zones
4大安全区域
Gathering
Processing Data
处理数据
Storing Data
Infrastructure
Running the World’s Internet Servers www.ChinaNetCloud.com
Processing Data
处理数据
Running the World’s Internet Servers www.ChinaNetCloud.com
Processing Data – Two parts
处理数据-2个方面
• Processing Data处理数据
• Map Reduce匹对
• Consolidating巩固
• Summarizing汇总
• Usually Hadoop
• Presentation演示
• Website网站
• Report报告
• Interactive 互动
Running the World’s Internet Servers www.ChinaNetCloud.com
Securing Hadoop
安全加固Hadoop
• Poor Authentication
认证环节薄弱
• Users & Services
用户和服务
• No privacy
无隐私
• No Integrity
不完整
• Arbitrary Code Exec
代码执行武断
• Exploits Exist
开发一直存在
Running the World’s Internet Servers www.ChinaNetCloud.com
Weg Code – OWASP Resources
代码 – OWASP 项目资源
• Info - 介绍
• Guides - 指引
• Tools - 工具
http://owasp.org.cn
Running the World’s Internet Servers www.ChinaNetCloud.com
Code – OWASP Top 10
代码-10大应用程序风险
Key Points要点
• A1 – Injection
• A2 – Auth & Session Mgmt
• A3 – XSS
• A7 – Function ACLs
• A8 – CSRF
• A9 – Insecure Components
http://owasp.org.cn
Running the World’s Internet Servers www.ChinaNetCloud.com
Processing Code – App Scanning
处理代码-APP扫描
• Best practice
最佳实践
• Find new problems
找到新问题
• As you update
更新
• Third parties
第三方
• New exploits
新的改进
Running the World’s Internet Servers www.ChinaNetCloud.com
4 Security Zones
4大安全区域
Gathering
Processing Data
Storing Data
存储数据
Infrastructure
Running the World’s Internet Servers www.ChinaNetCloud.com
Storing Data – Key Protection Point
存储数据-重要的保护点
• Easy to Steal
容易在以下几个地方被偷窃
• From DBMS
数据库管理系统
• From Storage
存储系统
• Privacy Also an Issue
隐私也是个问题
Running the World’s Internet Servers www.ChinaNetCloud.com
Storing Data – Two Levels
存储数据-2个层面
• DBMS Level
数据库管理系统层面
• Oracle, MySQL, etc.
• Operational Security
运维安全
• Users, Config, etc.
用户,配置等
• PII Separation / Sharding
PII隔离/分片
• Disk / SAN Level
磁盘/SAN层级
• Encrypt at Rest
全部加密
• Careful configuration
认真的配置
Running the World’s Internet Servers www.ChinaNetCloud.com
4 Security Zones
4大安全区域
Gathering
Processing Data
Storing Data
Infrastructure
底层设施
Running the World’s Internet Servers www.ChinaNetCloud.com
Infrastructure – Cloud & Servers
底层设施-云和物理服务器
• Services
服务软件
• Servers & OS
服务器和操作系统
• Cloud
云
• Network
网络
Running the World’s Internet Servers www.ChinaNetCloud.com
Cloud & Servers – Love & Respect Them
云和物理服务器-需要被关注
• Often forgotten
经常被遗忘
• Often use defaults
经常采取默认设置
• Or random Google search
或用谷歌搜索配置
• Source of great danger
风险的发源地
Running the World’s Internet Servers www.ChinaNetCloud.com
Infrastructure – Many Parts & Layers
基础设施-许多层级
• Internet – 互联网
• Firewalls - 防火墙
• Web/App Servers - 服务器
• Database - 数据库
• OS - 操作系统
• Servers /Cloud - 物理服务器/云
Running the World’s Internet Servers www.ChinaNetCloud.com
Firewall & WAF (Web App Firewall)
WAF – 网页应用防火墙
• Protect Networks
保护网络
• Protect Application Code
保护应用代码
• OWASP basics
• SQL, XSS
Running the World’s Internet Servers www.ChinaNetCloud.com
Under Your Application – Server & OS
应用之下-服务器&操作系统
• Hardened OS - 加固操作系统
• Iptables - 防火墙配置
• Run Users - 用户运行
• File permissions - 文件许可
• Logging - 日志
• Scanning (ClamAV) - 扫描
• Track activity - 轨迹追踪
• Automate - 自动
• System Updates - 系统升级
Running the World’s Internet Servers www.ChinaNetCloud.com
Under Your Application – Cloud
应用之下-云
• Best Practices - 最佳实践
• Control Access - 控制登录权限
• Can delete EVERYTHING
会意外删除一切
• Use Cloud Security Features
使用公共云上的安全服务
Running the World’s Internet Servers www.ChinaNetCloud.com
Audit is also Important
审计也很重要
Deep Check to Find Problems
深入检查,发现问题
Running the World’s Internet Servers www.ChinaNetCloud.com
Tools – Infosphere Guardium
工具
Running the World’s Internet Servers www.ChinaNetCloud.com
Summary
总结
• Security is Critically Important
安全非常重要
• Big Data is Vulnerable
大数据很容易被侵入
• Hard to Do Well
难以良好驾驭
• But more Tools
但,实用工具越来越多
• Details & Experts Help
要注重细节,并取得专家帮助!
Running the World’s Internet Servers www.ChinaNetCloud.com
Thank you!
谢谢
Running the World’s Internet Servers www.ChinaNetCloud.com
Thanks from ChinaNetCloud
来自云络的感谢
Pioneers in OaaS – Operations as a Service
运维即服务的先锋团队
ChinaNetCloud
Sales@ChinaNetCloud.com
www.ChinaNetCloud.com
Beijing Office:
北京办公室
Lee World Business Building #305
57 Happiness Village Road, Chaoyang District
朝阳区幸福村中路57号利世商务楼305室
Beijing, 100027 China
Silicon Valley Office:
硅谷办公室
California Avenue
Palo Alto, 94123 USA
Shanghai Headquarters:
上海办公室
X2 Space 1-601, 1238 Xietu Lu
Shanghai, 200032 China 斜土路1238号X2空间1号楼601室
T: +86-21-6422-1946 F: +86-21-6422-4911

Big Data Security (ChinaNetCloud - Guiyang Conference)

  • 1.
    Security for Big DataSystems 如何做好大数据的系统安全 By Steve Mushero May, 2015 Build & Manage Servers Optimize & Manage Servers Manage Cloud Servers Copyright © 2015 ChinaNetCloud
  • 2.
    Running the World’sInternet Servers www.ChinaNetCloud.com We have lots of data 我们有很多数据
  • 3.
    Running the World’sInternet Servers www.ChinaNetCloud.com We get data from everything 数据无处不在
  • 4.
    Running the World’sInternet Servers www.ChinaNetCloud.com From Every Part of Life 融入生活的每一部分
  • 5.
    Running the World’sInternet Servers www.ChinaNetCloud.com How to protect it ? 如何保护数据
  • 6.
    Running the World’sInternet Servers www.ChinaNetCloud.com How to protect it ? Like Gold ! 如何像保护黄金那样去保护它
  • 7.
    Running the World’sInternet Servers www.ChinaNetCloud.com Protecting Gold – Safes 保护黄金-保险箱
  • 8.
    Running the World’sInternet Servers www.ChinaNetCloud.com Protecting Gold – Vaults 保护黄金-地下保险库
  • 9.
    Running the World’sInternet Servers www.ChinaNetCloud.com Protecting Gold – Banks 保护黄金-银行
  • 10.
    Running the World’sInternet Servers www.ChinaNetCloud.com Protecting Gold – Forts 保护黄金-地堡
  • 11.
    Running the World’sInternet Servers www.ChinaNetCloud.com Those risks were physical 这些风险都是物理的
  • 12.
    Running the World’sInternet Servers www.ChinaNetCloud.com Today’s Risks are Digital 当今的风险是电子化的
  • 13.
    Running the World’sInternet Servers www.ChinaNetCloud.com Gold Thieves Arrive by Car with Guns 偷黄金的人是持枪驾车
  • 14.
    Running the World’sInternet Servers www.ChinaNetCloud.com Data Thieves Arrive by Cable 偷数据的人是接数据线
  • 15.
    Running the World’sInternet Servers www.ChinaNetCloud.com From Anywhere 从任何地方
  • 16.
    Running the World’sInternet Servers www.ChinaNetCloud.com By Anyone 任何人
  • 17.
    Running the World’sInternet Servers www.ChinaNetCloud.com Data Risks ? 数据风险是什么?
  • 18.
    Running the World’sInternet Servers www.ChinaNetCloud.com Data Risks ? 数据风险是什么?
  • 19.
    Running the World’sInternet Servers www.ChinaNetCloud.com Risk Types – Stolen Data 风险类型-偷数据
  • 20.
    Running the World’sInternet Servers www.ChinaNetCloud.com Risk Types – Tampered Data 风险类型-篡改数据
  • 21.
    Running the World’sInternet Servers www.ChinaNetCloud.com Risk Types – Privacy 风险类型-隐私侵犯
  • 22.
    Running the World’sInternet Servers www.ChinaNetCloud.com Risk Areas – Collecting Data 风险类型-收集数据
  • 23.
    Running the World’sInternet Servers www.ChinaNetCloud.com Risk Areas – Processing Data 风险类型-处理数据
  • 24.
    Running the World’sInternet Servers www.ChinaNetCloud.com Risk Areas – Storing Data 风险类型-存储数据
  • 25.
    Running the World’sInternet Servers www.ChinaNetCloud.com Risk Paths – Outsiders (Hackers) 风险路径-外部(黑客)
  • 26.
    Running the World’sInternet Servers www.ChinaNetCloud.com Risk Paths – Insiders (Employees) 风险路径-内部(员工)
  • 27.
    Running the World’sInternet Servers www.ChinaNetCloud.com Risk Paths – Sys Admin (Privileged Users) 风险路径-系统管理(特权用户)
  • 28.
    Running the World’sInternet Servers www.ChinaNetCloud.com What to do ? 怎么办
  • 29.
    Running the World’sInternet Servers www.ChinaNetCloud.com Securing Data – Difficult & Frustrating 安全加固数据 – 又难又麻烦
  • 30.
    Running the World’sInternet Servers www.ChinaNetCloud.com How to Secure it ? 如何来办
  • 31.
    Running the World’sInternet Servers www.ChinaNetCloud.com How to Balance Security vs. Usability ? 如何平衡数据安全和可用性 Usability – 可用性 Features - 特点 Performance - 性能 Convenience - 便捷 Security 安全
  • 32.
    Running the World’sInternet Servers www.ChinaNetCloud.com Every part should be good 要做好每一方面
  • 33.
    Running the World’sInternet Servers www.ChinaNetCloud.com Every part should be good 要做好每一方面 Weakest Link 最薄弱的环节
  • 34.
    Running the World’sInternet Servers www.ChinaNetCloud.com 4 Security Zones 4大安全区域 Gathering 收集收据 Processing Data 处理数据 Storing Data 存储数据 Infrastructure 底层设施
  • 35.
    Running the World’sInternet Servers www.ChinaNetCloud.com 4 Security Zones 4大安全区域 Gathering 收集数据 Processing Data Storing Data Infrastructure
  • 36.
    Running the World’sInternet Servers www.ChinaNetCloud.com Gathering & Ingesting Data 收集和摄取数据 • Secure gathering 安全收集 • Personal Identifying Info (PII) 个人身份信息 • Anonymisation 匿名
  • 37.
    Running the World’sInternet Servers www.ChinaNetCloud.com Data migration/ETL junctions 数据迁移/ETL结合 • Secure Systems 安全加固系统 • Process Validation处理验证
  • 38.
    Running the World’sInternet Servers www.ChinaNetCloud.com 4 Security Zones 4大安全区域 Gathering Processing Data 处理数据 Storing Data Infrastructure
  • 39.
    Running the World’sInternet Servers www.ChinaNetCloud.com Processing Data 处理数据
  • 40.
    Running the World’sInternet Servers www.ChinaNetCloud.com Processing Data – Two parts 处理数据-2个方面 • Processing Data处理数据 • Map Reduce匹对 • Consolidating巩固 • Summarizing汇总 • Usually Hadoop • Presentation演示 • Website网站 • Report报告 • Interactive 互动
  • 41.
    Running the World’sInternet Servers www.ChinaNetCloud.com Securing Hadoop 安全加固Hadoop • Poor Authentication 认证环节薄弱 • Users & Services 用户和服务 • No privacy 无隐私 • No Integrity 不完整 • Arbitrary Code Exec 代码执行武断 • Exploits Exist 开发一直存在
  • 42.
    Running the World’sInternet Servers www.ChinaNetCloud.com Weg Code – OWASP Resources 代码 – OWASP 项目资源 • Info - 介绍 • Guides - 指引 • Tools - 工具 http://owasp.org.cn
  • 43.
    Running the World’sInternet Servers www.ChinaNetCloud.com Code – OWASP Top 10 代码-10大应用程序风险 Key Points要点 • A1 – Injection • A2 – Auth & Session Mgmt • A3 – XSS • A7 – Function ACLs • A8 – CSRF • A9 – Insecure Components http://owasp.org.cn
  • 44.
    Running the World’sInternet Servers www.ChinaNetCloud.com Processing Code – App Scanning 处理代码-APP扫描 • Best practice 最佳实践 • Find new problems 找到新问题 • As you update 更新 • Third parties 第三方 • New exploits 新的改进
  • 45.
    Running the World’sInternet Servers www.ChinaNetCloud.com 4 Security Zones 4大安全区域 Gathering Processing Data Storing Data 存储数据 Infrastructure
  • 46.
    Running the World’sInternet Servers www.ChinaNetCloud.com Storing Data – Key Protection Point 存储数据-重要的保护点 • Easy to Steal 容易在以下几个地方被偷窃 • From DBMS 数据库管理系统 • From Storage 存储系统 • Privacy Also an Issue 隐私也是个问题
  • 47.
    Running the World’sInternet Servers www.ChinaNetCloud.com Storing Data – Two Levels 存储数据-2个层面 • DBMS Level 数据库管理系统层面 • Oracle, MySQL, etc. • Operational Security 运维安全 • Users, Config, etc. 用户,配置等 • PII Separation / Sharding PII隔离/分片 • Disk / SAN Level 磁盘/SAN层级 • Encrypt at Rest 全部加密 • Careful configuration 认真的配置
  • 48.
    Running the World’sInternet Servers www.ChinaNetCloud.com 4 Security Zones 4大安全区域 Gathering Processing Data Storing Data Infrastructure 底层设施
  • 49.
    Running the World’sInternet Servers www.ChinaNetCloud.com Infrastructure – Cloud & Servers 底层设施-云和物理服务器 • Services 服务软件 • Servers & OS 服务器和操作系统 • Cloud 云 • Network 网络
  • 50.
    Running the World’sInternet Servers www.ChinaNetCloud.com Cloud & Servers – Love & Respect Them 云和物理服务器-需要被关注 • Often forgotten 经常被遗忘 • Often use defaults 经常采取默认设置 • Or random Google search 或用谷歌搜索配置 • Source of great danger 风险的发源地
  • 51.
    Running the World’sInternet Servers www.ChinaNetCloud.com Infrastructure – Many Parts & Layers 基础设施-许多层级 • Internet – 互联网 • Firewalls - 防火墙 • Web/App Servers - 服务器 • Database - 数据库 • OS - 操作系统 • Servers /Cloud - 物理服务器/云
  • 52.
    Running the World’sInternet Servers www.ChinaNetCloud.com Firewall & WAF (Web App Firewall) WAF – 网页应用防火墙 • Protect Networks 保护网络 • Protect Application Code 保护应用代码 • OWASP basics • SQL, XSS
  • 53.
    Running the World’sInternet Servers www.ChinaNetCloud.com Under Your Application – Server & OS 应用之下-服务器&操作系统 • Hardened OS - 加固操作系统 • Iptables - 防火墙配置 • Run Users - 用户运行 • File permissions - 文件许可 • Logging - 日志 • Scanning (ClamAV) - 扫描 • Track activity - 轨迹追踪 • Automate - 自动 • System Updates - 系统升级
  • 54.
    Running the World’sInternet Servers www.ChinaNetCloud.com Under Your Application – Cloud 应用之下-云 • Best Practices - 最佳实践 • Control Access - 控制登录权限 • Can delete EVERYTHING 会意外删除一切 • Use Cloud Security Features 使用公共云上的安全服务
  • 55.
    Running the World’sInternet Servers www.ChinaNetCloud.com Audit is also Important 审计也很重要 Deep Check to Find Problems 深入检查,发现问题
  • 56.
    Running the World’sInternet Servers www.ChinaNetCloud.com Tools – Infosphere Guardium 工具
  • 57.
    Running the World’sInternet Servers www.ChinaNetCloud.com Summary 总结 • Security is Critically Important 安全非常重要 • Big Data is Vulnerable 大数据很容易被侵入 • Hard to Do Well 难以良好驾驭 • But more Tools 但,实用工具越来越多 • Details & Experts Help 要注重细节,并取得专家帮助!
  • 58.
    Running the World’sInternet Servers www.ChinaNetCloud.com Thank you! 谢谢
  • 59.
    Running the World’sInternet Servers www.ChinaNetCloud.com Thanks from ChinaNetCloud 来自云络的感谢 Pioneers in OaaS – Operations as a Service 运维即服务的先锋团队
  • 60.
    ChinaNetCloud Sales@ChinaNetCloud.com www.ChinaNetCloud.com Beijing Office: 北京办公室 Lee WorldBusiness Building #305 57 Happiness Village Road, Chaoyang District 朝阳区幸福村中路57号利世商务楼305室 Beijing, 100027 China Silicon Valley Office: 硅谷办公室 California Avenue Palo Alto, 94123 USA Shanghai Headquarters: 上海办公室 X2 Space 1-601, 1238 Xietu Lu Shanghai, 200032 China 斜土路1238号X2空间1号楼601室 T: +86-21-6422-1946 F: +86-21-6422-4911