Lassoing Big data - Chris & Greg Tinker, HP Master Technologists

Lassoing Big dataBIG Data and IT SolutionsChris & Greg Tinker, HP Master TechnologistsJune 2012© Copyright 2012 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice.

Lassoing Big DataAgenda• Defining Big Data• Challenges• Solution design• Scenarios• Take away & closing statements3 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Defining“Big Data” originating with analytics ‟ business intelligence (BI)• Traversing enormous diverse data types to spot patterns − 10s - 100s of terabytes (TB), petabytes (PB), and yes - even Exabytes (EB)• Business needing faster --“real time” (seconds - minutes vs. hours to days) analytic results − combining data from silos − Analyzing diverse data types − Connect data from various business units (cross analyze, access, & reference )• Growing at exponential rate − Structured data ‟ data stored in databases − Unstructured ‟ all other data including emails, social media, blogs, free form feedback, documents, transaction, multimedia (images, videos, etc.) − 90% of enterprise information is unstructured5 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

DefiningBig Data: growing at a massive scaleToday’s “Big Data” will not be considered the same in 5 years.By 2020, there will be 4 billion people online creating 50 trillion gigabytes of data*Data and its management is not just a concern for IT departments.„ ~4 trillion SMSes a month ~4 PB per month worldwide„ ~30 billion pieces of content shared on Facebook every month Structured Unstructured„ ~48 hours of video uploaded onto YouTube every minute Amount 10% 90% Growth 22% 62%In sixty seconds:• 1,820 TB of data is created; that’s enough data to fill up 2.6 million CDs**• 1.1 million conversations take place via instant Messenger***http://www.hpl.hp.com/research/intelligent_infrastructure.html**http://www.go-gulf.com/60scs_v2.jpg6 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

DefiningBig Data: Landscape Actionable intelligence HPCC + software Programming with more math and statistic Silo Data –Counts Structured Data –Averages Unstructured Data –Benchmarks Big –Sums i.e. Business units –Rates i.e. databases –Trends i.e. Social media Data Cloud Compute and Storage platforms7 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

ChallengesScaleBig Clouds – compute and storage platforms to transform data into actionable intelligence• fluctuating asset valuations − Convergence & Virtualization − Identifying untapped resources ‟utilization factors• cross-access, cross-analyze, and cross-reference − Reconcile data silos − Massive data• HPCC solutions − Hyperscale cluster solutions9 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

ChallengesGovernanceCompliance• SOX• Privacy directives ‟ Data access − US Federal ( HIPAA, FCRA, GLBA, DPPA, DOT, etc ) -- don’t forget the State addendums − UK (DPA,…)• Data retention and archives• Purging of data after expiration of legal retention• Ability to prove compliance upon request and proving data has not been manipulated, changed, or deleted• Restrictions and permutations of data models10 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

ChallengesArchitecture• Large, single name space file system(s) − Parallel access file system• Clustered file systems − Proprietary cluster volumes• Distance between data sources• Protocol(s) − ISCSI, IFCP, FCP, … − CIFS, NFS, …• Metadata management• Backups11 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

ChallengesAnalytics ‟ softwareHP software• HP Autonomy & (Information Data Operating Layer) IDOL10 − Natural language processing − unstructured• HP Vertica − StructuredOther software (examples)• Hadoop (both a file system and a map/reduce engine) − Hadoop map/reduce on HP IBRIX parallel single namespace file system − Data processing (no built in natural-language processing)• Apache, Cassandra, Cloudera, Lucene/Solr and many others12 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Solution designCapacityBig Data solutions must deliver optimal utilization of assets whileagile enough to support rapid scaling• Hierarchal storage management methods − Recent data must be readily available for real-time analytics − Performance − Reliability• Disaster recovery• Archival / backup management• Leverage open standards ‟ prevent lock-in14 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Solution designNear real-timeHistorically, analytics were derived from archived or aged data, today’s analytics requireCloud Compute and Storage platforms to achieve “nearly real-time” results• Limit data movement• Hyper-scale solutions: High Performance compute clusters (HPCC)• Cloud and virtualization• capacity scalability -- Just-in-time scalability• Parallel work streamsGet the analytics closer to the data…15 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Solution designPerformanceNeed fast and scalable access to dataHow tightly coupled the data is to the applications• Application bottlenecks − Message passing interfaces, network stacks, IO subsystem, Data layout• Parallelism ‟ aging applications which do not make use of threading• Data set size, quantity of objects, access patterns − How random is random? − File system(s) − Storage subsystem• Network• Processing16 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Solution designPerformance requirements influence scale constraintsService time ~1ms per I/O,Throughput ~ 8,000MB/secTransactions ~ 600,000 per day/hour/minute? (end to end? )Tempering IT solutions with Business Realities• Determine speed at which consumption and indexing of data types needs to take place• Close to real-time, seconds, minutes, hours, days• Utilize Enterprise Solutions − compress enormous volumes of data (via compression or de-duplication)• Volume of data available encumbers analysis ‟ SCOPE of data set• Capture of data -- Real-time/low latency17 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 1PowerProduction jobs executing for 4+ days when jobs began to fail…1,000+ users light up phone bankThough “near real-time” is a great sound byte, most complex analytics of large scale researchprojects take hours if not days. During which data at rest is expected to remain at rest.• 500TB• 2,000,000 directories• 60,000,000 files• Single file system• Storage subsystem experiences multiple component failure (PDU and UPS failure) File system linearly space representation. ???????19 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 1Power ‟ Big Data File system complications Dir US Patent # 6,782,389 … F1 F2 F3 Fn Subdir … S1 S2 S3 Sn Dir 1 S2 S1 4 Extremely High Aggregate S2 2 F2 Performance from a Single S1 Directory (and Single File) 3 F3 Subdir Sn … Sn 100 Fn S3 Segment20 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Segments Servers

Scenario 1PowerFile system metadata corruption• Part of disk subsystem failed while file system remained operational• Application IO errors combined with file system and SCSI IO errors• Disk subsystem was restored• No offline file system check was performed to fix metadataSolution/mitigation• Production Offline required to perform full check• Restore individual files which were marked for deletion and placed in lost+found• Replication21 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 2CorruptionA case that recently reached our desk ‟ corruption of Oracle 700TB databaseChallenge• Production down − database Down (corruption.. Would not start) − Application pointed to disk subsystem − 32 node farm − 50,000 LUNPATHS (we are seeing systems in excess of 200,000 LUNPATHS)• Restoration − Exactly what area is corrupt ‟ Data or temp/redo space?• Why/How?22 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 2Corruption: layers Without ASM With ASMTablesTablespacesFiles 0111000 0111000 00100… 00100… VxFS, .. .. ..,CFS Files and Disk Groups Managed by ASM,File Systems displayable in Oracle ViewsLogical Volumes (S)LVM, VxVolume Groups VM, CVMPhysical Volumes23 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 2 User Space ApplicationsCorruption: layers GNU C lib System Call Interface VFS (ext3, NTFS, VxFS, etc.) Kernel Space RAW Buffer Cache Upper Layer SD ST SR SG LVM, VxVM, Oracle ASM MPIO ‟ device mapper Blkdev SCSI IDE Etc… SCIS MID LAYER: GLUE SCSI Lower Layer FC ISCSI SAS Etc…24 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 2User corruptionPersistent naming is achieved by using the scsi_id ‟g ‟u ‟s UNIQUE wwid and placing it into the multipaths{} section of the multipath.conf filemultipaths { multipath { wwid 360060e8005709a000000709a000000c4 alias Oracle_vote1 } }Example:#> multipath -llOracle_vote1 (360060e8005709a000000709a000000c4) dm-11 HP,OPEN-V[size=513M][features=1 queue_if_no_path][hwhandler=0][rw]_ round-robin 0 [prio=0][active]_ 0:0:0:4 sde 8:64 [active][ready]_ 1:0:0:4 sdk 8:160 [active][ready]25 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 2Corruption ‟ result of human errorAdministrators manually modified /var/lib/multipath/bindingsThis file is a cache file created by multipath for persistent mapping of devices files• /var was it’s own filesystem• /var not mounted at boot time• / filesystem had it’s own /var/ which was covered up later in boot strap• Identification• Mitigation (establish definitions within multipath.conf)26 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 3Performance10Gb High Availability Network connections notmeeting expectationsPrevious solution was a 1GE port channel environmenton aging infrastructureUpgraded Servers to• Single Blade w/ 64 Cores ( 4 X 16 core processors )• 256GB memory• 4X 10Gb flex fabric interface ports• FCOE & ISCSIApplication performance was expected to be nearly 8X faster27 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 3Performance: applicationsAging applications are becoming bottlenecksFew older applications make use of parallel work streams leveraging the overall bandwidthcapacity of today’s servers• IT infrastructure at the time of application design• Home grown application scaled to unforeseen and unpredicted use − Production use throttled development indicatives − Closed source application vendor no longer exist28 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 3 Program design Application LayerPerformance: the stack Mapping Bus Stability Process model Integration Layer Message queues Data Access Scalability Data Access Data Layer Structured/Unstructured data OS Server Storage Infrastructure Layer Clustering Networking29 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 3PerformanceThroughput is achieved by optimizing parallel streams TCP # of streams TCP RTT (ms) KB/IO MTU TCP Segments/IO SCSI RTT KB/sec Mbit/sec Calculated MB/sec RTT (s) 1 0.009 1.46 1,500 1 0.01 162,222.22 1,267.36 158 0.000009 2 0.009 1.46 1,500 1 0.01 324,444.44 2,534.72 317 0.000009 3 0.009 1.46 1,500 1 0.01 486,666.67 3,802.08 475 0.000009 4 0.009 1.46 1,500 1 0.01 648,888.89 5,069.44 634 0.000009 5 0.009 1.46 1,500 1 0.01 811,111.11 6,336.81 792 0.000009 6 0.009 1.46 1,500 1 0.01 973,333.33 7,604.17 951 0.000009 7 0.009 1.46 1,500 1 0.01 1,135,555.56 8,871.53 1,109 0.000009 8 0.009 1.46 1,500 1 0.01 1,297,777.78 10,138.89 1,267 0.000009 ISCSI # of streams TCP RTT (ms) KB/IO MTU TCP Segments/IO SCSI RTT KB/sec Mbit/sec Calculated MB/sec SCSI SVC (s) 1 0.100 8.00 1,500 6 0.60 13,333.33 104.17 13 0.000600 2 0.100 8.00 1,500 6 0.60 26,666.67 208.33 26 0.000600 3 0.100 8.00 1,500 6 0.60 40,000.00 312.50 39 0.000600 4 0.100 8.00 1,500 6 0.60 53,333.33 416.67 52 0.000600 5 0.100 8.00 1,500 6 0.60 66,666.67 520.83 65 0.000600 6 0.100 8.00 1,500 6 0.60 80,000.00 625.00 78 0.000600 7 0.100 8.00 1,500 6 0.60 93,333.33 729.17 91 0.000600 8 0.100 8.00 1,500 6 0.60 106,666.67 833.33 104 0.00060030 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario 3Performance: the solutionProblem identified with the single threaded nature of the applicationTo achieve the desired performance the latency between the data and the analytics had to bereduced due to application rework was not an option. (Source lost)• Critical business application and data placed on local Storage − Latency maintained at or below ~0.1 msec where data set size allowed for such latency. • HP PCI based Smartarray battery backed disk controllers with SSD disks• Tiered storage model adopted• Utilized capacity of local server resources for special locality of application to reduce network latency − More cores and memory allows for application and OS virtualization on same physical machine − NOTE: virtual switch allows for network communication to not even leave the adapter when talking between guests on same vswitch31 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Take away Storage Design considerationsStorage Tech NAS NAS FC FC NASWork load Mixed Work loads Mixed Random and High Very high sequential bandwidth access to a Sequential Throughput single fileScale Depends on Change Rate Depends on Change Rate Depends on Change RateFile Types Many (millions) smaller to medium sized files Some large files (most Very large files (most over 100GByte) <100Gbyte) and some smaller Structured databases (data warehouses) files.Aggregate Throughput < 5Gbytes/sec 5 to 10 Gbytes/sec 10’s ‟ 100’s of Gbyte/sec requiredRequirementProtocols CIFS, NFS CIFS, NFS, FTP, HTTP, Webdav, FC ISCSI/block Access NAS w/ IB for low latency and throughput FC 33 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Take awayScalability Factors• Next Generation Data Centers − Power/heat − Scalable storage and compute power ‟ cloud platforms• Solution Designs − Availability − Scalability − Recovery − Performance34 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Take awayAlways on support from HPWho does your IT staff call?• Several levels − Foundation Care − Proactive Care − Datacenter Care − Lifecycle Event ServicesComplex Solution Team• Multi-vendor• Multi-solution35 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Test Drive HP Insight Online Win 1 of 12 HP Envy Spectre Ultrabooks and Monster Beats headset Demo #563NO PURCHASE NECESSARY & PURCHASE WILL NOT INCREASE CHANCE OF WINNING. OPEN ONLY TO Discover 2012 ‟ Las Vegas ATTENDEES; LEGAL RESIDENTS OF 50 UNITED STATES, & THE DC, 18 YEARS OLD. Void in Puerto Rico, U.S.territories, possessions and where prohibited by law. Employees of Sponsor, its subsidiaries, affiliates, their immediate family and household members, as well as public sector employees, are not eligible. Entry constitutesagreement to rules & Sponsor’s decisions. Participants enter by submitting lead inquiry to HP’s booth. Winners chosen by random drawing daily on or about June 4-6, 2012. One entry per person. Winners will be notified via email andmay have to sign and return an eligibility affidavit & liability release, unless prohibited. If eligible winners fail to sign and return required documents, prize may be forfeited. Prizes: One (1) of twelve (12) HP Envy Spectre 14 and Dr.Dre Beats Headsets (ARV $1699.00 each). No substitution, cash redemption or transfer of prizes, except in Sponsor’s discretion. Taxes are winners’ responsibility. Odds of winning depend on number of entries. Entrants release andhold harmless Copyrightits subsidiaries, affiliates, and their officers, directors, employees, agents from any claim subject to change without notice. or use. Sponsor: Hewlett-Packard Company, Attn: HP 11445 Compaq Center Drive 36 © Sponsor, 2012 Hewlett-Packard Development Company, L.P. The information contained herein is arising out of entry or prize receiptW, Houston, TX, USA 77070. Use this address for inquiries or requests for winner’s list.

Lassoing Big data - Chris & Greg Tinker, HP Master Technologists

by HP Enterprise on Jul 03, 2012

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 164

Statistics

Lassoing Big data - Chris & Greg Tinker, HP Master Technologists Presentation Transcript