Access2011 van garderen-suhonos-part2

BIG DATA

"datasets that grow solarge that they become difficult to work with using relationaldatabases and within atolerable elapsed time"

BIG DATA IS BIG

LIKE, REALLY BIG

FACEBOOK: 140 BILLION PHOTOSHUMAN GENOME: 3 BILLION BASE PAIRSGOOGLE: 50 BILLION WEB PAGESWORLDCAT: 1.5 BILLION ITEM RECORDS

NOT REALLY

EUROPEANA: 20 MILLION (715K / COUNTRY)LIBRARY OF 1.9 MILLIONCONGRESS:CANADIANA: 1 MILLIONLIBRARY AND 3.5 MILLIONARCHIVES CANADA: (ARCHIVAL DESCRIPTIONS)

BIG DATAIS COMPLICATED

NOT REALLY

ಠ_ಠ

SCALABILITY

● ICA-AtoM (LAMP)● BENCHMARK 3.5M RECORDS● 100% OPEN SOURCE SOFTWARE● COMMODITY HARDWARE

CAN WE DO IT?

WRITE SPEED

READ SPEED

WRITE MEMORY

READ MEMORY

NOSQL vs. SQL (a.k.a. ODM vs. ORM)● 4x - 10x FASTER● 50% - 90% LESS MEMORY

RELATIONAL IF YOUR DATADATABASES IS NOTSCALE WELL HIERARCHICALSOLR IF YOU HAVESCALES WELL INFINITE RAMBEWARE THE NOSQL IS ADOGMA OF SQL VIABLE OPTIONTHINK SIDEWAYS SCALE OUT →

THE CLOUD IS A LIE

“big data is less about size, and more about freedom” open source tools + distributed design = new opportunities

Access2011 van garderen-suhonos-part2

by peterVG999 on Oct 25, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

Access2011 van garderen-suhonos-part2 — Presentation Transcript