THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

•

1 like•626 views

Gigaom

Presentation from Robert Lefkowitz, 1010data #dataconf More at http://event.gigaom.com/structuredata/

THE TRILLION-ROW SPREADSHEET™
SPEAKER: Robert Lefkowitz
Director of Web
Development
1010data

Friday, July 27, 2012

The Trillion Row Spreadsheet
r0ml@1010data.com

2
Friday, July 27, 2012

Trillion Row Spreadsheet

5
Friday, July 27, 2012

Trillion Row Spreadsheet
Trillion Row Data Warehouse

6
Friday, July 27, 2012

Data Warehouse
• Operational -> Analytical
- Design/Model
- Data (cleaning)
- Multiple data sources
- Date (timeseries) analytics
- Reporting (business intelligence)

12
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

14
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

15
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Spreadsheet

16
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Spreadsheet

17
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Spreadsheet

18
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Spreadsheet

19
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Web based
Spreadsheet

20
Friday, July 27, 2012

Spreadsheets lack ... ?
• Content management
• Business process management
• Automated scheduling
• Authentication
• Audit
• Collaboration
• Enterprise level roll-up

21
Friday, July 27, 2012

Spreadsheets lack ... ?
• Scale

22
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Web-based
Spreadsheet

23
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse
Spreadsheet

Web-based
Spreadsheet

24
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse
Spreadsheet

If this is fast enough, you can
recalculate the summaries
from the detail on the fly....

Web-based
Spreadsheet

25
Friday, July 27, 2012

Operational Data

Operational Data Operational Data

Data Warehouse

Data Mart Data Mart Data Mart

Web-based
Spreadsheet

26
Friday, July 27, 2012

Spreadsheets lack ... ?
• Professional programming practices
- Requirements analysis
- Formal regression testing
- Model/View separation
- Object-Orientation
- Static typing
- Naming conventions

28
Friday, July 27, 2012

The Trillion Row Spreadsheet
r0ml@1010data.com

30
Friday, July 27, 2012

Similar to THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

ANALYZING LARGE-SCALE USER DATA from Structure:Data 2012

Gigaom

Digitale fabriek - I2 - Dsquare

Sirris

Investigative Analytics- What's in a Data Scientists Toolbox

Data Science London

PhD thesis defense of Ajith Ranabahu

Artificial Intelligence Institute at UofSC

SPONSORED WORKSHOP by Amplidata from Structure:Data 2012:

Gigaom

THE STATE OF GLOBAL INFRASTRUCTURE PERFORMANCE from Structure 2012

Gigaom

Data warehouse

Rishabh Dogra

Data warehouse

Rishabh Dogra

Jisc RSC Eastern Microsoft Briefing - Ridgian BI Jisc RSC Eastern 19th march ...

JISC RSC Eastern

Ms system centar 2012 hp-badurini prezentacijaTomislav Lulic

Evolutionary Development Methodology

Donna Kelly

THE 3V’S OF BIG DATA: VARIETY, VELOCITY, and VOLUME

Gigaom

Final Year Project Guidance

Varad Meru

Introduction to Data WarehousingTameem Ahmad

The 5 levels of embedded biMike Boyarski

Campus Party Recife - Sua primeira e próximas aplicações Android: como fazer ...

Nelson Glauber Leal

Similar to THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012 (16)

ANALYZING LARGE-SCALE USER DATA from Structure:Data 2012

Digitale fabriek - I2 - Dsquare

Investigative Analytics- What's in a Data Scientists Toolbox

PhD thesis defense of Ajith Ranabahu

SPONSORED WORKSHOP by Amplidata from Structure:Data 2012:

THE STATE OF GLOBAL INFRASTRUCTURE PERFORMANCE from Structure 2012

Data warehouse

Jisc RSC Eastern Microsoft Briefing - Ridgian BI Jisc RSC Eastern 19th march ...

Ms system centar 2012 hp-badurini prezentacija

Evolutionary Development Methodology

THE 3V’S OF BIG DATA: VARIETY, VELOCITY, and VOLUME

Final Year Project Guidance

Introduction to Data Warehousing

The 5 levels of embedded bi

Campus Party Recife - Sua primeira e próximas aplicações Android: como fazer ...

Recently uploaded

PCI PIN Basics Webinar from the Controlcase Team

ControlCase

Assure Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™

UiPathCommunity

In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni. 📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath: Autopilot per Studio Web Autopilot per Studio Autopilot per Apps Clipboard AI GenAI applicata alla Document Understanding 👨‍🏫👨‍💻 Speakers: Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath Andrei Tasca, RPA Solutions Team Lead @NTT Data

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024 https://alandix.com/academic/papers/synergy2024-epistemic/ As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.

PHP Frameworks: I want to break free (IPC Berlin 2024)

Ralf Eggert

In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development. This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

Securing your Kubernetes cluster_ a step-by-step guide to success !

KatiaHIMEUR1

Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster. However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks. In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Sri Ambati

Elevating Tactical DDD Patterns Through Object Calisthenics

Dorra BARTAGUIZ

After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!

zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs

Alex Pruden

This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second). Paper: https://eprint.iacr.org/2023/1886

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

BookNet Canada

The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more. Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/ Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.

Accelerate your Kubernetes clusters with Varnish Caching

Thijs Feryn

Leading Change strategies and insights for effective change management pdf 1.pdf

OnBoard

Introduction to CHERI technology - Cybersecurity

mikeeftimakis1

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

RESUME BUILDER APPLICATION Project for students

KAMESHS29

The Metaverse and AI: how can decision-makers harness the Metaverse for their...

Jen Stirrup

The Metaverse is popularized in science fiction, and now it is becoming closer to being a part of our daily lives through the use of social media and shopping companies. How can businesses survive in a world where Artificial Intelligence is becoming the present as well as the future of technology, and how does the Metaverse fit into business strategy when futurist ideas are developing into reality at accelerated rates? How do we do this when our data isn't up to scratch? How can we move towards success with our data so we are set up for the Metaverse when it arrives? How can you help your company evolve, adapt, and succeed using Artificial Intelligence and the Metaverse to stay ahead of the competition? What are the potential issues, complications, and benefits that these technologies could bring to us and our organizations? In this session, Jen Stirrup will explain how to start thinking about these technologies as an organisation.

Free Complete Python - A step towards Data Science

RinaMondal9

The Future of Platform Engineering

Jemma Hussein Allen

Climate Impact of Software Testing at Nordic Testing Days

Kari Kakkonen

My slides at Nordic Testing Days 6.6.2024 Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.

Recently uploaded (20)

PCI PIN Basics Webinar from the Controlcase Team

Assure Contact Center Experiences for Your Customers With ThousandEyes

Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™

Epistemic Interaction - tuning interfaces to provide information for AI support

PHP Frameworks: I want to break free (IPC Berlin 2024)

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

Securing your Kubernetes cluster_ a step-by-step guide to success !

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Elevating Tactical DDD Patterns Through Object Calisthenics

zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs

Transcript: Selling digital books in 2024: Insights from industry leaders - T...

Accelerate your Kubernetes clusters with Varnish Caching

Leading Change strategies and insights for effective change management pdf 1.pdf

Introduction to CHERI technology - Cybersecurity

FIDO Alliance Osaka Seminar: Overview.pdf

RESUME BUILDER APPLICATION Project for students

The Metaverse and AI: how can decision-makers harness the Metaverse for their...

Free Complete Python - A step towards Data Science

The Future of Platform Engineering

Climate Impact of Software Testing at Nordic Testing Days

THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

1. THE TRILLION-ROW SPREADSHEET™ SPEAKER: Robert Lefkowitz Director of Web Development 1010data Friday, July 27, 2012

2. The Trillion Row Spreadsheet r0ml@1010data.com 2 Friday, July 27, 2012

3. Friday, July 27, 2012

4. 4 Friday, July 27, 2012

5. Trillion Row Spreadsheet 5 Friday, July 27, 2012

6. Trillion Row Spreadsheet Trillion Row Data Warehouse 6 Friday, July 27, 2012

7. 7 Friday, July 27, 2012

8. 8 Friday, July 27, 2012

9. 9 Friday, July 27, 2012

10. 10 Friday, July 27, 2012

11. 11 Friday, July 27, 2012

12. Data Warehouse • Operational -> Analytical - Design/Model - Data (cleaning) - Multiple data sources - Date (timeseries) analytics - Reporting (business intelligence) 12 Friday, July 27, 2012

13. 13 Friday, July 27, 2012

14. Operational Data Operational Data Operational Data Data Warehouse 14 Friday, July 27, 2012

15. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart 15 Friday, July 27, 2012

16. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 16 Friday, July 27, 2012

17. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 17 Friday, July 27, 2012

18. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Spreadsheet 18 Friday, July 27, 2012

19. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet 19 Friday, July 27, 2012

20. Operational Data Operational Data Operational Data Data Warehouse Web based Spreadsheet 20 Friday, July 27, 2012

21. Spreadsheets lack ... ? • Content management • Business process management • Automated scheduling • Authentication • Audit • Collaboration • Enterprise level roll-up 21 Friday, July 27, 2012

22. Spreadsheets lack ... ? • Scale 22 Friday, July 27, 2012

23. Operational Data Operational Data Operational Data Data Warehouse Web-based Spreadsheet 23 Friday, July 27, 2012

24. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet Web-based Spreadsheet 24 Friday, July 27, 2012

25. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet If this is fast enough, you can recalculate the summaries from the detail on the fly.... Web-based Spreadsheet 25 Friday, July 27, 2012

26. Operational Data Operational Data Operational Data Data Warehouse Data Mart Data Mart Data Mart Web-based Spreadsheet 26 Friday, July 27, 2012

27. Operational Data Operational Data Operational Data Data Warehouse Spreadsheet If this is fast enough, you can recalculate the summaries from the detail on the fly.... Web-based Spreadsheet 27 Friday, July 27, 2012

28. Spreadsheets lack ... ? • Professional programming practices - Requirements analysis - Formal regression testing - Model/View separation - Object-Orientation - Static typing - Naming conventions 28 Friday, July 27, 2012

29. 29 Friday, July 27, 2012

30. The Trillion Row Spreadsheet r0ml@1010data.com 30 Friday, July 27, 2012

31. Friday, July 27, 2012

THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

Recommended

Recommended

More Related Content

Similar to THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012

Similar to THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012 (16)

More from Gigaom

More from Gigaom (20)

Recently uploaded

Recently uploaded (20)

THE TRILLION ROW SPREADSHEET(tm) from Structure:Data 2012