Big Data Marketing in the AWS Cloud: Improving Cross-Media Effectiveness was a webinar that covered:
1. Introductions of AWS database and big data services like DynamoDB, RDS, ElastiCache, and EMR.
2. Customer use cases and solutions from companies like MarketShare that use these AWS services.
3. How AWS services can help deliver cross-media analytics and improve marketing effectiveness.
4. An overview of MarketShare's platform built on AWS for big data modeling, simulation, and optimization.
To Each Their Own: How to Solve Analytic ComplexityInside Analysis
The Briefing Room with Shawn Rogers and Noetix
Slides from the Live Webcast on Aug. 14, 2012
One size will never fit all in the complex world of information management. In fact, the variety of information systems in use continues to expand. That includes all kinds of systems: data-producing applications, data-processing apps, and the downstream tools used for reporting and analytics. How can data-savvy organizations stay ahead of the curve?
Check out this episode of The Briefing Room to learn from Analyst Shawn Rogers of Enterprise Management Associates, who will explain how effective use of standard data models can solve the complexity of increasingly heterogeneous information architectures. Rogers will be briefed by Daryl Orts of Noetix who will tout his company’s wide range of industry and application-specific data models which can be used to satisfy the particular needs of today’s diverse user community.
For more information, visit: http://www.insideanalysis.com
To Each Their Own: How to Solve Analytic ComplexityInside Analysis
The Briefing Room with Shawn Rogers and Noetix
Slides from the Live Webcast on Aug. 14, 2012
One size will never fit all in the complex world of information management. In fact, the variety of information systems in use continues to expand. That includes all kinds of systems: data-producing applications, data-processing apps, and the downstream tools used for reporting and analytics. How can data-savvy organizations stay ahead of the curve?
Check out this episode of The Briefing Room to learn from Analyst Shawn Rogers of Enterprise Management Associates, who will explain how effective use of standard data models can solve the complexity of increasingly heterogeneous information architectures. Rogers will be briefed by Daryl Orts of Noetix who will tout his company’s wide range of industry and application-specific data models which can be used to satisfy the particular needs of today’s diverse user community.
For more information, visit: http://www.insideanalysis.com
Steve Sams (VP IBM Global Site & Facilities Services) presentation at Gartner Data Center Conference (Dec 2011). Learn more about IBM Smarter Data Center Services: ibm.co/smarterdc
Technology has evolved to make software & hardware Smarter, Faster & Cheaper. Business Intelligence space also is undergoing change and moving from Top Management to Strategic & Operational levels.
Democratization of BI is a presentation about how BI is changing and organizations are implementing BI Solutions even to Operational Level.
With the introduction of SAP HANA, what
was once just a possibility now becomes a
reality. This next generation of SAP’s
in-memory technology provides a
multipurpose, in-memory appliance, giving
organizations the power to gain instant
insight into business operations while
enabling them to react quickly to changing
business conditions. SAP HANA lets
business users immediately access, model
and analyze all of their transactional and
analytical data in real-time – from virtually
any data source – and in a single
environment, without affecting existing
applications or systems.
AORTA BI Solutions is specialized in implementing the Oracle BI Suite. The BI Server is the integrated information platform the suite is build on. Many people don't know it, but it's one of the best technologies Oracle has ever acquired.
PowerAdvisor’s open architecture provides feature richfunctionality that maximizes productivity and profitability.PowerPortal provides secure, customized web portals for viewing and retrieval of reports and other documents. Increase trade desk productivity with automated portfolio rebalancing and trade order processing with PowerTrade.
Datawarehouse på System z (IBM Systems z)IBM Danmark
Lær om datawarehouse-systemer baseret på system z og om, hvilken udviklingsstrategi IBM følger for fortsat at være først med lanceringen af næste generations platformløsninger.
Læs mere her: bit.ly/softwaredagsystemz5
Big Data Analytics in a Heterogeneous World - Joydeep Das of SybaseBigDataCloud
Big Data Analytics is characterized by analysis of data on three vectors: exploding data volume, proliferating data variety (relational, multi-media), and accelerating data velocity. However, other key vectors such as costs and skill set needed for Big Data Analytics are often overlooked. In this session, we will consider all five vectors by exploring various techniques where traditional but progressive technologies such as column store DBMS and Event Stream Processing is combined with open source frameworks such as Hadoop to exploit the full potential of Big Data Analytics.
Agenda:
- Big Data Analytics in the real world
- Commercial and Open Source techniques
- Bringing together Commercial and Open Source techniques
* Architectures
* Programming APIs
(e.g. embedded and federated MapReduce)
- Conclusions
A brief presentation on CRM. Speaks about the available CRM options, what to consider when buying CRM, CRM evaluation checklist, Benefits of CRM, Drivers of CRM.
IBM Cognos - IBM informations-integration för IBM Cognos användareIBM Sverige
Hur kan användare av IBM Cognos analys- och rapporteringsfunktioner känna 100% tillförsikt till den information de analyserar? De måste kunna se och få förklaringar till vad informationen betyder, var den kommer ifrån och vilken status den har. Lösningen på denna typ av krav, och fler därtill, är IBM InfoSphere Information Server, som är marknadens mest kompletta plattform för informationsintegration. Denna presentation hölls på IBM Cognos Performance 2010 av Mikael Sjöstedt, InfoSphere Specialist, IBM
[Webinar] Drawing insights from social mediaScupSocial
Slides for the launch Webinar of Scup's integration with GoodData. Learn how to draw and monetize insights from social media.
Access the recording of the full webinar here:
http://www.scup.com/en/access-drawing-insights-from-social-media-webinar/
Steve Sams (VP IBM Global Site & Facilities Services) presentation at Gartner Data Center Conference (Dec 2011). Learn more about IBM Smarter Data Center Services: ibm.co/smarterdc
Technology has evolved to make software & hardware Smarter, Faster & Cheaper. Business Intelligence space also is undergoing change and moving from Top Management to Strategic & Operational levels.
Democratization of BI is a presentation about how BI is changing and organizations are implementing BI Solutions even to Operational Level.
With the introduction of SAP HANA, what
was once just a possibility now becomes a
reality. This next generation of SAP’s
in-memory technology provides a
multipurpose, in-memory appliance, giving
organizations the power to gain instant
insight into business operations while
enabling them to react quickly to changing
business conditions. SAP HANA lets
business users immediately access, model
and analyze all of their transactional and
analytical data in real-time – from virtually
any data source – and in a single
environment, without affecting existing
applications or systems.
AORTA BI Solutions is specialized in implementing the Oracle BI Suite. The BI Server is the integrated information platform the suite is build on. Many people don't know it, but it's one of the best technologies Oracle has ever acquired.
PowerAdvisor’s open architecture provides feature richfunctionality that maximizes productivity and profitability.PowerPortal provides secure, customized web portals for viewing and retrieval of reports and other documents. Increase trade desk productivity with automated portfolio rebalancing and trade order processing with PowerTrade.
Datawarehouse på System z (IBM Systems z)IBM Danmark
Lær om datawarehouse-systemer baseret på system z og om, hvilken udviklingsstrategi IBM følger for fortsat at være først med lanceringen af næste generations platformløsninger.
Læs mere her: bit.ly/softwaredagsystemz5
Big Data Analytics in a Heterogeneous World - Joydeep Das of SybaseBigDataCloud
Big Data Analytics is characterized by analysis of data on three vectors: exploding data volume, proliferating data variety (relational, multi-media), and accelerating data velocity. However, other key vectors such as costs and skill set needed for Big Data Analytics are often overlooked. In this session, we will consider all five vectors by exploring various techniques where traditional but progressive technologies such as column store DBMS and Event Stream Processing is combined with open source frameworks such as Hadoop to exploit the full potential of Big Data Analytics.
Agenda:
- Big Data Analytics in the real world
- Commercial and Open Source techniques
- Bringing together Commercial and Open Source techniques
* Architectures
* Programming APIs
(e.g. embedded and federated MapReduce)
- Conclusions
A brief presentation on CRM. Speaks about the available CRM options, what to consider when buying CRM, CRM evaluation checklist, Benefits of CRM, Drivers of CRM.
IBM Cognos - IBM informations-integration för IBM Cognos användareIBM Sverige
Hur kan användare av IBM Cognos analys- och rapporteringsfunktioner känna 100% tillförsikt till den information de analyserar? De måste kunna se och få förklaringar till vad informationen betyder, var den kommer ifrån och vilken status den har. Lösningen på denna typ av krav, och fler därtill, är IBM InfoSphere Information Server, som är marknadens mest kompletta plattform för informationsintegration. Denna presentation hölls på IBM Cognos Performance 2010 av Mikael Sjöstedt, InfoSphere Specialist, IBM
[Webinar] Drawing insights from social mediaScupSocial
Slides for the launch Webinar of Scup's integration with GoodData. Learn how to draw and monetize insights from social media.
Access the recording of the full webinar here:
http://www.scup.com/en/access-drawing-insights-from-social-media-webinar/
Extending the reach of the business application to more users in the organization leads to better data management and better access to information.
After a short intro on Microsoft Dynamics AX you will learn about the new MS Office Add-ins for Microsoft Dynamics AX and discover the mobile capabilities for end users within the travel and expense module.
Learn how to create state-of-the-art, self-service BI solutions for Dynamics AX 2012 using SQL Server 2012 PowerPivot for SharePoint, BISM and Power View.
Information Management: Answering Today’s Enterprise ChallengeBob Rhubart
As presented by George Lumpkin at OTN Architect Day, Redwood Shores, CA, 7/22/09.
Find an OTN Architect Day event near you: http://www.oracle.com/technology/architect/archday.html
Interact with Architect Day presenters and participants on Oracle Mix: https://mix.oracle.com/groups/15511
Saleseffectivity and business intelligencemarekdan
some information about business intelligence second generation (in-memory) Tibco Spotfire and InfomatiX view how to use BI and mobile solutions to increase sales and marketing effectivennes
Karya develops mobile application services that fits the unique needs of your business. Our Mobile Application Services helps the users to better utilize the power of Mobile Technology.
An insight into how digital marketing organisations use Amazon Web Services and the benefits that our services bring to their business.
Phil Fitzsimons, Media Solutions Architect, AWS
Intergen - Dynamics CRM Roadmap and Social MediaIntergen
Earlier this year we saw the global launch of Dynamics CRM 2011, and according to key
analysts it’s already proving to be a world beater. In this session we’ll cover the key strengths
of CRM 2011, both on premise and online, as well as take a brief look into the future.
The session will also cover a deeper dive into the use of social media in the sales and
marketing arena. We’ll demonstrate how Dynamics CRM 2011 can help you create a central
view of social media activity as it relates to your business and how Dynamics CRM can help
extend your view of your customers and prospects.
TeleManagement Forum OSSera Case Study - AIS Thailand Service Manager Present...Mingxia Zhang, Ph.D.
Tuesday, February 7th, 5:30 - 5:50 PM
Using Frameworx in Implementing a Unified Service Management Tool –Improving Organizational Collaboration and Communication
Examining the drivers for developing a Unified Service Management Tool to improve business processes at the service level in the Strategy, Infrastructure, and Product (SIP) area as well as Operations.
Outlining the development of an enterprise-wide Service Management application, which enabled solidification of the Service Development and Management processes in the SIP area and Service Management and Operation processes
Quantifying the benefits in terms of information sharing, process unification/implementation, cost saving and revenue increasing in service management
Similar to Big Data Marketing in the AWS Cloud: Improving Cross-Media Effectiveness - Webinar (20)
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS.
In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup.
Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti.
Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.
Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi
Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
L’utilizzo dei container è in continua crescita.
Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili.
I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!
In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th.
Event Agenda :
Open banking so far (short recap)
• PSD2, OB UK, OB Australia, OB LATAM, OB Israel
Intro to Open Finance marketplace
• Scope
• Features
• Tech overview and Demo
The role of the Cloud
The Future of APIs
• Complying with regulation
• Monetizing data / APIs
• Business models
• Time to market
One platform for all: a Strategic approach
Q&A
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc.
AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta.
Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity.
AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet.
Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.
Dal riconoscimento facciale al riconoscimento di frodi o difetti di fabbricazione, l'analisi di immagini e video che sfruttano tecniche di intelligenza artificiale, si stanno evolvendo e raffinando a ritmi elevati. In questo webinar esploreremo le possibilità messe a disposizione dai servizi AWS per applicare lo stato dell'arte delle tecniche di computer vision a scenari reali.
Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti.
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti.
Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire.
Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito.
In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.
Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline.
Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.
Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.
2. Welcome
Sheri Sullivan
Senior Marketing Manager
Global SI Ecosystem
Amazon Web Services
3. Webinar Overview
• Submit Your Questions using the Q/A tool.
• A copy of today’s presentation will be made available on:
• AWS SlideShare Channel@
http://www.slideshare.net/AmazonWebServices/
• AWS YouTube Channel@
http://www.youtube.com/user/AmazonWebServices
Special Note: Today’s Webinar is being recorded.
4. What We’ll Cover
• Intro to AWS Database and Big Data Services
• Customer Use Cases and Solutions
• Delivering Cross-Media Analytics
• MarketShare Planner Platform
5. John Gannon
AWS Business
Development Manager
jgannon@amazon.com
6. Big Data and Databases on AWS
Managed services designed to reduce administration, accelerate
deployment, and minimize the cost of analysis and experimentation
DynamoDB
Schema-less data store that enables fast deployment of new applications
without the burden of database administration
Relational Database Service (RDS)
Manage existing database applications without the effort required to
provision, upgrade, backup and scale highly available instances
ElastiCache
Accelerate data retrieval performance by caching data in memory and
avoiding slower disk-based systems
Elastic MapReduce (EMR)
Hadoop-based infrastructure service enabling the parallel processing of
massive amounts of data
7. Amazon Relational Database
Service
RDS is a fully managed Relational database service that is
simple to deploy, easy to scale, reliable and cost-effective
Choice of Database Engines
Fully Managed Service
Push Button Scalability
Fault Tolerance with Multi-AZ
Works with EC2 & ElastiCache
8. Amazon DynamoDB
DynamoDB is a fully managed NoSQL database
service that provides extremely fast and
predictable performance with seamless scalability
Authors of NoSQL
Zero Administration
Low Latency SSD’s
Unlimited Potential
Storage and Throughput
9. AMAZON ELASTIC MAPREDUCE
Reduces complexity & cost of Hadoop Management
Integrates with AWS Services and 3rd Party vendors
Highly customizable
11. Amazon EMR is the #1
Enterprise Hadoop Solution
AWS is “the most
prominent Hadoop cloud
service provider” and
“leads the pack (of
Leaders) due to its
proven, feature-rich Elastic
MapReduce service…”
-The Forrester Wave™:
Enterprise Hadoop
Solutions Q1 2012
12. Success Story
Business Challenge
Needed a real-time analytics tool to determine dynamic live event pricing during the
ticket sales life cycle
Optimize event ticket pricing, improve yield management & generate incremental
revenue
AWS Services
Elastic Load Amazon Elastic
Amazon SimpleDB Amazon Simple
Balancer MapReduce Amazon CloudWatch
Email Service (SES)
Business Benefits
Ease of use, reducing developers’ infrastructure management time by 3 hours per day
Estimated 80% cost reduction annually, compared to fixed service costs
15. Who we are
MarketShare MarketShare
Planner™ Price™
The global marketer partner of choice MarketShare MarketShare
for understanding, optimizing and 360™ Optimizer™
driving revenue MarketShare Platform
Cloud modeling | Saas infrastructure | Data
connectors
• Recognized industry leader
Risky Strong
•
Bets Contenders Performers Leaders
Cloud-based software solutions Strong
• Over half the Fortune 100
• Strong media and agency Current
Offering
partnerships
• Global presence
Weak
Weak Strategy Strong
16. Terabytes per 1000+ variables
customer
Data
Architect
Client Data
ETL Reportin Modeling
g
Sim-Opt
FTP
Scale Complex Modeling Simulation Engineer
Modeling Sim-Opts Tool Stack Production
Stack Stack Tables Tables
Tables Tables Application
Modeler
100+ Customers 100+ data sources
17. Brand Product
Earned media
ETL Organic search Reporting Modeling
Innovation
Quality Events
Conferences
Controllable
Bing
WOM Google Trade shows
Sales
Blogs
Social media Twitter Awareness Training
Owned PR
Facebook Service
Support
media Commerce
Simulatio
Website Content Consideration Displays
FTP n
Shelf space In store
Google
Paid Search Bing Discounts
Purchase Bundles
Banner Ads
Coupons Promotions
Display Video Ads
Magazine Offering
Print Newspaper
Pricing Competition
TV
Applicati
Radio
on
Broadcast Signs
Interest
Seasonality
Digital
rates Non-
Stock market
signage Catalog Direct Mobile controllable
mail email
Paid media Economy
Outdoor
Direct
22. Many applications in
production
Marketing Efficiency Attribution
Dynamic Pricing
23. The Technology That Makes
It Possible
Elastic Cloud™ AWS
Amazon EC2 Amazon EC2
Permanent Instances On-Demand Instances
EC2 EC2 Amazon
Instance Instance Elastic MapReduce
Elastic Load
Balancer
Web App
Server Server
AWS
Amazon EC2 Amazon
Permanent Instances Managed Storage
EC2 EC2 RDS Database Amazon Simple
Instance Instance Instance Storage Service
(S3)
Web App
Serve Serve
r r
30. Summary
Design your data pipeline for a multi-cluster environment
• Write Configurable ETL to become independent, partitioned
workflows
• A cluster that stays up the entire month is not elastic
Save your intermediate results in low cost storage
• Think about compression
• Do not underestimate schema complexity
Loosely coupled architecture has failure points
• Save state obsessively
• Build restart-ability into your architecture
31. Programs to help you get started
with Big Data on AWS
Big Data
EMR
Discovery EMR Training
Bootcamp
Workshop
Identify and prioritize target Deploy a sample use case 3 day intensive
Big Data use cases with real customer data developer training
32. EMR Training Schedule
• Los Angeles, CA – 10/16-10/18
• Boston, MA – 10/30-11/1
• Mountain View, CA – 11/13-11/15
• Dallas, TX – 11/27-11/29
• New York, NY – 12/11-12/13
Visit http://bit.ly/AWS_EMR_Training for class details and registration
We’ve been operating the service for over 3 years now and in the last year alone we’ve operated over 2 MILLIONHadoop clusters
Forrester wave report named Amazon EMR the #1 enterprise hadoop solution because of it’s integration with various data stores, it’s ecosystem of vendors and the number of customers the service supports.
Hi, my name is Anupam Singh. I am the Vice President of Technology at MarketShare.
MarketShare builds solutions for marketing organizations at Fortune 100 companies. Our customers provide us data and we provide a cloud based analytic applications to improve the efficiency of our customer’s marketing.
So, what are the big challenges that we face? Our entire business is based on scaling complex data modeling. Our scaling challenges are across 4 major dimensions. Each customer has 10s of terabytes of data. The data comes from hundreds of data sources. This data has thousands of variables to analyze. And we need to do this for hundreds of customers. Let us look at the various stages to build a solution that scales.
The first stage is bringing the data together. Today’s marketing organization is faced with hundreds of data sources. Consider this picture where we bring together data from the customer’s website, the advertising logs from their vendors, revenue data from the ERP systems, variables like Seasonality & Economy. As you can see, we have to gather more than 40 data sources in this single picture. Just managing the storage for daily, weekly and monthly updates is a challenge.
A lot of this data is machine generated. And it is not ready for analytics. Each data source has to be scrubbed and cleaned through an ETL pipeline before doing analytics. Our ETL pipelines have 20-30 main stages with 100s of sub-stages. Scheduling these and correcting data errors is one of our biggest technical challenges. We will dive deeper into this later. Once the data has been cleaned, it is ready for analytics.
Many of our customers have never seen these data sources in a single dashboard. Even before running the data through our proprietary modeling platform, we can help our customers get dashboards on previous data black holes.
The term data scientist has been in vogue lately. At MarketShare, we have a large team of modelers who run modeling on the cloud. As the data has been cleaned up, the modelers run thousands of different equations. Many analytic applications stop their cloud usage at reporting. At MarketShare, we believe that reporting is not enough to answer the questions. Building a predictive model is key to answering business questions on terabytes of data. We use the cloud to build custom models for each one of our customers. We use the power of distributed systems to validate these models for accuracy.
Once the models have been prepared, they are deployed in an easy to use application. It should be noted that reducing big data should not mean that the user is lost in a forest of reports. At MarketShare, we believe in simplifying access to Big Data. We hide the model complexity behind easy to use applications that let our users build many different scenarios for their business.
So, what does all this give our customers? We have been able to release many different applications on top of this analytics pipeline. The first one is marketing efficiency. The second application is Attribution. The third one is Dynamic Pricing.
So, what makes this pipeline run? Our entire analytics workflow is built using various services from Amazon as building blocks. Our applications are deployed behind the elastic load balancer service. The data is stored in Storage services like S3, RDS and we are trying out Dynamo DB. Our analytics jobs are executed on dynamic clusters provided by elastic map reduce.
So, let us quickly go under the hood. 3 years ago, we started with a hadoop cluster to store all our data. Very quickly we noticed two important things with the cluster. The first observation is that however big we made the cluster, jobs kept running into each other. Try as we might, the cluster would get hot for some time when many different stages would start executing at the same time. The second observation was how unused the cluster was for large periods of our time. So, while we are spending a lot of dollars on this large cluster, our customers are still unhappy with the response times!
So, what was our solution? We rewrote our entire data pipeline to run many different clusters. So,
Big Data Discovery WorkshopBrainstorm pilot use casesIdentify data sources and formatsReview business and financial driversRecommended use casesRoadmap for data migration and production rolloutReference architectureEstimated pilot costNext stepsEMR BootcampInteractive onsite workshop (is not classroom training)Work w/customer to architect, install, and config EMRRun and debug production job flowsCustomer’s dataset(s) must be on S3