Building the Ideal Stack for Real-Time Analytics

•

4 likes•2,760 views

This document discusses building an ideal real-time analytics stack. It promotes the MemSQL database platform as being ranked #1 for operational data warehousing and being able to handle real-time analytics at massive scales. The ideal stack presented includes components for message queues, a transformation tier, data persistence with MemSQL, and real-time visualization. Use cases discussed include processing web logs, mobile apps, IoT data, and an energy company's drilling sensor data.

Building the Ideal Stack
for Real-Time Analytics
Steven Camiña, Principal Product Manager

MISSION
Make every company a real-time enterprise
PRODUCT
Database Platform: Real-Time | Scalable | Proven
Ranked #1 Operational Data Warehouse
ABOUT
Founders are former Facebook, SQL Server database engineers
YC alumni with $85m in funding from top tier investors
MemSQL at a Glance

REAL-TIME ANALYTICS
Accessing data patterns that provide meaning to a
business as they come into a system.

Today’s Real-Time Discussion:
Web Logs, Mobile Apps, IoT

Today’s Real-Time Discussion:
Web Logs, Mobile Apps, IoT
Huge Opportunity
Take Existing
Batch Processes
Real-Time

Other
Drive new revenue opportunities
Increase efficiency and reduce cost
Power new applications
(Example: IoT data processing)
Improve customer experience
Deliver real-time dashboards 52%
52%
43%
36%
29%
4%
Why are you pursuing real-time analytics?
6 MemSQL State of Real Time Survey 2016

7
The Ideal Real-Time Stack
Message Queue Transformation Tier Data Persistence Real-Time Visualization

11
Data Persistence
SSD
In-Memory and
Solid State
Distributed
Architecture
Data Source
Connectivity
Flexible
Deployment
SQL
Scalable SQL

12
The Ideal Real-Time Stack
Message Queue Transformation Tier Data Persistence Real-Time Visualization

A massively scalable database and ingest solution allowed for
massive growth, real-time analytic applications and faster, targeted.
+
MemSQL Confidential13

14
TECHNICAL BENEFITS
§ Instant accuracy to the latest re-pin
§ 1 GB/sec totaling 72 TB/day
THE PINTEREST REAL-TIME ARCHITECTURE
REAL-TIME
ANALYTICS

Real-time drilling sensor data to manage the high stakes of
producing oil in a depressed market and maximizing productivity.
+
15
Energy Company

16
TECHNICAL BENEFITS
§ Enabled machine learning scoring of streaming data for real-
time Predictive Analytics
§ Integrated SAS BI PMML for deep analytics
§ Joined multiple data types and third party sources including
geospatial and weather data
Energy Company

17
REAL-TIME
INPUTS
Spark MLlib Predictive Model
Raw Sensor 1 + Predictive Score 1
S1 P1
1
BUSINESS
LOGIC
Energy Company

New O’REILLY BOOK:
The Path to Predictive Analytics and
Machine Learning
Booth #308 or memsql.com/oreillyml
19

Thank You!
Try MemSQL: memsql.com/download
Contact: info@memsql.com
Twitter: @memsql

Machine Learning is not new, but its application across memory-optimized distributed systems has led to an explosion in both the number and capability of its uses. Pandora develops personalized content recommendations with machine learning algorithms, Tesla has produced the first widely distributed autonomous vehicle, and Amazon uses autonomous robots to move packages within its warehouses and even deliver packages. When coupled with real-time data, advanced analytics approaches like machine learning and deep learning create immediate business opportunities. Machine learning has never been more accessible—if your data pipelines support real-time analysis. Attendees will learn tools and techniques for integrating machine learning models across industries and organizations. Steven Camiña, MemSQL Product Manager, will walk through critical technologies needed in your technology ecosystem, including Python, Apache Kafka, Apache Spark, and a real-time database.

Driving the On-Demand Economy with Predictive Analytics

SingleStore

The Fast Path to Building Operational Applications with Spark

SingleStore

Machines and the Magic of Fast Learning

SingleStore

Human-machine interaction is no longer the exclusive province of science fiction. The advance of the internet and connected devices has inspired data scientists to create machine-learning applications to extract value from these new forms of data. So what's the next frontier? Join MemSQL Engineer Michael Andrews and Sr. Director Mike Boyarski to learn how to use real-time data as a vehicle for operationalizing machine-learning models. Michael and Mike will explore advanced tools, including TensorFlow, Apache Spark, and Apache Kafka, and compelling use cases demonstrating the power of machine learning to effect positive change. You will learn: Top technologies for building the ideal machine-learning stack How to power machine-learning applications with real-time data A use case and demo of machine learning for social good

Enabling Real-Time Analytics for IoT

SingleStore

Driving the On-Demand Economy with Predictive Analytics

SingleStore

Building an IoT Kafka Pipeline in Under 5 Minutes

SingleStore

The Real-Time CDO and the Cloud-Forward Path to Predictive Analytics

SingleStore

In the on-demand economy real-time analytics is both a necessity and a competitive advantage. The next evolution in the on-demand economy is in predictive analytics fueled by live streams of data—in effect knowing what customers want before they do. This session will feature technical examples of real-time pipelines, machine learning, and custom dashboards as well as off-the-shelf dashboards with Tableau.

Enabling Real-Time Analytics for IoT

SingleStore

Tapjoy: Building a Real-Time Data Science Service for Mobile Advertising

SingleStore

Real-Time Analytics with MemSQL and Spark

SingleStore

In-Memory Computing Webcast. Market Predictions 2017

SingleStore

Driving the On-Demand Economy with Spark and Predictive Analytics

SingleStore

Real-Time Geospatial Intelligence at Scale

SingleStore

Real-Time, Geospatial, Maps by Neil Dahlke

SingleStore

MemSQL - The Real-time Analytics Platform

SingleStore

Getting It Right Exactly Once: Principles for Streaming Architectures

SingleStore

Internet of Things and Multi-model Data Infrastructure

SingleStore

Introducing MemSQL 4

SingleStore

Scaling Production Machine Learning Pipelines with Databricks

Databricks

O'Reilly Media Webcast: Building Real-Time Data Pipelines

SingleStore

As our customers tap into new sources of data or modify to existing data pipelines, we are often asked questions like: What technologies should we consider? Where can we reduce data latency? How can we simplify our data architecture? To eliminate the guesswork, we teamed up with Ben Lorica, Chief Data Scientist at O’Reilly Media to host a webcast centered around building real-time data pipelines.

MemSQL

Ramzi Alqrainy

From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...

HostedbyConfluent

In renewable energy, like many other businesses, customers have come to expect real time data feeding their applications, products, and services. And internally, businesses need real time data to facilitate how we monitor our products proactively, reduce customer support costs, and provide customers with features they didn’t previously have access to. But traditional, legacy databases can’t handle the real-time requirements nor scale up to handle increasing amounts of data, and cloud monoliths and tightly-coupled systems prevent building the desired features. At SunPower, we set out to improve our cloud-based platform using Confluent and Kafka to increase the velocity of product development and unlock new features for our customers. In this session, we will share our journey to build a real-time monitoring platform based on Confluent and Kafka and how we’ve been able to improve customer satisfaction ratings and boost referral-based sales as a result.

Building the Next-gen Digital Meter Platform for Fluvius

Databricks

Fluvius is the network operator for electricity and gas in Flanders, Belgium. Their goal is to modernize the way people look at energy consumption using a digital meter that captures consumption and injection data from any electrical installation in Flanders ranging from households to large companies. After full roll-out there will be roughly 7 million digital meters active in Flanders collecting up to terabytes of data per day. Combine this with regulation that Fluvius has to maintain a record of these reading for at least 3 years, we are talking petabyte scale. delaware BeLux was assigned by Fluvius to setup a modern data platform and did so on Azure using Databricks as the core component to collect, store, process and serve these volumes of data to every single consumer and beyond in Flanders. This enables the Belgian energy market to innovate and move forward. Maarten took up the role as project manager and solution architect.

Zero Downtime App Deployment using Hadoop

DataWorks Summit/Hadoop Summit

Using Web Data to Drive Revenue and Reduce Costs

Connotate

This presentation is designed to help companies strengthen their competitive advantage by leveraging publicly available Web sources. Entrepreneurs, global industry leaders and enterprises of all sizes are turning Web data into lucrative opportunities – creating new revenue-generating products, reducing costs and re-engineering workflows to optimize pricing, streamline reporting, ensure compliance, engage interactively with clients and more. This presentation uses a variety of success stories to illustrate ways in which businesses can use Web data to drive revenue and streamline operations.

Spark Summit East Keynote by Anjul Bhambhri

Jen Aman

Partner Ecosystem Showcase for Apache Ranger and Apache Atlas

DataWorks Summit

The community for Apache Atlas and Apache Ranger, which are foundational components for Security and Governance across the Hadoop stack, has spawned a robust partner ecosystem of tools and platforms. These partner solutions build upon the extensibility offered in these platforms via open and robust APIs via integration patterns to provide innovative “better together” capabilities. In this talk, we will showcase how three of Hortonworks partners Talend, Protegrity, and Arcadia Data have effectively extended Apache Ranger and Apache Atlas frameworks to provide value added security and governance features to complement the Hadoop ecosystem. The talk will showcase partner-led demonstrations that will include how to enhance Apache Atlas lineage and metadata to cover ETL operations, how to build Apache Ranger authorizations of custom objects such as visualizations and how to enhance Apache Ranger’s data protection capabilities for encryption and masking. We will also provide a short overview of Hortonworks Gov Ready and Sec Ready programs and how partners can benefit from the certification process as part of this program.

Put Alternative Data to Use in Capital Markets 

Cloudera, Inc.

Alternative data for capital markets, such as satellite imagery, logistics data, and social media feeds, has been getting a lot of attention recently. Like any trending topic, its uses and benefits can be hyped up a bit but if the right plumbing and creativity is in place, those benefits can be realized. 3 things to learn: * Examples of alt data use cases, sources, and recent market trends * Why a big data platform that facilitates self service and collaboration is critical in monetizing alternative data * How alternative data can be applied to enhance current processes (Demo)

What's hot

CTO View: Driving the On-Demand Economy with Predictive Analytics

SingleStore

Enabling Real-Time Analytics for IoT

SingleStore

Tapjoy: Building a Real-Time Data Science Service for Mobile Advertising

SingleStore

Real-Time Analytics with MemSQL and Spark

SingleStore

In-Memory Computing Webcast. Market Predictions 2017

SingleStore

Driving the On-Demand Economy with Spark and Predictive Analytics

SingleStore

Real-Time Geospatial Intelligence at Scale

SingleStore

Real-Time, Geospatial, Maps by Neil Dahlke

SingleStore

MemSQL - The Real-time Analytics Platform

SingleStore

Getting It Right Exactly Once: Principles for Streaming Architectures

SingleStore

Internet of Things and Multi-model Data Infrastructure

SingleStore

Introducing MemSQL 4

SingleStore

Scaling Production Machine Learning Pipelines with Databricks

Databricks

O'Reilly Media Webcast: Building Real-Time Data Pipelines

SingleStore

MemSQL

Ramzi Alqrainy

From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...

HostedbyConfluent

Building the Next-gen Digital Meter Platform for Fluvius

Databricks

Zero Downtime App Deployment using Hadoop

DataWorks Summit/Hadoop Summit

Using Web Data to Drive Revenue and Reduce Costs

Connotate

Spark Summit East Keynote by Anjul Bhambhri

Jen Aman

What's hot (20)

CTO View: Driving the On-Demand Economy with Predictive Analytics

Enabling Real-Time Analytics for IoT

Tapjoy: Building a Real-Time Data Science Service for Mobile Advertising

Real-Time Analytics with MemSQL and Spark

In-Memory Computing Webcast. Market Predictions 2017

Driving the On-Demand Economy with Spark and Predictive Analytics

Real-Time Geospatial Intelligence at Scale

Real-Time, Geospatial, Maps by Neil Dahlke

MemSQL - The Real-time Analytics Platform

Getting It Right Exactly Once: Principles for Streaming Architectures

Internet of Things and Multi-model Data Infrastructure

Introducing MemSQL 4

Scaling Production Machine Learning Pipelines with Databricks

O'Reilly Media Webcast: Building Real-Time Data Pipelines

MemSQL

From Legacy SQL Server to High Powered Confluent & Kafka Monitoring System at...

Building the Next-gen Digital Meter Platform for Fluvius

Zero Downtime App Deployment using Hadoop

Using Web Data to Drive Revenue and Reduce Costs

Spark Summit East Keynote by Anjul Bhambhri

Viewers also liked

Partner Ecosystem Showcase for Apache Ranger and Apache Atlas

DataWorks Summit

Put Alternative Data to Use in Capital Markets 

Cloudera, Inc.

Real-Time Analytics Visualized w/ Kafka + Streamliner + MemSQL + ZoomData, An...

confluent

Building a real-time pipeline from scratch that is able to handle billion+ transactions per day, store, analyze and visualize it all in real-time has never been easier. In this build-as-we-go talk, we’ll create a front-to-back architecture that does exactly that. * we’ll start with a simple producer emitting a few messages and publishing them onto a Kafka queue * on consuming end of the queue a Spark-based Streamliner process will pick them up and store in MemSQL * ZoomData will connect to MemSQL for real-time visualization where we’ll be able to ask various questions and see answers change as data is flowing through the system * we’ll quickly make the entire pipeline more complex by increasing the amount of data as well as complexity of the data, until reaching 100K transactions per second As we walk through this demo, we will touch on cross data-center Kafka and MemSQL set-ups, speed limitations if any as well as echo back to real-life use cases of a similar set-up used in Goldman’s Asset Management division for the purposes of Portfolio Management & Trading.

빅데이터윈윈 컨퍼런스_데이터시각화자료

ABRC_DATA

Spark as part of a Hybrid RDBMS Architecture-John Leach Cofounder Splice Machine

Data Con LA

In this talk, we will discuss how we use Spark as part of a hybrid RDBMS architecture that includes Hadoop and HBase. The optimizer evaluates each query and sends OLTP traffic (including CRUD queries) to HBase and OLAP traffic to Spark. We will focus on the challenges of handling the tradeoffs inherent in an integrated architecture that simultaneously handles real-time and batch traffic. Lessons learned include: - Embedding Spark into a RDBMS - Running Spark on Yarn and isolating OLTP traffic from OLAP traffic - Accelerating the generation of Spark RDDs from HBase - Customizing the Spark UI The lessons learned can also be applied to other hybrid systems, such as Lambda architectures. Bio:- John Leach is the CTO and Co-Founder of Splice Machine. With over 15 years of software experience under his belt, John’s expertise in analytics and BI drives his role as Chief Technology Officer. Prior to Splice Machine, John founded Incite Retail in June 2008 and led the company’s strategy and development efforts. At Incite Retail, he built custom Big Data systems (leveraging HBase and Hadoop) for Fortune 500 companies. Prior to Incite Retail, he ran the business intelligence practice at Blue Martini Software and built strategic partnerships with integration partners. John was a key subject matter expert for Blue Martini Software in many strategic implementations across the world. His focus at Blue Martini was helping clients incorporate decision support knowledge into their current business processes utilizing advanced algorithms and machine learning. John received dual bachelor’s degrees in biomedical and mechanical engineering from Washington University in Saint Louis. Leach is the organizer emeritus for the Saint Louis Hadoop Users Group and is active in the Washington University Elliot Society.

Webinar - Sehr empfehlenswert: wie man aus Daten durch maschinelles Lernen We...

Cloudera, Inc.

Unternehmen sind heutzutage in der Lage ihre Daten mit relativer Leichtigkeit aufzunehmen und zu verwalten. Die Herausforderung besteht nun darin, die verborgenen Muster in den Daten zu erkennen und diese zu verstehen, um einen Mehrwert zu generieren. Aufgrund der großen Datenmengen gelingt dies mit traditionelle Ansätzen zumeist nicht. Das Ergebnis: Organisationen kämpfen, um wirklich zu innovieren und sich zu differenzieren.

Using Big Data to Transform Your Customer’s Experience - Part 1 

Cloudera, Inc.

Security implementation on hadoop

Wei-Chiu Chuang

Cloudera and Qlik: Big Data Analytics for Business

Data IQ Argentina

Apache Spark—Apache HBase Connector: Feature Rich and Efficient Access to HBa...

Spark Summit

Both Spark and HBase are widely used, but how to use them together with high performance and simplicity is a very challenging topic. Spark HBase Connector(SHC) provides feature rich and efficient access to HBase through Spark SQL. It bridges the gap between the simple HBase key value store and complex relational SQL queries and enables users to perform complex data analytics on top of HBase using Spark. SHC implements the standard Spark data source APIs, and leverages the Spark catalyst engine for query optimization. To achieve high performance, SHC constructs the RDD from scratch instead of using the standard HadoopRDD. With the customized RDD, all critical techniques can be applied and fully implemented, such as partition pruning, column pruning, predicate pushdown and data locality. The design makes the maintenance easy, while achieving a good tradeoff between performance and simplicity. In addition to fully supporting all the Avro schemas natively, SHC has also integrated natively with Phoenix data types. With SHC, Spark can execute batch jobs to read/write data from/into Phoenix tables. Phoenix can also read/write data from/into HBase tables created by SHC. For example, users can run a complex SQL query on top of an HBase table created by Phoenix inside Spark, perform a table join against an Dataframe which reads the data from a Hive table, or integrate with Spark Streaming to implement a more complicated system. In this talk, apart from explaining why SHC is of great use, we will also demo how SHC works, how to use SHC in secure/non-secure clusters, how SHC works with multiple secure HBase clusters, etc. This talk will also benefit people who use Spark and other data sources (besides HBase) as it inspires them with ideas of how to support high performance data source access at the Spark DataFrame level.

CWIN17 Frankfurt / Cloudera

Capgemini

The Evolution of Data Architecture

Wei-Chiu Chuang

Ibm watson

Vivek Mohan

Spark meetup - Zoomdata Streaming

Zoomdata

Softnix Messaging Server

Softnix Technology

Zoomdata

Vivek Mohan

Softnix Security Data Lake

Softnix Technology

MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...

Spark Summit

The use of large-scale machine learning and data mining methods is becoming ubiquitous in many application domains ranging from business intelligence and bioinformatics to self-driving cars. These methods heavily rely on matrix computations, and it is hence critical to make these computations scalable and efficient. These matrix computations are often complex and involve multiple steps that need to be optimized and sequenced properly for efficient execution. This work presents new efficient and scalable matrix processing and optimization techniques based on Spark. The proposed techniques estimate the sparsity of intermediate matrix-computation results and optimize communication costs. An evaluation plan generator for complex matrix computations is introduced as well as a distributed plan optimizer that exploits dynamic cost-based analysis and rule-based heuristics The result of a matrix operation will often serve as an input to another matrix operation, thus defining the matrix data dependencies within a matrix program. The matrix query plan generator produces query execution plans that minimize memory usage and communication overhead by partitioning the matrix based on the data dependencies in the execution plan. We implemented the proposed matrix techniques inside the Spark SQL, and optimize the matrix execution plan based on Spark SQL Catalyst. We conduct case studies on a series of ML models and matrix computations with special features on different datasets. These are PageRank, GNMF, BFGS, sparse matrix chain multiplications, and a biological data analysis. The open-source library ScaLAPACK and the array-based database SciDB are used for performance evaluation. Our experiments are performed on six real-world datasets are: social network data ( e.g., soc-pokec, cit-Patents, LiveJournal), Twitter2010, Netflix recommendation data, and 1000 Genomes Project sample. Experiments demonstrate that our proposed techniques achieve up to an order-of-magnitude performance.

Benefits of Transferring Real-Time Data to Hadoop at Scale

Hortonworks

Today’s Big Data teams demand solutions designed for Big Data that are optimized, secure, and adaptable to changing workload requirements. Working together, Hortonworks, IBM, and Attunity have designed an integrated solution that transfers large volumes of data to a platform that can handle rapid ingest, processing and analysis of data of all types from all sources, at scale. https://hortonworks.com/webinar/benefits-transferring-real-time-data-hadoop-scale-ibm-hortonworks-attunity/

[Big Data Spain] Apache Spark Streaming + Kafka 0.10: an Integration Story

Joan Viladrosa Riera

Viewers also liked (20)

Partner Ecosystem Showcase for Apache Ranger and Apache Atlas

Put Alternative Data to Use in Capital Markets 

Real-Time Analytics Visualized w/ Kafka + Streamliner + MemSQL + ZoomData, An...

빅데이터윈윈 컨퍼런스_데이터시각화자료

Spark as part of a Hybrid RDBMS Architecture-John Leach Cofounder Splice Machine

Webinar - Sehr empfehlenswert: wie man aus Daten durch maschinelles Lernen We...

Using Big Data to Transform Your Customer’s Experience - Part 1 

Security implementation on hadoop

Cloudera and Qlik: Big Data Analytics for Business

Apache Spark—Apache HBase Connector: Feature Rich and Efficient Access to HBa...

CWIN17 Frankfurt / Cloudera

The Evolution of Data Architecture

Ibm watson

Spark meetup - Zoomdata Streaming

Softnix Messaging Server

Zoomdata

Softnix Security Data Lake

MatFast: In-Memory Distributed Matrix Computation Processing and Optimization...

Benefits of Transferring Real-Time Data to Hadoop at Scale

[Big Data Spain] Apache Spark Streaming + Kafka 0.10: an Integration Story

Similar to Building the Ideal Stack for Real-Time Analytics

Digital Business Transformation in the Streaming Era

Attunity

Enterprises are rapidly adopting stream computing backbones, in-memory data stores, change data capture, and other low-latency approaches for end-to-end applications. As businesses modernize their data architectures over the next several years, they will begin to evolve toward all-streaming architectures. In this webcast, Wikibon, Attunity, and MemSQL will discuss how enterprise data professionals should migrate their legacy architectures in this direction. They will provide guidance for migrating data lakes, data warehouses, data governance, and transactional databases to support all-streaming architectures for complex cloud and edge applications. They will discuss how this new architecture will drive enterprise strategies for operationalizing artificial intelligence, mobile computing, the Internet of Things, and cloud-native microservices. Link to the Wikibon report - wikibon.com/wikibons-2018-big-data-analytics-trends-forecast Link to Attunity Streaming CDC Book Download - http://www.bit.ly/cdcbook Link to MemSQL's Free Data Pipeline Book - http://go.memsql.com/oreilly-data-pipelines

Slides: Case Study — How J.B. Hunt is Driving Efficiency with AI and Real-Tim...

DATAVERSITY

J.B. Hunt, one of the leading providers of transportation and logistics services in North America, recognizes the criticality of customer responsiveness, service quality, and operational efficiency for its success. However, with its data spread across multiple sources, including legacy mainframe systems, the organization was struggling to meet data requirements from multiple departments. They struggled to troubleshoot operational issues and respond to customers quickly. Join this webinar to hear about the optimized solution J. B. Hunt implemented, which automates real-time data pipelines for a reliable cloud data lake and provides multiple user groups an in-the-moment view of data without overwhelming internal operational systems. Discover how J.B. Hunt now leverages a modernized data environment to accelerate data delivery and drive various AI and analytics initiatives such as real-time service-pricing, competitive counterbidding, and improving their customer experience. Learn how you can: • Ingest data in real-time from legacy mainframe systems, enterprise applications, and more • Create a reliable cloud data lake to accelerate AI and Analytic Initiatives • Catalog, prepare, and provision data to empower data consumers • Drive operational efficiency and customer experience with AI-augmented insights

Enterprise Integration Patterns Revisited (EIP) for the Era of Big Data, Inte...

Kai Wähner

Data Culture Series - Keynote & Panel - 19h May - London

Jonathan Woodward

Big data. Small data. All data. You have access to an ever-expanding volume of data inside the walls of your business and out across the web. The potential in data is endless – from predicting election results to preventing the spread of epidemics. But how can you use it to your advantage to help move your business forward? Data is growing exponentially and it’s now possible to mine and unlock insights from data in new and unexpected ways. Empower your business to take advantage of this data by harnessing the rich capabilities of Microsoft SQL Server and the familiarity of Microsoft Office to help organize, analyze, and make sense of your data—no matter the size.

Building a Big Data Solution

James Serra

As a follow-on to the presentation "Building an Effective Data Warehouse Architecture", this presentation will explain exactly what Big Data is and its benefits, including use cases. We will discuss how Hadoop, the cloud and massively parallel processing (MPP) is changing the way data warehouses are being built. We will talk about hybrid architectures that combine on-premise data with data in the cloud as well as relational data and non-relational (unstructured) data. We will look at the benefits of MPP over SMP and how to integrate data from Internet of Things (IoT) devices. You will learn what a modern data warehouse should look like and how the role of a Data Lake and Hadoop fit in. In the end you will have guidance on the best solution for your data warehouse going forward.

Driving Datascience at scale using Postgresql, Greenplum and Dataiku - Greenp...

VMware Tanzu

From Spark to Ignition: Fueling Your Business on Real-Time Analytics

SingleStore

Enabling 360-degree Business Insights with SAP Data

Enterprise Management Associates

Create Your First SQL Server Cubes

Mark Kromer

Take Action: The New Reality of Data-Driven Business

Inside Analysis

The Briefing Room with Dr. Robin Bloor and WebAction Live Webcast on July 23, 2014 Watch the archive: https://bloorgroup.webex.com/bloorgroup/lsr.php?RCID=360d371d3a49ad256942f55350aa0a8b The waiting used to be the hardest part, but not anymore. Today’s cutting-edge enterprises can seize opportunities faster than ever, thanks to an array of technologies that enable real-time responsiveness across the spectrum of business processes. Early adopters are solving critical business challenges by enabling the rapid-fire design, development and production of very specific applications. Functionality can range from improved customer engagement to dynamic machine-to-machine interactions. Register for this episode of The Briefing Room to learn from veteran Analyst Dr. Robin Bloor, who will tout a new era in data-driven organizations, and why a data flow architecture will soon be critical for industry leaders. He’ll be briefed by Sami Akbay of WebAction, who will showcase his company’s real-time data management platform, which combines all the component parts needed to access, process and leverage data big and small. He’ll explain how this new approach can provide game-changing power to organizations of all types and sizes. Visit InsideAnlaysis.com for more information.

IMCSummit 2015 - Day 1 IT Business Track - From Spark to Ignition

In-Memory Computing Summit

Real-time is the next phase of big data. For the modern enterprise, processing and analyzing large volumes of data quickly is integral to success. MemSQL and Spark share design philosophies like memory-optimized data processing and scale out on commodity hardware that enable enterprises to build a real-time time data pipeline, where operational data is always online. This session shares hands-on 'how-to' recipes to build real-time pipelines around Apache Spark, with detailed production examples covered.

Webinar: Transforming Customer Experience Through an Always-On Data Platform

DataStax

According to Forrester Research, leaders in customer experience drive 5.1X revenue growth over laggards. And although 84% of companies aspire to be a leader in this space, only 1 in 5 successfully delivers good or great customer experience. Join us for our next webinar where Mike Gualtieri, VP and Principal Analyst at Forrester Research and Rajay Rai, Head of Digital Engineering at Macquarie Bank will share how Customer Experience can drive business results such as faster revenue growth, longer customer retention, greater employee engagement and improved profit margins. View webinar recording: https://youtu.be/eEc5tx-nHvI Explore past DataStax webinars: http://www.datastax.com/resources/webinars

DAMA Webinar: Turn Grand Designs into a Reality with Data Virtualization

Denodo

Watch full webinar here: https://buff.ly/2HMdbUp What started to evolve as the most agile and real-time enterprise data fabric, data virtualization is proving to go beyond its initial promise and is becoming one of the most important enterprise big data fabrics. Attend this session to learn: • What data virtualization really is, • How it differs from other enterprise data integration technologies • Real-world examples of data virtualization in action from companies such as Logitech, Autodesk and Festo.

How Businesses use Big Data to Impact the Bottom Line

Enterprise Management Associates

These slides - based on the webinar - shed light on how business stakeholders make the most of information from their big data environments and the requirements those stakeholders have to turn big data into business impact. Using recent big data end-user research from leading IT analyst firm Enterprise Management (EMA), data from Vertica’s recent benchmarks on SQL on Hadoop, and firsthand customer experiences, viewers will learn: - Use cases where end users around the world are using big data in their organizations - How maturity with big data strategies impact why and how business stakeholders use information from their big data environments - How Vertica empowers the use of information from big data environments

¿Cómo modernizar una arquitectura de TI con la virtualización de datos?

Denodo

Ver: https://bit.ly/347ImDf En la era digital, la gestión eficiente de los datos es un factor fundamental para optimizar la competitividad de las empresas. Sin embargo, la mayoría de ellas se enfrentan a silos de datos, lo que hace que su tratamiento sea lento y costoso. Además, la velocidad, la diversidad y el volumen de los datos pueden superar las arquitecturas de TI tradicionales. ¿Cómo mejorar la entrega de datos para extraer todo su valor? ¿Cómo conseguir que los datos estén disponibles y poder utilizarlos en tiempo real? Los expertos de Vault IT y Denodo te proponen este webinar para descubrir cómo la virtualización de datos permite modernizar una arquitectura de TI en un contexto de transformación digital.

Architecting for Real-Time Big Data Analytics

Rob Winters

Accelerate Self-Service Analytics with Data Virtualization and Visualization

Denodo

Watch full webinar here: https://bit.ly/3fpitC3 Enterprise organizations are shifting to self-service analytics as business users need real-time access to holistic and consistent views of data regardless of its location, source or type for arriving at critical decisions. Data Virtualization and Data Visualization work together through a universal semantic layer. Learn how they enable self-service data discovery and improve performance of your reports and dashboards. In this session, you will learn: - Challenges faced by business users - How data virtualization enables self-service analytics - Use case and lessons from customer success - Overview of the highlight features in Tableau

Develop a Custom Data Solution Architecture with NorthBay

Amazon Web Services

Organizations that have vast amounts of data in legacy applications often experience difficulties delivering that data to business unit end-users. Register to learn how Eliza Corporation and Scholastic overcame this challenge by leveraging a Data Lake solution from NorthBay on AWS to optimize data analytics and provide greater visibility. AWS and NorthBay will give you an in-depth overview of how you can use a Data Lake in conjunction with your existing on-premises or cloud-based Data Warehouse. NorthBay helps organizations scale their ETL and data warehousing workloads using Amazon EMR and Amazon Redshift. Join us to learn: • Best practices for using a Data Lake in conjunction with your existing data warehouse • The key aspects of introducing agile and scrum methodologies into an enterprise • The most impactful cost-savings levers that are addressed via a cloud data warehouse migration Who should attend: Heads of Analytics, Heads of BI, Analytics Managers, BI Teams, Senior Analysts

Devops the Microsoft Way

Patrick Chanezon

Devops is a cultural movement gathering developers and IT Pros responsible for operating applications around common values, goals, practices and tools in order to accelerate development and deployment cycles, creating fast feedback loops between development and operations. Like agility 15 years ago, Devops adoption, accelerated by Cloud platforms adoption, involves organizational, cultural, as well as tehcnical aspects. Emerging movement a few years ago, now well established at consumer web and mobile companies, Devops starts to get in the Enterprise. This presentation will explain the cultural and organizational aspects of the Devops movement, then will give an overview of the most common tools that are used to implement a Devops approach, showing that Microsoft is one of the few providers proposing a complete and integrated toolset, that works seamlessly for .NET developers, while integrating the most popular third party open source and proprietary tools, making Azure a great platform to implement a Devops approach for Linux, Java and open source workloads. We will talk about Visual Studio Online, Windows Azure, System Center, Windows Server, Azure Pack, PowerShell, NewRelic, Chef & Puppet integrations, Jenkins, … This deck was presented at Microsoft Techdays 2014, Read more at http://www.microsoft.com/france/mstechdays/programmes/2014/fiche-session.aspx?ID=07af5982-c413-46c3-8214-bba12365529b#0CDPXYrtwEbWxrgW.99

Devops a la sauce Microsoft

Microsoft

Devops est un mouvement culturel qui rassemble developpeurs et IT Pros specialistes des operations autour de valeurs, buts et pratiques communes, dans le but d’accelerer le cycle de developpement et deploiement des applications, en creant des boucles de feedback retroactives rapides entre developpement et operation. Comme l’agilite il y a 15 ans, l’adoption de Devops, acceleree par l’adoption des plateformes en nuage, met en jeu des aspects organisationnels, culturels, et techniques. Mouvement emergent il y a quelques annees, bien implante aujourd’hui chez les fournisseurs d’applications grand public, Devops commence a interesser les developpeurs en enterprise. Cette presentation fera un etat des lieux des aspects culturels et organisationnels du mouvement Devops, puis un etat de l’art des outils a la mode en ce moment pour implementer une demarche Devops, montrant que Microsoft est un des rares acteurs du marche a proposer une panoplie complete d’outils integres dans ce domaine, tout en s’integrant avec les outils existants, qu’ils soient proprietaires ou open source: Visual Studio Online, Windows Azure, System Center, Windows Server, Azure Pack, PowerShell, NewRelic, Chef & Puppet integrations, Jenkins, … Speaker : Patrick Chanezon (Microsoft)

Similar to Building the Ideal Stack for Real-Time Analytics (20)

Digital Business Transformation in the Streaming Era

Slides: Case Study — How J.B. Hunt is Driving Efficiency with AI and Real-Tim...

Enterprise Integration Patterns Revisited (EIP) for the Era of Big Data, Inte...

Data Culture Series - Keynote & Panel - 19h May - London

Building a Big Data Solution

Driving Datascience at scale using Postgresql, Greenplum and Dataiku - Greenp...

From Spark to Ignition: Fueling Your Business on Real-Time Analytics

Enabling 360-degree Business Insights with SAP Data

Create Your First SQL Server Cubes

Take Action: The New Reality of Data-Driven Business

IMCSummit 2015 - Day 1 IT Business Track - From Spark to Ignition

Webinar: Transforming Customer Experience Through an Always-On Data Platform

DAMA Webinar: Turn Grand Designs into a Reality with Data Virtualization

How Businesses use Big Data to Impact the Bottom Line

¿Cómo modernizar una arquitectura de TI con la virtualización de datos?

Architecting for Real-Time Big Data Analytics

Accelerate Self-Service Analytics with Data Virtualization and Visualization

Develop a Custom Data Solution Architecture with NorthBay

Devops the Microsoft Way

Devops a la sauce Microsoft

More from SingleStore

Five ways database modernization simplifies your data life

SingleStore

How Kafka and Modern Databases Benefit Apps and Analytics

SingleStore

Architecting Data in the AWS Ecosystem

SingleStore

Building the Foundation for a Latency-Free Life

SingleStore

Converging Database Transactions and Analytics

SingleStore

Building a Machine Learning Recommendation Engine in SQL

SingleStore

MemSQL 201: Advanced Tips and Tricks Webcast

SingleStore

Introduction to MemSQL

SingleStore

An Engineering Approach to Database Evaluations

SingleStore

Building a Fault Tolerant Distributed Architecture

SingleStore

Stream Processing with Pipelines and Stored Procedures

SingleStore

Curriculum Associates Strata NYC 2017

SingleStore

Image Recognition on Streaming Data

SingleStore

Learn how to leverage MPP technology and distributed data to deliver high volume transactional and analytical work loads which result in real time dashboards on rapidly changing data using standard SQL tools. Demonstrations will include the streaming of structured and JSON data from Kafka messages through a micro-batch ETL process into the MemSQL database where the data is then queried using standard SQL tools and visualized leveraging Tableau. This session will focus on image recognition, the techniques available, and how to put those techniques into production. It will further explore algebraic operations on tensors, and how that can assist in large-scale, high-throughput, highly-parallel image recognition. LIVE DEMO: Constructing and executing a real-time image recognition pipeline using Kafka and Spark. Speaker: Neil Dahlke, MemSQL Senior Solutions Engineer

Spark Summit Dublin 2017 - MemSQL - Real-Time Image Recognition

SingleStore

The State of the Data Warehouse in 2017 and Beyond

SingleStore

How Database Convergence Impacts the Coming Decades of Data Management

SingleStore

Teaching Databases to Learn in the World of AI

SingleStore

Gartner Catalyst 2017: The Data Warehouse Blueprint for ML, AI, and Hybrid Cloud

SingleStore

Gartner Catalyst 2017: Image Recognition on Streaming Data

SingleStore

Spark Summit West 2017: Real-Time Image Recognition with MemSQL and Spark

SingleStore

More from SingleStore (20)

Five ways database modernization simplifies your data life

How Kafka and Modern Databases Benefit Apps and Analytics

Architecting Data in the AWS Ecosystem

Building the Foundation for a Latency-Free Life

Converging Database Transactions and Analytics

Building a Machine Learning Recommendation Engine in SQL

MemSQL 201: Advanced Tips and Tricks Webcast

Introduction to MemSQL

An Engineering Approach to Database Evaluations

Building a Fault Tolerant Distributed Architecture

Stream Processing with Pipelines and Stored Procedures

Curriculum Associates Strata NYC 2017

Image Recognition on Streaming Data

Spark Summit Dublin 2017 - MemSQL - Real-Time Image Recognition

The State of the Data Warehouse in 2017 and Beyond

How Database Convergence Impacts the Coming Decades of Data Management

Teaching Databases to Learn in the World of AI

Gartner Catalyst 2017: The Data Warehouse Blueprint for ML, AI, and Hybrid Cloud

Gartner Catalyst 2017: Image Recognition on Streaming Data

Spark Summit West 2017: Real-Time Image Recognition with MemSQL and Spark

Recently uploaded

Climate Impact of Software Testing at Nordic Testing Days

Kari Kakkonen

My slides at Nordic Testing Days 6.6.2024 Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

FIDO Alliance

The Art of the Pitch: WordPress Relationships and Sales

Laura Byrne

Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes? All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf

Peter Spielvogel

Building better applications for business users with SAP Fiori. • What is SAP Fiori and why it matters to you • How a better user experience drives measurable business benefits • How to get started with SAP Fiori today • How SAP Fiori elements accelerates application development • How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities • How SAP Fiori paves the way for using AI in SAP apps

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Paige Cruz

Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack. While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack. I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:

Removing Uninteresting Bytes in Software Fuzzing

Aftab Hussain

Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process. In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds. - These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.

Monitoring Java Application Security with JDK Tools and JFR Events

Ana-Maria Mihalceanu

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

PCI PIN Basics Webinar from the Controlcase Team

ControlCase

Free Complete Python - A step towards Data Science

RinaMondal9

zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs

Alex Pruden

This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second). Paper: https://eprint.iacr.org/2023/1886

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DanBrown980551

Do you want to learn how to model and simulate an electrical network from scratch in under an hour? Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)! During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook. PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides: - A fully editable and extendable library for grid component modelling; - Visualization tools to display your network; - Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses; The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well. What you will learn during the webinar: - For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills; - For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.

Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx

nkrafacyberclub

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

SOFTTECHHUB

The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing. One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Sri Ambati

Assure Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance

Leading Change strategies and insights for effective change management pdf 1.pdf

OnBoard

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Prayukth K V

The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development. The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers: State of global ICS asset and network exposure Sectoral targets and attacks as well as the cost of ransom Global APT activity, AI usage, actor and tactic profiles, and implications Rise in volumes of AI-powered cyberattacks Major cyber events in 2024 Malware and malicious payload trends Cyberattack types and targets Vulnerability exploit attempts on CVEs Attacks on counties – USA Expansion of bot farms – how, where, and why In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East Why are attacks on smart factories rising? Cyber risk predictions Axis of attacks – Europe Systemic attacks in the Middle East Download the full report from here: https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/

Recently uploaded (20)