Ironically, Infrastructure Doesn't Matter - Quinton Anderson, Commonwealth Bank of Australia

•Download as PPTX, PDF•

1 like•427 views

Audience Level All levels Synopsis Our journey towards solving our Application and Infrastructure Problems using Immutability, Codification, Mesos, Docker and Ironic. OpenStack Australia Day Melbourne 2017 https://events.aptira.com/openstack-australia-day-melbourne-2017/

Ironically,
Infrastructure
doesn’t matter
Quinton Anderson
Head of Engineering
Analytics and Information
CBA

Downstream
systems
• Specialised
management
systems
• Reporting Systems
• Product
management
Channel &
product
systems
Master Data
Management
Hadoop
• Leverage all data & reduce
integration costs
• Comprehensive dataset –
internal & external, realtime &
batch, structured & unstructured
• Advanced analytics / machine
learning
Group Data
Warehouse
• Understand our business
• Accurate, conformed, and
reconciled data
• Access layer to support BI &
reporting
BI/Reporting
• User facing tools
• Regulatory reporting
• Dishoarding
• Self service BI for the
masses
Customer record &
insights
All data
Price,
conversation,
credit dec.
etc.
Financial Data
Subset of
data
User
access
Information for
people
Core Financial
Systems and
functions
• P&L
• Recon
• General Ledger
• Etc…
Closed loop,
automated ‘decisions’
Decisioning
• Personalise/optimise decisions,
maximise customer value
• E.g. price, credit decision, next
conversation, experience
Core information repositories
Analytics applications
Other systems

Hbase,
Cassandra,
HDFS,
Influx,
Elastic Search,
Kafka,
Etcd,
Zookeeper
OpenStack Swift

Dev,
Test,
Staging,
Prod 1,
Prod 2,
Etc…

- Unit Cost
- Cost of change
- Developer Satisfaction

Repo(s) CI/CD Apps
Docker Calico
Mesos + Marathon
Yarn
Spark, MR, Impala, etc
CI/CD
CI/CD
Repo(s)
Repo(s)
Open
Stack Nova/Ironic
OS
Firmware + Hardware + Tags
Vault

- VRFs with Route Leakage for migration
- Vlan and route separation of IPMI
- Tagged Vlan allows for IPMI
management servers to be logical

- Collaboration
- Track evolution
- Automation friendly
- Reproducibility

OS Image
Container X
Environment
CI Artefactory
CI Environment

CI
Environment
Branch PR
Merge
Dev
Deploy
Master
Deploy
Test
Change
Plan

Application
Infrastructure
Infrastructure

Learn how SolidFire storage can be used to deliver predictable application performance and improve infrastructure efficiency in your OpenStack environment. This presentation covers: - Consolidation of multiple mission critical applications on the same storage system - Integrated End-to-End Quality of Service allocation using OpenStack Cinder Volume Types to eliminate inconsistent performance - Boot from Cinder Volumes - Cinder snapshots and clone offloading with SolidFire

SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...Splunk

Low-latency real-time data processing at giga-scale with Kafka | John DesJard...

HostedbyConfluent

Data volumes continue to grow, demanding new, more scalable solutions for low-latency data processing. Previously, the default approach to deploying such systems was to throw a ton of hardware at the problem. However, that is no longer necessary, as newer technologies showcase a level of efficiency that enables smaller, more manageable clusters while handling extreme workloads. Processing billions of events per second on Kafka can now be done with a modest investment in compute resources. In this session, you will learn how to architect and build the fastest data processing applications that scale linearly, and combine streaming data and reference data data-in-motion and data-at-rest with machine learning. We will take you through the end-to-end framework and example application, built on the Hazelcast Platform, an open source software engine designed for ultra-fast performance. We will also show how you can leverage SQL to further explore the operational data in the solution including querying Kafka topics and key-value data on the in-memory data store. Attendees will also get access to the Github sample application shown.

Presto: Fast SQL-on-Anything Across Data Lakes, DBMS, and NoSQL Data Stores

Alluxio, Inc.

Kyligence Cloud 4 - An Overview

SamanthaBerlant

In January of this year, Kyligence announced the immediate availability of Kyligence Cloud 4, the first fully cloud-native, distributed OLAP platform. During our announcement, EMA analyst John Santaferraro said: “As the race for unified analytics heats up, Kyligence offers a solution that overcomes the challenges of querying data in both data lakes and data warehouses located both in the cloud and on premises.” Join Li Kang - VP of North America at Kyligence - as he provides an overview of the Kyligence Cloud 4 release that will show: --The new cloud native architecture that employs Apache Kylin, Apache Spark, and Apache Parquet to ensure optimal performance. --How KC4 delivers sub-second query responses on very large datasets using precomputed aggregate indexes (hyper-cubes) and table indexes. --The AI-Augmented engine that intelligently organizes your data and reduces data modeling time from days/weeks to minutes. In this presentation, we will present the Kyligence Cloud 4 story - high-speed analytics with unprecedented sub-second query response times against petabyte datasets.

Building the Next-gen Digital Meter Platform for Fluvius

Databricks

Fluvius is the network operator for electricity and gas in Flanders, Belgium. Their goal is to modernize the way people look at energy consumption using a digital meter that captures consumption and injection data from any electrical installation in Flanders ranging from households to large companies. After full roll-out there will be roughly 7 million digital meters active in Flanders collecting up to terabytes of data per day. Combine this with regulation that Fluvius has to maintain a record of these reading for at least 3 years, we are talking petabyte scale. delaware BeLux was assigned by Fluvius to setup a modern data platform and did so on Azure using Databricks as the core component to collect, store, process and serve these volumes of data to every single consumer and beyond in Flanders. This enables the Belgian energy market to innovate and move forward. Maarten took up the role as project manager and solution architect.

Cloudian and Rubrik - Hybrid Cloud based Disaster Recovery

Cloudian

Cloudian and Rubrik outline the benefits of using a modern hybrid cloud based approach for your VMware backups. While everyone is promising instant restores and shorter backup windows, Rubrik's solution with Cloudian additionally adds policy based management, reducing complexity. Tiering of data to Cloudian S3 Object Storage ensures that only the hottest backups are consuming SSD storage. Both solutions are scale-out so adding addition capacity is a breeze.

Enterprises are beginning to consider the deployment of data science and data warehouse platforms on hybrid (public cloud, private cloud, and on premises) infrastructure. This delivers the flexibility and freedom of choice to deploy your analytics anywhere you need it and to create an adaptable and agile analytics platform. But the market is conspiring against customer desire for innovation... Leading public cloud vendors are interested in pushing their new, but proprietary, analytic stacks, locking customers into subpar Analytics as a Service (AaaS) for years to come. In tandem, Legacy Data Warehouse vendors are trying to extend the lifecycle of their costly and aging appliances with new features of marginal value, simply imitating the same limiting models of public cloud vendors. New vendors are coming up with interesting ideas, but these ideas are often lacking critical features that don’t provide support for hybrid solutions, limiting the immediate value to users. It is 2017—you can, in fact, have your analytics cake and eat it too! Solve your short term costs and capabilities challenges, and establish a long term hybrid data strategy by running the same open source analytics platform on your infrastructure as it exists today. In this webinar you will learn how Pivotal can help you build a modern analytical architecture able to run on your public, private cloud, or on-premises platform of your choice, while fully leveraging proven open source technologies and supporting the needs of diverse analytical users. Let’s have a productive discussion about how to deploy a solid cloud analytics strategy. Presenter : Jacque Istok, Head of Data Technical Field for Pivotal https://content.pivotal.io/webinars/jul-20-how-to-build-modern-data-architectures-both-on-premises-and-in-the-cloud

Building Custom Big Data Integrations

Pat Patterson

Big data ingest frameworks ship with an array of connectors for common data origins and destinations, such as flat files, S3, HDFS, Kafka etc, but sometimes, you need to send data to, or receive data from a system that's not on the list. StreamSets includes template code for building your own connectors and processors; we'll walk through the process of building a simple destination that sends data to a REST web service, and show how it can be extended to target more sophisticated systems such as Salesforce Wave Analytics.

Extracting Value from IOT using Azure Cosmos DB, Azure Synapse Analytics and ...

HostedbyConfluent

Due to explosion of IoT, we have streaming data that needs to be processed in real-time. This needs to be made available for applications as well as analytics scenarios such as anomaly detection. This workshop presents a solution using Confluent Cloud on Azure, Azure Cosmos DB and Azure Synapse Analytics which can be connected in a secure way within Azure VNET using Azure Private link configured on Kafka clusters.

Seize Profits in the Cloud with SolidFire

NetApp

Big Data Case Study: Fortune 100 Telco

BlueData, Inc.

This Big Data case study outlines the Hadoop infrastructure deployment for a Fortune 100 media and telecommunications company. Hadoop adoption in this company had grown organically across multiple different teams, starting with “science projects” and lab initiatives that quickly grew and expanded. Going forward, some of the options they considered for their Big Data deployment included expanding their on-premises infrastructure and using a Hadoop-as-a-Service cloud offering. Fortunately, they realized that there is a third option: providing the benefits of Hadoop-as-a-Service with on-premises infrastructure. They selected the BlueData EPIC software platform to virtualize their Hadoop infrastructure and provide on-demand access to virtual Hadoop clusters in a secure, multi-tenant model. Learn more about this case study in the blog post at: http://www.bluedata.com/blog/2015/05/big-data-case-study-hadoop-infrastructure

Data stax webinar cassandra and titandb insights into datastax graph strategy...

DataStax

Cloud Stack & Cloud Storage

NetApp

Achieving cyber mission assurance with near real-time impact

Elasticsearch

See how Elastic and ECS support the Mission Assurance Decision Support System (MADSS) program for the Navy with observability, data enrichment, and powerful search in a containerized environment. By correlating data from diverse sources using web-based services and secure, automated data transformation services, MADSS improves responsiveness, predictions, and event analysis for critical network and infrastructure outages.

Cassandra-as-a-Service

Instaclustr

Denodo DataFest 2017: Integrating Big Data and Streaming Data with Enterprise...

Denodo

Watch live presentation here: https://goo.gl/UcZEHU Big data projects are becoming mature and consistent. However, they remain siloed compared to the enterprise data. In addition, now new streaming data needs to integrated as well. Watch this Denodo DataFest 2017 session to discover: • How big data projects can be combined with other enterprise data. • How to integrate streaming data into the mix. • Benefits of aggregating the data without having to move them into a centralized repository.

Cost Effectively Run Multiple Oracle Database Copies at Scale

NetApp

Complex Analytics with NoSQL Data Store in Real Time

Nati Shalom

NOSQL are often limited in the type of queries that they can support due to the distributed nature of the data. In this session we would learn patterns on how we can overcome this limitation and combine multiple query semantics with NoSQL based engines. We will demonstrate specifically a combination of key/value, SQL like, Document model and Graph based queries as well as more advanced topic such as handling partial update and query through projection. We will also demonstrate how we can create a meshaup between those API's i.e. write fast through Key/Value API and execute complex queries on that same data through SQL query. - See more at: http://nosql2014.dataversity.net/sessionPop.cfm?confid=81&proposalid=6335#sthash.PNSZi5TJ.dpuf

Webinar | How Clear Capital Delivers Always-on Appraisals on 122 Million Prop...

DataStax

In online residential and commercial real estate, even fractions of seconds in response times affect customer satisfaction and conversion to revenue. The need for continuous availability is paramount to deliver the levels of service customers demand from modern online applications. Join David Prinzing, Enterprise Architect at Clear Capital to discover why Clear Capital, a premium provider of real estate asset valuation and collateral risk assessment, chose DataStax as the database-backbone for their ClearCollateral Platform. David will discuss how DataStax Enterprise, the world’s fastest, most scalable distributed database technology built on Apache Cassandra ensures 100% uptime for over 122 million properties (90% of all the properties in the United States) and supports reporting on over 1 Billion total valuations while never going down. - The challenges in building real-time applications using relational technologies is forcing financial services firms to migrate to distributed database technologies - How Clear Capital delivers 100% availability and real-time decision support across multiple data centers in the Amazon Cloud using DataStax Enterprise - Why Apache Cassandra’s architecture delivers always-on, customer engaging applications that capture new business opportunities

Cortana Analytics Workshop: The "Big Data" of the Cortana Analytics Suite, Pa...

MSAdvAnalytics

Lance Olson. Cortana Analytics is a fully managed big data and advanced analytics suite that helps you transform your data into intelligent action. Come to this two-part session to learn how you can do "big data" processing and storage in Cortana Analytics. In the first part, we will provide an overview of the processing and storage services. We will then talk about the patterns and use cases which make up most big data solutions. In the second part, we will go hands-on, showing you how to get started today with writing batch/interactive queries, real-time stream processing, or NoSQL transactions all over the same repository of data. Crunch petabytes of data by scaling out your computation power to any sized cluster. Store any amount of unstructured data in its native format with no limits to file or account size. All of this can be done with no hardware to acquire or maintain and minimal time to setup giving you the value of "big data" within minutes. Go to https://channel9.msdn.com/ to find the recording of this session.

Pioneer Financial ver 1.0Pioneer Financial

Skillwise Big Data part 2

Skillwise Group

What's hot

Unified Data Access with Gimel

Alluxio, Inc.

How DataStax Enterprise and Azure Make Your Apps Scale from Day 1

DataStax

What? VDI without Nutanix and ControlUp?!

NEXTtour

Speeding Up Atlas Deep Learning Platform with Alluxio + Fluid

Alluxio, Inc.

Apache frameworks for Big and Fast Data

Naveen Korakoppa

IMC Summit 2016 Breakout - Steve Wikes - Making IMC Enterprise Grade

In-Memory Computing Summit

How to Build Modern Data Architectures Both On Premises and in the Cloud

VMware Tanzu

Building Custom Big Data Integrations

Pat Patterson

Extracting Value from IOT using Azure Cosmos DB, Azure Synapse Analytics and ...

HostedbyConfluent

Seize Profits in the Cloud with SolidFire

NetApp

Big Data Case Study: Fortune 100 Telco

BlueData, Inc.

Data stax webinar cassandra and titandb insights into datastax graph strategy...

DataStax

Cloud Stack & Cloud Storage

NetApp

Achieving cyber mission assurance with near real-time impact

Elasticsearch

Cassandra-as-a-Service

Instaclustr

Denodo DataFest 2017: Integrating Big Data and Streaming Data with Enterprise...

Denodo

Cost Effectively Run Multiple Oracle Database Copies at Scale

NetApp

Complex Analytics with NoSQL Data Store in Real Time

Nati Shalom

Webinar | How Clear Capital Delivers Always-on Appraisals on 122 Million Prop...

DataStax

Cortana Analytics Workshop: The "Big Data" of the Cortana Analytics Suite, Pa...

MSAdvAnalytics

What's hot (20)

Unified Data Access with Gimel

How DataStax Enterprise and Azure Make Your Apps Scale from Day 1

What? VDI without Nutanix and ControlUp?!

Speeding Up Atlas Deep Learning Platform with Alluxio + Fluid

Apache frameworks for Big and Fast Data

IMC Summit 2016 Breakout - Steve Wikes - Making IMC Enterprise Grade

How to Build Modern Data Architectures Both On Premises and in the Cloud

Building Custom Big Data Integrations

Extracting Value from IOT using Azure Cosmos DB, Azure Synapse Analytics and ...

Seize Profits in the Cloud with SolidFire

Big Data Case Study: Fortune 100 Telco

Data stax webinar cassandra and titandb insights into datastax graph strategy...

Cloud Stack & Cloud Storage

Achieving cyber mission assurance with near real-time impact

Cassandra-as-a-Service

Denodo DataFest 2017: Integrating Big Data and Streaming Data with Enterprise...

Cost Effectively Run Multiple Oracle Database Copies at Scale

Complex Analytics with NoSQL Data Store in Real Time

Webinar | How Clear Capital Delivers Always-on Appraisals on 122 Million Prop...

Cortana Analytics Workshop: The "Big Data" of the Cortana Analytics Suite, Pa...

Similar to Ironically, Infrastructure Doesn't Matter - Quinton Anderson, Commonwealth Bank of Australia

Pioneer Financial ver 1.0Pioneer Financial

Skillwise Big Data part 2

Skillwise Group

Decision support systems and business intelligence

Shwetabh Jaiswal

BI slides_Viatec DTAPP

Eric Anderson

Skilwise Big data

Skillwise Group

Business Intelligence Architecture

Philippe Julio

The essentials of business intelligence

Shwetabh Jaiswal

Hybrid Analytics in Healthcare: Leveraging Power BI and Office 365 to Make Sm...

Perficient, Inc.

As organizations realize the cost savings and scalability benefits of hybrid environments, the focus turns to implementing an analytics platform in these new environments. On-premises vs. cloud is the big choice, but how are some companies leveraging the best of both worlds? Perficient and UnityPoint Health discussed the benefits of Power BI and Office 365, and how one technology-savvy healthcare provider is leveraging its hybrid environment of Power BI, Excel-enabled dashboards and SharePoint 2013.

Big Data + PeopleSoft = BIG WIN!

Smart ERP Solutions, Inc.

Workshop: Make the Most of Customer Data Platforms - David Raab

Digital Customer Experience (DX) Summit

Marketers have more data available than ever but struggle to pull in together in a usable format. Customer Data Platforms promise to solve this problem by offering easy-to-deploy systems specializing in data unification and sharing. But can CDP really deliver on its promise? This workshop will equip you to understand the definition of a CDP, how CDPs differ from other systems, which features are shared by all CDPs and which are found in only some, the most important CDP use cases, how to select the right CDP, how to manage a successful deployment and where to look next for more information.

Decision support systems and business intelligence

Shwetabh Jaiswal

BizTrans SysTech_Analytics_Serv_SAP_v1.0BizTrans SysTech

Business analytics and data visualisation

Shwetabh Jaiswal

Basic concepts in mis

Renewable Energy Federation

Hadoop in the Cloud: Common Architectural Patterns

DataWorks Summit

The Business Value of Business Intelligence

Senturus

Learn about various BI architectures and approaches, as well as a comparison of different vendors’ BI offerings. See a demonstration of OLAP cube building. View the video recording and download this deck: http://www.senturus.com/resources/the-business-value-of-business-intelligence/ Senturus, a business analytics consulting firm, has a resource library with hundreds of free recorded webinars, trainings, demos and unbiased product reviews. Take a look and share them with your colleagues and friends: http://www.senturus.com/resources/.

Prez szabolcsDoina Draganescu

Migrating from Reports to Dashboards-an approach for the masses

Smart ERP Solutions, Inc.

Presentation from Smart ERP Solutions conveying the benefits of embedded analytic dashboards in PeopleSoft applications compared to offline static reports. Many organizations are hampered by continued use of inefficient static reports to run their business. Covered is how to easily advance beyond legacy static reports to more visual and interactive methods of accessing information to improve decision making. Covers a solution that embeds rich dynamic dashboards providing key analytics directly into your PeopleSoft applications, not in some separate costly reporting warehouse. Sample dashboards embedded into PeopleSoft applicatons include: Employee Onboarding; Employee Survey; Grants Distribution; Workflow Approval; Sales Review; and Tax Reconciliation.

BI and Predictive analytics 2011 shyam desigan presentation

Shyam Desigan

Business Analytics Training

Natalija Pavic

Similar to Ironically, Infrastructure Doesn't Matter - Quinton Anderson, Commonwealth Bank of Australia (20)

Pioneer Financial ver 1.0

Skillwise Big Data part 2

Decision support systems and business intelligence

BI slides_Viatec DTAPP

Skilwise Big data

Business Intelligence Architecture

The essentials of business intelligence

Hybrid Analytics in Healthcare: Leveraging Power BI and Office 365 to Make Sm...

Big Data + PeopleSoft = BIG WIN!

Workshop: Make the Most of Customer Data Platforms - David Raab

Decision support systems and business intelligence

BizTrans SysTech_Analytics_Serv_SAP_v1.0

Business analytics and data visualisation

Basic concepts in mis

Hadoop in the Cloud: Common Architectural Patterns

The Business Value of Business Intelligence

Prez szabolcs

Migrating from Reports to Dashboards-an approach for the masses

BI and Predictive analytics 2011 shyam desigan presentation

Business Analytics Training

More from OpenStack

Swinburne University of Technology - Shunde Zhang & Kieran Spear, Aptira

Ironically, Infrastructure Doesn't Matter - Quinton Anderson, Commonwealth Bank of Australia

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Ironically, Infrastructure Doesn't Matter - Quinton Anderson, Commonwealth Bank of Australia

Similar to Ironically, Infrastructure Doesn't Matter - Quinton Anderson, Commonwealth Bank of Australia (20)

More from OpenStack

More from OpenStack (20)

Recently uploaded

Recently uploaded (20)

Ironically, Infrastructure Doesn't Matter - Quinton Anderson, Commonwealth Bank of Australia