SlideShare a Scribd company logo
In a constantly changing industry you need
a forward-thinking technology partner
Introducing Mache – an Open
Source Big Data Cache
James Bowkett
@techwob
mache@excelian.com
What is Mache?
2
Mache is a NoSQL near cache with eventing; built from a 'mash-up' of
Open Source technologies with multiple pluggable NoSQL support and
multiple pluggable messaging platforms.We have used all the well
known Open-Source technologies.The heart of the system is akin to a
HashMap - usingGoogle's Guava cache and Spring Data.
The principle idea behind Mache was to abstract itself from the
Database layer. As a result it allows the two to remain somewhat
independent of each other, enabling concentration on development and
performance rather than scaling out the data layer. It has been
engineered to be as lightweight as possible, and because of theVendor
independence, Mache could offer a free caching solution to an already
deployed NoSQL database. It doesn’t need to know about the volume of
data which means that it can be as performant as the system allows.
It was also designed to be a piece of software which integrates well with
other software. It offers support for NoSQL programmes such as
Cassandra, MongoDB andCouchbase. But it also lends itself well to
support integration with pre-existing systems such as Kafka, RabbitMQ
andActiveMQ message services.
3
What is Mache?
Mache
(in-memory cache)
Cassandra
Mongo
Couchbase
Rabbit MQ
Kafka
Active MQ
Can write
through to…
Can send
notifications
to…
File
4
Example Use Cases
• Generally speaking, most grid applications will have a heavy reliance on read-mostly data. More relevant to
today’s challenges are MiFID II andCVA.
• Since the introduction of the MiFID II market makers must warehouse the audit data around their prices and
spread calculations.This inevitably leads to the storage of large volumes of data, which also offers the
opportunity for Big Data.Applications now need to read and write this data, and we believe Mache offers
the opportunity to help turn a ‘data lake’ into a localised manageable view of this data deluge.
• Take for instance, today’s trades or their price points. Mache fits naturally within the speed layer in a lambda
architecture.
• CreditValue Adjustment (CVA) requires large volumes of Monte Carlo data, and in some cases will need 300-
400GB of data for a single day.When stress testing or back testing is being performed, you could need 30+
days’ worth of data on each compute node. 400GB * 10k = a lot of data!
Why have you built Mache?
• The original idea was to provide an open source
alternative to Coherence
• Other caching API’s tend to offer their own
persistence formats, instead of integrating with the
leading NoSQL vendors
• The network is often the bottleneck within grid
platforms, near side caching is a solution
• Filtered views will open up use cases
– E.g. only view a trader’s trades for today
– Market/reference data distribution to processing nodes
5
How does Mache improve performance?
6
How does Mache improve performance?
7
How do I use Mache?
8
How does Mache compare with other existing technologies?
The following is a list of technologies that can fulfil similar use cases to Mache.While they are all leading technologies in
their field, it is important to note how they are distinctly different to Mache.
 Ehcache - is a distributed JSR-107-compliant cache that has pluggable storage mechanisms (on-heap/off-heap
and file-based). However it doesn't have out-of-the-box support for any NoSQL platforms.
 Gridgain - is a purely in-memory data and processing grid platform. It has no current support for NoSQL
platforms.
 Hazelcast - is another in-memory grid implementation, often marketed as a replacement for Oracle
Coherence.
 Infinispan - is a single node or clustered JSR-107-compliant in-memory cache implementation. Its persistence is
handled by pluggable adapters the application teams may have to write themselves. It provides no NoSQL
support as standard.
 Memcached - is a multi-node object cache with no persistence mechanism as standard.
 Redis - is an all-purpose in-memory data store with no persistence mechanism as standard.
In our view, there was no pre-existing solution that would provide for the simplest use case of near caching to abstract
away storage whilst still supporting the big 3 platforms. We proposed a solution that would adhere to the following
guiding principles:
 Lightweight - the Mache-core jar depends on 3 other libraries (in addition to any required vendor drivers for
messaging or persistence)
 Pluggable with existing NoSQL Platforms, and thereby creating no vendor lock-in
 Provide a small set of valuable features, adhering to the Unix principle of 'do one thing, and do it well'
9
So what is next for Mache?
We have quite a few enhancements on our list. In no particular order:
 JMX to provide configuration and management
 Overflow cache - so we can overflow from memory to disk using something like Berkley DB
 JSON/RestAPI - to provide a platform agnostic way of accessing local cached data which is run in a
service.
 Eventing straight from the database
 Filtered materialised views/Continuous query caches
 Shared cache instance between processes
 Distributed file system
10
What’s next for Mache?
For more information:
• email us at mache@excelian.com
• https://github.com/excelian/mache
• 0.6.1 available on jcenter/bintray
• Read more on our blog – Mache – Near Side Caching for NoSQL.
11
http://www.excelian.com/blog/mache---near-side-caching-for-nosql-/#.Vl2sME02q70
Cape Town Office
1st Floor, Corporate Place
13 Mispel Road, Bellville
Cape Town, 7530
South Africa
+27(0) 21 944 9900
Johannesburg Office
4th Floor, The Firs
Cradock Avenue. Rosebank
Johannesburg, SA
+27 (0) 11 759 4020
New York Office
8th Floor, 260 Madison Avenue
New York, NY 10016
United States
+1646 216 2065
Frankfurt Office
Brockenheimer Landstrasse 17/19
60325 Frankfurt
Germany
+49 (0) 69 710455 183
London Office
44 Featherstone Street
London, EC1Y 8RN
United Kingdom
Sydney Office
1 York Street
Level 2
Sydney, NSW 2000
Australia
+61 (2) 9191 7810
Toronto Office
100 King Street West, Suite 5600
Toronto, Ontario M5X 1C9
Canada
+1 (203) 295 5240
www.excelian.com

More Related Content

What's hot

NoSQL Databases
NoSQL DatabasesNoSQL Databases
NoSQL Databases
Ashish Karki
 
Assignment_4
Assignment_4Assignment_4
Assignment_4Kirti J
 
Quantitative Performance Evaluation of Cloud-Based MySQL (Relational) Vs. Mon...
Quantitative Performance Evaluation of Cloud-Based MySQL (Relational) Vs. Mon...Quantitative Performance Evaluation of Cloud-Based MySQL (Relational) Vs. Mon...
Quantitative Performance Evaluation of Cloud-Based MySQL (Relational) Vs. Mon...
Darshan Gorasiya
 
Software Defined storage
Software Defined storageSoftware Defined storage
Software Defined storageKirillos Akram
 
Backbone using Extensible Database APIs over HTTP
Backbone using Extensible Database APIs over HTTPBackbone using Extensible Database APIs over HTTP
Backbone using Extensible Database APIs over HTTP
Max Neunhöffer
 
Distributed storage system
Distributed storage systemDistributed storage system
Distributed storage system
Công Lợi Dương
 
Survey of distributed storage system
Survey of distributed storage systemSurvey of distributed storage system
Survey of distributed storage system
Zhichao Liang
 
Reducing large S3 API costs using Alluxio at Datasapiens
Reducing large S3 API costs using Alluxio at Datasapiens Reducing large S3 API costs using Alluxio at Datasapiens
Reducing large S3 API costs using Alluxio at Datasapiens
Alluxio, Inc.
 
NoSQL Consepts
NoSQL ConseptsNoSQL Consepts
NoSQL Consepts
Maynooth University
 
NOSQL- Presentation on NoSQL
NOSQL- Presentation on NoSQLNOSQL- Presentation on NoSQL
NOSQL- Presentation on NoSQL
Ramakant Soni
 
StorageQuery: federated querying on object stores, powered by Alluxio and Presto
StorageQuery: federated querying on object stores, powered by Alluxio and PrestoStorageQuery: federated querying on object stores, powered by Alluxio and Presto
StorageQuery: federated querying on object stores, powered by Alluxio and Presto
Alluxio, Inc.
 
Choosing The Right Database For Your Cloud Application
Choosing The Right Database For Your Cloud ApplicationChoosing The Right Database For Your Cloud Application
Choosing The Right Database For Your Cloud ApplicationNuoDB
 
NoSQL Databases: Why, what and when
NoSQL Databases: Why, what and whenNoSQL Databases: Why, what and when
NoSQL Databases: Why, what and when
Lorenzo Alberton
 
NetApp Se training storage grid webscale technical overview
NetApp Se training   storage grid webscale technical overviewNetApp Se training   storage grid webscale technical overview
NetApp Se training storage grid webscale technical overview
solarisyougood
 
Home For Gypsies – Storage for NoSQL Databases​
Home For Gypsies – Storage for NoSQL Databases​Home For Gypsies – Storage for NoSQL Databases​
Home For Gypsies – Storage for NoSQL Databases​
Atish Kathpal
 
NoSQL Data Architecture Patterns
NoSQL Data ArchitecturePatternsNoSQL Data ArchitecturePatterns
NoSQL Data Architecture Patterns
Maynooth University
 
OpenDrives_-_Product_Sheet_v13D (2) (1)
OpenDrives_-_Product_Sheet_v13D (2) (1)OpenDrives_-_Product_Sheet_v13D (2) (1)
OpenDrives_-_Product_Sheet_v13D (2) (1)Scott Eiser
 
Red hat, inc. open storage in the enterprise 0
Red hat, inc.   open storage in the enterprise 0Red hat, inc.   open storage in the enterprise 0
Red hat, inc. open storage in the enterprise 0
Tommy Lee
 
Introduction of Redis as NoSQL Database
Introduction of Redis as NoSQL DatabaseIntroduction of Redis as NoSQL Database
Introduction of Redis as NoSQL Database
Abhijeet Shekhar
 
Introduction to NoSQL
Introduction to NoSQLIntroduction to NoSQL
Introduction to NoSQL
Dimitar Danailov
 

What's hot (20)

NoSQL Databases
NoSQL DatabasesNoSQL Databases
NoSQL Databases
 
Assignment_4
Assignment_4Assignment_4
Assignment_4
 
Quantitative Performance Evaluation of Cloud-Based MySQL (Relational) Vs. Mon...
Quantitative Performance Evaluation of Cloud-Based MySQL (Relational) Vs. Mon...Quantitative Performance Evaluation of Cloud-Based MySQL (Relational) Vs. Mon...
Quantitative Performance Evaluation of Cloud-Based MySQL (Relational) Vs. Mon...
 
Software Defined storage
Software Defined storageSoftware Defined storage
Software Defined storage
 
Backbone using Extensible Database APIs over HTTP
Backbone using Extensible Database APIs over HTTPBackbone using Extensible Database APIs over HTTP
Backbone using Extensible Database APIs over HTTP
 
Distributed storage system
Distributed storage systemDistributed storage system
Distributed storage system
 
Survey of distributed storage system
Survey of distributed storage systemSurvey of distributed storage system
Survey of distributed storage system
 
Reducing large S3 API costs using Alluxio at Datasapiens
Reducing large S3 API costs using Alluxio at Datasapiens Reducing large S3 API costs using Alluxio at Datasapiens
Reducing large S3 API costs using Alluxio at Datasapiens
 
NoSQL Consepts
NoSQL ConseptsNoSQL Consepts
NoSQL Consepts
 
NOSQL- Presentation on NoSQL
NOSQL- Presentation on NoSQLNOSQL- Presentation on NoSQL
NOSQL- Presentation on NoSQL
 
StorageQuery: federated querying on object stores, powered by Alluxio and Presto
StorageQuery: federated querying on object stores, powered by Alluxio and PrestoStorageQuery: federated querying on object stores, powered by Alluxio and Presto
StorageQuery: federated querying on object stores, powered by Alluxio and Presto
 
Choosing The Right Database For Your Cloud Application
Choosing The Right Database For Your Cloud ApplicationChoosing The Right Database For Your Cloud Application
Choosing The Right Database For Your Cloud Application
 
NoSQL Databases: Why, what and when
NoSQL Databases: Why, what and whenNoSQL Databases: Why, what and when
NoSQL Databases: Why, what and when
 
NetApp Se training storage grid webscale technical overview
NetApp Se training   storage grid webscale technical overviewNetApp Se training   storage grid webscale technical overview
NetApp Se training storage grid webscale technical overview
 
Home For Gypsies – Storage for NoSQL Databases​
Home For Gypsies – Storage for NoSQL Databases​Home For Gypsies – Storage for NoSQL Databases​
Home For Gypsies – Storage for NoSQL Databases​
 
NoSQL Data Architecture Patterns
NoSQL Data ArchitecturePatternsNoSQL Data ArchitecturePatterns
NoSQL Data Architecture Patterns
 
OpenDrives_-_Product_Sheet_v13D (2) (1)
OpenDrives_-_Product_Sheet_v13D (2) (1)OpenDrives_-_Product_Sheet_v13D (2) (1)
OpenDrives_-_Product_Sheet_v13D (2) (1)
 
Red hat, inc. open storage in the enterprise 0
Red hat, inc.   open storage in the enterprise 0Red hat, inc.   open storage in the enterprise 0
Red hat, inc. open storage in the enterprise 0
 
Introduction of Redis as NoSQL Database
Introduction of Redis as NoSQL DatabaseIntroduction of Redis as NoSQL Database
Introduction of Redis as NoSQL Database
 
Introduction to NoSQL
Introduction to NoSQLIntroduction to NoSQL
Introduction to NoSQL
 

Viewers also liked

Cassandra and Hybrid Cloud - Introducing Mache
Cassandra and Hybrid Cloud - Introducing MacheCassandra and Hybrid Cloud - Introducing Mache
Cassandra and Hybrid Cloud - Introducing Mache
Excelian | Luxoft Financial Services
 
Без воды: презентация для первых лиц
Без воды: презентация для первых лицБез воды: презентация для первых лиц
Без воды: презентация для первых лиц
Marina Lopyrova
 
How to succeed at UPwork
How to succeed at UPworkHow to succeed at UPwork
How to succeed at UPwork
Marina Lopyrova
 
TriggMine - сервис возврата брошенных корзин
TriggMine - сервис возврата брошенных корзинTriggMine - сервис возврата брошенных корзин
TriggMine - сервис возврата брошенных корзин
Marina Lopyrova
 
Grid computing and trade analytics with elastic - Jay Chin's presentation at ...
Grid computing and trade analytics with elastic - Jay Chin's presentation at ...Grid computing and trade analytics with elastic - Jay Chin's presentation at ...
Grid computing and trade analytics with elastic - Jay Chin's presentation at ...
Excelian | Luxoft Financial Services
 
teoria de la arquitectura-Arquitectura postmoderna
teoria de la arquitectura-Arquitectura postmodernateoria de la arquitectura-Arquitectura postmoderna
teoria de la arquitectura-Arquitectura postmoderna
Walter Pacheco Javier
 
Excelian hyperledger fabric-feb17
Excelian hyperledger fabric-feb17Excelian hyperledger fabric-feb17
Excelian hyperledger fabric-feb17
Excelian | Luxoft Financial Services
 
Opportunities for Murex Consultants in 2016
Opportunities for Murex Consultants in 2016Opportunities for Murex Consultants in 2016
Opportunities for Murex Consultants in 2016
Excelian | Luxoft Financial Services
 
Excelian hyperledger walkthrough-feb17
Excelian hyperledger walkthrough-feb17Excelian hyperledger walkthrough-feb17
Excelian hyperledger walkthrough-feb17
Excelian | Luxoft Financial Services
 

Viewers also liked (9)

Cassandra and Hybrid Cloud - Introducing Mache
Cassandra and Hybrid Cloud - Introducing MacheCassandra and Hybrid Cloud - Introducing Mache
Cassandra and Hybrid Cloud - Introducing Mache
 
Без воды: презентация для первых лиц
Без воды: презентация для первых лицБез воды: презентация для первых лиц
Без воды: презентация для первых лиц
 
How to succeed at UPwork
How to succeed at UPworkHow to succeed at UPwork
How to succeed at UPwork
 
TriggMine - сервис возврата брошенных корзин
TriggMine - сервис возврата брошенных корзинTriggMine - сервис возврата брошенных корзин
TriggMine - сервис возврата брошенных корзин
 
Grid computing and trade analytics with elastic - Jay Chin's presentation at ...
Grid computing and trade analytics with elastic - Jay Chin's presentation at ...Grid computing and trade analytics with elastic - Jay Chin's presentation at ...
Grid computing and trade analytics with elastic - Jay Chin's presentation at ...
 
teoria de la arquitectura-Arquitectura postmoderna
teoria de la arquitectura-Arquitectura postmodernateoria de la arquitectura-Arquitectura postmoderna
teoria de la arquitectura-Arquitectura postmoderna
 
Excelian hyperledger fabric-feb17
Excelian hyperledger fabric-feb17Excelian hyperledger fabric-feb17
Excelian hyperledger fabric-feb17
 
Opportunities for Murex Consultants in 2016
Opportunities for Murex Consultants in 2016Opportunities for Murex Consultants in 2016
Opportunities for Murex Consultants in 2016
 
Excelian hyperledger walkthrough-feb17
Excelian hyperledger walkthrough-feb17Excelian hyperledger walkthrough-feb17
Excelian hyperledger walkthrough-feb17
 

Similar to Introducing Mache

The IBM Data Engine for NoSQL on IBM Power Systems™
The IBM Data Engine for NoSQL on IBM Power Systems™The IBM Data Engine for NoSQL on IBM Power Systems™
The IBM Data Engine for NoSQL on IBM Power Systems™
IBM Power Systems
 
Webcast Q&A- Big Data Architectures Beyond Hadoop
Webcast Q&A- Big Data Architectures Beyond HadoopWebcast Q&A- Big Data Architectures Beyond Hadoop
Webcast Q&A- Big Data Architectures Beyond Hadoop
Impetus Technologies
 
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, SmileOCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
OCCIware
 
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
Marc Dutoo
 
Hyper-converged infrastructure
Hyper-converged infrastructureHyper-converged infrastructure
Hyper-converged infrastructureIgor Malts
 
Introduction to NoSQL
Introduction to NoSQLIntroduction to NoSQL
Introduction to NoSQL
balwinders
 
8 Strategies For Building A Modern DataCenter
8 Strategies For Building A Modern DataCenter8 Strategies For Building A Modern DataCenter
8 Strategies For Building A Modern DataCenter
Envision Technology Advisors
 
HAWQ: a massively parallel processing SQL engine in hadoop
HAWQ: a massively parallel processing SQL engine in hadoopHAWQ: a massively parallel processing SQL engine in hadoop
HAWQ: a massively parallel processing SQL engine in hadoop
BigData Research
 
Microsoft Sql Server 2016 Is Now Live
Microsoft Sql Server 2016 Is Now LiveMicrosoft Sql Server 2016 Is Now Live
Microsoft Sql Server 2016 Is Now Live
Amber Moore
 
Vote NO for MySQL
Vote NO for MySQLVote NO for MySQL
Vote NO for MySQL
Ulf Wendel
 
OCCIware@POSS 2016 - an extensible, standard XaaS cloud consumer platform
OCCIware@POSS 2016 - an extensible, standard XaaS cloud consumer platformOCCIware@POSS 2016 - an extensible, standard XaaS cloud consumer platform
OCCIware@POSS 2016 - an extensible, standard XaaS cloud consumer platform
Marc Dutoo
 
SpringPeople - Introduction to Cloud Computing
SpringPeople - Introduction to Cloud ComputingSpringPeople - Introduction to Cloud Computing
SpringPeople - Introduction to Cloud Computing
SpringPeople
 
PaaS Emerging Technologies - October 2015
PaaS Emerging Technologies - October 2015PaaS Emerging Technologies - October 2015
PaaS Emerging Technologies - October 2015
Krishna-Kumar
 
Big_SQL_3.0_Whitepaper
Big_SQL_3.0_WhitepaperBig_SQL_3.0_Whitepaper
Big_SQL_3.0_WhitepaperScott Gray
 
OCCIware: Extensible and Standard-based XaaS Platform To Manage Everything in...
OCCIware: Extensible and Standard-based XaaS Platform To Manage Everything in...OCCIware: Extensible and Standard-based XaaS Platform To Manage Everything in...
OCCIware: Extensible and Standard-based XaaS Platform To Manage Everything in...
OW2
 
OCCIware, an extensible, standard-based XaaS consumer platform to manage ever...
OCCIware, an extensible, standard-based XaaS consumer platform to manage ever...OCCIware, an extensible, standard-based XaaS consumer platform to manage ever...
OCCIware, an extensible, standard-based XaaS consumer platform to manage ever...
OCCIware
 
Apache Mesos Overview and Integration
Apache Mesos Overview and IntegrationApache Mesos Overview and Integration
Apache Mesos Overview and Integration
Alex Baretto
 
E5 05 ijcite august 2014
E5 05 ijcite august 2014E5 05 ijcite august 2014
E5 05 ijcite august 2014
ijcite
 

Similar to Introducing Mache (20)

The IBM Data Engine for NoSQL on IBM Power Systems™
The IBM Data Engine for NoSQL on IBM Power Systems™The IBM Data Engine for NoSQL on IBM Power Systems™
The IBM Data Engine for NoSQL on IBM Power Systems™
 
Webcast Q&A- Big Data Architectures Beyond Hadoop
Webcast Q&A- Big Data Architectures Beyond HadoopWebcast Q&A- Big Data Architectures Beyond Hadoop
Webcast Q&A- Big Data Architectures Beyond Hadoop
 
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, SmileOCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
OCCIware presentation at EclipseDay in Lyon, November 2017, by Marc Dutoo, Smile
 
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
Model and pilot all cloud layers with OCCIware - Eclipse Day Lyon 2017
 
Hyper-converged infrastructure
Hyper-converged infrastructureHyper-converged infrastructure
Hyper-converged infrastructure
 
Introduction to NoSQL
Introduction to NoSQLIntroduction to NoSQL
Introduction to NoSQL
 
8 Strategies For Building A Modern DataCenter
8 Strategies For Building A Modern DataCenter8 Strategies For Building A Modern DataCenter
8 Strategies For Building A Modern DataCenter
 
HAWQ: a massively parallel processing SQL engine in hadoop
HAWQ: a massively parallel processing SQL engine in hadoopHAWQ: a massively parallel processing SQL engine in hadoop
HAWQ: a massively parallel processing SQL engine in hadoop
 
Microsoft Sql Server 2016 Is Now Live
Microsoft Sql Server 2016 Is Now LiveMicrosoft Sql Server 2016 Is Now Live
Microsoft Sql Server 2016 Is Now Live
 
Vote NO for MySQL
Vote NO for MySQLVote NO for MySQL
Vote NO for MySQL
 
Mysql wp memcached
Mysql wp memcachedMysql wp memcached
Mysql wp memcached
 
The NoSQL Movement
The NoSQL MovementThe NoSQL Movement
The NoSQL Movement
 
OCCIware@POSS 2016 - an extensible, standard XaaS cloud consumer platform
OCCIware@POSS 2016 - an extensible, standard XaaS cloud consumer platformOCCIware@POSS 2016 - an extensible, standard XaaS cloud consumer platform
OCCIware@POSS 2016 - an extensible, standard XaaS cloud consumer platform
 
SpringPeople - Introduction to Cloud Computing
SpringPeople - Introduction to Cloud ComputingSpringPeople - Introduction to Cloud Computing
SpringPeople - Introduction to Cloud Computing
 
PaaS Emerging Technologies - October 2015
PaaS Emerging Technologies - October 2015PaaS Emerging Technologies - October 2015
PaaS Emerging Technologies - October 2015
 
Big_SQL_3.0_Whitepaper
Big_SQL_3.0_WhitepaperBig_SQL_3.0_Whitepaper
Big_SQL_3.0_Whitepaper
 
OCCIware: Extensible and Standard-based XaaS Platform To Manage Everything in...
OCCIware: Extensible and Standard-based XaaS Platform To Manage Everything in...OCCIware: Extensible and Standard-based XaaS Platform To Manage Everything in...
OCCIware: Extensible and Standard-based XaaS Platform To Manage Everything in...
 
OCCIware, an extensible, standard-based XaaS consumer platform to manage ever...
OCCIware, an extensible, standard-based XaaS consumer platform to manage ever...OCCIware, an extensible, standard-based XaaS consumer platform to manage ever...
OCCIware, an extensible, standard-based XaaS consumer platform to manage ever...
 
Apache Mesos Overview and Integration
Apache Mesos Overview and IntegrationApache Mesos Overview and Integration
Apache Mesos Overview and Integration
 
E5 05 ijcite august 2014
E5 05 ijcite august 2014E5 05 ijcite august 2014
E5 05 ijcite august 2014
 

Recently uploaded

Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
UiPathCommunity
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
sonjaschweigert1
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles
 
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
Jen Stirrup
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
James Anderson
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance
 
UiPath Community Day Dubai: AI at Work..
UiPath Community Day Dubai: AI at Work..UiPath Community Day Dubai: AI at Work..
UiPath Community Day Dubai: AI at Work..
UiPathCommunity
 

Recently uploaded (20)

Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
 
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdfSmart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
 
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdfFIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
 
UiPath Community Day Dubai: AI at Work..
UiPath Community Day Dubai: AI at Work..UiPath Community Day Dubai: AI at Work..
UiPath Community Day Dubai: AI at Work..
 

Introducing Mache

  • 1. In a constantly changing industry you need a forward-thinking technology partner Introducing Mache – an Open Source Big Data Cache James Bowkett @techwob mache@excelian.com
  • 2. What is Mache? 2 Mache is a NoSQL near cache with eventing; built from a 'mash-up' of Open Source technologies with multiple pluggable NoSQL support and multiple pluggable messaging platforms.We have used all the well known Open-Source technologies.The heart of the system is akin to a HashMap - usingGoogle's Guava cache and Spring Data. The principle idea behind Mache was to abstract itself from the Database layer. As a result it allows the two to remain somewhat independent of each other, enabling concentration on development and performance rather than scaling out the data layer. It has been engineered to be as lightweight as possible, and because of theVendor independence, Mache could offer a free caching solution to an already deployed NoSQL database. It doesn’t need to know about the volume of data which means that it can be as performant as the system allows. It was also designed to be a piece of software which integrates well with other software. It offers support for NoSQL programmes such as Cassandra, MongoDB andCouchbase. But it also lends itself well to support integration with pre-existing systems such as Kafka, RabbitMQ andActiveMQ message services.
  • 3. 3 What is Mache? Mache (in-memory cache) Cassandra Mongo Couchbase Rabbit MQ Kafka Active MQ Can write through to… Can send notifications to… File
  • 4. 4 Example Use Cases • Generally speaking, most grid applications will have a heavy reliance on read-mostly data. More relevant to today’s challenges are MiFID II andCVA. • Since the introduction of the MiFID II market makers must warehouse the audit data around their prices and spread calculations.This inevitably leads to the storage of large volumes of data, which also offers the opportunity for Big Data.Applications now need to read and write this data, and we believe Mache offers the opportunity to help turn a ‘data lake’ into a localised manageable view of this data deluge. • Take for instance, today’s trades or their price points. Mache fits naturally within the speed layer in a lambda architecture. • CreditValue Adjustment (CVA) requires large volumes of Monte Carlo data, and in some cases will need 300- 400GB of data for a single day.When stress testing or back testing is being performed, you could need 30+ days’ worth of data on each compute node. 400GB * 10k = a lot of data!
  • 5. Why have you built Mache? • The original idea was to provide an open source alternative to Coherence • Other caching API’s tend to offer their own persistence formats, instead of integrating with the leading NoSQL vendors • The network is often the bottleneck within grid platforms, near side caching is a solution • Filtered views will open up use cases – E.g. only view a trader’s trades for today – Market/reference data distribution to processing nodes 5
  • 6. How does Mache improve performance? 6
  • 7. How does Mache improve performance? 7
  • 8. How do I use Mache? 8
  • 9. How does Mache compare with other existing technologies? The following is a list of technologies that can fulfil similar use cases to Mache.While they are all leading technologies in their field, it is important to note how they are distinctly different to Mache.  Ehcache - is a distributed JSR-107-compliant cache that has pluggable storage mechanisms (on-heap/off-heap and file-based). However it doesn't have out-of-the-box support for any NoSQL platforms.  Gridgain - is a purely in-memory data and processing grid platform. It has no current support for NoSQL platforms.  Hazelcast - is another in-memory grid implementation, often marketed as a replacement for Oracle Coherence.  Infinispan - is a single node or clustered JSR-107-compliant in-memory cache implementation. Its persistence is handled by pluggable adapters the application teams may have to write themselves. It provides no NoSQL support as standard.  Memcached - is a multi-node object cache with no persistence mechanism as standard.  Redis - is an all-purpose in-memory data store with no persistence mechanism as standard. In our view, there was no pre-existing solution that would provide for the simplest use case of near caching to abstract away storage whilst still supporting the big 3 platforms. We proposed a solution that would adhere to the following guiding principles:  Lightweight - the Mache-core jar depends on 3 other libraries (in addition to any required vendor drivers for messaging or persistence)  Pluggable with existing NoSQL Platforms, and thereby creating no vendor lock-in  Provide a small set of valuable features, adhering to the Unix principle of 'do one thing, and do it well' 9
  • 10. So what is next for Mache? We have quite a few enhancements on our list. In no particular order:  JMX to provide configuration and management  Overflow cache - so we can overflow from memory to disk using something like Berkley DB  JSON/RestAPI - to provide a platform agnostic way of accessing local cached data which is run in a service.  Eventing straight from the database  Filtered materialised views/Continuous query caches  Shared cache instance between processes  Distributed file system 10
  • 11. What’s next for Mache? For more information: • email us at mache@excelian.com • https://github.com/excelian/mache • 0.6.1 available on jcenter/bintray • Read more on our blog – Mache – Near Side Caching for NoSQL. 11 http://www.excelian.com/blog/mache---near-side-caching-for-nosql-/#.Vl2sME02q70
  • 12. Cape Town Office 1st Floor, Corporate Place 13 Mispel Road, Bellville Cape Town, 7530 South Africa +27(0) 21 944 9900 Johannesburg Office 4th Floor, The Firs Cradock Avenue. Rosebank Johannesburg, SA +27 (0) 11 759 4020 New York Office 8th Floor, 260 Madison Avenue New York, NY 10016 United States +1646 216 2065 Frankfurt Office Brockenheimer Landstrasse 17/19 60325 Frankfurt Germany +49 (0) 69 710455 183 London Office 44 Featherstone Street London, EC1Y 8RN United Kingdom Sydney Office 1 York Street Level 2 Sydney, NSW 2000 Australia +61 (2) 9191 7810 Toronto Office 100 King Street West, Suite 5600 Toronto, Ontario M5X 1C9 Canada +1 (203) 295 5240 www.excelian.com