BlueData EPIC 2.0 Overview
•
0 likes•3,102 views
This presentation provides an overview of what’s new in the 2.0 release of the BlueData EPIC software platform. BlueData’s EPIC software platform solves the infrastructure challenges and limitations that can slow down and stall on-premises Big Data deployments. With BlueData, you can spin up Hadoop or Spark clusters in minutes rather than months – with the data and analytical tools that your data scientists need. The BlueData EPIC 2.0 release leverages Docker containers to simplify Big Data clusters, supports Apache Zeppelin notebooks and other new functionality for Apache Spark, and includes an enhanced App Store that provides one-click access to Big Data distributions and analytics tools. Learn more about BlueData at http://www.bluedata.com
Report
Share
Report
Share
Recommended
BlueData Integration with Cloudera Manager
This presentation provides an overview of the BlueData integration with Cloudera Manager. With this integration, customers of our BlueData EPIC software platform can leverage the power of Cloudera Manager for end-to-end Hadoop systems management and administration.
When the BlueData EPIC platform provisions a virtual CDH cluster, Cloudera Manager can be provisioned as well – so you can easily deploy, manage, monitor and perform diagnostics on your Hadoop cluster. Our customers can take advantage of the Cloudera Manager GUI to monitor their cluster, troubleshoot issues, and administer their Hadoop deployment.
Learn more about BlueData at http://www.bluedata.com
Dell/EMC Technical Validation of BlueData EPIC with Isilon
The BlueData EPIC™ (Elastic Private Instant Clusters) software platform solves the infrastructure challenges and limitations that can slow down and stall Big Data deployments. With EPIC software, you can spin up Hadoop or Spark clusters – with the data and analytical tools that your data scientists need – in minutes rather than months. Leveraging the power of containers and the performance of bare-metal, EPIC delivers speed, agility, and cost-efficiency for Big Data infrastructure. It works with all of the major Apache Hadoop distributions as well as Apache Spark. It integrates with each of the leading analytical applications, so your data scientists can use the tools they prefer. You can run it with any shared storage environment, so you don’t have to move your
EMC Isilon Scale-out Storage Solutions for Hadoop combine a powerful yet simple and highly efficient storage platform with native Hadoop integration that allows you to accelerate analytics, gain new flexibility, and avoid the costs of a separate Hadoop infrastructure. BlueData EPIC Software combined with EMC Isilon shared storage provides a comprehensive solution for compute + storage.
BlueData and Isilon share several joint customers and opportunities at leading financial services, advanced research laboratories, healthcare and media/communication organizations.
This paper describes the process of validating Hadoop applications running in virtual clusters on the EPIC platform with data stored on the EMC Isilon storage device using either NFS or HDFS data access protocols
Spark Infrastructure Made Easy
BlueData makes on-premises Spark infrastructure easy.
With BlueData, you can spin up virtual Spark clusters within minutes – providing secure, on-demand access to Big Data analytics and infrastructure. You can use Spark with or without the Hadoop ecosystem of tools – using HDFS, Tachyon, or any shared storage system.
You can also build analytical pipelines and create Spark clusters using our RESTful APIs. BlueData’s software platform leverages virtualization and patent-pending innovations to make it simpler, faster, and more cost-effective to deploy Hadoop or Spark infrastructure on-premises.
Learn more at http://www.bluedata.com
Hadoop Virtualization - Intel White Paper
This white paper describes how BlueData enables virtualization of Hadoop and Spark workloads running on Intel architecture.
Even as virtualization has spread throughout the data center, Apache Hadoop continues to be deployed almost exclusively on bare-metal physical servers. Processing overhead and I/O latency typically associated with virtualization have prevented big data architects from virtualizing Hadoop implementations.
As a result, most Hadoop initiatives have been limited in terms of agility, with infrastructure changes such as provisioning a new server for Hadoop often taking weeks or even months. This infrastructure complexity continues to slow down adoption in enterprise deployments. Apache Spark is a relatively new big data technology, but interest is growing rapidly; many of these same deployment challenges apply to on-premises Spark implementations.
The BlueData EPIC software platform addresses these limitations, enabling data center operators to accelerate Hadoop and Spark implementations on Intel architecture-based servers.
For more information, visit intel.com/bigdata and bluedata.com
Big Data Case Study: Fortune 100 Telco
This Big Data case study outlines the Hadoop infrastructure deployment for a Fortune 100 media and telecommunications company.
Hadoop adoption in this company had grown organically across multiple different teams, starting with “science projects” and lab initiatives that quickly grew and expanded. Going forward, some of the options they considered for their Big Data deployment included expanding their on-premises infrastructure and using a Hadoop-as-a-Service cloud offering.
Fortunately, they realized that there is a third option: providing the benefits of Hadoop-as-a-Service with on-premises infrastructure. They selected the BlueData EPIC software platform to virtualize their Hadoop infrastructure and provide on-demand access to virtual Hadoop clusters in a secure, multi-tenant model.
Learn more about this case study in the blog post at: http://www.bluedata.com/blog/2015/05/big-data-case-study-hadoop-infrastructure
Complex Analytics with NoSQL Data Store in Real Time
NOSQL are often limited in the type of queries that they can support due to the distributed nature of the data. In this session we would learn patterns on how we can overcome this limitation and combine multiple query semantics with NoSQL based engines.
We will demonstrate specifically a combination of key/value, SQL like, Document model and Graph based queries as well as more advanced topic such as handling partial update and query through projection. We will also demonstrate how we can create a meshaup between those API's i.e. write fast through Key/Value API and execute complex queries on that same data through SQL query.
- See more at: http://nosql2014.dataversity.net/sessionPop.cfm?confid=81&proposalid=6335#sthash.PNSZi5TJ.dpuf
Spark and Couchbase– Augmenting the Operational Database with Spark
How do NoSQL Document-Oriented Databases like Couchbase fit in with Apache Spark? This set of slides gives a couple of use cases, shows why Couchbase works great with Spark, and sets up a scenario for a demo.
Recommended
BlueData Integration with Cloudera Manager
This presentation provides an overview of the BlueData integration with Cloudera Manager. With this integration, customers of our BlueData EPIC software platform can leverage the power of Cloudera Manager for end-to-end Hadoop systems management and administration.
When the BlueData EPIC platform provisions a virtual CDH cluster, Cloudera Manager can be provisioned as well – so you can easily deploy, manage, monitor and perform diagnostics on your Hadoop cluster. Our customers can take advantage of the Cloudera Manager GUI to monitor their cluster, troubleshoot issues, and administer their Hadoop deployment.
Learn more about BlueData at http://www.bluedata.com
Dell/EMC Technical Validation of BlueData EPIC with Isilon
The BlueData EPIC™ (Elastic Private Instant Clusters) software platform solves the infrastructure challenges and limitations that can slow down and stall Big Data deployments. With EPIC software, you can spin up Hadoop or Spark clusters – with the data and analytical tools that your data scientists need – in minutes rather than months. Leveraging the power of containers and the performance of bare-metal, EPIC delivers speed, agility, and cost-efficiency for Big Data infrastructure. It works with all of the major Apache Hadoop distributions as well as Apache Spark. It integrates with each of the leading analytical applications, so your data scientists can use the tools they prefer. You can run it with any shared storage environment, so you don’t have to move your
EMC Isilon Scale-out Storage Solutions for Hadoop combine a powerful yet simple and highly efficient storage platform with native Hadoop integration that allows you to accelerate analytics, gain new flexibility, and avoid the costs of a separate Hadoop infrastructure. BlueData EPIC Software combined with EMC Isilon shared storage provides a comprehensive solution for compute + storage.
BlueData and Isilon share several joint customers and opportunities at leading financial services, advanced research laboratories, healthcare and media/communication organizations.
This paper describes the process of validating Hadoop applications running in virtual clusters on the EPIC platform with data stored on the EMC Isilon storage device using either NFS or HDFS data access protocols
Spark Infrastructure Made Easy
BlueData makes on-premises Spark infrastructure easy.
With BlueData, you can spin up virtual Spark clusters within minutes – providing secure, on-demand access to Big Data analytics and infrastructure. You can use Spark with or without the Hadoop ecosystem of tools – using HDFS, Tachyon, or any shared storage system.
You can also build analytical pipelines and create Spark clusters using our RESTful APIs. BlueData’s software platform leverages virtualization and patent-pending innovations to make it simpler, faster, and more cost-effective to deploy Hadoop or Spark infrastructure on-premises.
Learn more at http://www.bluedata.com
Hadoop Virtualization - Intel White Paper
This white paper describes how BlueData enables virtualization of Hadoop and Spark workloads running on Intel architecture.
Even as virtualization has spread throughout the data center, Apache Hadoop continues to be deployed almost exclusively on bare-metal physical servers. Processing overhead and I/O latency typically associated with virtualization have prevented big data architects from virtualizing Hadoop implementations.
As a result, most Hadoop initiatives have been limited in terms of agility, with infrastructure changes such as provisioning a new server for Hadoop often taking weeks or even months. This infrastructure complexity continues to slow down adoption in enterprise deployments. Apache Spark is a relatively new big data technology, but interest is growing rapidly; many of these same deployment challenges apply to on-premises Spark implementations.
The BlueData EPIC software platform addresses these limitations, enabling data center operators to accelerate Hadoop and Spark implementations on Intel architecture-based servers.
For more information, visit intel.com/bigdata and bluedata.com
Big Data Case Study: Fortune 100 Telco
This Big Data case study outlines the Hadoop infrastructure deployment for a Fortune 100 media and telecommunications company.
Hadoop adoption in this company had grown organically across multiple different teams, starting with “science projects” and lab initiatives that quickly grew and expanded. Going forward, some of the options they considered for their Big Data deployment included expanding their on-premises infrastructure and using a Hadoop-as-a-Service cloud offering.
Fortunately, they realized that there is a third option: providing the benefits of Hadoop-as-a-Service with on-premises infrastructure. They selected the BlueData EPIC software platform to virtualize their Hadoop infrastructure and provide on-demand access to virtual Hadoop clusters in a secure, multi-tenant model.
Learn more about this case study in the blog post at: http://www.bluedata.com/blog/2015/05/big-data-case-study-hadoop-infrastructure
Complex Analytics with NoSQL Data Store in Real Time
NOSQL are often limited in the type of queries that they can support due to the distributed nature of the data. In this session we would learn patterns on how we can overcome this limitation and combine multiple query semantics with NoSQL based engines.
We will demonstrate specifically a combination of key/value, SQL like, Document model and Graph based queries as well as more advanced topic such as handling partial update and query through projection. We will also demonstrate how we can create a meshaup between those API's i.e. write fast through Key/Value API and execute complex queries on that same data through SQL query.
- See more at: http://nosql2014.dataversity.net/sessionPop.cfm?confid=81&proposalid=6335#sthash.PNSZi5TJ.dpuf
Spark and Couchbase– Augmenting the Operational Database with Spark
How do NoSQL Document-Oriented Databases like Couchbase fit in with Apache Spark? This set of slides gives a couple of use cases, shows why Couchbase works great with Spark, and sets up a scenario for a demo.
Manage Microservices & Fast Data Systems on One Platform w/ DC/OS
The application landscape inside our data center is changing: Along with the trend of moving toward microservices and containers, there are a number of new distributed data processing frameworks such as Kafka or Cassandra being released on a weekly basis. These changes have implications for the ways we think about infrastructure. With the growing need for computing power and the rise of distributed applications comes the need for a reliable and simple-use cluster manager and programming abstraction.
In this presentation, Mesosphere explains how to use DC/OS to manage microservices and fast data systems on a single platform. We will look at how container orchestration, including resource management and service management, can be streamlined to process fast data in a matter of seconds, allowing for predictive user interfaces, product recommendations, and billing charge back, among other modern app components.
The new big data
Slides from Scott Shaw's September 2020 presentation to the St. Louis Big Data IDEA meetup on Big Data in 2020
Architecting a Heterogeneous Data Platform Across Clusters, Regions, and Clouds
Alluxio Product School Webinar
January 27, 2022
For more Alluxio events: https://www.alluxio.io/events/
Speaker:
Adit Madan
Data platform teams are increasingly challenged with accessing multiple data stores that are separated from compute engines, such as Spark, Presto, TensorFlow or PyTorch. Whether your data is distributed across multiple datacenters and/or clouds, a successful heterogeneous data platform requires efficient data access. Alluxio enables you to embrace the separation of storage from compute and use Alluxio data orchestration to simplify adoption of the data lake and data mesh paradigms for analytics and AI/ML workloads.
Join Alluxio’s Sr. Product Mgr., Adit Madan, to learn:
- Key challenges with architecting a successful heterogeneous data platform
- How data orchestration can overcome data access challenges in a distributed, heterogeneous environment
- How to identify ways to use Alluxio to meet the needs of your own data environment and workload requirements
Triple C - Centralize, Cloudify and Consolidate Dozens of Oracle Databases (O...
Dozens of Oracle Databases - each health center location has one on its local server with the same data model and the same set of applications. These databases have to be centralized and cloudified and also be consolidated into one or as few databases as possible. To lower costs, ease operations and enable innovation. Each location can access only its own data, applications do not have to be changed and different locations can run different versions of applications and database objects. This is the story of a critical migration. About the cloud ready analysis, the Proofs of Concept with Oracle Database features VPD and Edition Based Redefinition, the scalability investigation, the redesign of change management, rollout and operational management processes and the careful modernization of a 25 year old platform on the latest database release and a shiny new, fully automated cloud platform. This is the story of an organization that had state of the art systems in the mid-90s. And they have these same systems today - no longer state of the art. They can keep the systems alive, but barely, and at increasing cost. In the Fall of 2020, we started an investigation into the feasibility of bringing the 100s of databases from each of the locations together, in a central location, in the cloud and finally: consolidated into one or at least as few database instances as possible. Using Oracle Database Virtual Private Database and Edition Based Redefinition, a smart database connection configuration in each site and a limited reimplementation of non-cloud/non-consolidated mechanisms (interaction with local file system for example) we have designed and proven a working new design and migration approach.
What's the Hadoop-la about Kubernetes?
There is increased interest in using Kubernetes, the open-source container orchestration system for modern, stateful Big Data analytics workloads. The promised land is a unified platform that can handle cloud native stateless and stateful Big Data applications. However, stateful, multi-service Big Data cluster orchestration brings unique challenges. This session will delve into the technical gaps and considerations for Big Data on Kubernetes.
Containers offer significant value to businesses; including increased developer agility, and the ability to move applications between on-premises servers, cloud instances, and across data centers. Organizations have embarked on this journey to containerization with an emphasis on stateless workloads. Stateless applications are usually microservices or containerized applications that don’t “store” data. Web services (such as front end UIs and simple, content-centric experiences) are often great candidates as stateless applications since HTTP is stateless by nature. There is no dependency on the local container storage for the stateless workload.
Stateful applications, on the other hand, are services that require backing storage and keeping state is critical to running the service. Hadoop, Spark and to lesser extent, noSQL platforms such as Cassandra, MongoDB, Postgres, and mySQL are great examples. They require some form of persistent storage that will survive service restarts...
Speakers
Anant Chintamaneni, VP Products, BlueData
Nanda Vijaydev, Director Solutions, BlueData
Lenovo Servers and Microsoft Azure: the future of the stack
When Lenovo servers inegrate with Microsoft Azure Stack
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Data Orchestration Summit 2020 organized by Alluxio
https://www.alluxio.io/data-orchestration-summit-2020/
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Sandipan Chakraborty, Director of Engineering (Rakuten)
About Alluxio: alluxio.io
Engage with the open source community on slack: alluxio.io/slack
Highlights of OpenStack Mitaka and the OpenStack Summit
This presentation explores the latest release of OpenStack (OpenStack Mitaka) and shares highlights from the OpenStack Summit held April 25-29, 2016 in Austin, TX.
OpenStack is a free and open-source software platform for building and managing cloud computing platforms for public and private clouds, mostly deployed as an infrastructure-as-a-service (IaaS). The software platform consists of interrelated components that control hardware pools of processing, storage, and networking resources throughout a data center. OpenStack works with popular enterprise and open source technologies making it ideal for heterogeneous infrastructure.
MOUG17: DB Security; Secure your Data
Midwest Oracle Users Group Training Day 2017 Presentation by Mike Messina, AdvizeX, Oracle Ace
Introducing Cloudian HyperStore 6.0
Cloudian HyperStore offer 100% S3 compatibility for low-cost, scalable smart object storage.
With HyperStore 6.0, we are focused on bringing down operational costs so that you can more effectively track, manage, and optimize your data storage as you scale.
Simplifying Hadoop with RecordService, A Secure and Unified Data Access Path ...
SFHUG presentation from February 2, 2016. One of the key values of the Hadoop ecosystem is its flexibility. There is a myriad of components that make up this ecosystem, allowing Hadoop to tackle otherwise intractable problems. However, having so many components provides a significant integration, implementation, and usability burden. Features that ought to work in all the components often require sizable per-component effort to ensure correctness across the stack.
Lenni Kuff explores RecordService, a new solution to this problem that provides an API to read data from Hadoop storage managers and return them as canonical records. This eliminates the need for components to support individual file formats, handle security, perform auditing, and implement sophisticated IO scheduling and other common processing that is at the bottom of any computation.
Lenni discusses the architecture of the service and the integration work done for MapReduce and Spark. Many existing applications on those frameworks can take advantage of the service with little to no modification. Lenni demonstrates how this provides fine grain (column level and row level) security, through Sentry integration, and improves performance for existing MapReduce and Spark applications by up to 5×. Lenni concludes by discussing how this architecture can enable significant future improvements to the Hadoop ecosystem.
About the speaker: Lenni Kuff is an engineering manager at Cloudera. Before joining Cloudera, he worked at Microsoft on a number of projects including SQL Server storage engine, SQL Azure, and Hadoop on Azure. Lenni graduated from the University of Wisconsin-Madison with degrees in computer science and computer engineering.
Backup multi-cloud solution based on named pipes
Backup solution based on named pipes to storreduce scheduled by Cronicle.
Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...
NTT Data is an IT service company.
Kirin is one of the largest beverages companies in Japan.
In this presentation, we will present the user story of migrating all applications from creaky infrastructure to OpenStack private cloud including actual challenges, know-hows and future prospects.
The key concept of this project is:
* Mission Critical: Migrate all Kirin enterprise applications to OpenStack private cloud.
* Think Big, Start Small: Start from small number of apps, and expand rapidly.
* Agility and elasticity: Adopt a PaaS-like automation approach, targeting 50% less development cost and 40% less operational cost.
In order to achieve all items above, we have decided to use OpenStack IaaS, ICO, which is an automation product by IBM, serverspec for testing, and Hinemos for monitoring management.
Starting from Aug 2014, the project expects 100 VM / 100 TB storage as the first-stage migration by end of 2015. We're planning to migrate 500 VM / 300 TB by end of 2016 and 2000 VM / 1 PB finally.
Microsoft ignite 2018 SQL Server 2019 big data clusters - intro session
SQL Server 2019 big data clusters brings HDFS and Spark into SQL Server for scalable compute and storage.
20150716 introduction to apache spark v3
Introduction toi Apache Spark presentation at Spark Ljubljana meetup
BlueData Hunk Integration: Splunk Analytics for Hadoop
BlueData is working in partnership with Splunk to streamline and accelerate the deployment and adoption of Hunk: Splunk Analytics for Hadoop. The BlueData EPIC software platform now integrates Hunk with Hadoop clusters running on virtualized on-premises infrastructure.
Using Hunk with the BlueData EPIC platform, our joint customers can quickly provision virtual Hadoop clusters together with Hunk in a matter of minutes – providing their data scientists and analysts with the ability to rapidly detect patterns and find anomalies across petabytes of raw data in Hadoop.
Learn more at http://www.bluedata.com
How to deploy Apache Spark in a multi-tenant, on-premises environment
Adoption of Apache Spark in the enterprise is increasing rapidly - it's become one of the fastest growing and most popular technologies in the Big Data ecosystem.
However, implementing an enterprise-ready, on-premises Spark deployment can be very complex and it requires expertise that is generally not available to all.
BlueData makes it easier to deploy Apache Spark on-premises. With BlueData, you can spin up virtual Spark clusters within minutes – providing secure, self-service, on-demand access to Big Data analytics and infrastructure. You can deploy Spark in standalone mode or with Hadoop / YARN. You can also build analytical pipelines and create Spark clusters using our RESTful APIs, and use web-based Zeppelin notebooks for interactive data analytics.
BlueData’s software platform leverages virtualization and Docker containers – combined with our own patent-pending innovations – to make it faster, and more cost-effective for enterprises to get up and running with a multi-tenant Spark deployment on-premises.
Learn more at www.bluedata.com
More Related Content
What's hot
Manage Microservices & Fast Data Systems on One Platform w/ DC/OS
The application landscape inside our data center is changing: Along with the trend of moving toward microservices and containers, there are a number of new distributed data processing frameworks such as Kafka or Cassandra being released on a weekly basis. These changes have implications for the ways we think about infrastructure. With the growing need for computing power and the rise of distributed applications comes the need for a reliable and simple-use cluster manager and programming abstraction.
In this presentation, Mesosphere explains how to use DC/OS to manage microservices and fast data systems on a single platform. We will look at how container orchestration, including resource management and service management, can be streamlined to process fast data in a matter of seconds, allowing for predictive user interfaces, product recommendations, and billing charge back, among other modern app components.
The new big data
Slides from Scott Shaw's September 2020 presentation to the St. Louis Big Data IDEA meetup on Big Data in 2020
Architecting a Heterogeneous Data Platform Across Clusters, Regions, and Clouds
Alluxio Product School Webinar
January 27, 2022
For more Alluxio events: https://www.alluxio.io/events/
Speaker:
Adit Madan
Data platform teams are increasingly challenged with accessing multiple data stores that are separated from compute engines, such as Spark, Presto, TensorFlow or PyTorch. Whether your data is distributed across multiple datacenters and/or clouds, a successful heterogeneous data platform requires efficient data access. Alluxio enables you to embrace the separation of storage from compute and use Alluxio data orchestration to simplify adoption of the data lake and data mesh paradigms for analytics and AI/ML workloads.
Join Alluxio’s Sr. Product Mgr., Adit Madan, to learn:
- Key challenges with architecting a successful heterogeneous data platform
- How data orchestration can overcome data access challenges in a distributed, heterogeneous environment
- How to identify ways to use Alluxio to meet the needs of your own data environment and workload requirements
Triple C - Centralize, Cloudify and Consolidate Dozens of Oracle Databases (O...
Dozens of Oracle Databases - each health center location has one on its local server with the same data model and the same set of applications. These databases have to be centralized and cloudified and also be consolidated into one or as few databases as possible. To lower costs, ease operations and enable innovation. Each location can access only its own data, applications do not have to be changed and different locations can run different versions of applications and database objects. This is the story of a critical migration. About the cloud ready analysis, the Proofs of Concept with Oracle Database features VPD and Edition Based Redefinition, the scalability investigation, the redesign of change management, rollout and operational management processes and the careful modernization of a 25 year old platform on the latest database release and a shiny new, fully automated cloud platform. This is the story of an organization that had state of the art systems in the mid-90s. And they have these same systems today - no longer state of the art. They can keep the systems alive, but barely, and at increasing cost. In the Fall of 2020, we started an investigation into the feasibility of bringing the 100s of databases from each of the locations together, in a central location, in the cloud and finally: consolidated into one or at least as few database instances as possible. Using Oracle Database Virtual Private Database and Edition Based Redefinition, a smart database connection configuration in each site and a limited reimplementation of non-cloud/non-consolidated mechanisms (interaction with local file system for example) we have designed and proven a working new design and migration approach.
What's the Hadoop-la about Kubernetes?
There is increased interest in using Kubernetes, the open-source container orchestration system for modern, stateful Big Data analytics workloads. The promised land is a unified platform that can handle cloud native stateless and stateful Big Data applications. However, stateful, multi-service Big Data cluster orchestration brings unique challenges. This session will delve into the technical gaps and considerations for Big Data on Kubernetes.
Containers offer significant value to businesses; including increased developer agility, and the ability to move applications between on-premises servers, cloud instances, and across data centers. Organizations have embarked on this journey to containerization with an emphasis on stateless workloads. Stateless applications are usually microservices or containerized applications that don’t “store” data. Web services (such as front end UIs and simple, content-centric experiences) are often great candidates as stateless applications since HTTP is stateless by nature. There is no dependency on the local container storage for the stateless workload.
Stateful applications, on the other hand, are services that require backing storage and keeping state is critical to running the service. Hadoop, Spark and to lesser extent, noSQL platforms such as Cassandra, MongoDB, Postgres, and mySQL are great examples. They require some form of persistent storage that will survive service restarts...
Speakers
Anant Chintamaneni, VP Products, BlueData
Nanda Vijaydev, Director Solutions, BlueData
Lenovo Servers and Microsoft Azure: the future of the stack
When Lenovo servers inegrate with Microsoft Azure Stack
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Data Orchestration Summit 2020 organized by Alluxio
https://www.alluxio.io/data-orchestration-summit-2020/
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Sandipan Chakraborty, Director of Engineering (Rakuten)
About Alluxio: alluxio.io
Engage with the open source community on slack: alluxio.io/slack
Highlights of OpenStack Mitaka and the OpenStack Summit
This presentation explores the latest release of OpenStack (OpenStack Mitaka) and shares highlights from the OpenStack Summit held April 25-29, 2016 in Austin, TX.
OpenStack is a free and open-source software platform for building and managing cloud computing platforms for public and private clouds, mostly deployed as an infrastructure-as-a-service (IaaS). The software platform consists of interrelated components that control hardware pools of processing, storage, and networking resources throughout a data center. OpenStack works with popular enterprise and open source technologies making it ideal for heterogeneous infrastructure.
MOUG17: DB Security; Secure your Data
Midwest Oracle Users Group Training Day 2017 Presentation by Mike Messina, AdvizeX, Oracle Ace
Introducing Cloudian HyperStore 6.0
Cloudian HyperStore offer 100% S3 compatibility for low-cost, scalable smart object storage.
With HyperStore 6.0, we are focused on bringing down operational costs so that you can more effectively track, manage, and optimize your data storage as you scale.
Simplifying Hadoop with RecordService, A Secure and Unified Data Access Path ...
SFHUG presentation from February 2, 2016. One of the key values of the Hadoop ecosystem is its flexibility. There is a myriad of components that make up this ecosystem, allowing Hadoop to tackle otherwise intractable problems. However, having so many components provides a significant integration, implementation, and usability burden. Features that ought to work in all the components often require sizable per-component effort to ensure correctness across the stack.
Lenni Kuff explores RecordService, a new solution to this problem that provides an API to read data from Hadoop storage managers and return them as canonical records. This eliminates the need for components to support individual file formats, handle security, perform auditing, and implement sophisticated IO scheduling and other common processing that is at the bottom of any computation.
Lenni discusses the architecture of the service and the integration work done for MapReduce and Spark. Many existing applications on those frameworks can take advantage of the service with little to no modification. Lenni demonstrates how this provides fine grain (column level and row level) security, through Sentry integration, and improves performance for existing MapReduce and Spark applications by up to 5×. Lenni concludes by discussing how this architecture can enable significant future improvements to the Hadoop ecosystem.
About the speaker: Lenni Kuff is an engineering manager at Cloudera. Before joining Cloudera, he worked at Microsoft on a number of projects including SQL Server storage engine, SQL Azure, and Hadoop on Azure. Lenni graduated from the University of Wisconsin-Madison with degrees in computer science and computer engineering.
Backup multi-cloud solution based on named pipes
Backup solution based on named pipes to storreduce scheduled by Cronicle.
Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...
NTT Data is an IT service company.
Kirin is one of the largest beverages companies in Japan.
In this presentation, we will present the user story of migrating all applications from creaky infrastructure to OpenStack private cloud including actual challenges, know-hows and future prospects.
The key concept of this project is:
* Mission Critical: Migrate all Kirin enterprise applications to OpenStack private cloud.
* Think Big, Start Small: Start from small number of apps, and expand rapidly.
* Agility and elasticity: Adopt a PaaS-like automation approach, targeting 50% less development cost and 40% less operational cost.
In order to achieve all items above, we have decided to use OpenStack IaaS, ICO, which is an automation product by IBM, serverspec for testing, and Hinemos for monitoring management.
Starting from Aug 2014, the project expects 100 VM / 100 TB storage as the first-stage migration by end of 2015. We're planning to migrate 500 VM / 300 TB by end of 2016 and 2000 VM / 1 PB finally.
Microsoft ignite 2018 SQL Server 2019 big data clusters - intro session
SQL Server 2019 big data clusters brings HDFS and Spark into SQL Server for scalable compute and storage.
20150716 introduction to apache spark v3
Introduction toi Apache Spark presentation at Spark Ljubljana meetup
What's hot (20)
Self-Service Provisioning and Hadoop Management with Apache Ambari
Self-Service Provisioning and Hadoop Management with Apache Ambari
Manage Microservices & Fast Data Systems on One Platform w/ DC/OS
Manage Microservices & Fast Data Systems on One Platform w/ DC/OS
Architecting a Heterogeneous Data Platform Across Clusters, Regions, and Clouds
Architecting a Heterogeneous Data Platform Across Clusters, Regions, and Clouds
Triple C - Centralize, Cloudify and Consolidate Dozens of Oracle Databases (O...
Triple C - Centralize, Cloudify and Consolidate Dozens of Oracle Databases (O...
Lenovo Servers and Microsoft Azure: the future of the stack
Lenovo Servers and Microsoft Azure: the future of the stack
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Highlights of OpenStack Mitaka and the OpenStack Summit
Highlights of OpenStack Mitaka and the OpenStack Summit
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
SplunkLive! Nutanix Session - Turnkey and scalable infrastructure for Splunk ...
Simplifying Hadoop with RecordService, A Secure and Unified Data Access Path ...
Simplifying Hadoop with RecordService, A Secure and Unified Data Access Path ...
Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...
Kirin User Story: Migrating Mission Critical Applications to OpenStack Privat...
Microsoft ignite 2018 SQL Server 2019 big data clusters - intro session
Microsoft ignite 2018 SQL Server 2019 big data clusters - intro session
Viewers also liked
BlueData Hunk Integration: Splunk Analytics for Hadoop
BlueData is working in partnership with Splunk to streamline and accelerate the deployment and adoption of Hunk: Splunk Analytics for Hadoop. The BlueData EPIC software platform now integrates Hunk with Hadoop clusters running on virtualized on-premises infrastructure.
Using Hunk with the BlueData EPIC platform, our joint customers can quickly provision virtual Hadoop clusters together with Hunk in a matter of minutes – providing their data scientists and analysts with the ability to rapidly detect patterns and find anomalies across petabytes of raw data in Hadoop.
Learn more at http://www.bluedata.com
How to deploy Apache Spark in a multi-tenant, on-premises environment
Adoption of Apache Spark in the enterprise is increasing rapidly - it's become one of the fastest growing and most popular technologies in the Big Data ecosystem.
However, implementing an enterprise-ready, on-premises Spark deployment can be very complex and it requires expertise that is generally not available to all.
BlueData makes it easier to deploy Apache Spark on-premises. With BlueData, you can spin up virtual Spark clusters within minutes – providing secure, self-service, on-demand access to Big Data analytics and infrastructure. You can deploy Spark in standalone mode or with Hadoop / YARN. You can also build analytical pipelines and create Spark clusters using our RESTful APIs, and use web-based Zeppelin notebooks for interactive data analytics.
BlueData’s software platform leverages virtualization and Docker containers – combined with our own patent-pending innovations – to make it faster, and more cost-effective for enterprises to get up and running with a multi-tenant Spark deployment on-premises.
Learn more at www.bluedata.com
Bare-metal performance for Big Data workloads on Docker containers
In a benchmark study, Intel® compared the performance of Big Data workloads running on a bare-metal deployment versus running in Docker* containers with the BlueData® EPIC™ software platform.
This in-depth study shows that performance ratios for container-based Hadoop workloads on BlueData EPIC are equal to — and in some cases, better than — bare-metal Hadoop. For example, benchmark tests showed that the BlueData EPIC platform demonstrated an average 2.33% performance gain over bare metal, for a configuration with 50 Hadoop compute nodes and 10 terabytes (TB) of data. These performance results were achieved without any modifications to the Hadoop software.
This is a revolutionary milestone, and the result of an ongoing collaboration between Intel and BlueData software engineering teams.
This white paper describes the software and hardware configurations for the benchmark tests, as well as details of the performance benchmark process and results.
Portworx + Rancher Webinar
Persistence and data services for containers, using Rancher + Portworx. My slides and audio coming.
Beniamino Pagliaro Keynote at Giornate dell'Infermiere 2007-Trieste
My keynote presented in Trieste (8 May 2007) talking about "Young people and troubles: real life and virtual life"
BlueData DataSheet
The BlueData EPIC™ software platform solves the challenges that can slow down and stall Big Data initiatives. It makes deployment of Big Data infrastructure easier, faster, and more
cost-effective – eliminating complexity as a barrier to adoption.
BlueData EPIC on AWS - Spec Sheet
The BlueData EPIC software platform makes deployment of Big Data infrastructure and applications easier, faster, and more cost-effective – whether on-premises or on the public cloud.
With BlueData EPIC on AWS, you can quickly and easily deploy your preferred Big Data applications, distributions and tools; leverage enterprise-class security and cost controls for multi-tenant deployments on the Amazon cloud; and tap into both Amazon S3 and on-premises storage for your Big Data analytics.
Sign up for a free two-week trial at www.bluedata.com/aws
Devops With Boxfuse and Shippable
Presentation for Walnut St Labs "iSchool" - Meant to be an inspiring and informative presentation about what is available to developers for full devops automation for FREE.
Why Virtualization is important by Tom Phelan of BlueData
This presentation will investigate the advantages and disadvantage of running an n-node Hadoop cluster in each of the following environments:
• Bare metal
• Private cloud using a traditional virtual machine environment
• Private cloud using a containerized virtual environment
Tom will follow this with a discussion of real Hadoop Use Cases and how each would, or would not, be suitable for running in each environment.
Stateful Containers: Flocker on CoreOS
Presented at the CoreOS / ClusterHQ meetup in San Francisco
Stephen Nguyen a Developer Evangelist for ClusterHQ reviews how volumes work and overviews the benefits of allowing Flocker to orchestrate your Volumes. (video coming soon)
Keynote version, added benefit of step by step animations for the more complex diagram and speaker's notes. (https://drive.google.com/file/d/0B7ZrB_ueOcMqSDY0Y3hOMmZxU00/view?usp=sharing_
PaaS Emerging Technologies - October 2015
An IEEE event Presentation - Paas Emerging Technologies -Described with several links pointing to the latest events.
Viewers also liked (13)
BlueData Hunk Integration: Splunk Analytics for Hadoop
BlueData Hunk Integration: Splunk Analytics for Hadoop
How to deploy Apache Spark in a multi-tenant, on-premises environment
How to deploy Apache Spark in a multi-tenant, on-premises environment
Bare-metal performance for Big Data workloads on Docker containers
Bare-metal performance for Big Data workloads on Docker containers
Beniamino Pagliaro Keynote at Giornate dell'Infermiere 2007-Trieste
Beniamino Pagliaro Keynote at Giornate dell'Infermiere 2007-Trieste
Why Virtualization is important by Tom Phelan of BlueData
Why Virtualization is important by Tom Phelan of BlueData
Similar to BlueData EPIC 2.0 Overview
Nanda Vijaydev, BlueData - Deploying H2O in Large Scale Distributed Environme...
This session was recorded in San Francisco on February 5th, 2019 and can be viewed here: https://youtu.be/CgoxjmdyMiU
This session will discuss how to get up and running quickly with containerized H2O environments (H2O Flow, Sparkling Water, and Driverless AI) at scale, in a multi-tenant architecture with a shared pool of resources using CPUs and/or GPUs. See how how you can spin up (and tear down) your H2O environments on-demand, with just a few mouse clicks. Find out how to enable quota management of GPU resources for greater efficiency, and easily connect your compute to your datasets for large-scale distributed machine learning. Learn how to operationalize your machine learning pipelines and deliver faster time-to-value for your AI initiative — while ensuring enterprise-grade security and high performance.
Bio: Nanda Vijaydev is senior director of solutions at BlueData (now HPE) - where she leverages technologies like Hadoop, Spark, and TensorFlow to build solutions for enterprise analytics and machine learning use cases. Nanda has 10 years of experience in data management and data science. Previously, she worked on data science and big data projects in multiple industries, including healthcare and media; was a principal solutions architect at Silicon Valley Data Science; and served as director of solutions engineering at Karmasphere. Nanda has an in-depth understanding of the data analytics and data management space, particularly in the areas of data integration, ETL, warehousing, reporting, and machine learning.
Lessons Learned from Dockerizing Spark Workloads
Many initiatives for running applications inside containers have been scoped to run on a single host. Using Docker containers for large-scale production environments poses interesting challenges, especially when deploying distributed Big Data applications like Apache Spark.
Some of these challenges include container lifecycle management, smart scheduling for optimal resource utilization, network configuration and security, and performance. BlueData is “all in” on Docker containers – with a specific focus on Spark applications. They’ve learned first-hand how to address these challenges for Fortune 500 enterprises and government organizations that want to deploy Big Data workloads using Docker.
This session at Spark Summit in February 2017 (by Thomas Phelan, co-founder and chief architect at BlueData) described lessons learned as well as some tips and tricks on how to Dockerize your Big Data applications in a reliable, scalable, and high-performance environment.
In this session, Tom described how to network Docker containers across multiple hosts securely. He discussed ways to achieve high availability across distributed Big Data applications and hosts in your data center. And since we’re talking about very large volumes of data, performance is a key factor. So Tom discussed some of the storage options that BlueData explored and implemented to achieve near bare-metal I/O performance for Spark using Docker.
https://spark-summit.org/east-2017/events/lessons-learned-from-dockerizing-spark-workloads
Deploying Big-Data-as-a-Service (BDaaS) in the Enterprise
Big-Data-as-a-Service (BDaaS) in an enterprise environment requires meeting the often contradictory goals of (1) providing your data scientists, analysts, and data engineers with a self-service consumption model; (2) delivering agile and scalable on-demand infrastructure for the rapidly evolving ecosystem of big data frameworks and application software; while (3) ensuring enterprise-grade capabilities for isolation, security, monitoring, etc.
In this presentation at our BDaaS meetup in Santa Clara, Tom Phelan (chief architect and co-founder of BlueData) reviewed these goals and how to resolve the potential contradictions. He also discussed the infrastructure, application, user experience, security, and maintainability considerations required before selecting (or designing and building) a Big-Data-as-a-Service platform for an enterprise big data deployment.
More info on this BDaaS meetup can be found at: http://www.meetup.com/Big-Data-as-a-Service/events/233999817
Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...
Many initiatives for running applications inside containers have been scoped to run on a single host. Using Docker containers for large-scale production environments poses interesting challenges, especially when deploying distributed Big Data applications like Apache Spark.
Some of these challenges include container lifecycle management, smart scheduling for optimal resource utilization, network configuration and security, and performance. At BlueData, we’re “all in” on Docker containers – with a specific focus on Spark applications. We’ve learned first-hand how to address these challenges for Fortune 500 enterprises and government organizations that want to deploy Big Data workloads using Docker.
In this session, you’ll learn about networking Docker containers across multiple hosts securely. We’ll discuss ways to achieve high availability across distributed Big Data applications and hosts in your data center. And since we’re talking about very large volumes of data, performance is a key factor. So we’ll discuss some of the storage options we explored and implemented at BlueData to achieve near bare-metal I/O performance for Spark using Docker. We’ll share our lessons learned as well as some tips and tricks on how to Dockerize your Big Data applications in a reliable, scalable, and high-performance environment.
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Find out how Hortonworks and IBM help you address these challenges to enable success to optimize your existing EDW environment.
https://hortonworks.com/webinar/modernize-existing-edw-ibm-big-sql-hortonworks-data-platform/
Lessons Learned Running Hadoop and Spark in Docker Containers
Many initiatives for running applications inside containers have been scoped to run on a single host. Using Docker containers for large-scale production environments poses interesting challenges, especially when deploying distributed big data applications like Apache Hadoop and Apache Spark. This session at Strata + Hadoop World in New York City (September 2016) explores various solutions and tips to address the challenges encountered while deploying multi-node Hadoop and Spark production workloads using Docker containers.
Some of these challenges include container life-cycle management, smart scheduling for optimal resource utilization, network configuration and security, and performance. BlueData is "all in” on Docker containers—with a specific focus on big data applications. BlueData has learned firsthand how to address these challenges for Fortune 500 enterprises and government organizations that want to deploy big data workloads using Docker.
This session by Thomas Phelan, co-founder and chief architect at BlueData, discusses how to securely network Docker containers across multiple hosts and discusses ways to achieve high availability across distributed big data applications and hosts in your data center. Since we’re talking about very large volumes of data, performance is a key factor, so Thomas shares some of the storage options implemented at BlueData to achieve near bare-metal I/O performance for Hadoop and Spark using Docker as well as lessons learned and some tips and tricks on how to Dockerize your big data applications in a reliable, scalable, and high-performance environment.
http://conferences.oreilly.com/strata/hadoop-big-data-ny/public/schedule/detail/52042
The ABC of Docker: The Absolute Best Compendium of Docker
This presentation is my contribution to the body of work around Docker.
It codifies my experience so far, with Docker. The goal is to provide a concise yet complete introduction to Docker and its ecosystem.
I explore various Docker objects, compare containers and virtualization, provide usage examples, and discuss critical concepts around Docker and Linux. The compendium part of this, is aspirational. I will update and add to it as I have time and my experience with the product evolves.
Let me know what you think. Feedback and Likes are always appreciated.
New Ceph capabilities and Reference Architectures
Have you heard about Inktank Ceph and are interested to learn some tips and tricks for getting started quickly and efficiently with Ceph? Then this is the session for you!
In this two part session you learn details of:
• the very latest enhancements and capabilities delivered in Inktank Ceph Enterprise such as a new erasure coded storage back-end, support for tiering, and the introduction of user quotas.
• best practices, lessons learned and architecture considerations founded in real customer deployments of Dell and Inktank Ceph solutions that will help accelerate your Ceph deployment.
Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?
Software Defined Storage, Big Data and Ceph - What Is all the Fuss About? By: Kamesh Pemmaraju,Neil Levine
Have you heard about Inktank Ceph and are interested to learn some tips and tricks for getting started quickly and efficiently with Ceph? Then this is the session for you! In this two part session you learn details of: • the very latest enhancements and capabilities delivered in Inktank Ceph Enterprise such as a new erasure coded storage back-end, support for tiering, and the introduction of user quotas. • best practices, lessons learned and architecture considerations founded in real customer deployments of Dell and Inktank Ceph solutions that will help accelerate your Ceph deployment.
Dell openstack cloud with inktank ceph – large scale customer deployment
This was my presentation at the OpenStack Summit in Hong Kong, November 2013. Learn detail around a unique deployment of the Dell OpenStack-Powered Cloud Solution with Inktank Ceph installed at a large nationally recognized American University that specializes in cancer and genomic research. The University had a need to provide a scalable, secure, centralized data repository to support approximately 900 researchers and an ever-expanding number of research projects and rapidly expanding universe of data. The Dell and Inktank cloud storage solution addresses these storage challenges with an open source solution that leverages the Dell Crowbar Framework and Reference Architecture. After assessing a number of traditional storage scenarios, the University partnered with Dell and Inktank to architect a centralized cloud storage platform that is capable of scaling seamlessly and rapidly, is cost-effective, and that can leverage a single hardware infrastructure, with Dell Power Edge R-720XD servers and the Dell Reference Architecture for their OpenStack compute and storage environment.
Azure Synapse Analytics Overview (r2)
Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.
Lessons learned from running Spark on Docker
Today, most any application can be “Dockerized.” However, there are special challenges when deploying a distributed application such as Spark on containers. This session will describe how to overcome these challenges in deploying Spark on Docker containers, with many practical tips and techniques for running Spark in a container environment.
Containers are typically used to run stateless applications on a single host. There are significant real-world enterprise requirements that need to be addressed when running a stateful, distributed application in a secure multi-host container environment.
There are decisions that need to be made concerning which tools and infrastructure to use. There are many choices with respect to container managers, orchestration frameworks, and resource schedulers that are readily available today and some that may be available tomorrow including:]
• Mesos
• Kubernetes
• Docker Swarm
Each has its own strengths and weaknesses; each has unique characteristics that may make it suitable, or unsuitable, for Spark. Understanding these differences is critical to the successful deployment of Spark on Docker containers.
This session will describe the work done by the BlueData engineering team to run Spark inside containers, on a distributed platform, including the evaluation of various orchestration frameworks and lessons learned. You will learn how to apply practical networking and storage techniques to achieve high performance and agility in a distributed, container environment.
Speaker
Thomas Phelan, Chief Architect, Blue Data, Inc
Introduction to Orchestration and DevOps with OpenStack
I would like to thank all who participates in the webinar, it was a great pleasure to share and contribute,
Below are the links to the record of the Webinar,
All the Webinar:
Just the Demo:
you can also find all the slides the HEAT template file, the CLI and all the materials used in this webinar here:
The OpenStack VM all-in-one: https://www.dropbox.com/s/501ul31o6ilnmv3/coa-aio-newton.ova?dl=0
All the materials: https://drive.google.com/drive/folders/1dTSe4n2m3VoevIHZGT_q8uZIV7_f9ZJt?usp=sharing
Thanks to Racim and to the ELIANIS TECHNOLOGIES team.
Special thanks to our REDHAT ARCHITECT Sir. Djelloul Bouida for attending the webinar and all our group member.
For those who didn't join our Group, here the link to our Group on Facebook: https://www.facebook.com/groups/475301352862998/
Best Practices for Running Kafka on Docker Containers
Docker containers provide an ideal foundation for running Kafka-as-a-Service on-premises or in the public cloud. However, using Docker containers in production environments for Big Data workloads using Kafka poses some challenges – including container management, scheduling, network configuration and security, and performance.
In this session at Kafka Summit in August 2017, Nanda Vijyaydev of BlueData shared lessons learned from implementing Kafka-as-a-Service with Docker containers.
https://kafka-summit.org/sessions/kafka-service-docker-containers
Introduction to Apache Mesos and DC/OS
Introduction to Apache Mesos and DC/OS present at Los Angeles UUAC and Los Angeles Mesos users group meetup on Jan 11, 2018
2016 08-05 - Intro to OpenStack
1-hour intro to OpenStack by Leando Guillén. Watch the video of the presentation here!: https://goo.gl/yv0G9m
The Time Has Come for Big-Data-as-a-Service
Enterprises have been using both Big Data and Cloud Computing technologies for years. Until recently, the two have not been combined. Now the agility and efficiency benefits of self-service elastic infrastructure are being extended to Big Data initiatives – whether on-premises or in the public cloud.
This session at Hadoop Summit in San Jose, California (June 2016) discusses the emerging category of Big-Data-as-a-Service (BDaaS) - representing the intersection of Big Data and Cloud Computing.
In this session, Kris Applegate (Cloud and Big Data Solution Architect at Dell) and Thomas Phelan (Co-Founder and Chief Architect at BlueData) outlined the following:
- Innovations that paved the way for Big-Data-as-a-Service
- Definition and categories of Big-Data-as-a-Service
- Key considerations for Big-Data-as-a-Service in the enterprise, including public cloud or on-premises deployment options
A video replay can also be found here: https://youtu.be/_ucPoTKuj8Q
Similar to BlueData EPIC 2.0 Overview (20)
Nanda Vijaydev, BlueData - Deploying H2O in Large Scale Distributed Environme...
Nanda Vijaydev, BlueData - Deploying H2O in Large Scale Distributed Environme...
Deploying Big-Data-as-a-Service (BDaaS) in the Enterprise
Deploying Big-Data-as-a-Service (BDaaS) in the Enterprise
Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...
Lessons Learned from Dockerizing Spark Workloads: Spark Summit East talk by T...
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Lessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker Containers
The ABC of Docker: The Absolute Best Compendium of Docker
The ABC of Docker: The Absolute Best Compendium of Docker
Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?
Software Defined Storage, Big Data and Ceph - What Is all the Fuss About?
Dell openstack cloud with inktank ceph – large scale customer deployment
Dell openstack cloud with inktank ceph – large scale customer deployment
Introduction to Orchestration and DevOps with OpenStack
Introduction to Orchestration and DevOps with OpenStack
Best Practices for Running Kafka on Docker Containers
Best Practices for Running Kafka on Docker Containers
More from BlueData, Inc.
Introduction to KubeDirector - SF Kubernetes Meetup
Presentation from San Francisco Kubernetes Meetup on October 30, 2018
https://www.meetup.com/San-Francisco-Kubernetes-Meetup/events/255431002
What is KubeDirector? - Tom Phelan & Joel Baxter, Bluedata
Kubernetes is clearly the container orchestrator of choice for cloud-native stateless applications. And with the introduction of StatefulSets and Persistent Volumes it is becoming possible to run stateful applications on Kubernetes.
Now the new KubeDirector project allows users to manage complex stateful clusters for AI, machine learning, and big data analytics on Kubernetes without writing a single line of GO code.
KubeDirector is an open source Apache project that uses the standard Kubernetes custom resource functionality and API extensions to deploy and manage complex stateful scale-out application clusters.
This session will provide an overview of the KubeDirector architecture, show how to author the metadata and artifacts required for an example stateful application (e.g. with Spark, Jupyter, and Cassandra), and demonstrate the deployment and management of the cluster on Kubernetes using KubeDirector.
https://github.com/bluek8s/kubedirector
Dell EMC Ready Solutions for Big Data
Dell EMC Ready Solutions for Big Data are powered by the BlueData EPIC software platform - for on-demand provisioning and automation. These integrated solutions enable a cloud-like experience for Big-Data-as-a-Service (BDaaS) while ensuring the enterprise-grade security and performance of on-premises infrastructure.
With Dell EMC Ready Solutions for Big Data, customers can rapidly deploy their analytics and machine learning workloads in a secure multi-tenant architecture, for multiple different user groups running on shared infrastructure. Their users can quickly and easily provision distributed environments for Cloudera, Hortonworks, Kafka, MapR, Spark, TensorFlow, as well as other tools.
The new Ready Solutions include everything that customers need to enable BDaaS on-premises – including BlueData EPIC software as well as Dell EMC hardware, consulting, deployment, and support services.
To learn more, visit www.dellemc.com/bdaas
BlueData and Hortonworks Data Platform (HDP)
With BlueData, you can spin up instant containerized environments for the Hortonworks Data Platform (HDP) and other Big Data analytics and machine learning workloads — providing your data science teams with on-demand environments for greater agility. You can decouple compute from storage resources, to improve efficiency and reduce costs. And you can ensure the enterprise-grade security and governance that your IT teams require.
BlueData has completed certification through the rigorous Hortonworks QATS (Quality Assured Testing Suite) program for deploying HDP in a containerized environment. This certification enables Hortonworks and BlueData to provide best-in-class support and high performance for their customers’ existing and future investments in HDP.
“We’ve seen rapidly growing interest in running HDP on containers, therefore it was key that we work closely with BlueData to benefit those users,” said Scott Andress, vice president of global channels & alliances at Hortonworks. “They passed our most rigorous QATS certification tests, validating that BlueData provides complete interoperability and high performance for customers running HDP in containerized environments.”
How to Protect Big Data in a Containerized Environment
Every enterprise spends significant resources to protect its data. This is especially true in the case of big data, since some of this data may include sensitive or confidential customer and financial information. Common methods for protecting data include permissions and access controls as well as the encryption of data at rest and in flight.
The Hadoop community has recently rolled out Transparent Data Encryption (TDE) support in HDFS. Transparent Data Encryption refers to the process whereby data is transparently encrypted by the big data application writing the data; it is not decrypted again until it is accessed by another application. The data is encrypted during its entire lifespan—in transit and at rest—except when it is being specifically accessed by a processing application.
TDE is an excellent approach for protecting data stored in data lakes built on the latest versions of HDFS. However, it does have its challenges and limitations. Systems that want to use TDE require tight integration with enterprise-wide Kerberos Key Distribution Center (KDC) services and Key Management Systems (KMS). This integration isn’t easy to set up or maintain. These issues can be even more challenging in a virtualized or containerized environment where one Kerberos realm may be used to secure the big data compute cluster and a different Kerberos realm may be used to secure the HDFS filesystem accessed by this cluster.
BlueData has developed significant expertise in configuring, managing, and optimizing access to TDE-protected HDFS. This session at the Strata Data Conference in March 2018 (by Thomas Phelan, co-founder and chief architect at BlueData) offers a detailed overview of how transparent data encryption works with HDFS, with a particular focus on containerized environments.
You’ll learn how HDFS TDE is configured and maintained in an environment where many big data frameworks run simultaneously (e.g., in a hybrid cloud architecture using Docker containers). Moreover, you’ll learn how KDC credentials can be managed in a Kerberos cross-realm environment to provide data scientists and analysts with the greatest flexibility in accessing data while maintaining complete enterprise-grade data security.
https://conferences.oreilly.com/strata/strata-ca/public/schedule/detail/63763
BlueData EPIC datasheet (en Français)
La plateforme logicielle BlueData EPIC™ simplifie, accélère et rend plus rentable le déploiement d’infrastructures et d’applications Big Data telles que Hadoop, Spark, Kafka, Cassandra, et plus, dans l’infrastructure locale ou dans le cloud public.
Solution Brief: Real-Time Pipeline Accelerator
Get started with Spark Streaming, Kafka, and Cassandra for real-time data analytics.
BlueData makes it easy to deploy Spark infrastructure and applications on- premises. The BlueData EPIC software platform is purpose-built to simplify and accelerate the deployment of Spark, Hadoop, and other tools for Big Data analytics—leveraging Docker containers and virtualized infrastructure.
Our new Real-Time Pipeline Accelerator solution provides the software and professional services you need for building data pipelines in a multi-tenant environment for Spark Streaming, Kafka, and Cassandra. With help from the BlueData team, you’ll also have two end-to-end real-time data pipelines as a starting point.
Learn more about BlueData at www.bluedata.com
Solution Brief: Big Data Lab Accelerator
Accelerate Hadoop and Spark deployment in a multi-tenant lab environment for dev/test/QA, evaluation of multiple tools for Big Data analytics, and other use cases. BlueData provides a turnkey on-premises solution with software and services to get up and running in two weeks.
The new Big Data Lab Accelerator solution provides a full enterprise license of BlueData EPIC software along with the professional services needed to deploy an on-premises multi-tenant Big Data lab. Within two weeks, customers will have a lab environment to evaluate Big Data tools and spin up multiple Hadoop or Spark clusters for development, testing and quality assurance. As part of this deployment, BlueData will also work with customers to implement initial use cases for Big Data analytics.
Learn more about BlueData at www.bluedata.com
More from BlueData, Inc. (7)
Introduction to KubeDirector - SF Kubernetes Meetup
Introduction to KubeDirector - SF Kubernetes Meetup
How to Protect Big Data in a Containerized Environment
How to Protect Big Data in a Containerized Environment
Recently uploaded
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Prosigns: Transforming Business with Tailored Technology Solutions
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
Globus Connect Server Deep Dive - GlobusWorld 2024
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Cracking the code review at SpringIO 2024
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
Launch Your Streaming Platforms in Minutes
The claim of launching a streaming platform in minutes might be a bit of an exaggeration, but there are services that can significantly streamline the process. Here's a breakdown:
Pros of Speedy Streaming Platform Launch Services:
No coding required: These services often use drag-and-drop interfaces or pre-built templates, eliminating the need for programming knowledge.
Faster setup: Compared to building from scratch, these platforms can get you up and running much quicker.
All-in-one solutions: Many services offer features like content management systems (CMS), video players, and monetization tools, reducing the need for multiple integrations.
Things to Consider:
Limited customization: These platforms may offer less flexibility in design and functionality compared to custom-built solutions.
Scalability: As your audience grows, you might need to upgrade to a more robust platform or encounter limitations with the "quick launch" option.
Features: Carefully evaluate which features are included and if they meet your specific needs (e.g., live streaming, subscription options).
Examples of Services for Launching Streaming Platforms:
Muvi [muvi com]
Uscreen [usencreen tv]
Alternatives to Consider:
Existing Streaming platforms: Platforms like YouTube or Twitch might be suitable for basic streaming needs, though monetization options might be limited.
Custom Development: While more time-consuming, custom development offers the most control and flexibility for your platform.
Overall, launching a streaming platform in minutes might not be entirely realistic, but these services can significantly speed up the process compared to building from scratch. Carefully consider your needs and budget when choosing the best option for you.
Top 7 Unique WhatsApp API Benefits | Saudi Arabia
Discover the transformative power of the WhatsApp API in our latest SlideShare presentation, "Top 7 Unique WhatsApp API Benefits." In today's fast-paced digital era, effective communication is crucial for both personal and professional success. Whether you're a small business looking to enhance customer interactions or an individual seeking seamless communication with loved ones, the WhatsApp API offers robust capabilities that can significantly elevate your experience.
In this presentation, we delve into the top 7 distinctive benefits of the WhatsApp API, provided by the leading WhatsApp API service provider in Saudi Arabia. Learn how to streamline customer support, automate notifications, leverage rich media messaging, run scalable marketing campaigns, integrate secure payments, synchronize with CRM systems, and ensure enhanced security and privacy.
Quarkus Hidden and Forbidden Extensions
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
Text summarization of braking news using fine tuning BART model
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos
https://www.amb-review.com/tubetrivia-ai
Exclusive Features:
AI-Powered Questions,
Wide Range of Categories,
Adaptive Difficulty,
User-Friendly Interface,
Multiplayer Mode,
Regular Updates.
#TubeTriviaAI
#QuizVideoMagic
#ViralQuizVideos
#AIQuizGenerator
#EngageExciteExplode
#MarketingRevolution
#BoostYourTraffic
#SocialMediaSuccess
#AIContentCreation
#UnlimitedTraffic
A Sighting of filterA in Typelevel Rite of Passage
Slide deck home: https://fpilluminated.com/deck/220
First Steps with Globus Compute Multi-User Endpoints
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate offers a comprehensive solution for locate ticket management by automating and streamlining the entire process. By integrating with Geospatial Information Systems (GIS), it provides accurate mapping and visualization of utility locations, enhancing decision-making and reducing the risk of errors. The system's advanced data analytics tools help identify trends, predict potential issues, and optimize resource allocation, making the locate ticket management process smarter and more efficient. Additionally, automated ticket management ensures consistency and reduces human error, while real-time notifications keep all relevant personnel informed and ready to respond promptly.
The system's ability to streamline workflows and automate ticket routing significantly reduces the time taken to process each ticket, making the process faster and more efficient. Mobile access allows field technicians to update ticket information on the go, ensuring that the latest information is always available and accelerating the locate process. Overall, Utilocate not only enhances the efficiency and accuracy of locate ticket management but also improves safety by minimizing the risk of utility damage through precise and timely locates.
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Recently uploaded (20)
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology Solutions
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
BlueData EPIC 2.0 Overview
- 1. INTRODUCING BLUEDATA EPIC 2.0 September 23, 2015
- 2. Big Data Traditional Assumptions Bare-metal Data locality HDFS on local disks New Realities with BlueData Docker containers Compute and storage separation In-place access on remote data stores (e.g. HDFS, NFS, Object) New Benefits and Value Elasticity and improved utilization Agility and cost savings Faster time-to-insights Big Data – New Realities
- 3. BlueData EPIC Software Platform
- 4. BlueData EPIC 2.0 Highlights • Docker-based platform for deployment flexibility – Multi-host deployments: supports physical servers and virtual machines – Single operating system: containers inherit host OS updates – Simplified packaging distributions and apps: Docker images • Apache Spark notebooks and other innovations – Enhanced self-service Spark deployment – Clusters pre-integrated with web-based notebooks (Zeppelin) – SparkR, Spark Streaming, MLlib, Spark SQL, and Spark Streaming-SQL • Big Data App Store with one-click deployment – Packaged with Hadoop and Spark distributions, new analytics tools – One-click deployment for select ISV partner applications – Application Workbench to create and test images
- 5. Docker Deployment Flexibility Physical Machines or VMs as HOSTS Docker containers as NODES
- 6. Big Data Innovation with Containers • Docker containers for transparent, lightweight virtualization – Enables multi-node Hadoop/Spark clusters on physical servers or in VMs – Customizable container flavors/sizes for CPU, memory and storage – Configurable root disk sizes (local storage for containers) – Pre-integrated Docker images for Hadoop ecosystem products • IP management and networking specific to Big Data use cases – Container provisioning and orchestration across multiple hosts – Leveraging hosts with single network interface – Retaining IP address after container (e.g. Hadoop node) re-boot – DHCP management • Policy engine for multi-tenancy – Controlling resources available for containers based on tenant quotas
- 7. Spark 1.4 and Zeppelin Notebooks
- 8. Big Data App Store
- 9. One-Click Big Data App Deployment
- 10. www.bluedata.com