Seagate - ceph day taiwan 2017 opening session
•
1 like•130 views
The document summarizes key points from a Ceph Day Taiwan 2017 opening session: 1. It discusses the evolution of computing platforms from mainframes to client-server to mobile-cloud and the emerging fourth platform of edge intelligence expanded by machines. 2. IDC identifies five key trends driving explosive data growth including the evolution of data to life-critical use cases and the rise of embedded systems, IoT, mobile and real-time data, and cognitive/AI systems. 3. Global datasphere size is projected to grow 10x from 2016 to 2025, from 16ZB to 163ZB as data creation increases across cloud, big data, machine learning, mobility, IoT, and more.
Report
Share
Report
Share
Download to read offline
Recommended
Arm - ceph on arm update
講者：
Jeff Chu (Director of Enterprise Solutions, ARM)
Kan Yan Rong (Technical Expert in Storage and Application) Technology, WDC/SanDisk)
概要：
Jeff from ARM will provide a brief update on the activities furthering Ceph on ARM including some recent progress from ARM as well some increased community activity. After that Chris and Yan from Western Digital/San Disk will be presenting the topic on Ceph Block Performance on Cavium ARM and SATA SSDs.
Ambedded - how to build a true no single point of failure ceph cluster
講者：
晨宇創新創辦人兼CEO - Aaron Joue
概要：
傳統的伺服器架構對於Ceph分散式儲存，有著許多待解決的問題。例如使用單一伺服器服務多個OSD daemon/container時，若是伺服器失效時會對整個叢集造成嚴重的影響。晨宇創新針對這些問題提出了創新的架構。 在這場演講中，Aaron將分享去中心化的Ceph Appliance 如何解決這些問題, 以及其為Ceph打造的管理介面，如何將Ceph的管理自動化，無負擔的管理Ceph叢集。
ThunderX ARMV8 Servers: Disruption and Innovation in the Server Market
Cavium joined Red Hat Storage Day New York on 1/19/16 to give the history of the ARM server ecosystem, explain the innovation of ThunderX, and describe scale out's influence on target workloads.
Red Hat Storage Day Boston - Red Hat Gluster Storage vs. Traditional Storage ...
Red Hat Gluster Storage provides a software-defined storage solution that is more cost efficient and flexible than traditional storage appliances. It leverages standard x86 hardware and has open source architecture with no vendor lock-in. A comparison shows Gluster Storage outperforms EMC Isilon on factors like cost, scalability, data protection methods, access protocols, and management capabilities. Gluster Storage is positioned to go beyond traditional storage by supporting containers, disaster recovery in cloud environments, and its roadmap includes additional advanced features.
Red Hat Storage Day Boston - Persistent Storage for Containers
Persistent storage is important for containerized applications. Red Hat provides container-ready storage using Red Hat Gluster Storage which provides scalable, distributed file storage for containers. It allows storage and containers to coexist on the same hardware, improving utilization and lowering costs. Red Hat Gluster Storage is optimized to provide container-native storage on OpenShift for workloads like databases to get the benefits of containers while ensuring persistent storage.
Red Hat Storage Day New York - Intel Unlocking Big Data Infrastructure Effici...
This document discusses using Ceph storage with Apache Hadoop to provide a scalable and efficient storage solution for big data workloads. It outlines the challenges of scaling Hadoop storage independently from compute resources using the native Hadoop Distributed File System. The solution presented is to use the open source Ceph storage system instead of direct-attached storage. This allows Hadoop compute and storage resources to scale independently and provides a centralized storage platform for all enterprise data workloads. Performance tests showed the Ceph and Hadoop configuration providing up to a 60% improvement in I/O performance when using Intel caching software and SSDs.
Red hat on_power-ibm _lop_day_2015
Chris Milsted from Red Hat's slides from the IBM Linux On Power Revolution event in IBM Southbank #LOPNow
Open CAPI, A New Standard for High Performance Attachment of Memory, Accelera...
In this deck from the Switzerland HPC Conference, Jeffrey Stuecheli from IBM presents: Open CAPI, A New Standard for High Performance Attachment of Memory, Acceleration, and Networks.
"OpenCAPI sets a new standard for the industry, providing a high bandwidth, low latency open interface design specification. This session will introduce the new standard and it's goals. This includes details on how the interface protocol provides unprecedented latency and bandwidth to attached devices."
Watch the video: http://wp.me/p3RLHQ-gDZ
Learn more: http://opencapi.org/
and
http://hpcadvisorycouncil.com/events/2017/swiss-workshop/agenda.php
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Recommended
Arm - ceph on arm update
講者：
Jeff Chu (Director of Enterprise Solutions, ARM)
Kan Yan Rong (Technical Expert in Storage and Application) Technology, WDC/SanDisk)
概要：
Jeff from ARM will provide a brief update on the activities furthering Ceph on ARM including some recent progress from ARM as well some increased community activity. After that Chris and Yan from Western Digital/San Disk will be presenting the topic on Ceph Block Performance on Cavium ARM and SATA SSDs.
Ambedded - how to build a true no single point of failure ceph cluster
講者：
晨宇創新創辦人兼CEO - Aaron Joue
概要：
傳統的伺服器架構對於Ceph分散式儲存，有著許多待解決的問題。例如使用單一伺服器服務多個OSD daemon/container時，若是伺服器失效時會對整個叢集造成嚴重的影響。晨宇創新針對這些問題提出了創新的架構。 在這場演講中，Aaron將分享去中心化的Ceph Appliance 如何解決這些問題, 以及其為Ceph打造的管理介面，如何將Ceph的管理自動化，無負擔的管理Ceph叢集。
ThunderX ARMV8 Servers: Disruption and Innovation in the Server Market
Cavium joined Red Hat Storage Day New York on 1/19/16 to give the history of the ARM server ecosystem, explain the innovation of ThunderX, and describe scale out's influence on target workloads.
Red Hat Storage Day Boston - Red Hat Gluster Storage vs. Traditional Storage ...
Red Hat Gluster Storage provides a software-defined storage solution that is more cost efficient and flexible than traditional storage appliances. It leverages standard x86 hardware and has open source architecture with no vendor lock-in. A comparison shows Gluster Storage outperforms EMC Isilon on factors like cost, scalability, data protection methods, access protocols, and management capabilities. Gluster Storage is positioned to go beyond traditional storage by supporting containers, disaster recovery in cloud environments, and its roadmap includes additional advanced features.
Red Hat Storage Day Boston - Persistent Storage for Containers
Persistent storage is important for containerized applications. Red Hat provides container-ready storage using Red Hat Gluster Storage which provides scalable, distributed file storage for containers. It allows storage and containers to coexist on the same hardware, improving utilization and lowering costs. Red Hat Gluster Storage is optimized to provide container-native storage on OpenShift for workloads like databases to get the benefits of containers while ensuring persistent storage.
Red Hat Storage Day New York - Intel Unlocking Big Data Infrastructure Effici...
This document discusses using Ceph storage with Apache Hadoop to provide a scalable and efficient storage solution for big data workloads. It outlines the challenges of scaling Hadoop storage independently from compute resources using the native Hadoop Distributed File System. The solution presented is to use the open source Ceph storage system instead of direct-attached storage. This allows Hadoop compute and storage resources to scale independently and provides a centralized storage platform for all enterprise data workloads. Performance tests showed the Ceph and Hadoop configuration providing up to a 60% improvement in I/O performance when using Intel caching software and SSDs.
Red hat on_power-ibm _lop_day_2015
Chris Milsted from Red Hat's slides from the IBM Linux On Power Revolution event in IBM Southbank #LOPNow
Open CAPI, A New Standard for High Performance Attachment of Memory, Accelera...
In this deck from the Switzerland HPC Conference, Jeffrey Stuecheli from IBM presents: Open CAPI, A New Standard for High Performance Attachment of Memory, Acceleration, and Networks.
"OpenCAPI sets a new standard for the industry, providing a high bandwidth, low latency open interface design specification. This session will introduce the new standard and it's goals. This includes details on how the interface protocol provides unprecedented latency and bandwidth to attached devices."
Watch the video: http://wp.me/p3RLHQ-gDZ
Learn more: http://opencapi.org/
and
http://hpcadvisorycouncil.com/events/2017/swiss-workshop/agenda.php
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Red Hat Storage Day LA - Persistent Storage for Linux Containers
This document discusses persistent storage options for Linux containers and how Red Hat Storage addresses the storage needs of containerized applications. It begins by explaining how containers package applications and dependencies for portability and ease of management. Typical workloads for containers often require persistent storage. The document then evaluates options like NFS, GlusterFS, Ceph RBD, iSCSI/FC and public cloud storage, noting that Red Hat Storage provides scalable, resilient, flexible software-defined storage. It presents Red Hat Storage and OpenShift Enterprise as a solution that allows applications and storage to run together on servers for improved utilization and costs. The document closes with two customer case studies demonstrating how Red Hat Storage supports containerized workloads at CapitalOne and Ver
Ceph Day KL - Ceph on ARM
This presentation discusses how ARM-based systems can provide scalable and efficient CEPH storage solutions. It outlines how the ARM ecosystem is innovating in storage, including low-cost enterprise SMB storage and energy efficient enterprise storage racks. Examples are given of CEPH implementations on ARM, such as a 504 node CEPH cluster using converged microservers and CEPH performance on ThunderX platforms comparable to x86 servers but with lower total cost of ownership. Overall, the presentation argues ARM-based systems can deliver scalable, portable, and optimized intelligent flexible cloud storage using CEPH.
OpenStack Resources and Capacity Management - Shimon Benattar, Mark Rasin - O...
OpenStack Resources and Capacity Management - Shimon Benattar, Mark Rasin - O...Cloud Native Day Tel Aviv
In this session we will present AT&T's vision on how to manage multiple OpenStacks.
By 2020 AT&T is planning to move 80% of its network to the cloud and to so will need ~1000 sites which will all be running OpenStack.
Managing multiple clouds raises resource management issues, these involv operative resource management and resource capacity planning.
In order to manage multiple projects, images, flavors we created ORM (OpenStack Resource Manager) which AT&T is also planning to Open Source in 2017.
In order to manage the capacity we need to understand the relationship between available, allocated and used resources, and their dependency on the timeline. We will review the information sources that can be useful for tracking cloud resources and look at the options to implement a reservation system that will help capacity planners to manage the resources.
If we have time left we will briefly review the OpenStack Blazar project and the OPNFV Promise project that aim to address the problem in the subject, and we will discuss the possibility to implement your enterprise solution based on these projects, or alternatively using some commercially available product, or even building your own. Ceph Day Seoul - Ceph on Arm Scaleable and Efficient
This document discusses how Ceph storage solutions can benefit from ARM-based platforms. It outlines how the ARM ecosystem provides increased efficiency and scale for Ceph through lower costs, higher energy efficiency, and simplified designs. Examples are given of various companies delivering Ceph clusters using ARM processors, including solutions optimized for microservers, converged infrastructure, and enterprise storage. The recent Jewel release of Ceph added support for the AARCH64 instruction set, opening up additional opportunities for Ceph on ARM platforms.
High Performance Interconnects: Landscape, Assessments & Rankings
“Dan Olds will present recent research into the history of High Performance Interconnects (HPI), the current state of the HPI market, where HPIs are going in the future, and how customers should evaluate HPI options today. This will be a highly informative and interactive session.”
Watch the video: http://insidehpc.com/2017/04/high-performance-interconnects-assessments-rankings-landscape/
Learn more: http://orionx.net
Sign up for our insidehpc.com/newsletter: http://insidehpc.com/newsletter
RDMA on ARM
"Applications, programming languages, and libraries that leverage sophisticated network hardware capabilities have a natural advantage when used in today’s and tomorrow’s high-performance and data center computer environments. Modern RDMA based network interconnects provides incredibly rich functionality (RDMA, Atomics, OS-bypass, etc.) that enable low-latency and high-bandwidth communication services. The functionality is supported by a variety of interconnect technologies such as InfiniBand, RoCE, iWARP, Intel OPA, Cray’s Aries/Gemini, and others. OFA organization and LinuxRDMA community have been playing a predominant role in the enablement efficient and vendor agnostic software stack for those interconnects. Over the last decade, the community has developed variety user/kernel level protocols and libraries that enable a variety of applications over RDMA including MPI, SHMEM, NFS over RDMA, IPoIB, and many others."
"With the emerging availability server platforms based on ARM CPU architecture, it is important to understand ARM integrates with RDMA hardware and software eco-system. In this talk, we will overview ARM architecture and system software stack. We will discuss how ARM CPU interacts with network devices and accelerators. In addition, we will share our experience in enabling RDMA software stack (OFED/MOFED Verbs) and one-sided communication libraries (Open UCX, OpenSHMEM/SHMEM) on ARM and share preliminary evaluation results."
Watch the video presentation: http://wp.me/p3RLHQ-gyO
Learn more: https://www.openfabrics.org/index.php/abstracts-agenda.html
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Red Hat Storage Day New York - What's New in Red Hat Ceph Storage
Red Hat Ceph Storage 2.0 will focus on usability improvements like the Red Hat Storage Console management tool, improved object storage features such as global clusters and authentication support, and performance enhancements from the new BlueStore backend. Future releases will add capabilities like block volume mirroring, CephFS support, and integration with OpenStack services. Upstream development will continue expanding Ceph's object, block, and file functionality.
Optimized HPC/AI cloud with OpenStack acceleration service and composable har...
Today data scientist is turning to cloud for AI and HPC workloads. However, AI/HPC applications require high computational throughput where generic cloud resources would not suffice. There is a strong demand for OpenStack to support hardware accelerated devices in a dynamic model.
In this session, we will introduce OpenStack Acceleration Service – Cyborg, which provides a management framework for accelerator devices (e.g. FPGA, GPU, NVMe SSD). We will also discuss Rack Scale Design (RSD) technology and explain how physical hardware resources can be dynamically aggregated to meet the AI/HPC requirements. The ability to “compose on the fly” with workload-optimized hardware and accelerator devices through an API allow data center managers to manage these resources in an efficient automated manner.
We will also introduce an enhanced telemetry solution with Gnnochi, bandwidth discovery and smart scheduling, by leveraging RSD technology, for efficient workloads management in HPC/AI cloud.
ARM HPC Ecosystem
"Algorithmic processing performed in High Performance Computing environments impacts the lives of billions of people, and planning for exascale computing presents significant power challenges to the industry. ARM delivers the enabling technology behind HPC. The 64-bit design of the ARMv8-A architecture combined with Advanced SIMD vectorization are ideal to enable large scientific computing calculations to be executed efficiently on ARM HPC machines. In addition ARM and its partners are working to ensure that all the software tools and libraries, needed by both users and systems administrators, are provided in readily available, optimized packages."
Learn more: https://developer.arm.com/hpc
and
http://hpcuserforum.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
RedisConf17- Zettaset + Redis - Protecting Redis Enterprise while Maintaining...
Zettaset provides encryption software that can protect Redis databases while maintaining high performance. It uses transparent encryption optimized for distributed systems and clouds. The software encrypts Redis data both in memory and persistent storage without impacting Redis' performance or scalability. When combined with Redis, Zettaset offers customers an industry-leading database with both maximum performance and security.
Ceph Day Tokyo - Ceph on ARM: Scaleable and Efficient
This document discusses the opportunity for Ceph storage solutions using ARM processors. It outlines how the ARM ecosystem enables scalable and efficient storage options through increased performance, lower costs, and greater energy efficiency. The recent Jewel release of Ceph added support for AARCH64 processors. Several companies are developing Ceph clusters using ARM-based platforms that demonstrate benefits like reduced power consumption and total cost of ownership compared to x86 solutions.
Intelligent Interconnect Architecture to Enable Next Generation HPC - Linaro ...
Speakers: Gilad Shainer and Scot Schultz
Company: Mellanox Technologies
Talk Title: Intelligent Interconnect Architecture to Enable Next
Generation HPC
Talk Abstract:
The latest revolution in HPC interconnect architecture is the development of In-Network Computing, a technology that enables handling and accelerating application workloads at the network level. By placing data-related algorithms on an intelligent network, we can overcome the new performance bottlenecks and improve the data center and applications performance. The combination of In-Network Computing and ARM based processors offer a rich set of capabilities and opportunities to build the next generation of HPC platforms.
Gilad Shainer Bio:
Gilad Shainer has served as Mellanox's vice president of marketing since March 2013. Previously, Mr. Shainer was Mellanox's vice president of marketing development from March 2012 to March 2013. Mr. Shainer joined Mellanox in 2001 as a design engineer and later served in senior marketing management roles between July 2005 and February 2012. Mr. Shainer holds several patents in the field of high-speed networking and contributed to the PCI-SIG PCI-X and PCIe specifications. Gilad Shainer holds a MSc degree (2001, Cum Laude) and a BSc degree (1998, Cum Laude) in Electrical Engineering from the Technion Institute of Technology in Israel.
Scot Schultz Bio:
Scot Schultz is a HPC technology specialist with broad knowledge in operating systems, high speed interconnects and processor technologies. Joining the Mellanox team in 2013, Schultz is 30-year veteran of the computing industry. Prior to joining Mellanox, he spent the past 17 years at AMD in various engineering and leadership roles in the area of high performance computing. Scot has also been instrumental with the growth and development of various industry organizations including the Open Fabrics Alliance, and continues to serve as a founding board-member of the OpenPOWER Foundation and Director of Educational Outreach and founding member of the HPC-AI Advisory Council.
Ceph Day Beijing- Ceph Community Update
This document provides a community update on Ceph activities in 2017. It discusses several Ceph Days conferences scheduled around the world, metrics on community growth, the User Committee and governance efforts. It also outlines technical talks, performance discussions, developer meetings and the new Ceph.com site. The document highlights contributions from the China community including code commits, meetups with over 1,000 attendees, an upcoming Ceph white paper, new books and the first Ceph training and certification program.
Mellanox High Performance Networks for Ceph
This document discusses how Mellanox networks enable high performance Ceph storage clusters. It notes that Ceph performance and scalability are dictated by the backend cluster network performance. It provides examples of customers deploying Ceph with Mellanox 40GbE and 10GbE interconnects, and highlights how these networks allow building scalable, high performing storage solutions. Specifically, it shows how 40GbE cluster networks and 40GbE client networks provide much higher throughput and IOPS compared to 10GbE. The document concludes by mentioning how RDMA offloads can free CPU for application processing, and how the Accelio library enables high performance RDMA for Ceph.
Ceph Day Taipei - How ARM Microserver Cluster Performs in Ceph
This document discusses how an ARM-based microserver cluster performs for Ceph storage. It outlines the issues with using a single server node with multiple Ceph OSDs, and describes how using a single micro server with one OSD reduces failure domains and bottlenecks. Test results show increased performance and lower power consumption compared to traditional servers. The document also introduces Ambedded's Ceph storage appliance and management software.
Ceph Day Taipei - Accelerate Ceph via SPDK
The document discusses accelerating Ceph storage performance using SPDK. SPDK introduces optimizations like asynchronous APIs, userspace I/O stacks, and polling mode drivers to reduce software overhead and better utilize fast storage devices. This allows Ceph to better support high performance networks and storage like NVMe SSDs. The document provides an example where SPDK helped XSKY's BlueStore object store achieve significant performance gains over the standard Ceph implementation.
Ceph Day Melabourne - Community Update
The document provides updates on the Ceph community and projects. It discusses that Ceph Day Melbourne 2015 was community-focused and planning has begun for 2016 events. It also summarizes that a Ceph hackathon was hosted by Intel, metrics platforms are being used and evolved to track community growth, and a user committee discusses community matters. Additional community initiatives discussed include Google Summer of Code, the CentOS Storage SIG, governance plans, Ceph tech talks, and the Ceph developer summit.
Ceph Day Beijing - Welcome to Beijing Ceph Day
This document contains an agenda for a Ceph conference. It lists the scheduled time, duration, topic, and speakers for each presentation during the day. Talks will cover various topics related to optimizing Ceph performance using Intel technologies, all-flash array design using Ceph, global deduplication solutions for Ceph, and experiences deploying large-scale Ceph clusters at companies like Alibaba and China Mobile. The day includes keynotes, presentations, breaks for networking, and a closing session.
Ceph Day Tokyo - Delivering cost effective, high performance Ceph cluster
This document discusses an all-NVMe Ceph cluster configuration for MySQL hosting. It describes a 5-node Ceph cluster with Intel Xeon processors, 128GB of RAM, and 20 Intel SSDs providing 80 object storage devices (OSDs) for a total effective capacity of 19TB. Benchmark results show the cluster achieving over 1.4 million IOPS for 4K random reads with an average latency of 1ms, and over 220K IOPS for 4K random writes with 5ms latency. Sysbench tests of MySQL databases on the cluster using 16KB IOs showed response times under 10ms for query depths from 2 to 8.
Storage for Containerized Applications
Persistent storage for containerized applications
Linux containers are on course to change DevOps forever. Container technology will also impact how we think about persistent storage for applications and microservices.
In turn, software-defined storage will impact how storage is dynamically provisioned and managed for containerized applications. Using close integration with orchestration frameworks, such as Kubernetes and private Platform-as-a-Service (PaaS) such as OpenShift, Red Hat® Storage extends the current limit of performance by providing seamless, enterprise-grade storage for critical applications in containers.
bigdata.pptx
This document provides an overview of a seminar on big data. It begins with an introduction on how big data has become important in the IT world. It then defines big data, noting its characteristics of volume, velocity, and variety. It discusses storing, selecting, and processing big data using tools like Hadoop. It covers sources of big data and applications such as smarter healthcare, manufacturing, and traffic control. Both the risks and benefits of big data are mentioned, such as being overwhelmed by data but also making better decisions. Finally, it discusses the future of big data and its importance for the Indian market.
Big data
This document provides an overview of big data presented by five individuals. It defines big data, discusses its three key characteristics of volume, velocity and variety. It explains how big data is stored, selected and processed using techniques like Hadoop and MapReduce. Examples of big data sources and tools are provided. Applications of big data across various industries are highlighted. Both the risks and benefits of big data are summarized. The future growth of big data and its impact on IT is also outlined.
More Related Content
What's hot
Red Hat Storage Day LA - Persistent Storage for Linux Containers
This document discusses persistent storage options for Linux containers and how Red Hat Storage addresses the storage needs of containerized applications. It begins by explaining how containers package applications and dependencies for portability and ease of management. Typical workloads for containers often require persistent storage. The document then evaluates options like NFS, GlusterFS, Ceph RBD, iSCSI/FC and public cloud storage, noting that Red Hat Storage provides scalable, resilient, flexible software-defined storage. It presents Red Hat Storage and OpenShift Enterprise as a solution that allows applications and storage to run together on servers for improved utilization and costs. The document closes with two customer case studies demonstrating how Red Hat Storage supports containerized workloads at CapitalOne and Ver
Ceph Day KL - Ceph on ARM
This presentation discusses how ARM-based systems can provide scalable and efficient CEPH storage solutions. It outlines how the ARM ecosystem is innovating in storage, including low-cost enterprise SMB storage and energy efficient enterprise storage racks. Examples are given of CEPH implementations on ARM, such as a 504 node CEPH cluster using converged microservers and CEPH performance on ThunderX platforms comparable to x86 servers but with lower total cost of ownership. Overall, the presentation argues ARM-based systems can deliver scalable, portable, and optimized intelligent flexible cloud storage using CEPH.
OpenStack Resources and Capacity Management - Shimon Benattar, Mark Rasin - O...
OpenStack Resources and Capacity Management - Shimon Benattar, Mark Rasin - O...Cloud Native Day Tel Aviv
In this session we will present AT&T's vision on how to manage multiple OpenStacks.
By 2020 AT&T is planning to move 80% of its network to the cloud and to so will need ~1000 sites which will all be running OpenStack.
Managing multiple clouds raises resource management issues, these involv operative resource management and resource capacity planning.
In order to manage multiple projects, images, flavors we created ORM (OpenStack Resource Manager) which AT&T is also planning to Open Source in 2017.
In order to manage the capacity we need to understand the relationship between available, allocated and used resources, and their dependency on the timeline. We will review the information sources that can be useful for tracking cloud resources and look at the options to implement a reservation system that will help capacity planners to manage the resources.
If we have time left we will briefly review the OpenStack Blazar project and the OPNFV Promise project that aim to address the problem in the subject, and we will discuss the possibility to implement your enterprise solution based on these projects, or alternatively using some commercially available product, or even building your own. Ceph Day Seoul - Ceph on Arm Scaleable and Efficient
This document discusses how Ceph storage solutions can benefit from ARM-based platforms. It outlines how the ARM ecosystem provides increased efficiency and scale for Ceph through lower costs, higher energy efficiency, and simplified designs. Examples are given of various companies delivering Ceph clusters using ARM processors, including solutions optimized for microservers, converged infrastructure, and enterprise storage. The recent Jewel release of Ceph added support for the AARCH64 instruction set, opening up additional opportunities for Ceph on ARM platforms.
High Performance Interconnects: Landscape, Assessments & Rankings
“Dan Olds will present recent research into the history of High Performance Interconnects (HPI), the current state of the HPI market, where HPIs are going in the future, and how customers should evaluate HPI options today. This will be a highly informative and interactive session.”
Watch the video: http://insidehpc.com/2017/04/high-performance-interconnects-assessments-rankings-landscape/
Learn more: http://orionx.net
Sign up for our insidehpc.com/newsletter: http://insidehpc.com/newsletter
RDMA on ARM
"Applications, programming languages, and libraries that leverage sophisticated network hardware capabilities have a natural advantage when used in today’s and tomorrow’s high-performance and data center computer environments. Modern RDMA based network interconnects provides incredibly rich functionality (RDMA, Atomics, OS-bypass, etc.) that enable low-latency and high-bandwidth communication services. The functionality is supported by a variety of interconnect technologies such as InfiniBand, RoCE, iWARP, Intel OPA, Cray’s Aries/Gemini, and others. OFA organization and LinuxRDMA community have been playing a predominant role in the enablement efficient and vendor agnostic software stack for those interconnects. Over the last decade, the community has developed variety user/kernel level protocols and libraries that enable a variety of applications over RDMA including MPI, SHMEM, NFS over RDMA, IPoIB, and many others."
"With the emerging availability server platforms based on ARM CPU architecture, it is important to understand ARM integrates with RDMA hardware and software eco-system. In this talk, we will overview ARM architecture and system software stack. We will discuss how ARM CPU interacts with network devices and accelerators. In addition, we will share our experience in enabling RDMA software stack (OFED/MOFED Verbs) and one-sided communication libraries (Open UCX, OpenSHMEM/SHMEM) on ARM and share preliminary evaluation results."
Watch the video presentation: http://wp.me/p3RLHQ-gyO
Learn more: https://www.openfabrics.org/index.php/abstracts-agenda.html
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Red Hat Storage Day New York - What's New in Red Hat Ceph Storage
Red Hat Ceph Storage 2.0 will focus on usability improvements like the Red Hat Storage Console management tool, improved object storage features such as global clusters and authentication support, and performance enhancements from the new BlueStore backend. Future releases will add capabilities like block volume mirroring, CephFS support, and integration with OpenStack services. Upstream development will continue expanding Ceph's object, block, and file functionality.
Optimized HPC/AI cloud with OpenStack acceleration service and composable har...
Today data scientist is turning to cloud for AI and HPC workloads. However, AI/HPC applications require high computational throughput where generic cloud resources would not suffice. There is a strong demand for OpenStack to support hardware accelerated devices in a dynamic model.
In this session, we will introduce OpenStack Acceleration Service – Cyborg, which provides a management framework for accelerator devices (e.g. FPGA, GPU, NVMe SSD). We will also discuss Rack Scale Design (RSD) technology and explain how physical hardware resources can be dynamically aggregated to meet the AI/HPC requirements. The ability to “compose on the fly” with workload-optimized hardware and accelerator devices through an API allow data center managers to manage these resources in an efficient automated manner.
We will also introduce an enhanced telemetry solution with Gnnochi, bandwidth discovery and smart scheduling, by leveraging RSD technology, for efficient workloads management in HPC/AI cloud.
ARM HPC Ecosystem
"Algorithmic processing performed in High Performance Computing environments impacts the lives of billions of people, and planning for exascale computing presents significant power challenges to the industry. ARM delivers the enabling technology behind HPC. The 64-bit design of the ARMv8-A architecture combined with Advanced SIMD vectorization are ideal to enable large scientific computing calculations to be executed efficiently on ARM HPC machines. In addition ARM and its partners are working to ensure that all the software tools and libraries, needed by both users and systems administrators, are provided in readily available, optimized packages."
Learn more: https://developer.arm.com/hpc
and
http://hpcuserforum.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
RedisConf17- Zettaset + Redis - Protecting Redis Enterprise while Maintaining...
Zettaset provides encryption software that can protect Redis databases while maintaining high performance. It uses transparent encryption optimized for distributed systems and clouds. The software encrypts Redis data both in memory and persistent storage without impacting Redis' performance or scalability. When combined with Redis, Zettaset offers customers an industry-leading database with both maximum performance and security.
Ceph Day Tokyo - Ceph on ARM: Scaleable and Efficient
This document discusses the opportunity for Ceph storage solutions using ARM processors. It outlines how the ARM ecosystem enables scalable and efficient storage options through increased performance, lower costs, and greater energy efficiency. The recent Jewel release of Ceph added support for AARCH64 processors. Several companies are developing Ceph clusters using ARM-based platforms that demonstrate benefits like reduced power consumption and total cost of ownership compared to x86 solutions.
Intelligent Interconnect Architecture to Enable Next Generation HPC - Linaro ...
Speakers: Gilad Shainer and Scot Schultz
Company: Mellanox Technologies
Talk Title: Intelligent Interconnect Architecture to Enable Next
Generation HPC
Talk Abstract:
The latest revolution in HPC interconnect architecture is the development of In-Network Computing, a technology that enables handling and accelerating application workloads at the network level. By placing data-related algorithms on an intelligent network, we can overcome the new performance bottlenecks and improve the data center and applications performance. The combination of In-Network Computing and ARM based processors offer a rich set of capabilities and opportunities to build the next generation of HPC platforms.
Gilad Shainer Bio:
Gilad Shainer has served as Mellanox's vice president of marketing since March 2013. Previously, Mr. Shainer was Mellanox's vice president of marketing development from March 2012 to March 2013. Mr. Shainer joined Mellanox in 2001 as a design engineer and later served in senior marketing management roles between July 2005 and February 2012. Mr. Shainer holds several patents in the field of high-speed networking and contributed to the PCI-SIG PCI-X and PCIe specifications. Gilad Shainer holds a MSc degree (2001, Cum Laude) and a BSc degree (1998, Cum Laude) in Electrical Engineering from the Technion Institute of Technology in Israel.
Scot Schultz Bio:
Scot Schultz is a HPC technology specialist with broad knowledge in operating systems, high speed interconnects and processor technologies. Joining the Mellanox team in 2013, Schultz is 30-year veteran of the computing industry. Prior to joining Mellanox, he spent the past 17 years at AMD in various engineering and leadership roles in the area of high performance computing. Scot has also been instrumental with the growth and development of various industry organizations including the Open Fabrics Alliance, and continues to serve as a founding board-member of the OpenPOWER Foundation and Director of Educational Outreach and founding member of the HPC-AI Advisory Council.
Ceph Day Beijing- Ceph Community Update
This document provides a community update on Ceph activities in 2017. It discusses several Ceph Days conferences scheduled around the world, metrics on community growth, the User Committee and governance efforts. It also outlines technical talks, performance discussions, developer meetings and the new Ceph.com site. The document highlights contributions from the China community including code commits, meetups with over 1,000 attendees, an upcoming Ceph white paper, new books and the first Ceph training and certification program.
Mellanox High Performance Networks for Ceph
This document discusses how Mellanox networks enable high performance Ceph storage clusters. It notes that Ceph performance and scalability are dictated by the backend cluster network performance. It provides examples of customers deploying Ceph with Mellanox 40GbE and 10GbE interconnects, and highlights how these networks allow building scalable, high performing storage solutions. Specifically, it shows how 40GbE cluster networks and 40GbE client networks provide much higher throughput and IOPS compared to 10GbE. The document concludes by mentioning how RDMA offloads can free CPU for application processing, and how the Accelio library enables high performance RDMA for Ceph.
Ceph Day Taipei - How ARM Microserver Cluster Performs in Ceph
This document discusses how an ARM-based microserver cluster performs for Ceph storage. It outlines the issues with using a single server node with multiple Ceph OSDs, and describes how using a single micro server with one OSD reduces failure domains and bottlenecks. Test results show increased performance and lower power consumption compared to traditional servers. The document also introduces Ambedded's Ceph storage appliance and management software.
Ceph Day Taipei - Accelerate Ceph via SPDK
The document discusses accelerating Ceph storage performance using SPDK. SPDK introduces optimizations like asynchronous APIs, userspace I/O stacks, and polling mode drivers to reduce software overhead and better utilize fast storage devices. This allows Ceph to better support high performance networks and storage like NVMe SSDs. The document provides an example where SPDK helped XSKY's BlueStore object store achieve significant performance gains over the standard Ceph implementation.
Ceph Day Melabourne - Community Update
The document provides updates on the Ceph community and projects. It discusses that Ceph Day Melbourne 2015 was community-focused and planning has begun for 2016 events. It also summarizes that a Ceph hackathon was hosted by Intel, metrics platforms are being used and evolved to track community growth, and a user committee discusses community matters. Additional community initiatives discussed include Google Summer of Code, the CentOS Storage SIG, governance plans, Ceph tech talks, and the Ceph developer summit.
Ceph Day Beijing - Welcome to Beijing Ceph Day
This document contains an agenda for a Ceph conference. It lists the scheduled time, duration, topic, and speakers for each presentation during the day. Talks will cover various topics related to optimizing Ceph performance using Intel technologies, all-flash array design using Ceph, global deduplication solutions for Ceph, and experiences deploying large-scale Ceph clusters at companies like Alibaba and China Mobile. The day includes keynotes, presentations, breaks for networking, and a closing session.
Ceph Day Tokyo - Delivering cost effective, high performance Ceph cluster
This document discusses an all-NVMe Ceph cluster configuration for MySQL hosting. It describes a 5-node Ceph cluster with Intel Xeon processors, 128GB of RAM, and 20 Intel SSDs providing 80 object storage devices (OSDs) for a total effective capacity of 19TB. Benchmark results show the cluster achieving over 1.4 million IOPS for 4K random reads with an average latency of 1ms, and over 220K IOPS for 4K random writes with 5ms latency. Sysbench tests of MySQL databases on the cluster using 16KB IOs showed response times under 10ms for query depths from 2 to 8.
Storage for Containerized Applications
Persistent storage for containerized applications
Linux containers are on course to change DevOps forever. Container technology will also impact how we think about persistent storage for applications and microservices.
In turn, software-defined storage will impact how storage is dynamically provisioned and managed for containerized applications. Using close integration with orchestration frameworks, such as Kubernetes and private Platform-as-a-Service (PaaS) such as OpenShift, Red Hat® Storage extends the current limit of performance by providing seamless, enterprise-grade storage for critical applications in containers.
What's hot (20)
Red Hat Storage Day LA - Persistent Storage for Linux Containers
Red Hat Storage Day LA - Persistent Storage for Linux Containers
OpenStack Resources and Capacity Management - Shimon Benattar, Mark Rasin - O...
OpenStack Resources and Capacity Management - Shimon Benattar, Mark Rasin - O...
Ceph Day Seoul - Ceph on Arm Scaleable and Efficient
Ceph Day Seoul - Ceph on Arm Scaleable and Efficient
High Performance Interconnects: Landscape, Assessments & Rankings
High Performance Interconnects: Landscape, Assessments & Rankings
Red Hat Storage Day New York - What's New in Red Hat Ceph Storage
Red Hat Storage Day New York - What's New in Red Hat Ceph Storage
Optimized HPC/AI cloud with OpenStack acceleration service and composable har...
Optimized HPC/AI cloud with OpenStack acceleration service and composable har...
RedisConf17- Zettaset + Redis - Protecting Redis Enterprise while Maintaining...
RedisConf17- Zettaset + Redis - Protecting Redis Enterprise while Maintaining...
Ceph Day Tokyo - Ceph on ARM: Scaleable and Efficient
Ceph Day Tokyo - Ceph on ARM: Scaleable and Efficient
Intelligent Interconnect Architecture to Enable Next Generation HPC - Linaro ...
Intelligent Interconnect Architecture to Enable Next Generation HPC - Linaro ...
Ceph Day Taipei - How ARM Microserver Cluster Performs in Ceph
Ceph Day Taipei - How ARM Microserver Cluster Performs in Ceph
Ceph Day Tokyo - Delivering cost effective, high performance Ceph cluster
Ceph Day Tokyo - Delivering cost effective, high performance Ceph cluster
Similar to Seagate - ceph day taiwan 2017 opening session
bigdata.pptx
This document provides an overview of a seminar on big data. It begins with an introduction on how big data has become important in the IT world. It then defines big data, noting its characteristics of volume, velocity, and variety. It discusses storing, selecting, and processing big data using tools like Hadoop. It covers sources of big data and applications such as smarter healthcare, manufacturing, and traffic control. Both the risks and benefits of big data are mentioned, such as being overwhelmed by data but also making better decisions. Finally, it discusses the future of big data and its importance for the Indian market.
Big data
This document provides an overview of big data presented by five individuals. It defines big data, discusses its three key characteristics of volume, velocity and variety. It explains how big data is stored, selected and processed using techniques like Hadoop and MapReduce. Examples of big data sources and tools are provided. Applications of big data across various industries are highlighted. Both the risks and benefits of big data are summarized. The future growth of big data and its impact on IT is also outlined.
Cloud Revolution Conitnues
The document summarizes Bill Barney's presentation on the future of cloud computing. Some key points:
1. Cloud computing has enabled unprecedented data growth and new forms of data generation. By 2020, there will be over 50 billion internet connected devices generating huge amounts of data.
2. Emerging markets will be the main drivers of future growth as 75% of the world's population lives in emerging markets that are still underserved. Whoever can make emerging markets their home market will have a huge advantage.
3. There is no current leader providing cloud infrastructure globally. The opportunities exist to build fiber networks, data centers, and orchestration capabilities around the world. Whoever masters these areas will be well positioned
Big data ppt
This document provides an overview of big data in a seminar presentation. It defines big data, discusses its key characteristics of volume, velocity and variety. It describes how big data is stored, selected and processed. Examples of big data sources and tools used are provided. The applications and risks of big data are summarized. Benefits to organizations from big data analytics are outlined, as well as its impact on IT and future growth prospects.
Special issues on big data
Research Report on Big Data.
Abstract data,Methods along with example.
Wikepedia is the best example of BIG DATA.
Big data
This document discusses big data, defining it as large volumes of diverse data that are growing rapidly and requiring new techniques to capture, curate, manage, and analyze. It covers the key characteristics of big data including volume, velocity, and variety. The document also outlines common sources of big data, tools used to manage and analyze it, applications of big data analytics, risks and benefits, and the future growth of big data.
IBM Private Cloud Platform - Setting Foundation for Hybrid (JUKE, 2015)
This is the slide for IBM Partner Event, November 2015.
Digital Transformation, Innovation, and Industry Transformer through Hybrid Cloud. IBM Scenarios of Hybrid Cloud and Roadmap .An example of how Enterprise can get into Hybrid Cloud through simple Dev/Test Private Cloud as the start.
Big data 2017 final
COMEX2017 Smart Talks by Amjid Ali , Muscat, Oman. Covering Introduction to big data, Big Data Definitions, Big Data Revolution, Big Data Timeline, Hadoop and Map Reduce covers importance of storage and DNA, Oceanstore 9000, Microsoft R, Spark,
IoT Architectural Overview - 3 use case studies from InfluxData
This SlideShare reviews how an IoT Data platform fits in with any IoT Architecture to manage the data requirements of every IoT implementation. It is based on the learnings from existing IoT practitioners that have adopted an IoT Data platform using InfluxData. These clients have a range of solutions–from home automation (thermostat monitoring & management), to infrastructure management (solar panel monitoring and control) to manufacturing (equipment monitoring & control) as well as environmental management (green wall monitoring & control).
These learnings will help IoT adopters avoid the common pitfalls current clients faced on their journey to developing their IoT solution.
Big data ppt
This document provides an overview of big data. It defines big data as large volumes of diverse data that are growing rapidly and require new techniques to capture, store, distribute, manage, and analyze. The key characteristics of big data are volume, velocity, and variety. Common sources of big data include sensors, mobile devices, social media, and business transactions. Tools like Hadoop and MapReduce are used to store and process big data across distributed systems. Applications of big data include smarter healthcare, traffic control, and personalized marketing. The future of big data is promising with the market expected to grow substantially in the coming years.
Big Data ppt
This document provides an overview of big data, including its definition, characteristics, sources, tools, applications, risks and benefits. It defines big data as large volumes of diverse data that can be analyzed to reveal patterns and trends. The three key characteristics are volume, velocity and variety. Examples of big data sources include social media, sensors and user data. Tools used for big data include Hadoop, MongoDB and analytics programs. Big data has many applications and benefits but also risks regarding privacy and regulation. The future of big data is strong with the market expected to grow significantly in coming years.
Big Data
Big data comes from various sources in different formats and structures. It includes social media data, search engine data, medical records, and IoT device data. YouTube alone generates billions of views from over 1 billion users uploading 1.2 million videos daily. Big data is characterized by its variety, velocity, and volume. Hadoop is an open source framework used to process and store big data. It has two core components - HDFS for storage and MapReduce for processing. Hadoop distributes data and tasks across nodes, provides fault tolerance, and scales dynamically with hardware.
Big data Presentation
Content:
Introduction
What is Big Data?
Big Data facts
Three Characteristics of Big Data
Storing Big Data
THE STRUCTURE OF BIG DATA
WHY BIG DATA
HOW IS BIG DATA DIFFERENT?
BIG DATA SOURCES
BIG DATA ANALYTICS
TYPES OF TOOLS USED IN BIG-DATA
Application Of Big Data analytics
HOW BIG DATA IMPACTS ON IT
RISKS OF BIG DATA
BENEFITS OF BIG DATA
Future of big data
Big Data PPT by Rohit Dubey
This document provides an introduction to big data. It defines big data as large and complex data sets that are difficult to process using traditional data management tools. It discusses the three V's of big data - volume, variety and velocity. Volume refers to the large scale of data. Variety means different data types. Velocity means the speed at which data is generated and processed. The document outlines topics that will be covered, including Hadoop, MapReduce, data mining techniques and graph databases. It provides examples of big data sources and challenges in capturing, analyzing and visualizing large and diverse data sets.
Developing robust & enterprise io t applications
Review of development issues involve in IoT based applications. All the factors that need to be considered are covered.
big-data-8722-m8RQ3h1.pptx
This document outlines a seminar presentation on big data. It begins with an introduction that defines big data and notes how it emerged in the early 21st century mainly through online firms. It then covers the three key characteristics of big data - volume, velocity and variety. Other sections discuss storing, selecting and processing big data, as well as tools used and applications. Risks, benefits and the future impact and growth of big data are also summarized. The presentation provides an overview of the key concepts regarding big data.
Bigdatappt 140225061440-phpapp01
This document provides an overview of big data including:
- It defines big data and describes its three key characteristics: volume, velocity, and variety.
- It explains how big data is stored, selected, and processed using techniques like Hadoop and NoSQL databases.
- It discusses some common sources of big data, tools used to analyze it, and applications of big data analytics across different industries.
presentation Comstor IoT_RTL
The document discusses how digital transformation and the Internet of Everything are disrupting businesses. It notes that by 2020, 75% of businesses will be fully digital or in preparation to become digital. However, only 30% of digitization efforts will be successful. The Internet of Everything connects people, processes, data and things to deliver the right information to the right person or machine at the right time. It provides examples of how the Internet of Everything can transform smart cities and smart buildings through connecting various systems and devices to improve operations and resource management.
Big Communications Event, Keynote Communications Track
Focus on Key Technology Trends at BCE, Austin, TX. How Open Source, Big Data, Containers, IOT and Microservices are a catalyst of change, creating a virtuous cycle.
Bigdata
This presentation provides an overview of big data. It introduces the group members presenting and defines big data as large amounts of data that cannot be analyzed using traditional methods due to the volumes of data and the speed at which it is generated and needs to be processed. It describes the three main characteristics of big data as volume, velocity and variety. It also discusses storing, selecting, processing and analyzing big data and provides examples of big data sources and tools used. Potential applications and risks/benefits of big data are also summarized.
Similar to Seagate - ceph day taiwan 2017 opening session (20)
IBM Private Cloud Platform - Setting Foundation for Hybrid (JUKE, 2015)
IBM Private Cloud Platform - Setting Foundation for Hybrid (JUKE, 2015)
IoT Architectural Overview - 3 use case studies from InfluxData
IoT Architectural Overview - 3 use case studies from InfluxData
Developing robust & enterprise io t applications
Developing robust & enterprise io t applications
Big Communications Event, Keynote Communications Track
Big Communications Event, Keynote Communications Track
More from inwin stack
Migrating to Cloud Native Solutions
The document discusses migrating to cloud native solutions. It defines cloud native as an approach that exploits the advantages of cloud computing using containers, microservices, and other modern technologies. This allows applications to be scalable, resilient, and manageable. The document outlines the benefits of cloud native and provides a "trail map" to transitioning applications. It also discusses common challenges like technical debt and failing to meet CI/CD expectations, and provides recommendations to address them such as automating processes and simplifying architectures.
Cloud Native 下的應用網路設計
This document discusses how Citrix Application Delivery Management (ADM) can be used to manage Citrix ADC instances at scale in cloud-native environments. Key points include:
- Citrix ADM allows controlling and gaining insights from one to thousands of Citrix ADC instances (VPX, MPX, CPX), across container platforms like Mesos/Marathon and Kubernetes.
- Metadata from Citrix ADCs provides valuable information to Citrix ADM for an "App Health Score", including user experience metrics, security threats, and device health.
- Citrix ADM provides capabilities for app-centric lifecycles, configuration at scale, visibility, and security across Citrix ADC instances.
當電子發票遇見 Google Cloud Function
This document summarizes a presentation about electronic invoices and Google Cloud Functions. It discusses electronic invoices in Taiwan, how Google Cloud Functions work and can be used to process events like changes to files in Cloud Storage buckets or messages published to Pub/Sub topics. It also provides examples of using Cloud Functions to process electronic invoice notifications received via Gmail, extract data from the CSV attachments, and insert the data into a Google Sheet. The document concludes with some tips, like needing to decode the CSV attachments which use ANSI encoding, and how to set permissions for Cloud Pub/Sub.
運用高效、敏捷全新平台極速落實雲原生開發
1. The document discusses using a new high-efficiency and agile platform to quickly implement cloud-native development.
2. It introduces Pivotal Container Service (PKS) which provides a production-ready container platform for deploying Kubernetes clusters on VMware vSphere.
3. PKS leverages technologies like VMware NSX-T and BOSH to provide capabilities like security, high availability, and auto-scaling for container workloads.
The last mile of digital transformation AI大眾化：數位轉型的最後一哩
1) The document discusses democratizing AI and digital transformation through building an AI cloud platform in Taiwan with ASUS, Gigabyte, and Taiwan Mobile to provide computing resources.
2) It describes ASUS WebStorage, a personal cloud storage service with features like cross-device syncing, backup, photo uploading, sharing, and file searching. Storage is free up to 5GB with more for ASUS device owners.
3) The last mile of digital transformation is the democratization of AI through addressing challenges of computing power, tools, data, and commercialization. ASUS is working to provide cloud infrastructure, data platforms, AI platforms, and develop vertical ecosystems and applications.
整合Cloud Foundry 和 Kubernetes 技術打造企業級雲應用平台解決方案
The document discusses the history and development of Cloud Foundry and Kubernetes container technologies. It provides steps to deploy SUSE Cloud Foundry on Kubernetes, including adding Helm charts, deploying UAA and Cloud Foundry, and checking pod statuses. The benefits highlighted are providing the familiar Cloud Foundry developer experience on Kubernetes, leveraging innovations to improve agility, and allowing customers to simplify their digital transformations.
An Open, Open source way to enable your Cloud Native Journey
The document discusses SUSE's open source strategy and product portfolio. It outlines that SUSE is committed to open source, being a leader in the community, and delivering open and flexible technology to customers. It then provides an overview of SUSE's products for application delivery, infrastructure management, software-defined infrastructure, and container management.
維運Kubernetes的兩三事
This document provides an overview and outline of topics related to operating and maintaining Kubernetes in production environments. It discusses considerations for self-hosting Kubernetes versus using managed Kubernetes services, techniques for managing stateful applications, logging and monitoring, continuous delivery practices, troubleshooting, and trends in Kubernetes technologies. The document also provides references and advice for using kubectl effectively when managing multiple Kubernetes clusters and namespaces.
Serverless framework on kubernetes
This document summarizes Serverless Framework for Kubernetes called Fission. Fission is an open source Kubernetes-native serverless framework that allows running serverless applications on Kubernetes clusters both on-premises and on cloud providers. It provides portability, a flexible cost model through optimization features like autoscaling, and integrates with DevOps pipelines. Fission treats functions as core objects along with triggers, environments and uses Kubernetes custom resource definitions to store configuration. It supports various programming languages and event sources through a pool-based or new-deployment execution model to provide tunable cost and performance tradeoffs. The document demonstrates creating and testing functions through Fission's development workflow and deployment features like canary deployments.
Web後端技術的演變
This document discusses the evolution of web backend technologies. It covers the history and concepts of infrastructure as code, immutable infrastructure, blue-green deployments, and canary deployments. It also discusses tools for physical delivery, virtual machines, configuration management, continuous integration/delivery, Docker, and Kubernetes. Kubernetes makes it easy to implement infrastructure as code practices and deployment strategies like blue-green and canary deployments through features like deployments and services.
以 Kubernetes 部屬 Spark 大數據計算環境
This document summarizes using Kubernetes to deploy a Spark big data computing environment. It discusses why Kubernetes is preferable to other solutions like Cloudera for managing Spark. The architecture of running Spark on Kubernetes is shown, with the Spark master and worker controllers. Performance is compared between Spark on Kubernetes and standalone Spark using the SparkPI and WordCount examples. Support for Spark 2.3.0 on Kubernetes is now official.
Setup Hybrid Clusters Using Kubernetes Federation
This document summarizes how to setup hybrid clusters using Kubernetes Federation. It discusses the benefits of federation such as keeping applications synced across multiple clusters and configuring network resources to route traffic. It then describes the federation architecture including the federation control plane and federated resources/clusters. Finally, it provides steps to setup a demo federation including initializing the control plane and joining clusters from different regions.
基於 K8S 開發的 FaaS 專案 - riff
This document introduces riff, an open source project that provides a serverless platform for executing functions in response to events. It discusses using riff to run functions on Kubernetes in a polyglot way and scale them based on concurrent event load. Example use cases for event-driven functions include stream processing, web events, integration and FaaS. The document also covers integrating riff with Spring Cloud Function and using riff to transform a monolithic application into microservices.
使用 Prometheus 監控 Kubernetes Cluster
The document discusses monitoring Kubernetes clusters using Prometheus and Grafana. It describes how Prometheus scrapes metrics using exporters like Node Exporter and stores them in a time series database. Grafana is used to build dashboards and visualize the metrics collected by Prometheus. It provides configuration details for deploying Prometheus, Node Exporter, and Grafana as Kubernetes deployments and accessing the services.
利用K8S實現高可靠應用
This document discusses using Kubernetes to implement highly reliable applications. It begins with an agenda that includes an overview of microservices, an introduction to Kubernetes, and using NodeRed and Kubernetes to build a chatbot. It then provides background on microservices architecture, explaining how applications have evolved from huge monolithic applications to independent microservices that can be deployed and updated more quickly. It introduces Kubernetes concepts like pods, deployments, statefulsets, daemonsets and jobs. It also discusses using Kubernetes to run NodeRed chatbot containers as a deployment, including load balancing, self-healing and scaling benefits. Challenges with logging and maintaining chat conversations across containers are noted.
Integrate Kubernetes into CORD(Central Office Re-architected as a Datacenter)
- CORD aims to virtualize telecom central offices using open source software and commodity hardware. Kubernetes could help integrate NFV apps but challenges remain.
- Issues include converting existing VM-based NFVs to containers, supporting both OpenStack and Kubernetes, and ensuring the SDN controller ONOS can communicate with Kubernetes network components.
- The presenter's team addressed these by designing a multi-interface CNI plugin and centralized IPAM using Etcd to integrate ONOS and provide pod networking. Further work is needed to fully integrate ONOS control and test the solution.
Distributed tensorflow on kubernetes
This document discusses running distributed TensorFlow on Kubernetes. It provides an introduction to Kubernetes and how it can schedule GPUs. It then discusses distributed TensorFlow, how to set it up to run across multiple workers and parameter servers. Finally, it discusses how to package the TensorFlow code into a Docker container and deploy it on Kubernetes, taking advantage of Kubernetes' scaling, load balancing and fault tolerance.
More from inwin stack (20)
The last mile of digital transformation AI大眾化：數位轉型的最後一哩
The last mile of digital transformation AI大眾化：數位轉型的最後一哩
An Open, Open source way to enable your Cloud Native Journey
An Open, Open source way to enable your Cloud Native Journey
Extend the Kubernetes API with CRD and Custom API Server
Extend the Kubernetes API with CRD and Custom API Server
Integrate Kubernetes into CORD(Central Office Re-architected as a Datacenter)
Integrate Kubernetes into CORD(Central Office Re-architected as a Datacenter)
Build your own kubernetes apiserver and resource type
Build your own kubernetes apiserver and resource type
Recently uploaded
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
How to Get CNIC Information System with Paksim Ga.pptx
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentationsAlt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
DevOps and Testing slides at DASA Connect
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
GridMate - End to end testing is a critical piece to ensure quality and avoid...
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
UiPath Test Automation using UiPath Test Suite series, part 6
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
“I’m still / I’m still / Chaining from the Block”
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Recently uploaded (20)
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
GridMate - End to end testing is a critical piece to ensure quality and avoid...
GridMate - End to end testing is a critical piece to ensure quality and avoid...
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Generative AI Deep Dive: Advancing from Proof of Concept to Production
Seagate - ceph day taiwan 2017 opening session
- 1. 7/20/2017 July, 2017 Ceph Day Taiwan 2017 Opening Session Jones Lin Sales Lead, Taiwan Channel
- 2. 1960 1980 2000 2020 1960 1980 2000 2020 1st Platform: Mainframe Expanded by Companies • Millions of Users • Thousands of Apps 3rd Platform: Mobile-cloud Expanded by People • Billions of Users • Millions of Apps 2nd Platform: Client-server Expanded by Systems • Hundreds of Millions of Users • Tens of Thousands of Apps 4th Platform: Edge Intelligence Expanded by Machines • Trillions of Things Compute Platform CENTRALIZED DISTRIBUTED Seagate Confidential | 2
- 3. Enabling Explosive Data Growth 3RD PLATFORM ECO-SYSTEM IDC FIVE KEY TRENDS 1. The evolution of data from business background to life-critical 2. Embedded systems and IoT 3. Mobile and real-time data 4. Cognitive / AI systems that change the landscape 5. Security as a critical foundation Cloud / enterprise Big Data Machine Learning CLOUD Big Data Machine Learning END POINTS Mobility IoT In 2016 16ZB In 2025 163ZB10X Growth in Datasphere Seagate Confidential | 3 Datasphere: Sum of all data created, captured, and replicated
- 4. Server SAN in the Edge for Growing
- 7. OpenStack A-Team Taiwan • The Power of Partnership 7 More ………. & Join us 晨宇創新 數位無限
- 8. OpenStack & Ceph Taiwan Ecosystem Alliances THE POWER OF PARTNERSHIP OpenStack Day Taiwan Event Committee 晨宇創新 數位無限 More ………. & Join us