1) FlexSC implements efficient and flexible system calls for the multicore era by introducing exception-less system calls and syscall threads.
2) Results show MySQL throughput increased up to 40% and latency reduced 30%, and Apache throughput increased up to 115% and latency reduced 50%.
3) Synchronous system calls are expensive due to direct mode switching costs and indirect processor structure pollution costs, which exception-less system calls help reduce.
The document discusses exception-less system calls which can improve the performance of event-driven servers. It describes how event-driven servers use non-blocking I/O and asynchronous callbacks instead of threads to handle requests. The authors modified memcached and nginx to use their library called libflexsc that provides asynchronous system calls and callbacks. This approach speeds up memcached by 25-35% and nginx by 70-120% by reducing mode switches between user and kernel mode.
Falcon Storage Engine Designed For Speed Presentationelliando dias
This document discusses the Falcon database system and its approach to achieving high performance. It provides an overview of Falcon's architecture including its use of record caching, multi-version concurrency control, and separate storage of indexes and data. It then covers Falcon's history and goals, techniques for handling different workload characteristics, challenges in competing with InnoDB on benchmarks, approaches to multi-threading, and its use of cycle locking to enable concurrent operations while avoiding locking conflicts.
The document summarizes steps taken to update the database (db) nodes on an Exadata system from software version 11.2.3.2.1 to 11.2.3.3.0 using the dbnodeupdate.sh utility. It provides details of prerequisites, running the utility in update mode with the -u flag and specified media source, and validations performed during the process including dependency checks. The summary includes checking for known issues, collecting diagnostics, and noting next steps after upgrade completion.
Upgrading MySQL databases do not come without risk. There is no guarantee that no problems will happen if you move to a new major MySQL version.
Should we just upgrade and rollback immediately if problems occur? But what if these problems only happen a few days after migrating to this new version?
You might have a database environment that is risk-adverse, where you really have to be sure that this new MySQL version will handle the workload properly.
Examples:
- Both MySQL 5.6 and 5.7 have a lot of changes in the MySQL Optimizer. It is expected that this improves performance of my queries, but is it really the case? What if there is a performance regression? How will this affect my database performance?
- Also, there are a lot of incompatible changes which are documented in the release notes, how do I know if I'm affected by this in my workload? It's a lot to read..
- Can I go immediately from MySQL 5.5 to 5.7 and skip MySQL 5.6 even though the MySQL documentation states that this is not supported?
- Many companies have staging environments, but is there a QA team and do they really test all functionality, under a similar workload?
This presentation will show you a process, using open source tools, of these types of migrations with a focus on assessing risk and fixing any problems you might run into prior to the migration.
This process can then be used for various changes:
- MySQL upgrades for major version upgrades
- Switching storage engines
- Changing hardware architecture
Additionally, we will describe ways to do the actual migration and rollback with the least amount of downtime.
기존에 저희 회사에서 사용하던 모니터링은 Zabbix 였습니다.
컨테이너 모니터링 부분으로 옮겨가면서 변화가 필요하였고, 이에 대해서 프로메테우스를 활용한 모니터링 방법을 자연스럽게 고민하게 되었습니다.
이에 이영주님께서 테크세션을 진행하였고, 이에 발표자료를 올립니다.
5개의 부분으로 구성되어 있으며, 세팅 방법에 대한 내용까지 포함합니다.
01. Prometheus?
02. Usage
03. Alertmanager
04. Cluster
05. Performance
The document summarizes the steps to upgrade an Oracle VM (OVM) 2.2 server and manager to OVM 3.0.1. It involves installing a new OVM 3.0.1 manager on a Linux system using VirtualBox. Then installing new OVM 3.0.1 servers and importing existing virtual servers, templates, and resources from the 2.2 environment. It provides details on the manager and server installation and configuration, including network settings and access URLs for the new OVM 3.0.1 environment.
vSphere defines VMware's virtualization product suite, including the ESXi hypervisor, vCenter management server, and vSphere Client interface. ESXi uses a proprietary kernel called vmkernel along with some open source components. Key features of vSphere include VMware HA, vMotion, and DRS for managing and migrating VMs across hosts. Troubleshooting performance issues involves tools like esxtop to monitor CPU, memory, and swap usage on ESXi hosts and VMs.
Stackless Python is used extensively in EVE Online to provide cooperative multitasking. It allows tasks to be split across multiple tasklets that run concurrently without preemption. Channels are used to synchronize tasklets in a way similar to coroutines. This approach allows EVE's massive multiplayer server to handle over 120,000 concurrent users on a single database shard using Stackless Python's lightweight tasklets and channels for communication and synchronization.
The document discusses exception-less system calls which can improve the performance of event-driven servers. It describes how event-driven servers use non-blocking I/O and asynchronous callbacks instead of threads to handle requests. The authors modified memcached and nginx to use their library called libflexsc that provides asynchronous system calls and callbacks. This approach speeds up memcached by 25-35% and nginx by 70-120% by reducing mode switches between user and kernel mode.
Falcon Storage Engine Designed For Speed Presentationelliando dias
This document discusses the Falcon database system and its approach to achieving high performance. It provides an overview of Falcon's architecture including its use of record caching, multi-version concurrency control, and separate storage of indexes and data. It then covers Falcon's history and goals, techniques for handling different workload characteristics, challenges in competing with InnoDB on benchmarks, approaches to multi-threading, and its use of cycle locking to enable concurrent operations while avoiding locking conflicts.
The document summarizes steps taken to update the database (db) nodes on an Exadata system from software version 11.2.3.2.1 to 11.2.3.3.0 using the dbnodeupdate.sh utility. It provides details of prerequisites, running the utility in update mode with the -u flag and specified media source, and validations performed during the process including dependency checks. The summary includes checking for known issues, collecting diagnostics, and noting next steps after upgrade completion.
Upgrading MySQL databases do not come without risk. There is no guarantee that no problems will happen if you move to a new major MySQL version.
Should we just upgrade and rollback immediately if problems occur? But what if these problems only happen a few days after migrating to this new version?
You might have a database environment that is risk-adverse, where you really have to be sure that this new MySQL version will handle the workload properly.
Examples:
- Both MySQL 5.6 and 5.7 have a lot of changes in the MySQL Optimizer. It is expected that this improves performance of my queries, but is it really the case? What if there is a performance regression? How will this affect my database performance?
- Also, there are a lot of incompatible changes which are documented in the release notes, how do I know if I'm affected by this in my workload? It's a lot to read..
- Can I go immediately from MySQL 5.5 to 5.7 and skip MySQL 5.6 even though the MySQL documentation states that this is not supported?
- Many companies have staging environments, but is there a QA team and do they really test all functionality, under a similar workload?
This presentation will show you a process, using open source tools, of these types of migrations with a focus on assessing risk and fixing any problems you might run into prior to the migration.
This process can then be used for various changes:
- MySQL upgrades for major version upgrades
- Switching storage engines
- Changing hardware architecture
Additionally, we will describe ways to do the actual migration and rollback with the least amount of downtime.
기존에 저희 회사에서 사용하던 모니터링은 Zabbix 였습니다.
컨테이너 모니터링 부분으로 옮겨가면서 변화가 필요하였고, 이에 대해서 프로메테우스를 활용한 모니터링 방법을 자연스럽게 고민하게 되었습니다.
이에 이영주님께서 테크세션을 진행하였고, 이에 발표자료를 올립니다.
5개의 부분으로 구성되어 있으며, 세팅 방법에 대한 내용까지 포함합니다.
01. Prometheus?
02. Usage
03. Alertmanager
04. Cluster
05. Performance
The document summarizes the steps to upgrade an Oracle VM (OVM) 2.2 server and manager to OVM 3.0.1. It involves installing a new OVM 3.0.1 manager on a Linux system using VirtualBox. Then installing new OVM 3.0.1 servers and importing existing virtual servers, templates, and resources from the 2.2 environment. It provides details on the manager and server installation and configuration, including network settings and access URLs for the new OVM 3.0.1 environment.
vSphere defines VMware's virtualization product suite, including the ESXi hypervisor, vCenter management server, and vSphere Client interface. ESXi uses a proprietary kernel called vmkernel along with some open source components. Key features of vSphere include VMware HA, vMotion, and DRS for managing and migrating VMs across hosts. Troubleshooting performance issues involves tools like esxtop to monitor CPU, memory, and swap usage on ESXi hosts and VMs.
Stackless Python is used extensively in EVE Online to provide cooperative multitasking. It allows tasks to be split across multiple tasklets that run concurrently without preemption. Channels are used to synchronize tasklets in a way similar to coroutines. This approach allows EVE's massive multiplayer server to handle over 120,000 concurrent users on a single database shard using Stackless Python's lightweight tasklets and channels for communication and synchronization.
Instruction on creating a cluster on jboss eap environmentMadhusudan Pisipati
The document describes setting up an active-passive JBoss EAP cluster configuration with two active nodes in one cluster and two passive nodes in another cluster. Key steps include:
1. Creating users on each node for management and application access.
2. Configuring the first node as the domain controller and defining server groups and profiles for the active and passive clusters.
3. Configuring the second active node and passive nodes to connect to the domain controller and join the appropriate cluster.
4. Installing Apache HTTP Server on a separate machine and configuring mod_cluster to load balance between the active clusters.
This document provides a summary of the state of JBoss EAP/WildFly application servers. It discusses the history and key releases of JBoss AS, including the path to Java EE 6 compliance and the major changes and improvements in JBoss AS 7. It then outlines the goals and key features for the next major versions, WildFly 8 and JBoss EAP 6, including support for Java EE 7, single instance patching, role-based access control, and a new web container.
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...Виталий Стародубцев
##Что такое Storage Replica
##Архитектура и сценарии
##Синхронная и асинхронная репликация
##Междисковая, межсерверная, внутрикластерная и межкластерная репликация
##Дизайн и проектирование Storage Replica
##Нововведения в Windows Server 2016 TP5
##Графический интерфейс управления, и другие возможности - демонстрация и планы развития
##Интеграция Storage Replica с Storage Spaces Direct
This document provides an overview of asynchronous I/O programming. It begins with an outline and introduction to asynchronous I/O. It then discusses specific asynchronous I/O APIs like Berkeley sockets, select, poll, epoll, KQueue, and Posix AIO. It covers advantages and drawbacks of asynchronous programming. Examples are provided of asynchronous programming with these different APIs. The document also discusses libraries like libevent and frameworks like Twisted that provide asynchronous functionality.
Visão geral sobre Citrix XenServer 6 - Ferramentas e LicenciamentoLorscheider Santiago
This document provides an overview of Citrix XenServer, including:
- Why use XenServer over VMware, with XenServer having leadership in the market share and lower costs.
- An overview of XenServer's key features like virtual memory licensing, clusters and pools, live migration, snapshots, and high availability.
- A comparison of XenServer and VMware features around licensing, importing VMs, backup solutions, and more.
- Details on newer versions of XenServer that include integrated disaster recovery, provisioning services, and monitoring solutions.
Benchmark emc vnx7500, emc fast suite, emc snap sure and oracle rac on v-mwaresolarisyougood
This document describes a scalable virtualized Oracle RAC 11g database deployment using EMC VNX7500 storage with EMC FAST Suite. Testing showed that using FAST Cache improved transactions per minute by 133% and response time by over 90%, while FAST Suite improved TPM by 136% and response time by over 95%. The solution also enabled rapid provisioning of Oracle databases through SnapSure checkpoints and Oracle dNFS clonedb. It provided high availability with automatic failover during network or storage hardware failures.
The document discusses new features in Windows Server 2016 related to cluster rolling upgrades. It describes the process for performing an in-place upgrade of a Hyper-V cluster from Windows Server 2012 R2 to Windows Server 2016 without downtime. The process involves pausing each node, upgrading its OS, then rejoining it to the cluster. Once all nodes are upgraded, the cluster functional level can be upgraded to enable new Windows Server 2016 features. The document also covers new storage replication capabilities in Windows Server 2016 technical preview called Storage Replica.
It's presentation for technet 2015 in korea.
I changed the format to pptx,
목차는 아래와 같습니다.
Openstack 인프라 구축 (4 node 구성) [ 30분]
Openstack 위에 VM 생성 [ 20분 ]
docker 구축 기초 [ 30분]
오픈스택에 docker를 연결 [ 30분]
Docker로 WEB서비스 구축 [ 15분]
Openstack 위에 Docker로 WEB서비스 구축 [ 15분]
Docker로 jenkins 구현 [30분]
Windows Internals for Linux Kernel DevelopersKernel TLV
Agenda:
The Windows kernel has an honorable history of more than a quarter of a century. Since its inception in 1989, Windows NT supported a variety of modern OS features -- symmetric multiprocessing, interrupt prioritization, virtual memory, deferred interrupt processing, and many others. In this talk, targeted for Linux kernel developers, we will highlight the key features of the Windows NT kernel that are interesting or different from Linux's perspective. We will begin with a brief overview of processes, threads, and virtual memory on Windows. Next, we will talk about interrupt handling, interrupt priorities (IRQLs), bottom-half processing (DPC, APC, kernel worker threads, kernel thread pool), and I/O request flow. Among other things, we will look at device driver structure on Windows, application to driver communication (handles, IOCTLs), and the logical \DosDevices filesystem. Finally, we will discuss some features introduced in newer Windows versions, such as user-mode drivers (UMDF).
Speaker:
Sasha is the CTO of Sela Group, a training and consulting company based in Israel that employs over 400 developers world-wide. Most of Sasha's work revolves around performance optimization, production debugging, and low-level system diagnostics, but he also dabbles in mobile application development on iOS and Android. Sasha is the author of two books and three Pluralsight courses, and a contributor to multiple open-source projects. He blogs at http://blog.sashag.net.
This document provides instructions for implementing an Oracle 11g R2 Real Application Cluster on a Red Hat Enterprise Linux 5.0 system using a two-node configuration. It describes pre-installation steps including hardware and network configuration, installing prerequisite packages and libraries, and configuring the Oracle ASM library driver. Detailed steps are provided for installing Oracle Grid Infrastructure and database software, and configuring the single client access name and storage area network.
The document summarizes troubleshooting techniques for the XenServer virtualization platform. It describes the XenServer architecture including the Xen domain 0, guest OS internals, XAPI management interface, and storage manager. It then covers various troubleshooting tools and techniques such as useful Linux commands, XenServer log files, debugging XenCenter logs, and examining the object model and CLI.
Uponor Exadata e-Business Suite Migration Case StudySimo Vilmunen
Uponor, a plumbing solutions company, migrated their Oracle E-Business Suite and Oracle Business Intelligence environments from traditional hardware to Oracle Exadata in order to improve performance, scalability, availability and manageability. The migration was completed within 3 months and resulted in significant performance gains across key business processes. Lessons learned included benefits of using Exadata-specific tools and configurations and importance of testing database-specific functionality during migration.
Consistency between Engine and Binlog under Reduced DurabilityYoshinori Matsunobu
- When MySQL instances fail and recover, the binary logs and storage engines can become inconsistent due to different levels of durability settings. This can cause issues when trying to rejoin instances to replication.
- The document discusses challenges in ensuring consistency between binary logs and storage engines like InnoDB under reduced durability settings. It also addresses issues that can occur when restarting masters or replicas due to potential inconsistencies.
- Solutions discussed include using the max GTID from the storage engine to determine where to start replication, truncating binary logs on restart if they are ahead of the engines, and using idempotent recovery techniques to handle potential duplicate or missing rows. Ensuring consistency across multiple storage engines is also challenging.
This document discusses techniques for troubleshooting issues with Red Hat JBoss EAP 6. It covers generating thread dumps, heap dumps, and log files to analyze where threads are stuck or understand memory usage. The JBoss Diagnostic Reporter (JDR) subsystem can collect troubleshooting information. Byteman allows inserting extra Java code into applications to aid debugging. The log subsystem level and GC logging can be configured for additional troubleshooting data in log files.
The document discusses upcoming releases of the WildFly application server. It notes that WildFly 8 is Java EE7 certified and includes performance and usability improvements. WildFly 9 is still in development and will include splitting the codebase into modular feature packs, improved security features like Elytron, and support for capabilities and requirements. The goal is to make WildFly more customizable and modular and reduce testing time.
WildFly v9 - State of the Union Session at Voxxed, Istanbul, May/9th 2015.Dimitris Andreadis
This document summarizes the history and future of the WildFly application server project. It discusses the evolution from JBoss AS to WildFly, recaps key features of WildFly 8 including full Java EE 7 certification. It provides an update on WildFly 9 including new features like the WildFly-Core project and distribution, and previews plans for WildFly 10 such as replacing HornetQ with Artemis and improving security with the new Elytron module.
This document provides an overview and introduction to virtual storage concepts in VMware vSphere, including NFS, iSCSI, VMFS, and Virtual SAN datastores. It discusses storage protocols, multipathing, and best practices for configuring and managing different types of datastores. The document is divided into several sections covering storage concepts, iSCSI, NFS, VMFS, and Virtual SAN datastores.
This document provides an overview of troubleshooting storage performance issues in vSphere environments. It discusses using vCenter performance charts and ESXTop to analyze latency and I/O statistics at the storage path, disk, and LUN level. The document also covers topics like disk alignment, considerations for using SCSI versus SATA disks, identifying APD issues, multipathing, and how VMware uses SCSI reservations for metadata locking on shared VMFS datastores.
20 Real-World Use Cases to help pick a better MySQL Replication scheme (2012)Darpan Dinker
MySQL Replication: Pros and Cons
Achieve Higher Performance, Uptime, Reliability and Simplicity
for Real-World Use Cases.
Synchronous, semi-synchronous, asynchronous replication with parallel Slave appliers.
High availability.
Fastsocket is a software that improves the scalability and performance of socket-based applications on multicore systems. It addresses kernel inefficiencies like synchronization overhead that consume over 90% of CPU cycles. Fastsocket introduces techniques like receive flow delivery, local listen/established tables, and a fastsocket-aware VFS to partition resources and process connections locally on each CPU core. In production at SINA, Fastsocket improved HTTP load balancing throughput by 45% on a 16-core system. Future work aims to further optimize performance through techniques like improved interrupt handling and system call batching.
Instruction on creating a cluster on jboss eap environmentMadhusudan Pisipati
The document describes setting up an active-passive JBoss EAP cluster configuration with two active nodes in one cluster and two passive nodes in another cluster. Key steps include:
1. Creating users on each node for management and application access.
2. Configuring the first node as the domain controller and defining server groups and profiles for the active and passive clusters.
3. Configuring the second active node and passive nodes to connect to the domain controller and join the appropriate cluster.
4. Installing Apache HTTP Server on a separate machine and configuring mod_cluster to load balance between the active clusters.
This document provides a summary of the state of JBoss EAP/WildFly application servers. It discusses the history and key releases of JBoss AS, including the path to Java EE 6 compliance and the major changes and improvements in JBoss AS 7. It then outlines the goals and key features for the next major versions, WildFly 8 and JBoss EAP 6, including support for Java EE 7, single instance patching, role-based access control, and a new web container.
Технологии работы с дисковыми хранилищами и файловыми системами Windows Serve...Виталий Стародубцев
##Что такое Storage Replica
##Архитектура и сценарии
##Синхронная и асинхронная репликация
##Междисковая, межсерверная, внутрикластерная и межкластерная репликация
##Дизайн и проектирование Storage Replica
##Нововведения в Windows Server 2016 TP5
##Графический интерфейс управления, и другие возможности - демонстрация и планы развития
##Интеграция Storage Replica с Storage Spaces Direct
This document provides an overview of asynchronous I/O programming. It begins with an outline and introduction to asynchronous I/O. It then discusses specific asynchronous I/O APIs like Berkeley sockets, select, poll, epoll, KQueue, and Posix AIO. It covers advantages and drawbacks of asynchronous programming. Examples are provided of asynchronous programming with these different APIs. The document also discusses libraries like libevent and frameworks like Twisted that provide asynchronous functionality.
Visão geral sobre Citrix XenServer 6 - Ferramentas e LicenciamentoLorscheider Santiago
This document provides an overview of Citrix XenServer, including:
- Why use XenServer over VMware, with XenServer having leadership in the market share and lower costs.
- An overview of XenServer's key features like virtual memory licensing, clusters and pools, live migration, snapshots, and high availability.
- A comparison of XenServer and VMware features around licensing, importing VMs, backup solutions, and more.
- Details on newer versions of XenServer that include integrated disaster recovery, provisioning services, and monitoring solutions.
Benchmark emc vnx7500, emc fast suite, emc snap sure and oracle rac on v-mwaresolarisyougood
This document describes a scalable virtualized Oracle RAC 11g database deployment using EMC VNX7500 storage with EMC FAST Suite. Testing showed that using FAST Cache improved transactions per minute by 133% and response time by over 90%, while FAST Suite improved TPM by 136% and response time by over 95%. The solution also enabled rapid provisioning of Oracle databases through SnapSure checkpoints and Oracle dNFS clonedb. It provided high availability with automatic failover during network or storage hardware failures.
The document discusses new features in Windows Server 2016 related to cluster rolling upgrades. It describes the process for performing an in-place upgrade of a Hyper-V cluster from Windows Server 2012 R2 to Windows Server 2016 without downtime. The process involves pausing each node, upgrading its OS, then rejoining it to the cluster. Once all nodes are upgraded, the cluster functional level can be upgraded to enable new Windows Server 2016 features. The document also covers new storage replication capabilities in Windows Server 2016 technical preview called Storage Replica.
It's presentation for technet 2015 in korea.
I changed the format to pptx,
목차는 아래와 같습니다.
Openstack 인프라 구축 (4 node 구성) [ 30분]
Openstack 위에 VM 생성 [ 20분 ]
docker 구축 기초 [ 30분]
오픈스택에 docker를 연결 [ 30분]
Docker로 WEB서비스 구축 [ 15분]
Openstack 위에 Docker로 WEB서비스 구축 [ 15분]
Docker로 jenkins 구현 [30분]
Windows Internals for Linux Kernel DevelopersKernel TLV
Agenda:
The Windows kernel has an honorable history of more than a quarter of a century. Since its inception in 1989, Windows NT supported a variety of modern OS features -- symmetric multiprocessing, interrupt prioritization, virtual memory, deferred interrupt processing, and many others. In this talk, targeted for Linux kernel developers, we will highlight the key features of the Windows NT kernel that are interesting or different from Linux's perspective. We will begin with a brief overview of processes, threads, and virtual memory on Windows. Next, we will talk about interrupt handling, interrupt priorities (IRQLs), bottom-half processing (DPC, APC, kernel worker threads, kernel thread pool), and I/O request flow. Among other things, we will look at device driver structure on Windows, application to driver communication (handles, IOCTLs), and the logical \DosDevices filesystem. Finally, we will discuss some features introduced in newer Windows versions, such as user-mode drivers (UMDF).
Speaker:
Sasha is the CTO of Sela Group, a training and consulting company based in Israel that employs over 400 developers world-wide. Most of Sasha's work revolves around performance optimization, production debugging, and low-level system diagnostics, but he also dabbles in mobile application development on iOS and Android. Sasha is the author of two books and three Pluralsight courses, and a contributor to multiple open-source projects. He blogs at http://blog.sashag.net.
This document provides instructions for implementing an Oracle 11g R2 Real Application Cluster on a Red Hat Enterprise Linux 5.0 system using a two-node configuration. It describes pre-installation steps including hardware and network configuration, installing prerequisite packages and libraries, and configuring the Oracle ASM library driver. Detailed steps are provided for installing Oracle Grid Infrastructure and database software, and configuring the single client access name and storage area network.
The document summarizes troubleshooting techniques for the XenServer virtualization platform. It describes the XenServer architecture including the Xen domain 0, guest OS internals, XAPI management interface, and storage manager. It then covers various troubleshooting tools and techniques such as useful Linux commands, XenServer log files, debugging XenCenter logs, and examining the object model and CLI.
Uponor Exadata e-Business Suite Migration Case StudySimo Vilmunen
Uponor, a plumbing solutions company, migrated their Oracle E-Business Suite and Oracle Business Intelligence environments from traditional hardware to Oracle Exadata in order to improve performance, scalability, availability and manageability. The migration was completed within 3 months and resulted in significant performance gains across key business processes. Lessons learned included benefits of using Exadata-specific tools and configurations and importance of testing database-specific functionality during migration.
Consistency between Engine and Binlog under Reduced DurabilityYoshinori Matsunobu
- When MySQL instances fail and recover, the binary logs and storage engines can become inconsistent due to different levels of durability settings. This can cause issues when trying to rejoin instances to replication.
- The document discusses challenges in ensuring consistency between binary logs and storage engines like InnoDB under reduced durability settings. It also addresses issues that can occur when restarting masters or replicas due to potential inconsistencies.
- Solutions discussed include using the max GTID from the storage engine to determine where to start replication, truncating binary logs on restart if they are ahead of the engines, and using idempotent recovery techniques to handle potential duplicate or missing rows. Ensuring consistency across multiple storage engines is also challenging.
This document discusses techniques for troubleshooting issues with Red Hat JBoss EAP 6. It covers generating thread dumps, heap dumps, and log files to analyze where threads are stuck or understand memory usage. The JBoss Diagnostic Reporter (JDR) subsystem can collect troubleshooting information. Byteman allows inserting extra Java code into applications to aid debugging. The log subsystem level and GC logging can be configured for additional troubleshooting data in log files.
The document discusses upcoming releases of the WildFly application server. It notes that WildFly 8 is Java EE7 certified and includes performance and usability improvements. WildFly 9 is still in development and will include splitting the codebase into modular feature packs, improved security features like Elytron, and support for capabilities and requirements. The goal is to make WildFly more customizable and modular and reduce testing time.
WildFly v9 - State of the Union Session at Voxxed, Istanbul, May/9th 2015.Dimitris Andreadis
This document summarizes the history and future of the WildFly application server project. It discusses the evolution from JBoss AS to WildFly, recaps key features of WildFly 8 including full Java EE 7 certification. It provides an update on WildFly 9 including new features like the WildFly-Core project and distribution, and previews plans for WildFly 10 such as replacing HornetQ with Artemis and improving security with the new Elytron module.
This document provides an overview and introduction to virtual storage concepts in VMware vSphere, including NFS, iSCSI, VMFS, and Virtual SAN datastores. It discusses storage protocols, multipathing, and best practices for configuring and managing different types of datastores. The document is divided into several sections covering storage concepts, iSCSI, NFS, VMFS, and Virtual SAN datastores.
This document provides an overview of troubleshooting storage performance issues in vSphere environments. It discusses using vCenter performance charts and ESXTop to analyze latency and I/O statistics at the storage path, disk, and LUN level. The document also covers topics like disk alignment, considerations for using SCSI versus SATA disks, identifying APD issues, multipathing, and how VMware uses SCSI reservations for metadata locking on shared VMFS datastores.
20 Real-World Use Cases to help pick a better MySQL Replication scheme (2012)Darpan Dinker
MySQL Replication: Pros and Cons
Achieve Higher Performance, Uptime, Reliability and Simplicity
for Real-World Use Cases.
Synchronous, semi-synchronous, asynchronous replication with parallel Slave appliers.
High availability.
Fastsocket is a software that improves the scalability and performance of socket-based applications on multicore systems. It addresses kernel inefficiencies like synchronization overhead that consume over 90% of CPU cycles. Fastsocket introduces techniques like receive flow delivery, local listen/established tables, and a fastsocket-aware VFS to partition resources and process connections locally on each CPU core. In production at SINA, Fastsocket improved HTTP load balancing throughput by 45% on a 16-core system. Future work aims to further optimize performance through techniques like improved interrupt handling and system call batching.
Achieving Performance Isolation with Lightweight Co-KernelsJiannan Ouyang, PhD
This slides were presented at the 24th International Symposium on High-Performance Parallel and Distributed Computing (HPDC '15)
Performance isolation is emerging as a requirement for High Performance Computing (HPC) applications, particularly as HPC architectures turn to in situ data processing and application composition techniques to increase system throughput. These approaches require the co-location of disparate workloads on the same compute node, each with different resource and runtime requirements. In this paper we claim that these workloads cannot be effectively managed by a single Operating System/Runtime (OS/R). Therefore, we present Pisces, a system software architecture that enables the co-existence of multiple independent and fully isolated OS/Rs, or enclaves, that can be customized to address the disparate requirements of next generation HPC workloads. Each enclave consists of a specialized lightweight OS co-kernel and runtime, which is capable of independently managing partitions of dynamically assigned hardware resources. Contrary to other co-kernel approaches, in this work we consider performance isolation to be a primary requirement and present a novel co-kernel architecture to achieve this goal. We further present a set of design requirements necessary to ensure performance isolation, including: (1) elimination of cross OS dependencies, (2) internalized management of I/O, (3) limiting cross enclave communication to explicit shared memory channels, and (4) using virtualization techniques to provide missing OS features. The implementation of the Pisces co-kernel architecture is based on the Kitten Lightweight Kernel and Palacios Virtual Machine Monitor, two system software architectures designed specifically for HPC systems. Finally we will show that lightweight isolated co-kernels can provide better performance for HPC applications, and that isolated virtual machines are even capable of outperforming native environments in the presence of competing workloads.
VMware ESXi - Intel and Qlogic NIC throughput difference v0.6David Pasek
We are observing different network throughputs on Intel X710 NICs and QLogic FastLinQ QL41xxx NIC. ESXi hardware supports NIC hardware offloading and queueing on 10Gb, 25Gb, 40Gb and 100Gb NIC adapters. Multiple hardware queues per NIC interface (vmnic) and multiple software threads on ESXi VMkernel is depicted and documented in this paper which may or may not be the root cause of the observed problem. The key objective of this document is to clearly document and collect NIC information on two specific Network Adapters and do a comparison to find the difference or at least root cause hypothesis for further troubleshooting.
The sFlow Standard: Scalable, Unified Monitoring of Networks, Systems and App...netvis
This document discusses using sFlow for scalable, unified monitoring of networks, systems, and applications. sFlow exports standard performance counters from network devices, hosts, and applications. It uses a lightweight "push" protocol over UDP that is scalable and cloud-friendly. In addition to counters, sFlow also exports random packet samples which provide insight into issues like top URLs, clients, and servers without high overhead. Tagged, a social networking company, uses sFlow for comprehensive monitoring across their infrastructure through integration with tools like Ganglia.
Anton Moldovan "Building an efficient replication system for thousands of ter...Fwdays
For one of our projects, we needed to improve the current content delivery system for terminals. In this talk, I will share our experience in building an efficient data replication system for thousands of terminals. We will touch on architecture decisions and tradeoffs, technologies that we used, and a bit of load testing.
Spoiler: We didn't use Kafka.
Shak larry-jeder-perf-and-tuning-summit14-part2-finalTommy Lee
This document provides an overview of performance analysis and tuning techniques in Red Hat Enterprise Linux (RHEL). It discusses the tuned profile packages and how they optimize systems for different workloads. Specific topics covered include disk I/O tuning, memory tuning, network performance tuning, and power management techniques. A variety of Linux performance analysis tools are also introduced, including tuned, turbostat, netsniff-ng, and Performance Co-Pilot.
High Performance Storage Devices in the Linux KernelKernel TLV
Agenda:
In this talk we will present the Linux kernel storage layers and dive into blk-mq, a scalable, parallel block layer for high performance block devices, and how it is used to unleash the performance of NVMe, flash and beyond.
Speaker:
Evgeny Budilovsky, Kernel Developer at E8 Storage
https://www.linkedin.com/company/e8-storage
The document discusses several technologies for using MySQL as a NoSQL database system by bypassing SQL overhead: HandlerSocket allows direct access to indexes; the NDB API provides a NoSQL interface to MySQL Cluster; the BLOB Streaming Engine improves BLOB handling; the Handler Interface skips parsing and optimization; and the OQGRAPH storage engine enables graph queries and computations. Benchmarks show these techniques providing significant performance gains over standard SQL.
Load Balancing MySQL with HAProxy - SlidesSeveralnines
Agenda:
* What is HAProxy?
* SQL Load balancing for MySQL
* Failure detection using MySQL health checks
* High Availability with Keepalived and Virtual IP
* Use cases: MySQL Cluster, Galera Cluster and MySQL Replication
* Alternative methods: Database drivers with inbuilt cluster support, MySQL proxy, MaxScale, ProxySQL
This document provides an introduction to parallel synchronous replication using Percona XtraDB Cluster (PXC). It discusses the limitations of traditional MySQL replication and how PXC implements a data-centric approach with synchronous multi-master replication between nodes. Key features of PXC highlighted include parallel replication, data consistency, and automatic provisioning of new nodes. The document also covers integration with load balancers and limitations to be aware of for write-intensive or large transaction workloads.
A talk I gave at the Boston Web Performance Meetup in August 2014.
Performance is one of the most challenging issues in modern web app design, in large part because modeling, testing, and validating performance before deploying to production is so challenging. While many ops teams have nailed down the problem of re-creating pre-production environments that closely mimic production, those environments frequently rely on known-good components beyond the application code itself: AWS ELB, F5 load balancers, CDNs, Varnish, and more.
Testing plug-in components like that can be challenging, because their performance characteristics don't directly align with application metrics.
- How many simultaneous users can my load balancer support? - What sort of network load will I put on my CDN (i.e., how much will it cost?) - How do different user behavior patterns affect performance?
In this meetup, we'll introduce a novel tool in this toolbox: tcpreplay, an open-source tool for replaying packet capture files back at an application. By replaying user traffic to a staging environment, you can test the effects of
- Network saturation to the load balancer - High numbers of users / IPs - Lots of traffic to your other monitoring tools!
Chicago Flink Meetup: Flink's streaming architectureRobert Metzger
This document summarizes the architecture of Apache Flink's streaming runtime. Flink is a stream processor that embraces the streaming nature of data with low latency, high throughput, and exactly-once guarantees. It achieves this through pipelining to keep data moving efficiently and distributed snapshots for fault tolerance. Flink also supports batch processing as a special case of streaming by running bounded streams as a single global window.
Direct Code Execution - LinuxCon Japan 2014Hajime Tazaki
Direct Code Execution (DCE) is a userspace kernel network stack that allows running real network stack code in a single process. DCE provides a testing platform that enables reproducible testing, fine-grained parameter tuning, and a development framework for network protocols. It achieves this through a virtualization core layer that runs multiple network nodes within a single process, a kernel layer that replaces the kernel with a shared library, and a POSIX layer that redirects system calls to the kernel library. This allows full control and observability for testing and debugging the network stack.
Apache Flink(tm) - A Next-Generation Stream ProcessorAljoscha Krettek
In diesem Vortrag wird es zunächst einen kurzen Überblick über den aktuellen Stand im Bereich der Streaming-Datenanalyse geben. Danach wird es mit einer kleinen Einführung in das Apache-Flink-System zur Echtzeit-Datenanalyse weitergehen, bevor wir tiefer in einige der interessanten Eigenschaften eintauchen werden, die Flink von den anderen Spielern in diesem Bereich unterscheidet. Dazu werden wir beispielhafte Anwendungsfälle betrachten, die entweder direkt von Nutzern stammen oder auf unserer Erfahrung mit Nutzern basieren. Spezielle Eigenschaften, die wir betrachten werden, sind beispielsweise die Unterstützung für die Zerlegung von Events in einzelnen Sessions basierend auf der Zeit, zu der ein Ereignis passierte (event-time), Bestimmung von Zeitpunkten zum jeweiligen Speichern des Zustands eines Streaming-Programms für spätere Neustarts, die effiziente Abwicklung bei sehr großen zustandsorientierten Streaming-Berechnungen und die Zugänglichkeit des Zustandes von außerhalb.
SR-IOV: The Key Enabling Technology for Fully Virtualized HPC ClustersGlenn K. Lockwood
How well does InfiniBand virtualized with SR-IOV really perform? SDSC carried out some initial application benchmarking studies and compared to the best-available commercial alternative to determine whether or not SR-IOV was a viable technology for closing the performance gap of virtualized HPC. The results were promising, and this technology will be used in Comet, SDSC's two-petaflop supercomputer being deployed in 2015.
VMworld 2013
Lenin Singaravelu, VMware
Haoqiang Zheng, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
The document discusses different types of MySQL replication including asynchronous, semi-synchronous, and synchronous. It provides pros and cons of each type and describes how they handle transaction ordering, parallelism, flow control, and consistency. The key points are that asynchronous replication has potential for data loss and stale reads, semi-synchronous reduces but does not eliminate data loss risk, while SchoonerSQL's synchronous replication guarantees no data loss and failover without stalled transactions.
Similar to FlexSC: Exception-Less System Calls - presented @ OSDI 2010 (20)
FlexSC: Exception-Less System Calls - presented @ OSDI 2010
1. FlexSC
Flexible System Call Scheduling with
Exception-Less System Calls
Livio Soares and Michael Stumm
University of Toronto
2. Motivation
The synchronous system call interface is a
legacy from the single core era
Expensive! Costs are:
➔ direct: mode-switch
➔ indirect: processor
structure pollution
FlexSC implements efficient and flexible
system calls for the multicore era
2
3. FlexSC overview
Two contributions: FlexSC and FlexSC-Threads
Results in:
1) MySQL throughput increase of up to 40%
and latency reduction of 30%
2) Apache throughput increase of up to 115%
and latency reduction of 50% 3
4. Performance impact of synchronous syscalls
➔ Xalan from SPEC CPU 2006
➔ Virtually no time in the OS
➔ Linux on Intel Core i7 (Nehalem)
➔ Injected exceptions with varying frequencies
➔ Direct: emulate null system call
Direct
➔ Indirect: emulate “write()” system call
Indirect
➔ Measured only user-mode time
➔ Kernel time ignored
Ideally, user-mode performance is unaltered
4
5. Degradation due to sync. syscalls
Degradation (lower is faster)
Xalan (SPEC CPU 2006)
70%
60%
Apache Indirect
50% Direct
40% MySQL
30%
20%
10%
0%
1K 2K 5K 10K 20K 50K 100K
user-mode instructions between exceptions
(log scale)
System calls can half processor efficiency;
indirect cause is major contributor
5
6. Processor state pollution
➔ Key source of performance impact
➔ On a Linux write() call:
rd
➔ up to 2/3 of the L1 data cache and data
TLB are evicted
➔ Kernel performance equally affected
➔ Processor efficiency for OS code is also cut
in half
6
7. Synchronous system calls are expensive
User
Kernel
Traditional system calls are synchronous
and use exceptions to cross domains
7
8. Alternative: side-step the boundary
User
Kernel
Exception-less syscalls remove synchronicity
by decoupling invocation from execution
8
9. Benefits of exception-less system calls
➔ Significantly reduce direct costs
➔ Fewer mode switches
User
➔ Allow for batching
Kernel
➔ Reduce indirect costs
➔ Allow for dynamic multicore specialization
➔ Further reduce direct and indirect costs
9
13. Syscall threads
➔ Kernel-only threads
➔ Part of application process
➔ Execute requests from syscall page
➔ Schedulable on a per-core basis
13
14. System call batching
Request as many system calls as possible
Switch to kernel-mode
Start executing all posted system calls
Avoids direct and indirect costs,
even on a single core 14
16. What programs can benefit from FlexSC?
Event-driven servers
(e.g., memcached, nginx webserver)
➔ Use asynchoronous calls, similar to FlexSC
➔ Can use FlexSC directly
➔ Mix sync and exception-less system calls
Multi-threaded servers: FlexSC-Threads
➔ Thread library, compatible with Pthreads
➔ No changes to app. code or recompilation required
➔ Transparently converts legacy syscalls into
exception-less ones
16
17. FlexSC-Threads library
➔ Hybrid (M-on-N) threading model
➔ One kernel visible thread per core
➔ Many user threads per kernel-visible thread
➔ Redirects system calls (libc wrappers)
➔ Posts exception-less syscall to syscall page
➔ Switches to other user-level thread
➔ Resumes thread upon syscall completion
Benefits of exception-less syscalls
while maintaining sequential syscall interface
17
31. Discussion
➔ New OS architecture not necessary
➔ Exception-less syscalls can coexist with legacy ones
➔ Foundation for non-blocking system calls
➔ select() / poll() in user-space
➔ Interesting case of non-blocking free()
➔ Multicore ultra -specialization
➔ TCP Servers (Rutgers; Iftode et.al), FS Servers
➔ Single-ISA asymmetric cores
➔ OS-friendly cores (HP Labs; Mogul et. al)
31
32. Concluding Remarks
➔ System calls degrade server performance
➔ Processor pollution is inherent to synchronous
system calls
➔ Exception-less syscalls
➔ Flexible and efficient system call execution
➔ FlexSC-Threads
➔ Leverages exception-less syscalls
➔ No modifications to multi-threaded applications
➔ Throughput & latency gains
➔ 2x throughput improvement for Apache and BIND
➔ 1.4x throughput improvement for MySQL 32
33. FlexSC
Flexible System Call Scheduling with
Exception-Less System Calls
Livio Soares and Michael Stumm
University of Toronto