Databases are a key part of any application. The storage subsystem contributes most to performance of the database. In recent days, new storage technologies like Solid State Storage (SSD) and high performance drives are becoming cheaper and more accessible, but it takes a lot of planning to use these technologies in a cost effective way for best price-performance.
Databases are a key part of any application. The storage subsystem contributes most to performance of the database. In recent days, new storage technologies like Solid State Storage (SSD) and high performance drives are becoming cheaper and more accessible, but it takes a lot of planning to use these technologies in a cost effective way for best price-performance.
Optimizing Oracle databases with SSD - April 2014Guy Harrison
Presentation on using Solid State Disk (SSD) with Oracle databases, including the 11GR2 db flash cache and using flash in Exadata. Last given at Collaborate 2014 #clv14.
Azure VM 101 - HomeGen by CloudGen Verona - Marco ObinuMarco Obinu
Slides presented during HomeGen by CloudGen Verona, about how to properly size an Azure IaaS VM, with an additional focus on high availability and cost-saving topics.
Session recording: https://youtu.be/C8v6c6EkJ9A
Demo: https://github.com/OmegaMadLab/SqlIaasVmPlayground
Webinar slides: The Holy Grail Webinar: Become a MySQL DBA - Database Perform...Severalnines
You’re running MySQL as backend database, how do you tune it to make best use of the hardware? How do you optimize the Operating System? How do you best configure MySQL for a specific database workload?
Do these questions sound familiar to you? Maybe you’re having to deal with that type of situation yourself?
In this webinar, we’ve discussed some of the settings that are most often tweaked and which can bring you significant improvement in the performance of your MySQL database. We also covered some of the variables which are frequently modified even though they should not. Performance tuning is not easy, but you can go a surprisingly long way with a few basic guidelines.
AGENDA
Database tuning - the what and why
Principles of the tuning process
Tuning the Operating System configuration
Tuning the MySQL configuration
Useful tools
pt-summary
pt-mysql-summary
What to avoid when tuning OS and MySQL configuration
SPEAKER
Krzysztof Książek, Senior Support Engineer at Severalnines, is a MySQL DBA with experience managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
This webinar is based on our popular blog series ‘Become a MySQL DBA’.
Databases are a key part of any application. The storage subsystem contributes most to performance of the database. In recent days, new storage technologies like Solid State Storage (SSD) and high performance drives are becoming cheaper and more accessible, but it takes a lot of planning to use these technologies in a cost effective way for best price-performance.
Optimizing Oracle databases with SSD - April 2014Guy Harrison
Presentation on using Solid State Disk (SSD) with Oracle databases, including the 11GR2 db flash cache and using flash in Exadata. Last given at Collaborate 2014 #clv14.
Azure VM 101 - HomeGen by CloudGen Verona - Marco ObinuMarco Obinu
Slides presented during HomeGen by CloudGen Verona, about how to properly size an Azure IaaS VM, with an additional focus on high availability and cost-saving topics.
Session recording: https://youtu.be/C8v6c6EkJ9A
Demo: https://github.com/OmegaMadLab/SqlIaasVmPlayground
Webinar slides: The Holy Grail Webinar: Become a MySQL DBA - Database Perform...Severalnines
You’re running MySQL as backend database, how do you tune it to make best use of the hardware? How do you optimize the Operating System? How do you best configure MySQL for a specific database workload?
Do these questions sound familiar to you? Maybe you’re having to deal with that type of situation yourself?
In this webinar, we’ve discussed some of the settings that are most often tweaked and which can bring you significant improvement in the performance of your MySQL database. We also covered some of the variables which are frequently modified even though they should not. Performance tuning is not easy, but you can go a surprisingly long way with a few basic guidelines.
AGENDA
Database tuning - the what and why
Principles of the tuning process
Tuning the Operating System configuration
Tuning the MySQL configuration
Useful tools
pt-summary
pt-mysql-summary
What to avoid when tuning OS and MySQL configuration
SPEAKER
Krzysztof Książek, Senior Support Engineer at Severalnines, is a MySQL DBA with experience managing complex database environments for companies like Zendesk, Chegg, Pinterest and Flipboard.
This webinar is based on our popular blog series ‘Become a MySQL DBA’.
JetStor 8 Series FC units 16G FC Active-Active 12G SAS
Features & Highlights
DataBolt™ Bandwidth Optimizer for balancing faster host & slower SAS or SATA devices
Supports SSD automatic monitor clone (AMC) to extend SSD lifetime
Supports Self-Encryption Drive (SED)
Supports Clone disk
Encryption at Rest Support ISE, TCG, SED, SSD with Block Erase & Password Protect
JetStor X Storage Products 2017! New HOT products!Gene Leyzarovich
JetStor Brand new 12, 16, 24 and 26bay AFA, Hybrid Storage!
JetStor FX series features a Dual-Active controller architecture, both controllers concurrently provide storage services in real time. Active-Active architecture doubles the available host bandwith and cache hit ratio, ensuring maximum utilization of system resources and maximized throughput. If one controller fails, the other controller can transparently take over all storage services. In addition to storage services, management service can transparently pass to the secondary controller.
The JetStor FX series is a highly-available SAN storage system. All of the critical components in the FX series are hot pluggable and designed with full redundancy. This design allows the FX series to withstand multiple component failures and achieve 99.999% availability.
TrioNAS LX U300 consolidate NAS and SAN offers multiple enterprise-level features including DeDup & Compression, Unlimited Snapshot, Thin Provisioning, Online Capacity Expansion and SSD caching.
For years Qsan has won plenty of proven records in enterprise markets and numerous vertical industries. Based on expertise in delivering in-house iSCSI & RAID stack, TrioNAS LX U300 deliver the best price-performance value to meet enterprise IT budget and specific needs.
For more detail please visit: http://www.qsantechnology.com/en/raidsystem_view.php?RSTID=AQ000108
The purpose of this guide is to share insights on industrial vs commercial storage and assist you to make an informed buying decisions during the selection process
JetStor 8 Series FC units 16G FC Active-Active 12G SAS
Features & Highlights
DataBolt™ Bandwidth Optimizer for balancing faster host & slower SAS or SATA devices
Supports SSD automatic monitor clone (AMC) to extend SSD lifetime
Supports Self-Encryption Drive (SED)
Supports Clone disk
Encryption at Rest Support ISE, TCG, SED, SSD with Block Erase & Password Protect
JetStor X Storage Products 2017! New HOT products!Gene Leyzarovich
JetStor Brand new 12, 16, 24 and 26bay AFA, Hybrid Storage!
JetStor FX series features a Dual-Active controller architecture, both controllers concurrently provide storage services in real time. Active-Active architecture doubles the available host bandwith and cache hit ratio, ensuring maximum utilization of system resources and maximized throughput. If one controller fails, the other controller can transparently take over all storage services. In addition to storage services, management service can transparently pass to the secondary controller.
The JetStor FX series is a highly-available SAN storage system. All of the critical components in the FX series are hot pluggable and designed with full redundancy. This design allows the FX series to withstand multiple component failures and achieve 99.999% availability.
TrioNAS LX U300 consolidate NAS and SAN offers multiple enterprise-level features including DeDup & Compression, Unlimited Snapshot, Thin Provisioning, Online Capacity Expansion and SSD caching.
For years Qsan has won plenty of proven records in enterprise markets and numerous vertical industries. Based on expertise in delivering in-house iSCSI & RAID stack, TrioNAS LX U300 deliver the best price-performance value to meet enterprise IT budget and specific needs.
For more detail please visit: http://www.qsantechnology.com/en/raidsystem_view.php?RSTID=AQ000108
The purpose of this guide is to share insights on industrial vs commercial storage and assist you to make an informed buying decisions during the selection process
ADITECH CUSTOMER MEET-2015 was held at Hotel RAMADA, Millennium Business Park, Navi Mumbai. This event was sponsored by Intel and Innodisk Taiwan, Event was attended by 39 System Integrator partners from Mumbai, Pune, Delhi, Surat and Banglore. Intel has presented the IOT opportunities for SME. Innodisk has enlightened SI partners on latest technologies used in Industrial grade SSD. Aditech has demonstrated Industrial grade solutions and transportation solutions. Aditech's presentation was on Industrial grade Panel PC's and industrial communication. The event is ended with lucky draw and group photograph followed by networking dinner and ADITECH office visit
Weight loss formulas are all the rage, especially for busy people who want to start losing weight without having to go through an exercise routine or strict diet. Xenadrine is one such weight loss formula that promises quick weight loss. But how effective is Xenadrine, and is it safe?
Read more: http://dietopia.net/xenadrine.html#ixzz2RGeMk68G
NAND Flash is the dominant memory for storage in mobile and SSD applications. The presentation is a review of the players from the device level to the enterprise application level
Presentación realizada por TIC Hoteles a la Correduría de Seguros CosmoBrok, con sedes en Barcelona, Coruña, Santiago, Terrassa, Sabadell, Mollerusa, Tarrega, Ames, Porto do Son, Milladoiro
Wie Visualisierungen uns die Augen Öffnen OOP 2016Olaf Lewitz
Visualisierungen helfen uns, im Alltag dort hinzuschauen, wo wir sonst etwas übersehen, wir uns unsicher fühlen oder wo es wehtut. Visualisierung sehen wir alle, und gemeinsam sind wir mutiger und aufmerksamer! Warum sind Visualisierungen effektiv? Welche Tools sind für welche Situationen geeignet? Wir stellen verschiedene Werkzeuge und Tools vor, die ein gemeinsames Handlungsbewusstsein erzeugen. Wir machen uns bewusst, was gute Visualisierungen in unserem Geist bewirken. Einige davon probieren wir gleich aus.
Wir benutzen im agilen Umfeld so viele Visualisierungen und reden auch prominent von der Wichtigkeit von "Information Radiators" - wenigen ist klar, warum das eigentlich so wichtig ist. Wir sind kürzlich über die Information gestolpert, dass unser visueller Kortex (weil er evolutionär viele Millionen Jahre älter ist und die Natur Zeit zum Optimieren hatte) um Größenordnungen schneller und energiesparender im Zugriff ist als unser Sprachzentrum. Das hat uns neugierig gemacht, und wir haben uns mit der Neurobiologie von Individuen und Gruppen näher beschäftigt.
Unsere Erkenntnisse erklären, warum bestimmte agile (oder auch klassische wie Modellierung am Whiteboard) Visualisierungen hilfreich sind. Mit diesem Wissen steigt die Motivation, Visualisierungen im Alltag einzusetzen. Die Teilnehmer können diese Methoden gezielter anwenden und verbessern.
Protect and maintain databases
حماية و صيانة قواعد البيانات
1. ضغط و اصلاح قاعدة البيانات تساعدنا في تحسين أداء قاعدة البيانات و اصلاح الملف عند حدوث المشاكل.
2. تشفير قاعدة البيانات بتطبيق كلمة مرور لتقييد الوصول, فقط المستخدمون الذين يعرفون كلمة المرور يستطيعون الوصول الى قاعدة البيانات.
3. العمل مع النسخ الاحتياطي.
4. دمج قواعد البيانات.
5. تقسيم قاعدة البيانات و هي خطوة تطبق خصيصا للمشاركة مع عدة مستخدمين.
إنشاء الاستعلامات الإجرائية
تشرح الأقسام الثلاثة المقبلة أنواع مختلفة من الاستعلامات الإجرائية:
استعلام تكوين جدول، استعلام إلحاقي، و استعلامات التحديث و الحذف.
غالبا ما تستخدم الاستعلامات الإجرائية للمساعدة في إدارة السجلات في قاعدة
البيانات. على سبيل المثال، يمكنك استخدام استعلام تحديد لاسترداد السجلات
لجميع المنتجات المتوقفة. يمكنك استخدام-تكوين جدول أو استعلام إلحاقي لأرشفة
تلك السجلات، ثم قم بتشغيل استعلام حذف على جدول المنتجات لإزالة سجلات
لتلك المنتجات.
إدارة السجلات
يمكنك إدارة السجلات في الجداول من خلال عرض ورقة البيانات. يشرح هذا القسم كيفية إضافة, تحديث, و حذف السجلات,
و أيضا كيفية البحث, الفرز, و تصفية السجلات عندما تحتاج الى العمل مع السجلات التي تناسب معايير مخصصة.
يشرح هذا القسم أيضا كيفية إلحاق سجلات الى جدول موجود.
Data deduplication is a hot topic in storage and saves significant disk space for many environments, with some trade offs. We’ll discuss what deduplication is and where the Open Source solutions are versus commercial offerings. Presentation will lean towards the practical – where attendees can use it in their real world projects (what works, what doesn’t, should you use in production, etcetera).
Demystifying Storage - Building large SANsDirecti Group
From http://wiki.directi.com/x/hQAa - This is a fairly detailed presentation I made at BarCamp Mumbai on building large storage networks and different SAN topologies. It covers fundamentals of selecting harddrives, RAID levels and performance of various storage architectures. This is Part I of a 3-part series
This presentation provides an overview of the Dell PowerEdge R730xd server performance results with Red Hat Ceph Storage. It covers the advantages of using Red Hat Ceph Storage on Dell servers with their proven hardware components that provide high scalability, enhanced ROI cost benefits, and support of unstructured data.
A 45 minute talk discussing various in production performance enhancements for PostgreSQL. We touch on hard drives including SSD and RAID. We also discuss Memory, PostgreSQL settings and various other topics such as DASvsNASvsSAN.
From http://wiki.directi.com/x/hQAa - This is a fairly detailed presentation I made at BarCamp Mumbai on building large storage networks and different SAN topologies. It covers fundamentals of selecting harddrives, RAID levels and performance of various storage architectures. This is Part I of a 3-part series.
Running Apache Spark on a High-Performance Cluster Using RDMA and NVMe Flash ...Databricks
Effectively leveraging fast networking and storage hardware (e.g., RDMA, NVMe, etc.) in Apache Spark remains challenging. Current ways to integrate the hardware at the operating system level fall short, as the hardware performance advantages are shadowed by higher layer software overheads. This session will show how to integrate RDMA and NVMe hardware in Spark in a way that allows applications to bypass both the operating system and the Java virtual machine during I/O operations. With such an approach, the hardware performance advantages become visible at the application level, and eventually translate into workload runtime improvements. Stuedi will demonstrate how to run various Spark workloads (e.g, SQL, Graph, etc.) effectively on 100Gbit/s networks and NVMe flash.
Nagios Conference 2012 - Dan Wittenberg - Case Study: Scaling Nagios Core at ...Nagios
Dan Wittenberg's presentation on using Nagios at a Fortune 50 Company
The presentation was given during the Nagios World Conference North America held Sept 25-28th, 2012 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna
AC&NC provides full product line up of Network Attached Storage (NAS) systems that are all built for reliability and ease of use. AC&NC also offers combined NAS and Storage Area Networks (SAN) into a single system, allowing for a consolidated storage and network environment.
Focused intently on storage without distractions of tape backup or bundled servers, AC&NC manufacturers in-house and delivers complete solutions in 24-48 hours from in-stock JetStor RAID, iSCSI, FC, NAS / Unified, All Flash and JBOD SAS systems that set the bar for performance.
Greg Smith
Cover how to use simple low-level tools such as memtest86, dd, bonnie++, and sysbench to benchmark the hardware of a server intended for database use. A heavy dose of vendor management suggestions will be included as well, for the inevitable time when your shiny new server fails to deliver the performance it should.
Similar to Database performance tuning for SSD based storage (20)
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Welcome to the first live UiPath Community Day Dubai! Join us for this unique occasion to meet our local and global UiPath Community and leaders. You will get a full view of the MEA region's automation landscape and the AI Powered automation technology capabilities of UiPath. Also, hosted by our local partners Marc Ellis, you will enjoy a half-day packed with industry insights and automation peers networking.
📕 Curious on our agenda? Wait no more!
10:00 Welcome note - UiPath Community in Dubai
Lovely Sinha, UiPath Community Chapter Leader, UiPath MVPx3, Hyper-automation Consultant, First Abu Dhabi Bank
10:20 A UiPath cross-region MEA overview
Ashraf El Zarka, VP and Managing Director MEA, UiPath
10:35: Customer Success Journey
Deepthi Deepak, Head of Intelligent Automation CoE, First Abu Dhabi Bank
11:15 The UiPath approach to GenAI with our three principles: improve accuracy, supercharge productivity, and automate more
Boris Krumrey, Global VP, Automation Innovation, UiPath
12:15 To discover how Marc Ellis leverages tech-driven solutions in recruitment and managed services.
Brendan Lingam, Director of Sales and Business Development, Marc Ellis
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
2. Agenda
Performance issues in Storage
Hybrid Storage (Disks, SSDs, Memory)
ZFS - Not Just Another File System
Tuning for databases (General principles)
Tuning for MySQL
Tuning for PostgreSQL
Tuning for Oracle
3. Why?
• Some very practical advice based on
> recent test results
> Improved pgbench results from 70 tps for pure disk to 5003 tps with
SSD and tuning
> Improved sysbench results from 425 tps to 1811 tps with SSD and
tuning for read/write.
> Improved sysbench results from 786 tps to 3085 tps with SSD and
tuning for read.
> collection of tuning knowledge from Sun performance
engineers and the community
• Some very good resources at the end of the talk for
further study
4. Storage Performance
Cache
Memory
0,0 00 X t ial
10 en
fer
e dif
nc
Disk rma
High Cache
e rfo
Performance Disks P
Large Capacity
Disks
5. Latency Comparison
Bridging the DRAM to HDD Gap
1S
100mS
10m
S
1mS
100uS
10uS TAPE
1u HDD
S
100nS FLASH/
SSD
10nS
DRAM
1n
S CPU
11. Administering ZFS in two slides
As easy as pie
• zpool commands
> create a single disk pool:
# zpool create newpool diskname
> create a pool with a mirror
# zpool create newpool mirror disk1name disk2name
> Add device to a pool:
# zpool add poolname diskname
> Replace a bad disk
# zpool replace poolname baddiskname newdiskname
> History of commands on the pool:
# zpool history poolname
> How is my pool performing:
# zpool iostat poolname
No format command, No fdisk partitions, No volumes
12. Administering ZFS in two slides
As easy as pie
• zfs commands
> create a filesystem:
# zfs create poolname/fs-name
> set filesystem property:
# zfs set quota=size poolname/fs-name
# zfs set compression=on poolname/fs-name
# zfs set nfsshare=on poolname/fs-name
# zfs set recordsize=16k poolname/fs-name
> get filesystem property:
# zfs get compressratio poolname/fs-name
# zfs get all poolname/fs-name
> snapshot the filesystem:
# zfs snapshot poolname/fs-name@snapshotname
No newfs, No mkfs, No /etc/vfstab, No fsck
13. ZFS and Hybrid Storage
As easy as pie
• Read side
> Add ssd as a read side cache
> # zpool add poolname cache ssd-device
• Write side
> Add SSD as a ZFS Intent Log device
> # zpool add poolname log ssd-device
14. ZFS Performance Features
• Copy-on-write
> Turns Random writes to Sequential writes
• Dynamic Striping across all devices
> Maximize throughput
• Multiple Block Sizes
> Automatically chosen to match workload
• IO Pipelining
> Priority/Deadline scheduling, sorting, aggregation
• Intelligent prefetch
• Compression - Improves performance & Capacity
• Can safely use write cache on disks
15. Databases
Not Just Another Application
• Most Databases do their own buffering
> Filesystem caching can get in the way
> “double buffer” problem
• Most Databases do “prefetch”
> Filesystems prefetch can cause extra IO
> “directio” gets filesystem out of the way
• Have their own “log” mechanism.
> Interesting interaction with a transaction based filesystem
• Multiple blocks sizes
> Database & Transaction log, block sizes are normally different
16. Tuning ZFS for Databases
Tuning is Evil - Long live Tuning
• In general tuning is evil. Let ZFS do it for you.
• A few fine tuning tips for databases
> Get to the latest update of OS
> Set the recordsize to match database
block size
> Separate Transaction logs and data
onto separate zpools
> [Note: This will be addressed with the ZIL bypass property fix]
> Reduce the impact of double buffering by changing the caching
method to “metadata only”
> Use separate ZIL (ZFS Intent Log) preferably SSD
> Use SSD as secondary cache - L2ARC (Level 2. Adaptive
Replacement Cache)
17. ZFS tuning for MySQL
• Many tuning depends on storage engine
• For Innodb
> Prefer to cache in Innodb rather than ARC
zfs set primarycache=metadata poolname/database
> Set recordsize to 16k for data and 128k for log
zfs set recordsize=16k poolname/database
(Note: do this before you load any data)
> Turn off prefetch
set zfs:zfs_prefetch_disable = 1 (in /etc/system)
(File level prefetch not triggered if you change record size to 16k)
> Use raid0 or mirror over raidz
raidz is no suitable for random IO
> Add SSDs for either read side or write side based on workload
zpool add datapool cache ssd-disk
zpool create logpool ssd-disk3
In my.cnf set innodb_data_home_dir & innodb_log_group_home_dir
18. ZFS tuning for MySQL
• More tuning for Innodb
> Some device vendors flush cache even
when not needed. (eg. battery backed cache)
set zfs:zfs_nocacheflush = 1
> Turn on compression
zfs set compression=on poolname/database
ZFS does not turn on compression if less than 12.5% saving.
IO reduction may offset the extra cpu cost
> Disable double writes
innodb_doublewrite=0 (in my.cnf)
ZFS does not allow any partial writes so no need to guard against it.
19. ZFS tuning for PostgreSQL
• Postgres tuning hints
> Set recordsize to 8k
zfs set recordsize=8k poolname/database
> Turn down ARC cache.
set zfs:zfs_arc_max in /etc/system
> Add SSDs for either read side or write side based on workload
zpool add poolname cache ssd-name
zpool add poolname log ssd-name
> Use separate pool for log (preferably one with SSD) & data
initdb -X log_directory_name
create tablespace datatbs location 'database_directory_name'
create database mydb with tablespace datatbs
> Don’t forget to basic Postgres tuning on Solaris - (huge gains)
Set shared_buffers, temp_buffers, work_mem, maintenance_work_mem,
wal_sync_method, synchronous_commits etc
see: http://blogs.sun.com/jkshah/entry/best_practices_with_postgresql_8
20. ZFS tuning for Oracle
• Oracle tuning hints
> Set recordsize to match db_block_size (default 8k)
zfs set recordsize=8k poolname/database
> Use separate pool for Oracle logs
make sure record size of the log filesystem is left to the 128k default
> Add SSDs for either read side or write side based on workload
zpool add poolname cache ssd-name
zpool add poolname log ssd-name
21. Benchmark results
• Hardware
> Sun x4150 2 x Quad core 2.3 GHz Xeon
12 GB ram
3 x 10000 rpm drives
3 x 32 GB SSDs
• Software
> OpenSolaris 2009.06
> Postgres 8.3.7
> MySQL 5.4 beta
22. Benchmark results
• pgbench & Postgres
> command line: pgbench -c 10 -s 10 -t 10000 pgbench
Description TPS
Single disk ZFS 72 tps
2 Raid 0 disk + SSD as level 2 cache 241 tps
Above + general postgres optimization 2026 tps
+ all the data on SSD 2603 tps
+ data on hdd & log on SSD 4372 tps
+ primarycache=metadata 5003 tps
23. Benchmark results
• sysbench & mysql 5.4
> read/write test: sysbench --max-time=300 --max-requests=0 --test=oltp --
oltp-dist-type=special --oltp-table-size=10000000 --num-threads=20 run
Description TPS
Single disk ZFS 425 tps
raid0 ZFS 670 tps
+ SSD cache 788 tps
+ Separate intent log 1352 tps
+ With optimization 1809 tps
24. Benchmark results
• sysbench & mysql 5.4
> read test: sysbench --max-time=300 --max-requests=0 --test=oltp --oltp-
dist-type=special --oltp-table-size=10000000 --num-threads=20 --oltp-read-
only=on run
Description TPS
Single disk ZFS 786 tps
2 disk raid0 ZFS 1501 tps
+ SSD cache 1981 tps
+ Separate intent log on SSD 2567 tps
+ optimization 3065 tps
26. Getting these systems at a discount
Sun Startup Essentials
• Exclusive program for startups
• Eligibility <6 yrs. Old, <150
sun.com/startup employees
• Co-marketing opportunities
• Funding assistance
• Deeply discounted storage and
servers certified for Linux,
Windows, and Solaris
• Hosting starting at $40
• Open source software, and
discounted MySQL
• Free email based tech support
• Free and discounted training on
Sun technologies
• Member-only webinars
27. Resources
• ZFS info: http://www.opensolaris.org/os/community/zfs/
• ZFS Best Practices Guide:
http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide
• ZFS Evil Tuning Guide:
http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide
• Blogs of note:
> All things performance tuning:
http://blogs.sun.com/realneel
http://blogs.sun.com/roch
> Postgres tuning - Jignesh’s Blog
http://blogs.sun.com/jkshah
> Angelo’s blog
http://blogs.sun.com/angelo