The document discusses the behavior of the Oracle log writer (LGWR) process under different conditions. In idle mode, LGWR sleeps for 3 seconds at a time on a semaphore without writing to the redo log buffer. When a transaction is committed, LGWR may write the committed redo entries to disk either before or after the foreground process waits on a "log file sync" event, depending on whether LGWR has already flushed the data. The document also compares the "post-wait" and "polling" modes used for the log file sync wait.
Troubleshooting Complex Oracle Performance Problems with Tanel PoderTanel Poder
Troubleshooting Complex Oracle Performance Problems hacking session & presentation by Tanel Poder.
This presentation is about a complex performance issue where the initial symptoms pointed somewhere else than the root cause. Only when systematically following through the troubleshooting drilldown method, we get to the root cause of the problem. This session aims to help you understand (and reason about) the Oracle’s multi-process & multi-layer system behavior, preparing you for independent troubleshooting of such complex performance issues in the future.
Video recordings of this presentation are in my YouTube channel:
1) Hacking Session: https://www.youtube.com/watch?v=INQewGJMdCI
2) Presentation: https://www.youtube.com/watch?v=aaHZ8A8Ygdg
Tanel's blog and training information: https://blog.tanelpoder.com/seminar
Troubleshooting Complex Oracle Performance Problems with Tanel PoderTanel Poder
Troubleshooting Complex Oracle Performance Problems hacking session & presentation by Tanel Poder.
This presentation is about a complex performance issue where the initial symptoms pointed somewhere else than the root cause. Only when systematically following through the troubleshooting drilldown method, we get to the root cause of the problem. This session aims to help you understand (and reason about) the Oracle’s multi-process & multi-layer system behavior, preparing you for independent troubleshooting of such complex performance issues in the future.
Video recordings of this presentation are in my YouTube channel:
1) Hacking Session: https://www.youtube.com/watch?v=INQewGJMdCI
2) Presentation: https://www.youtube.com/watch?v=aaHZ8A8Ygdg
Tanel's blog and training information: https://blog.tanelpoder.com/seminar
Oracle Latch and Mutex Contention TroubleshootingTanel Poder
This is an intro to latch & mutex contention troubleshooting which I've delivered at Hotsos Symposium, UKOUG Conference etc... It's also the starting point of my Latch & Mutex contention sections in my Advanced Oracle Troubleshooting online seminar - but we go much deeper there :-)
AWR Ambiguity: Performance reasoning when the numbers don't add upJohn Beresniewicz
A close look at an AWR report where DB Time is exceeded by the sum of DB CPU and foreground wait time. We recall core Oracle performance principles and instrumentation design on the way to untangling the confusion.
Troubleshooting Complex Performance issues - Oracle SEG$ contentionTanel Poder
From Tanel Poder's Troubleshooting Complex Performance Issues series - an example of Oracle SEG$ internal segment contention due to some direct path insert activity.
Awr1page - Sanity checking time instrumentation in AWR reportsJohn Beresniewicz
Discusses Oracle time-based performance instrumentation as presented in AWR reports and inconsistencies between instrumentation sources that can cause confusion as conflicting information is presented. The cognitive load of investigating and reasoning about such conundrums is very high, discouraging even senior performance experts. A program (AWR1page) is discussed that consumes an AWR report and produces a 1-page normalized time summary by instrumentation source, precisely designed for reasoning about instrumentation inconsistencies in AWR reports.
How to Avoid Pitfalls in Schema Upgrade with GaleraSveta Smirnova
Galera Cluster for MySQL is a 100% synchronized cluster in regards to data modification operations (DML). It is ensured by the optimistic locking model and ability to rollback a transaction, which cannot be applied on all nodes. However, schema changes (DDL operations) are not transactional in MySQL, which adds complexity when you need to perform an upgrade or change schema of the database.
Changes made by DDL may affect results of the queries. Therefore all modifications must replicate on all nodes prior next data access. For operations which run momentarily it can be easily achieved, but schema changes may take hours to apply. Therefore in addition to safest synchronous blocking schema upgrade method TOI Galera also supports more relaxed, thought not safe, method RSU.
In her talk Sveta will describe which pitfalls you can hit while performing the change using one or another method, why and how to avoid them.
Presented at MariaDB Day Brussels 0202 2020: https://mariadb.org/mariadb-day-brussels-0202-2020-provisional-schedule/
This is a recording of my Advanced Oracle Troubleshooting seminar preparation session - where I showed how I set up my command line environment and some of the main performance scripts I use!
Understanding Average Active Sessions (AAS) is critical to understanding Oracle performance at the systemic level. This is my first presentation on the topic done at RMOUG Training Days in 2007. Later I will upload a more recent presentation on AAS from 2013.
Tanel Poder Oracle Scripts and Tools (2010)Tanel Poder
Tanel Poder's Oracle Performance and Troubleshooting Scripts & Tools presentation initially presented at Hotsos Symposium Training Day back in year 2010
Oracle Latch and Mutex Contention TroubleshootingTanel Poder
This is an intro to latch & mutex contention troubleshooting which I've delivered at Hotsos Symposium, UKOUG Conference etc... It's also the starting point of my Latch & Mutex contention sections in my Advanced Oracle Troubleshooting online seminar - but we go much deeper there :-)
AWR Ambiguity: Performance reasoning when the numbers don't add upJohn Beresniewicz
A close look at an AWR report where DB Time is exceeded by the sum of DB CPU and foreground wait time. We recall core Oracle performance principles and instrumentation design on the way to untangling the confusion.
Troubleshooting Complex Performance issues - Oracle SEG$ contentionTanel Poder
From Tanel Poder's Troubleshooting Complex Performance Issues series - an example of Oracle SEG$ internal segment contention due to some direct path insert activity.
Awr1page - Sanity checking time instrumentation in AWR reportsJohn Beresniewicz
Discusses Oracle time-based performance instrumentation as presented in AWR reports and inconsistencies between instrumentation sources that can cause confusion as conflicting information is presented. The cognitive load of investigating and reasoning about such conundrums is very high, discouraging even senior performance experts. A program (AWR1page) is discussed that consumes an AWR report and produces a 1-page normalized time summary by instrumentation source, precisely designed for reasoning about instrumentation inconsistencies in AWR reports.
How to Avoid Pitfalls in Schema Upgrade with GaleraSveta Smirnova
Galera Cluster for MySQL is a 100% synchronized cluster in regards to data modification operations (DML). It is ensured by the optimistic locking model and ability to rollback a transaction, which cannot be applied on all nodes. However, schema changes (DDL operations) are not transactional in MySQL, which adds complexity when you need to perform an upgrade or change schema of the database.
Changes made by DDL may affect results of the queries. Therefore all modifications must replicate on all nodes prior next data access. For operations which run momentarily it can be easily achieved, but schema changes may take hours to apply. Therefore in addition to safest synchronous blocking schema upgrade method TOI Galera also supports more relaxed, thought not safe, method RSU.
In her talk Sveta will describe which pitfalls you can hit while performing the change using one or another method, why and how to avoid them.
Presented at MariaDB Day Brussels 0202 2020: https://mariadb.org/mariadb-day-brussels-0202-2020-provisional-schedule/
This is a recording of my Advanced Oracle Troubleshooting seminar preparation session - where I showed how I set up my command line environment and some of the main performance scripts I use!
Understanding Average Active Sessions (AAS) is critical to understanding Oracle performance at the systemic level. This is my first presentation on the topic done at RMOUG Training Days in 2007. Later I will upload a more recent presentation on AAS from 2013.
Tanel Poder Oracle Scripts and Tools (2010)Tanel Poder
Tanel Poder's Oracle Performance and Troubleshooting Scripts & Tools presentation initially presented at Hotsos Symposium Training Day back in year 2010
ARC202 Architecting for High Availability - AWS re: Invent 2012Amazon Web Services
Amazon Web Services (AWS) provides a platform that is ideally suited for building highly available systems, enabling you to build reliable, affordable, fault-tolerant systems that operate with a minimal amount of human interaction. This session covers many of the high-availability and fault-tolerance concepts and features of the various services that you can use to build highly reliable and highly available applications in the AWS Cloud: architectures involving multiple Availability Zones, including EC2 best practices and RDS Multi-AZ deployments; loosely coupled and self-healing systems involving SQS and Auto Scaling; networking best practices for high availability, including Elastic IP addresses, load balancing, and DNS; leveraging services that inherently are built with high-availability and fault tolerance in mind, including S3, Elastic Beanstalk and more.
How to find and fix your Oracle application performance problemCary Millsap
How long does your code take to run? Is it changing? When it is slow, WHY is it slow? Is it your fault, or somebody else's? Can you prove it? How much faster could your code be? Do you know how to measure the performance of your code as user workloads and data volumes increase? These are fundamental questions about performance, but the vast majority of Oracle application developers can't answer them. The most popular performance tools available to them—and to the database administrators that run their code in production—are incapable of answering any of these questions. But the Oracle Database can give you exactly what you need to answer these questions and many more. You can know exactly where YOUR CODE is spending YOUR TIME. This session explains how.
Oracle ACFS 12c now supports Oracle Database files. Which new opportunities can come out from this combination?
In this session you will discover the new features of ACFS 12c that are useful when deploying Oracle Databases. You will then learn how to use them through examples and live demos.
Worst Practices in Data Warehouse DesignKent Graziano
This presentation was given at OakTable World 2014 (#OTW14) in San Francisco. After many years of designing data warehouses and consulting on data warehouse architectures, I have seen a lot of bad design choices by supposedly experienced professional. A sense of professionalism, confidentiality agreements, and some sense of common decency have prevented me from calling people out on some of this. No more! In this session I will walk you through a typical bad design like many I have seen. I will show you what I see when I reverse engineer a supposedly complete design and walk through what is wrong with it and discuss options to correct it. This will be a test of your knowledge of data warehouse best practices by seeing if you can recognize these worst practices.
You need to investigate some performance issues on an Oracle database, but you have no access to Oracle Enterprise Manager (OEM), or what is worse, not even to SQL*Plus. Where do you start? What is the first query you ask the DBA to execute for you? What is your second and third? Do you request an AWR report? For which snapshots? This scenario is not uncommon or unheard, it happens often to 3rd party consultants, or even to internal DBAs and Developers when the administration of the database has been subcontracted and access has been restricted.
Performance is not the only case where you may need to reach a database and struggle with access; doing a database health-check or collecting historical performance for a capacity planning exercise may face the same issues. What you wish you had is access to a restricted SQL*Plus account, even if remote, and a toolset to collect as much information as possible from the database of interest; or to ask a DBA with access to this database to simply run this toolset and give you back all the output so you can find answers to most of your questions.
eDB360 is a free tool that installs nothing on the database, executes through a SQL*Plus connection, and produces a zip file with a comprehensive report that provides a 360-degree view of an Oracle database. This session is about edb360. It covers what is included on its output, how you execute this tool, and how it can be used to gain a fair understanding of an Oracle database. This session is for DBAs, Developers and Consultants.
Benefits:
1. Learn how to get a fair 360-degree view of a database
2. Gather enough database information to start a health-check
3. Learn which performance to collect for a sizing exercise
In addition to running databases in Amazon EC2, AWS customers can choose among a variety of managed database services. These services save effort, save time, and unlock new capabilities and economies. In this session, we make it easy to understand how they differ, what they have in common, and how to choose one or more. We explain the fundamentals of Amazon DynamoDB, a fully managed NoSQL database service; Amazon RDS, a relational database service in the cloud; Amazon ElastiCache, a fast, in-memory caching service in the cloud; and Amazon Redshift, a fully managed, petabyte-scale data-warehouse solution that can be surprisingly economical. We will cover how each service might help support your application, how much each service costs, and how to get started.
A session in the DevNet Zone at Cisco Live, Berlin. At the moment, this is the DoE: DevOps of Everything. DevOps is about culture first but many people take shortcuts to tools and workflow. They forgot the essence of DevOps which is about people and not only from Dev to Ops. In this session, we will show you how we are currently building a DevOps culture with a focus on continuous improvement.
Application Logging in the 21st century - 2014.keyTim Bunce
Slides for my talk at the Austrian Perl Workshop in Salzburg on October 10th.
A video of the talk can be found at https://www.youtube.com/watch?v=4Qj-_eimGuE
Up to MySQL 5.5, replication was not crash safe: it would fail with “dup.key” or “not found” error (or data corruption). So 5.6 is better, right? Maybe: it is possible, but not the default. MySQL 5.7 is not much better, 8.0 has safer defaults but it is still easy to get things wrong.
Crash safety is impacted by positioning (File+Pos or GTID), type (single/multi-threaded), MTS settings (Db/Logical Clock, and preserve commit order), the sync-ing of relay logs, the presence of binlogs, log-slave-updates and their sync-ing. This is complicated and even the manual is confused about it.
In this talk, I will explain above with details on replication internals, so you might learn a thing or two.
Docker Logging and analysing with Elastic StackJakub Hajek
Collecting logs from the entire stateless environment is challenging parts of the application lifecycle. Correlating business logs with operating system metrics to provide insights is a crucial part of the entire organization. What aspects should be considered while you design your logging solutions?
Docker Logging and analysing with Elastic Stack - Jakub Hajek PROIDEA
Collecting logs from the entire stateless environment is challenging parts of the application lifecycle. Correlating business logs with operating system metrics to provide insights is a crucial part of the entire organization. We will see the technical presentation on how to manage a large amount of the data in a typical environment with microservices.
Up to MySQL 5.5, replication was not crash safe: after a crash, it would fail with "duplicate key" or "row not found" error, or might generate silent data corruption. It looks like 5.6 is much better, right? The short answer is maybe: in the simplest case, it is possible to achieve replication crash safety but it is not the default setting. MySQL 5.7 is not much better, 8.0 has safer defaults but it is still easy to get things wrong.
Crash safety is impacted by replication positioning (File+Pos or GTID), type (single-threaded or MTS), MTS settings (Database or Logical Clock, and with or without slave preserve commit order), the sync-ing of relay logs, the presence of binary logs, log-slave-updates and their sync-ing. This is very complicated stuff and even the manual is confused about it.
In this talk, I will explain the impact of above and help you finding the path to crash safety nirvana. I will also give details about replication internals, so you might learn a thing or two.
FPGA based 10G Performance Tester for HW OpenFlow SwitchYutaka Yasuda
SDN operators need to measure the performance of OF HW switch on their site. Cause there is 1000 times differences in latency, depends on the specified flow entry. ASIC can forward in several μsecs but the software (CPU) may take msec.
To protect yourself from unexpected performance plunge, monitor your switches healthiness on your site.
Riding the Binlog: an in Deep Dissection of the Replication StreamJean-François Gagné
Binary Logs are the cornerstone of MySQL Replication, but is it fully understood ? To start apprehending this, we can think of the binary logs as a transport for a Stream of Transactions. Traveling from master to slave, sometimes via Intermediate Masters, this stream evolves: it can shrink by the application of filters, can grow by the addition of slave-local transactions, and two streams can merge by the usage of multi-source replication. After presenting the binary logs Stream Model, the different MySQL use-cases will be mapped to the model, which can serve as a validation of the model. After this validation, the model will be used to make prediction on new use-cases/features that could emerge in the future.
Redis - for duplicate detection on real time streamCodemotion
Roberto "frank" Franchini presenta a Codemotion Techmeetup Torino Redis, un data structure server che può utilizzare come chiavi stringhe, hashes, lists, sets, sorted sets, bitmaps e hyperloglogs
.
One of the great challenges of of monitoring any large cluster is how much data to collect and how often to collect it. Those responsible for managing the cloud infrastructure want to see everything collected centrally which places limits on how much and how often. Developers on the other hand want to see as much detail as they can at as high a frequency as reasonable without impacting the overall cloud performance.
To address what seems to be conflicting requirements, we've chosen a hybrid model at HP. Like many others, we have a centralized monitoring system that records a set of key system metrics for all servers at the granularity of 1 minute, but at the same time we do fine-grained local monitoring on each server of hundreds of metrics every second so when there are problems that need more details than are available centrally, one can go to the servers in question to see exactly what was going on at any specific time.
The tool of choice for this fine-grained monitoring is the open source tool collectl, which additionally has an extensible api. It is through this api that we've developed a swift monitoring capability to not only capture the number of gets, put, etc every second, but using collectl's colmux utility, we can also display these in a top-like formact to see exactly what all the object and/or proxy servers are doing in real-time.
We've also developer a second cability that allows one to see what the Virtual Machines are doing on each compute node in terms of CPU, disk and network traffic. This data can also be displayed in real-time with colmux.
This talk will briefly introduce the audience to collectl's capabilities but more importantly show how it's used to augment any existing centralized monitoring infrastructure.
Speakers
Mark Seger
You Can't Correlate what you don't have - ArcSight Protect 2011Scott Carlson
In this presentation we discuss gathering data with syslog-ng in order to properly feed your SIEM system such as ArcSight ESM. This presentation is from HP/ArcSight Protect 2011.
Similar to Profiling the logwriter and database writer (20)
Successfully convince people with data visualizationKyle Hailey
Successfully convince people with data visualization
video of presentation available at https://www.youtube.com/watch?v=3PKjNnt14mk
from Data by the Bay conference
top nidhi software solution freedownloadvrstrong314
This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.
Your Digital Assistant.
Making complex approach simple. Straightforward process saves time. No more waiting to connect with people that matter to you. Safety first is not a cliché - Securely protect information in cloud storage to prevent any third party from accessing data.
Would you rather make your visitors feel burdened by making them wait? Or choose VizMan for a stress-free experience? VizMan is an automated visitor management system that works for any industries not limited to factories, societies, government institutes, and warehouses. A new age contactless way of logging information of visitors, employees, packages, and vehicles. VizMan is a digital logbook so it deters unnecessary use of paper or space since there is no requirement of bundles of registers that is left to collect dust in a corner of a room. Visitor’s essential details, helps in scheduling meetings for visitors and employees, and assists in supervising the attendance of the employees. With VizMan, visitors don’t need to wait for hours in long queues. VizMan handles visitors with the value they deserve because we know time is important to you.
Feasible Features
One Subscription, Four Modules – Admin, Employee, Receptionist, and Gatekeeper ensures confidentiality and prevents data from being manipulated
User Friendly – can be easily used on Android, iOS, and Web Interface
Multiple Accessibility – Log in through any device from any place at any time
One app for all industries – a Visitor Management System that works for any organisation.
Stress-free Sign-up
Visitor is registered and checked-in by the Receptionist
Host gets a notification, where they opt to Approve the meeting
Host notifies the Receptionist of the end of the meeting
Visitor is checked-out by the Receptionist
Host enters notes and remarks of the meeting
Customizable Components
Scheduling Meetings – Host can invite visitors for meetings and also approve, reject and reschedule meetings
Single/Bulk invites – Invitations can be sent individually to a visitor or collectively to many visitors
VIP Visitors – Additional security of data for VIP visitors to avoid misuse of information
Courier Management – Keeps a check on deliveries like commodities being delivered in and out of establishments
Alerts & Notifications – Get notified on SMS, email, and application
Parking Management – Manage availability of parking space
Individual log-in – Every user has their own log-in id
Visitor/Meeting Analytics – Evaluate notes and remarks of the meeting stored in the system
Visitor Management System is a secure and user friendly database manager that records, filters, tracks the visitors to your organization.
"Secure Your Premises with VizMan (VMS) – Get It Now"
Advanced Flow Concepts Every Developer Should KnowPeter Caitens
Tim Combridge from Sensible Giraffe and Salesforce Ben presents some important tips that all developers should know when dealing with Flows in Salesforce.
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Hivelance Technology
Cryptocurrency trading bots are computer programs designed to automate buying, selling, and managing cryptocurrency transactions. These bots utilize advanced algorithms and machine learning techniques to analyze market data, identify trading opportunities, and execute trades on behalf of their users. By automating the decision-making process, crypto trading bots can react to market changes faster than human traders
Hivelance, a leading provider of cryptocurrency trading bot development services, stands out as the premier choice for crypto traders and developers. Hivelance boasts a team of seasoned cryptocurrency experts and software engineers who deeply understand the crypto market and the latest trends in automated trading, Hivelance leverages the latest technologies and tools in the industry, including advanced AI and machine learning algorithms, to create highly efficient and adaptable crypto trading bots
Understanding Globus Data Transfers with NetSageGlobus
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?XfilesPro
Worried about document security while sharing them in Salesforce? Fret no more! Here are the top-notch security standards XfilesPro upholds to ensure strong security for your Salesforce documents while sharing with internal or external people.
To learn more, read the blog: https://www.xfilespro.com/how-does-xfilespro-make-document-sharing-secure-and-seamless-in-salesforce/
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
Modern design is crucial in today's digital environment, and this is especially true for SharePoint intranets. The design of these digital hubs is critical to user engagement and productivity enhancement. They are the cornerstone of internal collaboration and interaction within enterprises.
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
Why React Native as a Strategic Advantage for Startup Innovation.pdfayushiqss
Do you know that React Native is being increasingly adopted by startups as well as big companies in the mobile app development industry? Big names like Facebook, Instagram, and Pinterest have already integrated this robust open-source framework.
In fact, according to a report by Statista, the number of React Native developers has been steadily increasing over the years, reaching an estimated 1.9 million by the end of 2024. This means that the demand for this framework in the job market has been growing making it a valuable skill.
But what makes React Native so popular for mobile application development? It offers excellent cross-platform capabilities among other benefits. This way, with React Native, developers can write code once and run it on both iOS and Android devices thus saving time and resources leading to shorter development cycles hence faster time-to-market for your app.
Let’s take the example of a startup, which wanted to release their app on both iOS and Android at once. Through the use of React Native they managed to create an app and bring it into the market within a very short period. This helped them gain an advantage over their competitors because they had access to a large user base who were able to generate revenue quickly for them.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
2. `whoami`
• Frits
Hoogland
• Working
with
Oracle
products
since
1996
!
• Blog:
hEp://fritshoogland.wordpress.com
• TwiEer:
@fritshoogland
• Email:
frits.hoogland@enkitec.com
• Oracle
ACE
Director
• OakTable
Member
2
3. Goals
&
prerequisites
• Goal:
learn
about
typical
behaviour
of
both
lgwr
and
dbwr,
both
visible
(wait
events)
and
inner-‐
working.
!
• Prerequisites:
– Understanding
of
(internal)
execu[on
of
C
programs.
– Understanding
of
Oracle
tracing
mechanisms.
– Understanding
of
interac[on
between
procs.
with
the
Oracle
database.
3
4. Test
system
• The
tests
and
inves[ga[on
is
done
in
a
VM:
– Host:
Mac
OSX
10.9
/
VMWare
Fusion
6.0.2.
– VM:
Oracle
Linux
x86_64
6u5
(UEK3
3.8.13).
– Oracle
Grid
11.2.0.4
with
ASM/External
redundancy.
– Oracle
database
11.2.0.4.
!
– Unless
specified
otherwise.
4
5. Logwriter,
concepts
guide
• From
the
concepts
guide:
– The
lgwr
manages
the
redolog
buffer
!
– The
lgwr
writes
all
redo
entries
that
have
been
copied
in
the
buffer
since
the
last
[me
it
wrote
when:
• User
commits.
• Logswitch.
• Three
seconds
since
last
write.
• Buffer
1/3th
full
or
1MB
filled.
• dbwr
must
write
modified
(‘dirty’)
buffers.
5
6. Logwriter,
idle
• The
general
behaviour
of
the
log
writer
can
easily
be
shown
by
pugng
a
10046/8
on
lgwr:
!
SYS@v11204 AS SYSDBA> @who
…
130,1,@1 2243 oracle@ol65-ora11204-asm.local (LGWR)
…
SYS@v11204 AS SYSDBA> oradebug setospid 2243
Oracle pid: 11, Unix process pid: 2243, image: oracle@ol65-ora11204-asm.local (LGWR)
SYS@v11204 AS SYSDBA> oradebug unlimit
Statement processed.
SYS@v11204 AS SYSDBA> oradebug event 10046 trace name context forever, level 8;
Statement processed.
6
9. Logwriter,
idle
• It
does
look
in
the
/proc
filesystem
to
the
‘stat’
file
of
a
certain
process:
!
open("/proc/2218/stat", O_RDONLY) = 21
read(21, "2218 (oracle) S 1 2218 2218 0 -1"..., 999) = 209
close(21)
!
• It
does
so
every
20th
[me
(20*3)=
60
sec.
• The
PID
is
PMON.
9
10. Logwriter,
idle
• Recap:
– In
an
idle
database.
– The
lgwr
sleeps
on
a
semaphore
for
3
seconds.
• Then
wakes
up,
and
sets
up
the
semaphore/sleep
again.
• Processes
sleeping
on
a
semaphore
do
not
spend
CPU
– Every
minute,
lgwr
reads
pmon's
process
stats.
– lgwr
doesn’t
write
if
there’s
no
need.
• But
what
happens
when
we
insert
a
row
of
data,
and
commit
that?
10
13. Logwriter,
commit
-‐
actual
13
time
semctl(458755, 15, SETVAL, 0x7fff00000001)
commit;
foreground
logwriter
semtimedop(458755, {{15, -1, 0}}, 1, {3, 0})
io_submit(139981752844288, 1,
{{0x7f5008e23480, 0, 1, 0, 256}})
no ‘log file sync’ wait!
No semtimedop()
No semctl()
io_getevents(139981752844288, 1, 128,
{{0x7f5008e23480, 0x7f5008e23480, 3584, 0}},
{600, 0})
14. Logwriter,
commit
• Inves[ga[on
shows:
– Foreground
scans
log
writer
progress
up
to
3
[mes.
• kcrf_commit_force()
>
kcscur3()
!
– If
its
data*
in
the
redo
log
buffer
is
not
wriEen:
• It
no[fies
the
lgwr
that
it
is
going
to
sleep
on
a
semaphore.
• sem[medop()
for
100ms,
un[l
posted
by
lgwr.
– If
its
data*
has
been
wriEen:
• No
need
to
wait
on
it.
• No
‘log
file
sync’
wait.
14
15. Logwriter,
commit
• Wait!!!
– This
(no
log
file
sync)
turned
out
to
be
an
edge
case.
• I
traced
the
kcrf_commit_force()
and
kcscur3()
calls
using
breaks
in
gdb.
!
– In
normal
situa[ons,
the
wait
will
appear.
• Depending
on
log
writer
and
FG
progress.
• The
sem[medop()
call
in
the
FG
can
be
absent.
– As
a
result,
lgwr
will
not
semctl()
15
17. adap[ve
log
file
sync
• Feature
of
Oracle
11.2
– Parameter
'_use_adap[ve_log_file_sync'
• Set
to
FALSE
up
to
11.2.0.2
• Set
to
TRUE
star[ng
from
11.2.0.3
• Third
value
‘POLLING_ONLY’
– Makes
Oracle
adap[vely
switch
between
‘post-‐wait’
and
polling.
– The
log
writer
writes
a
no[fica[on
in
its
logfile
if
it
switches
between
modes
(if
param
=
‘TRUE’)
17
19. Logwriter,
post-‐wait
vs.
polling
– No
wait
event
‘log
file
sync’
if:
• Lgwr
was
able
to
flush
the
commiEed
data
before
the
foreground
has
issued
kcscur3()
2/3
[mes
in
kcrf_commit_force().
– If
not,
the
foreground
starts
a
‘log
file
sync’
wait.
• If
in
“post-‐wait”
mode
(default),
it
will
record
it’s
wai[ng
state
in
the
post-‐wait
queue,
sleep
in
sem[medop()
for
100ms
at
a
[me,
wai[ng
to
be
posted
by
lgwr.
• If
in
“polling”
mode,
it
will
sleep
in
nanosleep()
for
computed
[me*,
then
check
lgwr
progress,
if
lgwr
write
has
progressed
beyond
its
commiEed
data
SCN:
end
wait,
else
start
sleeping
in
nanosleep()
again.
19
20. Logwriter
• The
main
task
of
lgwr
is
to
flush
data
in
the
logbuffer
to
disk.
– The
lgwr
is
idle
when
wai[ng
on
‘rdbms
ipc
message’.
– There
are
two
main*
indicators
of
lgwr
business:
• CPU
[me.
• Wait
event
‘log
file
parallel
write’.
!
!
• The
lgwr
needs
to
be
able
to
get
onto
the
CPU
in
order
to
do
process!
20
25. Logwriter
-‐
wri[ng
25
logwriter
log file parallel write
io_submit(139981752844288, n,
{{0x7f5008e23480, 0, 1, 0, 256}})
io_getevents(139981752844288, n, 128,
{{0x7f5008e23480, 0x7f5008e23480, 3584, 0}},
{0, 0})
io_getevents(139981752844288, n, 128,
{{0x7f5008e23480, 0x7f5008e23480, 3584, 0}},
{600, 0})
With the linux ‘strace’ utility,
the non-blocking syscall is visible
OR
the blocking one syscall is visible.
26. • The
Logwriter
-‐
wri[ng
event
‘log
file
parallel
write’
is
an
indicator
of
IO
wait
[me
for
the
lgwr.
– NOT
IO
latency
[me!!
26
io_submit() io_getevents() io_getevents()
log file parallel write: 9.85ms
real IO time: 7ms
real IO time: 10ms
27. Logwriter
wait
events
• rdbms
ipc
message
– [meout:
300
(cen[seconds;
3
seconds).
– process
sleeping
~
3
seconds
on
semaphore.
!
• log
file
parallel
write
– files:
number
of
log
file
members.
– blocks:
total
number
of
log
blocks
wriEen.
– requests:
?
• I’ve
seen
this
differ
from
the
actual
numer
of
IO
requests.
27
28. Logwriter
wait
events
• Let’s
switch
the
database
to
synchronous
IO.
– Some
plaxorms
have
difficulty
with
AIO
(HPUX!)
– Got
to
check
if
your
config
does
use
AIO.
• Found
out
by
accident
that
ASM+NFS
has
no
AIO
by
default.
!
– Good
to
understand
what
the
absence
of
AIO
means.
!
• If
you
can’t
use
AIO
today,
you
are
doing
it
WRONG!
28
30. log
file
parallel
write
30
disk_asynch_io=FALSE
(no
AIO)
Let’s
add
100ms
to
the
IOs
(shell
sleep
0.1)
!
kslwtbctx
semtimedop - 458755 semid, timeout: $3 = {tv_sec = 2, tv_nsec = 900000000}
kslwtectx -- Previous wait time: 2905568: rdbms ipc message
!
pwrite64 - fd, size - 256,512
>sleep 0.1
pwrite64 - fd, size - 256,512
>sleep 0.1
!
kslwtbctx
kslwtectx -- Previous wait time: 545: log file parallel write
Two writes again.
In the break a sleep of 100ms is
added.
This should make the timing at
least 200’000
The timing is 545 (0.5ms):
timing is off.
31. log
file
parallel
write
• Conclusion:
– For
at
least
Oracle
version
11.2.
– When
synchronous
IO
(pwrite64())
is
issued.
• disk_asynch_io
=
FALSE
(ASM)
• filesystemio_op[ons
!=
“setall”
or
“asynch”
– The
wait
event
does
not
[me
the
IO
requests.
!
• How
about
the
other
log
writer
wait
events?
31
32. control
file
sequen[al
read
32
disk_asynch_io=FALSE
(no
AIO)
!
kslwtbctx
pread64 - fd, size - 256,16384
>sleep 0.1
kslwtectx -- Previous wait time: 100323: control file sequential read
!
This
event
is
correctly
[med.
33. control
file
parallel
write
33
disk_asynch_io=FALSE
(no
AIO)
!
pwrite64 - fd, size - 256,16384
>sleep 0.1
kslwtbctx
kslwtectx -- Previous wait time: 705: control file parallel write
!
This
event
is
incorrectly
[med!
34. log
file
single
write
34
disk_asynch_io=FALSE
(no
AIO)
!
kslwtbctx
pwrite64 - fd, size - 256,512
>sleep 0.1
kslwtectx -- Previous wait time: 104594: log file single write
!
This
event
is
correctly
[med.
35. Logwriter
wait
events
logswitch
• Some
of
these
waits
typically
show
up
during
a
logswitch.
– This
are
all
the
waits
which
are
normally
seen:
• os
thread
startup
(semctl()-‐sem[medop())
• control
file
sequen[al
read
(pread64())
• control
file
parallel
write
(io_submit()-‐io_getevents())
• log
file
sequen[al
read
(pread64())
• log
file
single
write
(pwrite64())
• KSV
master
wait
(semctl()
post
to
dbwr)
!
• This
is
with
AIO
enabled!
35
36. Logwriter,
[meout
message
• Warning:
!
Warning: log write elapsed time 523ms, size 2760KB
!
• Printed
in
logwriter
tracefile
(NOT
alert.log)
• This
is
instrumented
with
the
‘log
write
parallel
write’
event.
• Threshold
set
with
parameter:
– _side_channel_batch_[meout_ms
(500ms)
36
37. Logwriter,
[meout
message
• Warning
(RAC!):
!
Warning: log write broadcast wait time 2913ms (SCN 0xb86.cd638134)
!
• Printed
in
logwriter
tracefile
(NOT
alert.log)
• This
is
instrumented
with
the
‘wait
for
scn
ack’
event.
37
38. Logwriter:
disable
logging
• The
“forbidden
switch”:
_disable_logging
– Do
not
use
this
for
anything
else
than
tests!
!
• Everything
is
done
the
same
—
no
magic
– Except
the
write
by
the
lgwr
to
the
logfiles
– No
‘log
file
parallel
write’
– Redo/control/data
files
are
synced
with
shut
normal
!
• A
way
to
test
if
lgwr
IO
influences
db
processing
38
40. Logwriter:
exadata
kslwtbctx
semtimedop - 3309577 semid, timeout: $24 = {tv_sec = 2, tv_nsec = 970000000}
kslwtectx -- Previous wait time: 2973630: rdbms ipc message
!
$25 = "oss_write"
$26 = "oss_write"
$27 = "oss_write"
$28 = "oss_write"
!
kslwtbctx
$29 = "oss_wait"
$30 = "oss_wait"
$31 = "oss_wait"
$32 = "oss_wait"
kslwtectx -- Previous wait time: 2956: log file parallel write
!
kslwtbctx
semtimedop - 3309577 semid, timeout: $33 = {tv_sec = 3, tv_nsec = 0}
kslwtectx -- Previous wait time: 3004075: rdbms ipc message
40
The dbwr semaphore sleep.
The writes are issued here.
There is no io_submit like wait. This is not timed.
The wait is log file parallel write, identical to non-exadata.
It seems to wait for all issued IOs
41. Database
writer
• From
the
Oracle
11.2
concepts
guide:
– The
DBWn
process
writes
dirty
buffers
to
disk
under
the
following
condi[ons:
• When
a
server
process
cannot
find
a
clean
reusable
buffer
a|er
scanning
a
threshold
of
buffers,
it
signals
DBWn
to
write.
DBWn
writes
dirty
buffers
to
disk
asynchronously
if
possible
while
performing
other
processing.
• DBWn
periodically
writes
buffers
to
advance
the
checkpoint,
which
is
the
posi[on
in
the
redo
thread
from
which
instance
recovery
begins.
The
log
posi[on
of
the
checkpoint
is
determined
by
the
oldest
dirty
buffer
in
the
buffer
cache.
41
44. Database
writer,
idle
• It
does
look
in
the
/proc
filesystem
to
the
‘stat’
file
of
a
certain
process:
!
open("/proc/2218/stat", O_RDONLY) = 21
read(21, "2218 (oracle) S 1 2218 2218 0 -1"..., 999) = 209
close(21)
!
• It
does
so
every
20th
[me
(20*3)=
60
sec.
• The
PID
is
PMON.
44
45. Database
writer,
idle
• Recap:
– In
an
idle
database.
– The
dbwr
sleeps
on
a
semaphore
for
3
seconds.
• Then
wakes
up,
and
sets
up
the
semaphore/sleep
again.
• Processes
sleeping
on
a
semaphore
do
not
spend
CPU
– Every
minute,
dbwr
reads
pmon's
process
stats.
– dbwr
doesn’t
write
if
there’s
no
need.
45
46. Database
writer,
force
write
• We
can
force
the
dbwr
to
write:
– Dirty
some
blocks
(insert
a
row
into
a
table).
– Force
a
thread
checkpoint
(alter
system
checkpoint).
!
!
!
!
!
*
There
are
mul[ple
ways,
this
is
one
of
them.
46
47. Database
writer,
force
write
10046/8
trace:
*** 2014-01-03 03:37:47.473
WAIT #0: nam='rdbms ipc message' ela= 3000957 timeout=300 p2=0 p3=0 obj#=-1
tim=1388716667473024
!
*** 2014-01-03 03:37:49.735
WAIT #0: nam='rdbms ipc message' ela= 2261867 timeout=300 p2=0 p3=0 obj#=-1
tim=1388716669735046
WAIT #0: nam='db file async I/O submit' ela= 0 requests=3 interrupt=0 timeout=0
obj#=-1 tim=1388716669735493
WAIT #0: nam='db file parallel write' ela= 21 requests=1 interrupt=0
timeout=2147483647 obj#=-1 tim=1388716669735566
!
*** 2014-01-03 03:37:50.465
WAIT #0: nam='rdbms ipc message' ela= 729110 timeout=73 p2=0 p3=0 obj#=-1
tim=1388716670464967
47
elapsed time = 2.26 sec.
So the dbwr is posted!
db file async I/O submit?!
It looks like the io_submit() call is instrumented for the dbwr!
But what does ‘requests=3’ mean for a single row update
checkpoint?
And the write, via the event ‘db file parallel write’.
48. dbwr,
sql_trace
+
strace
• Let’s
take
a
look
at
the
Oracle
wait
events,
together
with
the
actual
system
calls.
!
• That
is:
– Segng
a
10046/8
event
for
trace
and
waits.
– Execute
strace
with
‘-‐e
write=all
-‐e
all’
48
51. dbwr,
sql_trace
+
strace
io_submit(140195085938688, 3, {{0x7f81b622ab10, 0, 1, 0, 256}, {0x7f81b622a8a0, 0,
1, 0, 256}, {0x7f81b622a630, 0, 1, 0, 256}}) = 3
write(13, "WAIT #0: nam='db file async I/O "..., 108) = 108
| 00000 57 41 49 54 20 23 30 3a 20 6e 61 6d 3d 27 64 62 WAIT #0: nam='db |
| 00010 20 66 69 6c 65 20 61 73 79 6e 63 20 49 2f 4f 20 file as ync I/O |
| 00020 73 75 62 6d 69 74 27 20 65 6c 61 3d 20 31 20 72 submit' ela= 1 r |
| 00030 65 71 75 65 73 74 73 3d 33 20 69 6e 74 65 72 72 equests= 3 interr |
| 00040 75 70 74 3d 30 20 74 69 6d 65 6f 75 74 3d 30 20 upt=0 ti meout=0 |
| 00050 6f 62 6a 23 3d 2d 31 20 74 69 6d 3d 31 33 38 38 obj#=-1 tim=1388 |
| 00060 39 37 37 36 35 31 38 30 34 32 36 31 97765180 4261 |
io_getevents(140195085938688, 1, 128, {{0x7f81b622ab10, 0x7f81b622ab10, 8192, 0},
{0x7f81b622a8a0, 0x7f81b622a8a0, 8192, 0}, {0x7f81b622a630, 0x7f81b622a630, 8192,
0}}, {600, 0}) = 3
write(13, "WAIT #0: nam='db file parallel w"..., 116) = 116
| 00000 57 41 49 54 20 23 30 3a 20 6e 61 6d 3d 27 64 62 WAIT #0: nam='db |
| 00010 20 66 69 6c 65 20 70 61 72 61 6c 6c 65 6c 20 77 file pa rallel w |
| 00020 72 69 74 65 27 20 65 6c 61 3d 20 35 38 20 72 65 rite' el a= 58 re |
| 00030 71 75 65 73 74 73 3d 31 20 69 6e 74 65 72 72 75 quests=1 interru |
| 00040 70 74 3d 30 20 74 69 6d 65 6f 75 74 3d 32 31 34 pt=0 tim eout=214 |
| 00050 37 34 38 33 36 34 37 20 6f 62 6a 23 3d 2d 31 20 7483647 obj#=-1 |
| 00060 74 69 6d 3d 31 33 38 38 39 37 37 36 35 31 38 30 tim=1388 97765180 |
| 00070 34 35 37 39 4579 |
51
This is the MINIMAL number of requests to reap before successful.
(min_nr - see man io_getevents) ?
?
?
The timeout for io_getevents() is set to 600 seconds.
struct timespec { sec, nsec }
Despite only needing 1 request, this call returned all 3.
This information is NOT EXTERNALISED (!!)
52. dbwr,
db
file
async
I/O
submit
• Let’s
take
a
look
at
the
what
the
documenta;on
says
about
“db
file
async
I/O
submit”:
!
52
(That’s
right…nothing)
!
53. dbwr,
db
file
async
I/O
submit
• My
Oracle
Support
on
“db
file
async
I/O
submit”:
!
'db file async I/O submit' when FILESYSTEMIO_OPTIONS=NONE
[Article ID 1274737.1]
How To Address High Wait Times for the 'Direct Path Write Temp ' Wait Event
[Article ID 1576956.1]
!
• Both
don’t
describe
what
this
event
is.
• 1st
note
is
only
for
filesystemio_op;ons=NONE
and
describes
the
event
not
being
tracked
prior
to
version
11.2.0.2.
53
54. dbwr,
db
file
async
I/O
submit
• So
the
ques;on
is:
!
– What
DOES
the
event
“db
file
async
I/O
submit”
mean?
!
54
• The
obvious
answer
is:
– Instrumenta;on
of
the
io_submit()
call.
• My
answer
is:
– Don’t
know.
– But
NOT
the
instrumenta;on
of
io_submit().
55. dbwr,
db
file
async
I/O
submit
• This
is
a
trace
of
the
relevant
C
func;ons:
!
kslwtbctx
kslwtectx -- Previous wait time: 236317: rdbms ipc message
!
io_submit - 3,45e5a000 - nr,ctx
kslwtbctx
kslwtectx -- Previous wait time: 688: db file async I/O submit
!
kslwtbctx
io_getevents - 1,45e5a000 - minnr,ctx,timeout: $3 = {tv_sec = 600, tv_nsec = 0}
skgfr_return64 - 3 IOs returned
kslwtectx -- Previous wait time: 9604: db file parallel write
55
Waiting on a semaphore to be posted.
io_submit() for 3 IOs
The begin of the wait starts AFTER the io_submit()?
io_getevevents() is properly timed. min_nr=1, got 3 IOs
56. dbwr,
db
file
async
I/O
submit
• Trace
with
sleep
0.1
in
the
break
on
io_submit()
!
kslwtbctx
kslwtectx -- Previous wait time: 385794: rdbms ipc message
!
io_submit - 3,45e5a000 - nr,ctx
> sleep 0.1
kslwtbctx
kslwtectx -- Previous wait time: 428: db file async I/O submit
!
kslwtbctx
io_getevents - 1,45e5a000 - minnr,ctx,timeout: $37 = {tv_sec = 600, tv_nsec = 0}
skgfr_return64 - 3 IOs returned
kslwtectx -- Previous wait time: 8053: db file parallel write
56
Waiting on a semaphore to be posted.
io_submit() for 3 IOs + sleep of 100’000
Wait time too low. io_submit() is not timed.
57. dbwr,
db
file
parallel
write
• Let’s
look
at
the
“db
file
parallel
write”
event.
57
58. dbwr,
db
file
parallel
write
• Descrip;on
from
the
Reference
Guide:
!
db file parallel write
This event occurs in the DBWR. It indicates that the DBWR is performing a parallel write to
files and blocks. When the last I/O has gone to disk, the wait ends.
Wait Time: Wait until all of the I/Os are completed
Parameter
Description
requests: This indicates the total number of I/O requests, which will be the same as blocks
interrupt:
timeout: This indicates the timeout value in hundredths of a second to wait for the I/O
completion.
58
Correct Correct…but only if AIO is enabled.
Incorrect
Incorrect
Incorrect
Empty?
Probably incorrect.
Or does a timeout of
2’147’483’647
/100/60/60/24=
248.55 days
Make sense to
*anybody*?
59. dbwr,
db
file
parallel
write
• Recap
of
previous
traced
calls:
!
kslwtbctx
kslwtectx -- Previous wait time: 236317: rdbms ipc message
!
io_submit - 3,45e5a000 - nr,ctx
kslwtbctx
kslwtectx -- Previous wait time: 688: db file async I/O submit
!
kslwtbctx
io_getevents - 1,45e5a000 - minnr,ctx,timeout: $3 = {tv_sec = 600, tv_nsec = 0}
skgfr_return64 - 3 IOs returned
kslwtectx -- Previous wait time: 9604: db file parallel write
59
So….how about severely limiting OS IO capacity and see what happens?
60. dbwr,
db
file
parallel
write
• Database
writer
—
severely
limited
IO
(
1
IOPS)
io_submit - 366,45e5a000 - nr,ctx
kslwtbctx
kslwtectx -- Previous wait time: 1070: db file async I/O submit
!
kslwtbctx
io_getevents - 100,45e5a000 - minnr,ctx,timeout: $7 = {tv_sec = 600, tv_nsec = 0}
skgfr_return64 - 100 IOs returned
kslwtectx -- Previous wait time: 109334845: db file parallel write
!
io_getevents - 128,45e5a000 - minnr,ctx,timeout: $8 = {tv_sec = 0, tv_nsec = 0}
io_getevents - 128,45e5a000 - minnr,ctx,timeout: $9 = {tv_sec = 0, tv_nsec = 0}
!
io_submit - 73,45e5a000 - nr,ctx
kslwtbctx
kslwtectx -- Previous wait time: 486: db file async I/O submit
60
366 IO requests are submitted onto the OS.
But only 100 IOs are needed to satisfy io_getevents()
Which it does in this case… leaving outstanding IOs
The dbwr starts issuing non-blocking calls to reap IOs!
It seems to be always 2 if outstanding IOs remain.
Minnr = # outstanding IOs, max 128.
61. dbwr,
db
file
parallel
write
• This
got
me
thinking…
• The
dbwr
submits
the
IOs
it
needs
to
write.
!
• But
it
waits
for
a
variable
amount
of
IOs
to
finish.
– Wait
event
‘db
file
parallel
write’.
– Amount
seems
33-‐25%
of
submifed
IOs*
– Aher
that,
2
tries
to
reap
the
remaining
IOs*
– Then
either
submit
again,
DFPW
un;l
IOs
reaped
or
back
to
sleeping
on
semaphore.
61
62. dbwr,
db
file
parallel
write
• This
means
‘db
file
parallel
write’
is
not:
– Physical
IO
indicator.
– IO
latency
;ming
!
• I’ve
come
to
the
conclusion
that
the
blocking
io_getevents
call
for
a
number
of
IOs
of
the
dbwr
is
an
IO
limiter.
• …and
’db
file
parallel
write’
is
the
;ming
of
it.
62
63. dbwr,
synchronous
IO
• Let’s
turn
AIO
off
again.
– To
simulate
this,
I’ve
set
disk_asynch_io
to
FALSE.
!
• And
set
a
10046/8
trace
and
strace
on
the
dbwr.
• And
issue
the
SQLs
as
before:
– insert
into
followed
by
commit
– alter
system
checkpoint
63
64. dbwr,
synchronous
IO
pwrite(256, "62420020726100hG1700020063R001000009U10[G170"...,
8192, 8053121024) = 8192
pwrite(256, "22420024603000hG1700014220T0030r0j
30020301717"..., 8192, 7443103744) = 8192
pwrite(256, "&2420024003000hG
170002437221600000000000000"..., 8192, 7443054592) = 8192
write(11, "WAIT #0: nam='db file parallel w"..., 107) = 107
| 00000 57 41 49 54 20 23 30 3a 20 6e 61 6d 3d 27 64 62 WAIT #0: nam='db |
| 00010 20 66 69 6c 65 20 70 61 72 61 6c 6c 65 6c 20 77 file pa rallel w |
| 00020 72 69 74 65 27 20 65 6c 61 3d 20 32 30 20 72 65 rite' el a= 20 re |
| 00030 71 75 65 73 74 73 3d 33 20 69 6e 74 65 72 72 75 quests=3 interru |
| 00040 70 74 3d 30 20 74 69 6d 65 6f 75 74 3d 30 20 6f pt=0 tim eout=0 o |
| 00050 62 6a 23 3d 2d 31 20 74 69 6d 3d 31 33 38 39 38 bj#=-1 t im=13898 |
| 00060 30 32 32 38 31 34 35 36 37 34 30 02281456 740 |
write(11, "WAIT #0: nam='db file parallel w"..., 106) = 106
| 00000 57 41 49 54 20 23 30 3a 20 6e 61 6d 3d 27 64 62 WAIT #0: nam='db |
| 00010 20 66 69 6c 65 20 70 61 72 61 6c 6c 65 6c 20 77 file pa rallel w |
| 00020 72 69 74 65 27 20 65 6c 61 3d 20 31 20 72 65 71 rite' el a= 1 req |
| 00030 75 65 73 74 73 3d 33 20 69 6e 74 65 72 72 75 70 uests=3 interrup |
| 00040 74 3d 30 20 74 69 6d 65 6f 75 74 3d 30 20 6f 62 t=0 time out=0 ob |
| 00050 6a 23 3d 2d 31 20 74 69 6d 3d 31 33 38 39 38 30 j#=-1 ti m=138980 |
| 00060 32 32 38 31 34 35 38 34 39 32 22814584 92 |
64
3 pwrite() calls. This is synchronous IO!
The db file parallel write wait event shows 3 requests!
But why a second db file parallel write wait event?
65. dbwr,
synchronous
IO
• There’s
no
‘db
file
async
I/O
submit’
wait
anymore.
– Which
is
good,
because
SIO
has
no
submit
phase.
• The
‘db
file
parallel
write’
waits
seem
suspicious.
– It
seems
like
the
wait
for
DFPW
is
issued
twice.
– Further
inves;ga;on
shows
that
it
does.
• My
guess
this
is
a
bug
in
the
sync.
IO
implementa;on.
• Let’s
look
a
level
deeper
and
see
if
there’s
more
to
see.
65
66. dbwr,
synchronous
IO
kslwtbctx
semtimedop - 458755 semid, timeout: $18 = {tv_sec = 3, tv_nsec = 0}
kslwtectx -- Previous wait time: 1239214: rdbms ipc message
!
pwrite64 - fd, size - 256,8192
pwrite64 - fd, size - 256,8192
pwrite64 - fd, size - 256,8192
!
kslwtbctx
kslwtectx -- Previous wait time: 949: db file parallel write
!
kslwtbctx
kslwtectx -- Previous wait time: 650: db file parallel write
!
kslwtbctx
semtimedop - 458755 semid, timeout: $19 = {tv_sec = 1, tv_nsec = 620000000}
66
This is clearly the semaphore being posted: timeout=3s,
wait time = 1239,2ms
3 IO’s in serial using pwrite().
The only possibility if there isn’t AIO of course.
Two db file parallel write (which aren’t parallel) for which
both the begin of the waits are started AFTER the IO (!!)
After the IOs are done, the dbwr continues sleeping.
67. dbwr,
synchronous
IO
• Let’s
do
the
same
trick
as
done
earlier:
– In
gdb,
add
“shell
sleep
0.1”
to
the
pwrite
call.
– This
makes
the
execu;on
of
this
call
take
100ms
longer.
– To
see
if
there’s
s;ll
some
way
Oracle
;mes
it
properly.
67
68. dbwr,
synchronous
IO
kslwtbctx
semtimedop - 458755 semid, timeout: $23 = {tv_sec = 3, tv_nsec = 0}
kslwtectx -- Previous wait time: 92080: rdbms ipc message
!
pwrite64 - fd, size - 256,8192
> shell sleep 0.1
pwrite64 - fd, size - 256,8192
> shell sleep 0.1
pwrite64 - fd, size - 256,8192
> shell sleep 0.1
!
kslwtbctx
kslwtectx -- Previous wait time: 478: db file parallel write
!
kslwtbctx
kslwtectx -- Previous wait time: 495: db file parallel write
!
kslwtbctx
semtimedop - 458755 semid, timeout: $24 = {tv_sec = 2, tv_nsec = 460000000}
68
The 3 IOs again, each sleeps in pwrite() for 100ms (0.1s)
Yet the ‘db file parallel write’ wait shows a waiting time of 478;
which is 0.478ms: the timing is wrong.
69. dbwr,
synchronous
IO
• So,
my
conclusion
on
the
wait
events
for
the
dbwr
with
synchronous
IO:
– The
events
are
not
properly
;med
– It
seems
like
the
wait
for
DFPW
is
issued
twice.
– Further
inves;ga;on
shows
that
it
does.
• My
guess
this
is
a
bug
in
the
sync.
IO
implementa;on.
69
71. dbwr:
exadata
kslwtbctx
semtimedop - 3309577 semid, timeout: $389 = {tv_sec = 3, tv_nsec = 0}
kslwtectx -- Previous wait time: 1266041: rdbms ipc message
$390 = "oss_write"
$391 = "oss_write"
$392 = "oss_write"
$393 = "oss_write"
$394 = "oss_write"
$395 = "oss_write"
!
kslwtbctx
kslwtectx -- Previous wait time: 684: db file async I/O submit
!
kslwtbctx
$396 = "oss_wait"
$397 = "oss_wait"
kslwtectx -- Previous wait time: 2001: db file parallel write
$398 = "oss_wait"
$399 = "oss_wait"
$400 = "oss_wait"
$401 = "oss_wait"
!
semctl - 3309577,23,16 - semid, semnum, cmd
kslwtbctx
semtimedop - 3309577 semid, timeout: $402 = {tv_sec = 1, tv_nsec = 630000000}
kslwtectx -- Previous wait time: 1634299: rdbms ipc message
71
The dbwr semaphore sleep.
The writes are issued here.
This is not timed.
There is the db file async I/O submit.
Again, it doesn’t seem to time any of the typical IO calls!
And there we got the db file parallel write.
It does seem to always time two oss_wait() calls…
But it looks for more IOs to finish, alike the trailing io_getevents() calls.
I am quite sure oss_wait() is a blocking call…
72. Conclusion
• Logwriter:
– When
idle,
is
sleeping
on
a
semaphore/rdbms
ipc
message
– Gets
posted
with
semctl()
to
do
work.
– Only
writes
when
it
needs
to
do
so.
– Version
11.2.0.3:
two
methods
for
pos;ng
FGs:
– Polling
and
post/wait.
– Post/wait
is
default,
might
switch
to
polling.
– No;fica;on
of
switch
is
in
log
writer
trace
file.
– Polling/nanosleep()
;me
is
variable.
72
73. Conclusion
• Logwriter:
– Log
file
parallel
write
– AIO:
two
io_getevents()
calls.
– AIO:
;me
wai;ng
for
all
lgwr
submifed
IOs
to
finish.
– Not
IO
latency
-me!
– SIO:
does
not
do
parallel
writes,
but
serial.
– SIO:
does
not
;me
IO.
73
74. Conclusion
• Logwriter:
– Wait
event
IO
;ming:
– All
the
‘*
parallel
read’
and
‘*
parallel
write’
events
do
not
seem
to
;me
IO
correctly
with
synchronous
IO.
– All
the
events
which
cover
single
block
IOs
do
use
synchronous
IO
calls,
even
with
asynchronous
IO
set.
– Logwriter
writes
a
warning
when
IO
;me
and
SCN
broadcast
ack
;me
exceeds
500ms
in
the
log
writer
trace
file.
– _disable_logging
*only*
disables
write
to
logs.
74
75. Conclusion
• Database
writer:
– When
idle,
is
sleeping
on
a
semaphore/rdbms
ipc
message
– Gets
posted
with
semctl()
to
do
work.
– Only
writes
when
it
needs
to
do
so.
– Since
version
11.2.0.2,
event
‘db
file
async
I/O
submit’:
– Is
not
shown
with
synchronous
I/O.
– Shows
the
actual
amount
of
IOs
submifed.
– Does
not
;me
io_submit()
– Unknown
what
or
if
it
;mes
something.
75
76. Conclusion
• Database
writer:
– Event
‘db
file
parallel
write’:
– Shows
the
minimal
number
io_getevents()
waits
for.
– The
number
of
requests
it
waits
for
varies,
but
mostly
seems
to
be
~
25-‐33%
of
submifed
IOs.
– Aher
the
;med,
blocking,
io_getevents()
call,
it
issues
two
non-‐blocking
io_getevents()
calls
for
the
remaining
non-‐reaped
IOs,
if
any.
– My
current
idea
is
the
blocking
io_getevents()
call
is
an
IO
throfle
mechanism.
76
77. Conclusion
• Database
writer:
– Event
‘db
file
parallel
write’,
with
synchronous
IO:
– pwrite64()
calls
are
issued
serially.
– These
are
not
;med.
– The
event
is
triggered
twice.
!
– On
exadata,
two
out
of
the
total
number
of
oss_wait()
calls
are
;med
with
the
event
‘db
file
parallel
write’.
77
78. Update:
Oracle
12.1.0.2
• I
tried
to
present
this
findings
to
Oracle
Support
– With
no
effect…
!
• Luckily
I
could
to
raise
awareness
of
the
IO
;ming
inefficiencies
inside
Oracle.
– With
as
result
that
the
IO
;ming
changed
with
12.1.0.2.
– I
like
to
think
I
am
responsible
for
that,
but
it
might
be
just
coincidence.
78
79. log
file
parallel
write,
AIO
12.1.0.2
!
kslwtbctx
io_submit - 2,fe614000 - nr,ctx
fd: 256, nbytes: 512
fd: 256, nbytes: 512
io_getevents (libaio) - min_nr: 2, ctx: fe614000, timeout { 0,0 }
skgfr_return64 - 0 IOs returned
skgfr_return64 - 32 IOs returned
kslwtectx -> event=log file parallel write, wait_time=48873, files=2, blocks=2,
requests=2
!
• Change:
• Wait
includes
io_submit().
• Wait
;me:
• Time
for
all
log
writer
IOs
to
finish,
total
latency
;me
of
slowest
IO.
79
82. db
file
parallel
write,
AIO
12.1.0.2
!
kslwtbctx
io_getevents (libaio) - min_nr: 1, ctx: ef59000, timeout { 600,0 }
skgfr_return64 - 0 IOs returned
kslwtectx -> event=db file parallel write, wait_time=15012, requests=1, interrupt=0,
timeout=2147483647
!
• Change:
• Nothing.
• Wait
;me:
• Time
spend
in
io_getevents()
in
blocking
mode,
wai;ng
for
min_nr
IO
requests
to
finish.
82
83. db
file
parallel
write,
SIO
12.1.0.2
!
kslwtbctx
pwrite64 - fd, size - 256,8192
kslwtectx -> event=db file parallel write, wait_time=59805, requests=1, interrupt=0,
timeout=0
kslwtbctx
kslwtectx -> event=db file parallel write, wait_time=512, requests=1, interrupt=0,
timeout=0
kslwtbctx
!
• Change:
• pwrite()
calls
are
included
in
first
wait.
• Wait
;me:
• Total
;me
of
pwrite()
calls
executed
sequen;ally.
Second
wait
s;ll
there,
with
no
IO
calls,
same
#
of
requests.
83
84. others,
SIO
12.1.0.2
!
• I
haven’t
spend
a
lot
of
;me
looking
for
others.
!
• I
have
looked
at
‘direct
path
read’
however.
!
• This
wait
with
synchronous
IO
now
includes
the
actual
pread64()
call.
84