The document discusses the Optimized Row Columnar (ORC) file format. ORC provides column-based storage with optimizations for performance and compression. These include column-level compression, row groups for parallel reads, protobuf metadata, vectorization, and integration with Hive ACID transactions and Apache Tez for distributed execution. The document outlines the performance benefits ORC has provided for companies like Facebook and Spotify. It also details ongoing work to further optimize ORC using techniques like JDK8 SIMD, dynamic compression, row indexes, and low-latency analytics processing.
Learn from the author of SQLTXPLAIN the fundamentals of SQL Tuning: 1) Diagnostics Collection; 2) Root Cause Analysis (RCA); and 3) Remediation.
SQL Tuning is a complex and intimidating area of knowledge, and it requires years of frequent practice to master it. Nevertheless, there are some concepts and practices that are fundamental to succeed. From basic understanding of the Cost-based Optimizer (CBO) and the Execution Plans, to more advance topics such as Plan Stability and the caveats of using SQL Profiles and SQL Plan Baselines, this session is full of advice and experience sharing. Learn what works and what doesn't when it comes to SQL Tuning.
Participants of this session will also learn about several free tools (besides SQLTXPLAIN) that can be used to diagnose a SQL statement performing poorly, and some others to improve Execution Plan Stability.
Either if your are a novice DBA, or an experienced DBA or Developer, there will be something new for you on this session. And if this is your first encounter with SQL Tuning, at least you will learn the basic concepts and steps to succeed in your endeavor.
「TECH x GAME COLLEGE #21」 2019/06/05 登壇資料
https://techxgamecollege.connpass.com/event/130420/
【Laravelを用いたゲームサーバーのチューニング】
主にDB周りの話題で、
・速度を出す
・水平分散をする
という観点で
Laravelをゲームサーバーで用いた際にチューニングしてきたことを紹介します。
The document discusses the Optimized Row Columnar (ORC) file format. ORC provides column-based storage with optimizations for performance and compression. These include column-level compression, row groups for parallel reads, protobuf metadata, vectorization, and integration with Hive ACID transactions and Apache Tez for distributed execution. The document outlines the performance benefits ORC has provided for companies like Facebook and Spotify. It also details ongoing work to further optimize ORC using techniques like JDK8 SIMD, dynamic compression, row indexes, and low-latency analytics processing.
Learn from the author of SQLTXPLAIN the fundamentals of SQL Tuning: 1) Diagnostics Collection; 2) Root Cause Analysis (RCA); and 3) Remediation.
SQL Tuning is a complex and intimidating area of knowledge, and it requires years of frequent practice to master it. Nevertheless, there are some concepts and practices that are fundamental to succeed. From basic understanding of the Cost-based Optimizer (CBO) and the Execution Plans, to more advance topics such as Plan Stability and the caveats of using SQL Profiles and SQL Plan Baselines, this session is full of advice and experience sharing. Learn what works and what doesn't when it comes to SQL Tuning.
Participants of this session will also learn about several free tools (besides SQLTXPLAIN) that can be used to diagnose a SQL statement performing poorly, and some others to improve Execution Plan Stability.
Either if your are a novice DBA, or an experienced DBA or Developer, there will be something new for you on this session. And if this is your first encounter with SQL Tuning, at least you will learn the basic concepts and steps to succeed in your endeavor.
「TECH x GAME COLLEGE #21」 2019/06/05 登壇資料
https://techxgamecollege.connpass.com/event/130420/
【Laravelを用いたゲームサーバーのチューニング】
主にDB周りの話題で、
・速度を出す
・水平分散をする
という観点で
Laravelをゲームサーバーで用いた際にチューニングしてきたことを紹介します。
There exist some valid reasons to rebuild indexes on an Oracle database (not many). This presentation is about some of those reasons and how to automate such online index rebuild.
The document discusses various disaster recovery strategies for SQL Server including failover clustering, database mirroring, and peer-to-peer transactional replication. It provides advantages and disadvantages of each approach. It also outlines the steps to configure replication for Always On Availability Groups which involves setting up publications and subscriptions, configuring the availability group, and redirecting the original publisher to the listener name.
Oracle applications r12.2, ebr, online patching means lot of work for devel...Ajith Narayanan
This document discusses the implications of Oracle Applications R12.2's architecture and online patching capabilities for developers. It begins with introducing the speaker and providing an overview of R12.2's dual application tier architecture. It then covers key concepts around Edition Based Redefinition (EBR) and the rules and steps required for online patching. The main implications for developers are that objects must be uniquely named at 29 bytes or less, references to editioned objects require special handling, and code must work across editions that can change without warning during patching. Extra steps are required for table, seed data, view, trigger and other changes to support editions. Close coordination is needed when multiple developers are working concurrently.
This presentation describes how to efficiently load data into Hive. I cover partitioning, predicate pushdown, ORC file optimization and different loading schemes
Understanding SQL Trace, TKPROF and Execution Plan for beginnersCarlos Sierra
The three fundamental steps of SQL Tuning are: 1) Diagnostics Collection; 2) Root Cause Analysis (RCA); and 3) Remediation. This introductory session on SQL Tuning is for novice DBAs and Developers that are required to investigate a piece of an application performing poorly.
On this session participants will learn about producing a SQL Trace then a summary TKPROF report. A sample TKPROF is navigated with the audience, where the trivial and the no so trivial is exposed and explain. Execution Plans are also navigated and explained, so participants can later untangle complex Execution Plans and start diagnosing SQL performing badly.
This session encourages participants to ask all kind of questions that could be potential road-blocks for deeper understanding of how to address a SQL performing poorly.
This document discusses Oracle database performance tuning. It covers identifying common Oracle performance issues such as CPU bottlenecks, memory issues, and inefficient SQL statements. It also outlines the Oracle performance tuning method and tools like the Automatic Database Diagnostic Monitor (ADDM) and performance page in Oracle Enterprise Manager. These tools help administrators monitor performance, identify bottlenecks, implement ADDM recommendations, and tune SQL statements reactively when issues arise.
Survey of some free Tools to enhance your SQL Tuning and Performance Diagnost...Carlos Sierra
Several free tools are available to help with SQL tuning and performance diagnostics, including SQLd360, SQLT, and eDB360. SQLd360 and SQLT are good for diagnosing a single SQL statement, while eDB360 provides a 360-degree view of an entire Oracle database. Snapper and TUNAs360 can diagnose sessions and database activity. Standalone scripts like planx and sqlmon provide specialized diagnostics for individual cases. These free tools vary in size and capabilities, but all aim to help tune and diagnose SQL and database performance issues.
Mysql is a popular open source database system. It can be downloaded from the mysql website for free. Mysql allows users to create, manipulate and store data in databases. A database contains tables which store data in a structured format. Structured Query Language (SQL) is used to perform operations like querying and manipulating data within mysql databases. Some common sql queries include select, insert, update and delete.
Hadoop con 2015 hadoop enables enterprise data lakeJames Chen
Mobile Internet, Social Media 以及 Smart Device 的發展促成資訊的大爆炸,伴隨產生大量的非結構化及半結構化的資料,不但資料的格式多樣,產生的速度極快,對企業的資訊架構帶來了前所未有的挑戰,面對多樣的資料結構及多樣的分析工具,我們應該採用什麼樣的架構互相整合,才能有效的管理資料生命週期,提取資料價值,Hadoop 生態系統,無疑的在這個大架構裡,將扮演最基礎的資料平台的角色,實現企業的 Data Lake。
There exist some valid reasons to rebuild indexes on an Oracle database (not many). This presentation is about some of those reasons and how to automate such online index rebuild.
The document discusses various disaster recovery strategies for SQL Server including failover clustering, database mirroring, and peer-to-peer transactional replication. It provides advantages and disadvantages of each approach. It also outlines the steps to configure replication for Always On Availability Groups which involves setting up publications and subscriptions, configuring the availability group, and redirecting the original publisher to the listener name.
Oracle applications r12.2, ebr, online patching means lot of work for devel...Ajith Narayanan
This document discusses the implications of Oracle Applications R12.2's architecture and online patching capabilities for developers. It begins with introducing the speaker and providing an overview of R12.2's dual application tier architecture. It then covers key concepts around Edition Based Redefinition (EBR) and the rules and steps required for online patching. The main implications for developers are that objects must be uniquely named at 29 bytes or less, references to editioned objects require special handling, and code must work across editions that can change without warning during patching. Extra steps are required for table, seed data, view, trigger and other changes to support editions. Close coordination is needed when multiple developers are working concurrently.
This presentation describes how to efficiently load data into Hive. I cover partitioning, predicate pushdown, ORC file optimization and different loading schemes
Understanding SQL Trace, TKPROF and Execution Plan for beginnersCarlos Sierra
The three fundamental steps of SQL Tuning are: 1) Diagnostics Collection; 2) Root Cause Analysis (RCA); and 3) Remediation. This introductory session on SQL Tuning is for novice DBAs and Developers that are required to investigate a piece of an application performing poorly.
On this session participants will learn about producing a SQL Trace then a summary TKPROF report. A sample TKPROF is navigated with the audience, where the trivial and the no so trivial is exposed and explain. Execution Plans are also navigated and explained, so participants can later untangle complex Execution Plans and start diagnosing SQL performing badly.
This session encourages participants to ask all kind of questions that could be potential road-blocks for deeper understanding of how to address a SQL performing poorly.
This document discusses Oracle database performance tuning. It covers identifying common Oracle performance issues such as CPU bottlenecks, memory issues, and inefficient SQL statements. It also outlines the Oracle performance tuning method and tools like the Automatic Database Diagnostic Monitor (ADDM) and performance page in Oracle Enterprise Manager. These tools help administrators monitor performance, identify bottlenecks, implement ADDM recommendations, and tune SQL statements reactively when issues arise.
Survey of some free Tools to enhance your SQL Tuning and Performance Diagnost...Carlos Sierra
Several free tools are available to help with SQL tuning and performance diagnostics, including SQLd360, SQLT, and eDB360. SQLd360 and SQLT are good for diagnosing a single SQL statement, while eDB360 provides a 360-degree view of an entire Oracle database. Snapper and TUNAs360 can diagnose sessions and database activity. Standalone scripts like planx and sqlmon provide specialized diagnostics for individual cases. These free tools vary in size and capabilities, but all aim to help tune and diagnose SQL and database performance issues.
Mysql is a popular open source database system. It can be downloaded from the mysql website for free. Mysql allows users to create, manipulate and store data in databases. A database contains tables which store data in a structured format. Structured Query Language (SQL) is used to perform operations like querying and manipulating data within mysql databases. Some common sql queries include select, insert, update and delete.
Hadoop con 2015 hadoop enables enterprise data lakeJames Chen
Mobile Internet, Social Media 以及 Smart Device 的發展促成資訊的大爆炸,伴隨產生大量的非結構化及半結構化的資料,不但資料的格式多樣,產生的速度極快,對企業的資訊架構帶來了前所未有的挑戰,面對多樣的資料結構及多樣的分析工具,我們應該採用什麼樣的架構互相整合,才能有效的管理資料生命週期,提取資料價值,Hadoop 生態系統,無疑的在這個大架構裡,將扮演最基礎的資料平台的角色,實現企業的 Data Lake。
This document provides an overview of SQL Server 2017 In-Memory OLTP for developers. It discusses table types that can be used, including schema-only tables that are faster than schema-and-data tables. It also covers range and hash indexes that provide better performance than disk-based indexes when used in-memory. Natively compiled stored procedures can more efficiently access memory-optimized tables and avoid recompilation during restart. The document recommends setting the transaction isolation level to read committed and including retry logic to handle conflicts when using snapshot isolation.
The document outlines steps for preparing a local database including restoring from backups, fixing schemas and initial data, adding any missing SQL scripts, and restoring from another backup file if needed. It then mentions preparing load-ready SQL files and migrating the local database to integration, QA, staging, and production databases using EF Core migrations.
This document provides an overview of Kafka including its architecture, key concepts, and performance tuning. It describes how Kafka is a distributed streaming platform popular for use cases like logging, metrics, and messaging. The architecture explained includes Kafka brokers that make up clusters, Zookeeper for coordination, producers that publish messages, consumers that subscribe to messages, and topics for categorizing data. It also covers message delivery guarantees, monitoring tools, and ways to optimize producer, broker, consumer, and JVM performance such as configuration settings for throughput, latency, and durability.
Grafana is an open source analytics and monitoring tool that uses InfluxDB to store time series data and provide visualization dashboards. It collects metrics like application and server performance from Telegraf every 10 seconds, stores the data in InfluxDB using the line protocol format, and allows users to build dashboards in Grafana to monitor and get alerts on metrics. An example scenario is using it to collect and display load time metrics from a QA whitelist VM.