This document provides an introduction and overview of Apache Spark with Python (PySpark). It discusses key Spark concepts like RDDs, DataFrames, Spark SQL, Spark Streaming, GraphX, and MLlib. It includes code examples demonstrating how to work with data using PySpark for each of these concepts.
This document introduces Apache Cassandra, a distributed column-oriented NoSQL database. It discusses Cassandra's architecture, data model, query language (CQL), and how to install and run Cassandra. Key points covered include Cassandra's linear scalability, high availability and fault tolerance. The document also demonstrates how to use the nodetool utility and provides guidance on backing up and restoring Cassandra data.
This document provides an introduction and overview of Apache Spark with Python (PySpark). It discusses key Spark concepts like RDDs, DataFrames, Spark SQL, Spark Streaming, GraphX, and MLlib. It includes code examples demonstrating how to work with data using PySpark for each of these concepts.
This document introduces Apache Cassandra, a distributed column-oriented NoSQL database. It discusses Cassandra's architecture, data model, query language (CQL), and how to install and run Cassandra. Key points covered include Cassandra's linear scalability, high availability and fault tolerance. The document also demonstrates how to use the nodetool utility and provides guidance on backing up and restoring Cassandra data.
SQL or noSQL - Oracle Cloud Day IstanbulGokhan Atil
This document discusses SQL and NoSQL databases. It defines NoSQL as non-relational databases that are not only SQL. It describes some key properties of NoSQL databases including being schema-less, horizontally scalable, having a simple API, and following BASE semantics rather than ACID. The document outlines several types of NoSQL databases, discusses the CAP theorem, and contrasts ACID and BASE. It concludes by advising readers to consider their needs around constraints, ACID support, data modeling, scalability, and resources when deciding between SQL and NoSQL.
EM13c: Write Powerful Scripts with EMCLIGokhan Atil
This document provides an overview and introduction to using EMCLI (Enterprise Manager Command Line Interface) to write scripts that access Oracle Enterprise Manager functionality. It discusses installing and configuring EMCLI, the various EMCLI verbs (commands) across different categories, and provides examples of Python scripts using EMCLI verbs to automate tasks like clearing alerts, applying management templates, and changing passwords.
Oracle Enterprise Manager Cloud Control 13c for DBAsGokhan Atil
This document provides an overview and introduction to Oracle Enterprise Manager Cloud Control 13c (EM13c) for database administrators (DBAs). It discusses the key features and benefits of EM13c for monitoring, performance tuning, and provisioning databases. The document outlines the architecture and components of EM13c and why it is useful for centralized management. It also provides tips for DBAs on using features like monitoring, incident management, ASH analytics, provisioning, patching, and best practices for installation, configuration, and maintenance of an EM13c environment.
This document provides an overview of essential Linux commands for database administrators (DBAs). It covers commands for quick system health checks like uptime, free, top, vmstat, iostat, mpstat, pidstat, sar, and dmesg. It also covers tools for profiling and tracing like perf, strace, ltrace, and pstack. Finally, it discusses other useful commands like file, dd, hexdump, strings, fuser, lsof, ipcs, and ldd. The document is intended as an introduction to core Linux commands that do not require external repositories or root privileges.
Oracle Enterprise Manager Cloud Control 13c for DBAsGokhan Atil
This document provides an overview of Oracle Enterprise Manager Cloud Control 13c for database administrators. It begins with introductions to the presenter and an agenda. It then discusses what Enterprise Manager is, its architecture involving agents, management server, and repository. Some key benefits for DBAs are standardized automation of tasks using a single tool. The document outlines several top features for DBAs, including monitoring, metrics/alerts, incident management, corrective actions, provisioning, patching, ASH analytics, and AWR warehouse. It provides guidance on installing EM13c and post-install tasks. Finally, it covers maintaining EM through tasks like backups, agent management, and keeping everything updated.
Enterprise Manager: Write powerful scripts with EMCLIGokhan Atil
The document provides an introduction to the EMCLI (Enterprise Manager Command Line Interface) tool. Some key points:
- EMCLI allows accessing Enterprise Manager functionality from a command line instead of the web interface. It comes pre-installed on the OMS server and can also be installed on other machines.
- There are three modes of EMCLI - Standard, Interactive, and Scripting. Scripting mode allows running Python scripts containing EMCLI verbs.
- Over 400 verbs are available across 75+ categories to perform various monitoring and management tasks. Common verbs include get_targets, add_target, update_db_password.
- Python is introduced as the scripting language used with EMCLI in Script
This document provides an overview of the Oracle Enterprise Manager Command Line Interface (EM CLI).
It discusses the different modes of EM CLI including standard, interactive, and scripting modes. It also covers EM CLI verbs, formatting output, fetching information from the EM repository, and provides examples of Bash and Python scripts using EM CLI.
Sample scripts demonstrated include clearing stateless alerts, changing database passwords, and promoting unmanaged databases to managed targets. Fundamentals of Python programming are also introduced for effective EM CLI scripting.
This document provides an overview of using the Oracle Enterprise Manager Command Line Interface (EM CLI). It discusses the different modes of EM CLI including standard, interactive, and scripting modes. It also covers installing and configuring EM CLI, common verbs, formatting and filtering output, and provides examples of using EM CLI to clear alerts, create blackouts, and change passwords via scripts. The document is intended as a crash course on getting started with EM CLI.
Oracle DB Standard Edition: Başka Bir Arzunuz?Gokhan Atil
Enterprise Edition'a gerçekten ihtiyacınız var mı? Standard Edition ile arasındaki farkları biliyor musunuz? Her projede Enterprise Edition kullanmak doğru mu?
Using APEX to Create a Mobile User Interface for Enterprise Manager 12cGokhan Atil
This document discusses creating a mobile user interface for Oracle Enterprise Manager 12c using Oracle Application Express (APEX). It outlines the need for a mobile app to access incidents and problems from the EM12c repository. It then provides details on using APEX to build a basic mobile app with pages to view open incidents and problems, and see incident and problem details. The app allows accessing key monitoring data from EM12c in a mobile-friendly way.
2. Gökhan Atıl
• DBA Takım Lideri
• 10 yıldan fazla deneyim
• 10g/11g/R12 OCP DBA
• Oracle Certified Expert (SQL & Exadata)
• Comptia Linux+
• Blogger (Ekim 2008’den beri) gokhanatil.com
• Co-Founder of Turkey Oracle User Group (TROUG)
• Oracle ACE (Ekim 2011)
3. Expert Oracle Enterprise Manager 12c
Kellyn Pot'vin
Anand Akela
Gökhan Atıl
Bobby Curtis
Alex Gorbachev
Niall Litchfield
Leighton Nelson
Pete Sharman
25 Haziran 2013
4. Sunum İçeriği
Oracle Database In-Memory Nedir?
Neden In-Memory?
Oracle Database In-Memory’nin Yapısı
Performans Karşılaştırması
Nasıl Kullanılır?
Son Söz
5. 12.1.0.1
Major Database Release Number (significant new functionality)
Database Maintenance Release Number (some new features)
Application Server Release Number
Component-Specific Release Number (patch sets or interim releases)
6. Oracle Database In-Memory Nedir?
Database (storage structures)
Instance
(System Global Area)
Background Process
PMONSMONDBWn LGWRCKPT ARCn
Database
Buffer
Cache Shared Pool
In-Memory
Area
IMCO
SP LP JP RL
nK
7. Neden In-Memory?
• Memory’den 1 MB sırasız veri okuma 0.25 ms
• SSD’den 1 MB sırasız veri okuma 1 ms
• Sabit Disk’ten 1 MB sırasız veri okuma 20 ms
9. Oracle ve Kolon Formatı?
• Sadece In-memory’de
kullanılıyor
• Sıkıştırma sayesinde veri
x2-x20 kat daha az yer
kaplıyor (MEMCOMPRESS)
• Logging Gerektirmiyor
(recovery edilmesine gerek
yok çünkü satır formatı
halinde zaten diskte
tutuluyor)
• Startup sırasında tablolar
belleğe alınıyorlar
(PRIORITY)
Memory
Kolon formatı
10. SIMD
Neden Kolon Formatı?
In-Memory Column Store
Şehir SatışlarSales
Örnek: İstanbul’daki satış toplamını bulmak
1. Sadece gerekli kolon verisine
erişim!
2. Yeni algoritma sayesinde
sıkıştırılmış veri içinde arama
3. IM Storage Index sayesinde ilgisiz
verinin filtrelenmesi (IMCU)
4. SIMD Vektör İşleme ile değerlerin
tek komut ile karşılaştırılması
İstanbul 5,000,000 TL
11. SIMD Vektör İşleme Nedir?
• SIMD (Single Instruction processing Multiple Data values) vektör
işleme, SIMD desteği olan sunucularda her kolondaki veriyi tek tek
işlemek yerine, birden fazla kolondaki verinin beraberce tek bir
CPU dönüşünde ile işlenmesini sağlar.
• Modern Intel işlemciler 16-32 SIMD registerlarına sahiptir.
12. İki Kopya (Satır+Kolon) Yavaş Olmaz mı?
Tablo
1-3
OLTP içim
İndeksler
5-15
Analitik Sorgular için
İndeksler
13. Performans Karşılaştırması
• Tanel Põder & Kerry Osborne’nun Exadata
üzerinde yaptığı testler:
SELECT COUNT(*), SUM(order_total)
FROM
orders
WHERE
warehouse_id BETWEEN 500 AND 510
8 GB’lık tablonun %1’ini
seçmek istiyoruz
14. Test Sonuçları:
Erişim metodu Toplam CPU(ms)
index (range scan) 265,203 37,438
full table scan (buffered) 132,075 48,944
full table scan (direct path) 155,67 11,808
full table scan (smart scan) 2,102 729
full table scan (inmemory) 155 155
15. In-Memory Hızlı Peki ya Buffer Cache?
Erişim metodu Toplam CPU(ms)
full table scan (smart scan) 2,102 729
full table scan (inmemory) 155 155
full table scan (buffer cache) 7,850 7,831
1. Sadece gerekli kolonlara erişim
2. Yeni algoritma sayesinde sıkıştırılmış veri içinde arama
3. IM Storage Index sayesinde ilgisiz verinin filtrelenmesi
4. SIMD Vektör İşleme
16. In-Memory ve Verilerin İşlenmesi
• Veriye erişim her zaman hızlı olmasına karşın,
özellikle TEMP tablespace’i kullanılan
sorgularda verinin işlenmesi darboğaz
oluşturabilir:
Sorgularınızın TEMP tablespace’i kullanmadığına
emin olun
Verinin filtrelenmesini önce yapın
PGA_AGGREGATE_TARGET’ı arttırın
Paralel sorgular kullanın
17. Nasıl Aktif Edilir?
• Database In-Memory Opsiyonunu
etkinleştirmek:
ALTER SYSTEM SET inmemory_size = 20G
SCOPE=spfile;
• Veritabanını restart etmeniz gerekir.
• SGA’nin yeterince büyük olmasına dikkat edin!
18. Tablolar In-Memory’ye Yükleyin
• İstediğiniz tablo için In-Memory opsiyonunu aktif
hale getirmek:
ALTER TABLE sales INMEMORY;
• Tablolar in-memory area’ya (IM Column Store)
varsayılan olarak on-demand olarak yüklenir.
Bunun olması için tabloya full scan yapan bir
sorgu çalışmalıdır.
19. In-Memory’ye Otomatik Yükleme
• Tablonun otomatik olarak In-Memory’ye
yüklenmesi için:
ALTER TABLE sales INMEMORY PRIORITY
CRITICAL;
• PRIORITY şunlardan biri olabilir: NONE
(varsayılan), LOW, MEDIUM, HIGH, CRITICAL
20. Tablolarım In-Memory’de mi?
• V$IM_SEGMENTS veya V$IM_USER_SEGMENTS’ı
sorgulayarak tablolarınız in-memory’de mi
görebilirsiniz:
SELECT
OWNER, SEGMENT_NAME,
INMEMORY_SIZE, BYTES, BYTES_NOT_POPULATED,
POPULATE_STATUS
FROM v$im_segments;
21. Kısıtlamalar Nelerdir?
Şu objeler In-Memory’de tutulamaz:
• SYSTEM or SYSAUX tablespace’indeki SYS
objeleri
• Index Organized Tablolar (IOTs)
• Clustered Tablolar
• LONG veritipindeki kolonlar
22. Kolon/Partition Bazlı In-Memory
• İstediğiniz kolon ve partitionların In-Memory area’ya
yüklenmesini engelleyebilir veya kaldırabilirsiniz:
ALTER TABLE sales INMEMORY
NO INMEMORY (prod_id);
ALTER TABLE sales MODIFY PARTITION SALES_1998 NO
INMEMORY;
ALTER TABLE sales INMEMORY NO INMEMORY;
23. In-Memory ve Sıkıştırma
• Tablonun, partitionın veya kolonun nasıl
sıkıştırılacağını belirtebilirsiniz:
ALTER TABLE sales INMEMORY
MEMCOMPRESS FOR QUERY;
ALTER TABLE sales MODIFY PARTITION
SALES_1998 INMEMORY NOMEMCOMPRESS;
ALTER TABLE sales INMEMORY
MEMCOMPRESS FOR QUERY (prod_id);
24. Sıkıştırma Seçenekleri
• MEMCOMPRESS parametresi:
NOCOMPRESS Sıkıştırma yok
MEMCOMPRESS FOR DML
DML işlemleri için optimize edilmiş
minimum sıkıştırma
MEMCOMPRESS FOR QUERY LOW Varsayılan (Sorgu performansı için optimize)
MEMCOMPRESS FOR QUERY HIGH
Hem sorgu performansı hem yer kullanımı
açısından iyi
MEMCOMPRESS FOR CAPACITY LOW
Sorgu performansından çok yer kullanımını
düşünen
MEMCOMPRESS FOR CAPACITY HIGH
Tamamen yer kullanımını azaltmak için
tasarlanmış
25. Kolonlar Nasıl Sıkıştırılmış?
• Kolonlarla ilgili sıkıştırma seçeneklerini
V$IM_COLUMN_LEVEL viewinden
sorgulayabilirsiniz:
SELECT COLUMN_NAME,
INMEMORY_COMPRESSION
FROM v$im_column_level
WHERE TABLE_NAME = ‘SALES’;
28. Son Söz
• Fonksiyonel: Her hangi bir SQL kısıtlaması yok
• Uygulanması Kolay: Verinin aktarılmasına
veya değiştirilmesine gerek yok
• Tamamen Uyumlu: Uygulamalarda her hangi
bir değişiklik gerektirmiyor
• Partitioning ile Birlikte Enterprise Edition
kullanmak için en geçerli sebep
29. KAYNAKLAR:
@gokhanatil http://gokhanatil.com
Tanel Põder & Kerry Osborne’sunumu Oracle Database In‐Memory Option in Action
Oracle Database In-Memory blog https://blogs.oracle.com/In-Memory/
Oracle Database In-Memory White Paper Ekim 2014
Editor's Notes
This template can be used as a starter file for presenting training materials in a group setting.
Sections
Right-click on a slide to add sections. Sections can help to organize your slides or facilitate collaboration between multiple authors.
Notes
Use the Notes section for delivery notes or to provide additional details for the audience. View these notes in Presentation View during your presentation.
Keep in mind the font size (important for accessibility, visibility, videotaping, and online production)
Coordinated colors
Pay particular attention to the graphs, charts, and text boxes.
Consider that attendees will print in black and white or grayscale. Run a test print to make sure your colors work when printed in pure black and white and grayscale.
Graphics, tables, and graphs
Keep it simple: If possible, use consistent, non-distracting styles and colors.
Label all graphs and tables.
This is another option for an Overview slides using transitions.