Automatic data optimization with oracle database 12c

1. Automatic Data Optimization with Oracle Database 12c
As of automatic-data-optimization 12c in relate to Information Lifecycle
Management (ILM) :
In Oracle Database 12c ILM related features Advanced Compression Option.
Heat Map automatically tracks modification and query timestamps at the row
and segment levels, providing detailed insights into how data is being accessed.
Automatic Data Optimization (ADO)automatically moves and compresses data
according to user defined policies based on the information collected by Heat
Map
Heat Map and ADO make it easy to use existing innovations in Oracle
Database Compression and Partitioning technologies, which help reduce the
cost of managing large amounts of data, while also improving application and
database performance. Together these capabilities help to implement first
++++++++++++++++++++++++++++
TESTING: compression policy
++++++++++++++++++++++++++++
Step1: First create tablespace to store SH.SALES data and insert rows:
SQL> create tablespace tstado datafile
'/BPELAIT2/MONDB01/MONDB01/tstado1.dbf' size 400M reuse autoextend
off extent management loc al;
Tablespace created.
Step2: Then creating another tablespace, space_press where SH.SALES table
data may be moved for space pressure
SQL> create tablespace space_press datafile
'/BPELAIT2/MONDB01/MONDB01/sp1.dbf' size 150M;
Tablespace created.
SQL> grant dba to sh;
Grant succeeded.
Step 3: create a procedure which simulates the passage of time in this example
so that the table qualifies for ADO action (compression) even though actually
20 days have not elapsed.

2. SQL> CREATE OR REPLACE PROCEDURE set_stat (object_id number,
2 data_object_id number,
3 n_days number,
4 p_ts# number,
5 p_segment_access number)
6 as
7 begin
8 insert into sys.heat_map_stat$
9 (obj#,
10 dataobj#,
11 track_time,
12 segment_access,
13 ts#)
14 values
15 (object_id,
16 data_object_id,
17 sysdate - n_days,
18 p_segment_access,
19 p_ts# );
20 commit;
21 end;
22 /
Procedure created.
SQL> grant execute on set_stat to public;
Grant succeeded.
SQL> grant select any dictionary to SH;
Grant succeeded.
Step 4: Create a view used based on COMPRESSION_STAT$ table to retrieve
segment compression results after ADO action. Grant select on the view to
public and create a public synonym for the view.
SQL> CREATE OR REPLACE VIEW user_compression_stats
2 (object_name , subobject_name, avgsize_uncomp, avgsize_disk,
3 nblocks_uncmp, nblocks_oltp, nblocks_ehcc, nrows_uncmp,
4 nrows_oltp, nrows_ehcc)
5 as select b.name,
6 b.subname,
7 a.AVGROWSIZE_NC,
8 a.AVGROWSIZE_C,
9 a.NBLK_NC,

3. 10 a.NBLK_ADVANCED,
11 a.NBLK_EHCC,
12 a.NROWS_NC,
13 a.NROWS_ADVANCED,
14 a.NROWS_EHCC
15 from sys.compression_stat$ a,
16 sys.obj$ b
17 where a.obj# = b.obj# and
18 b.owner# = userenv('SCHEMAID');
View created.
SQL> GRANT select ON user_compression_stats TO PUBLIC;
Grant succeeded.
SQL> CREATE OR REPLACE PUBLIC SYNONYM user_compression_stats FOR
sys.user_compression_stats;
Synonym created.
Step 6: Enable heat map tracking, and set the heat map tracking start time
back 25 days to ensure statistics logged after this time are valid and considered
by Automatic Data Optimization (ADO).
SQL> alter system set heat_map=on scope=both;
exec dbms_ilm_admin.set_heat_map_start(start_date => sysdate - 20);
SQL> create table sales_ado as select * from sales;
Table created.
SQL> alter table sales_ado move tablespace tstado;
Table altered.
Populte the table with some more data :
SQL>
declare
sql_test clob;
begin
for i in 1..7
loop sql_test := 'insert /*+ append */ into sh.sales_ado select * from sh.sales
where CUST_ID=3548';
execute immediate sql_test;

4. commit;
end loop;
end;
/
PL/SQL procedure successfully completed.
SQL> select count (*) from sh.sales_ado;
COUNT(*)
----------
920376
Step 7: Verify that heat map tracking collected statistics for the sh.sales_ado
table.
SQL>
select OBJECT_NAME,SEGMENT_WRITE_TIME , SEGMENT_READ_TIME,
FULL_SCAN
FROM dba_heat_map_segment
WHERE OBJECT_NAME='SALES_ADO'
AND OWNER = 'SH';SQL> 2 3 4
OBJECT_NAME
--------------------------------------------------------------------------------
SEGMENT_WRITE_TIME SEGMENT_READ_TIME FULL_SCAN
------------------ ------------------ ------------------
SALES_ADO
10-oct-13 10:36:25
Check Segment level
-----------------------------
col "Segment write" format A14
col "Full Scan" format A12
col "Lookup Scan" format a12
select object_name, track_time "Tracking Time",
segment_write "Segment write",
full_scan "Full Scan",
lookup_scan "Lookup Scan"
from DBA_HEAT_MAP_SEG_HISTOGRAM
WHERE OBJECT_NAME='SALES_ADO'
AND OWNER = 'SH';

5. OBJECT_NAME Tracking Time Segment write Full Scan Lookup Scan
------------------ -------------- ------------ ------------
SALES_ADO 10-oct-13 10:37:49 NO YES NO
CHECK THE COMPRESSION attribute
-------------------------------------------------
SQL> select compression, compress_for from dba_tables where table_name
='SALES_ADO'AND OWNER = 'SH';
COMPRESS COMPRESS_FOR
-------- ------------------------------
DISABLED
Step 8:
SQL> analyze table SH.SALES_ADO compute statistics;
Table analyzed.
SQL>
select object_name, nrows_uncmp, nrows_oltp, nrows_ehcc from
sys.user_compression_stats;SQL>
no rows selected
CREATE COMPRESSION POLICY on table SH.SALES_ADO
---------------------------------------------------
SQL> alter table SH.SALES_ADO
ILM ADD POLICY
ROW STORE COMPRESS ADVANCED
SEGMENT
AFTER 20 DAYS OF NO MODIFICATION;
Table altered.
SQL> show user
USER is "SH"
SQL> select policy_name, action_type, scope, compression_level,
condition_type, condition_days
from user_ilmdatamovementpolicies
order by policy_name;
POLICY_NAME ACTION_TYPE SCOPE COMPRESSION_LEVEL CONDITION_TYPE CONDITION_DAYS
--------------------------------------------------------------------------------------------------------------------------------------------------
P1 COMPRESSION SEGMENT ADVANCED LAST MODIFICATION TIME 20

6. SQL> select policy_name, object_name, inherited_from, enabled
from user_ilmobjects;
POLICY_NAME OBJECT_NAME INHERITED_FROM ENA
-------------------- ------------------------------------------------
P1 SALES_ADO POLICY NOT INHERITED YES
Step 9: We are simulating a situation where no modification has been done on
SH.SALES_ADO table for the last 20 days. Use the procedure sys.set_stat to
fake the passage of time so that compression policy qualifies for ILM action.
We are setting heat map statistics clock back by 20 days.
As SYS:
SQL> conn sys/*** as sysdba
Connected.
SQL> alter session set nls_date_format='dd-mon-yy hh:mi:ss';
Session altered.
SQL> declare
2 v_obj# number;
3 v_dataobj# number;
4 v_ts# number;
5 begin
6 select object_id, data_object_id into v_obj#, v_dataobj#
7 from all_objects
8 where object_name = 'SALES_ADO'
9 and owner = 'SH';
10 select ts# into v_ts#
11 from sys.ts$ a,
12 dba_segments b
13 where a.name = b.tablespace_name
14 and b.segment_name = 'SALES_ADO';
15 commit;
16 sys.set_stat
17 (object_id => v_obj#,
18 data_object_id => v_dataobj#,
19 n_days => 20,
20 p_ts# => v_ts#,
21 p_segment_access => 1);
22 end;
23 /
PL/SQL procedure successfully completed.

7. select object_name, segment_write_time
from dba_heat_map_segment
where object_name='SALES_ADO';
OBJECT_NAME SEGMENT_WRITE_TIME
------------------------------------------------------------
SALES_ADO 20-sep-13 11:03:46
16 rows selected.
SQL> !date
Thursday, 10 October 2013 11:05:30 AM EST
statistics are showing that the table was last accessed more 20 days back.
Step 10: To open and trigger the ADO policies jobs [Note: Not waiting for the
maintenance window for testing]
================================================================
SQL> conn sh/sh
Connected.
SQL> declare
v_executionid number;
begin
dbms_ilm.execute_ILM (ILM_SCOPE => dbms_ilm.SCOPE_SCHEMA,
execution_mode => dbms_ilm.ilm_execution_offline,
task_id => v_executionid);
end;
/
PL/SQL procedure successfully completed.
Step 11:
View the results of the job that completed the compression operation.
================================================================
SQL> select task_id, start_time as start_time from user_ilmtasks;
TASK_ID
----------
START_TIME
---------------------------------------------------------------------------
2
10-OCT-13 11.09.07.952584 AM

8. SQL> SQL> select task_id, job_name, job_state, completion_time completion
from user_ilmresults;
TASK_ID JOB_NAME JOB_STATE COMPLETION
---------------------------------------------------------------------------
2 ILMJOB2 COMPLETED SUCCESSFULLY 10-OCT-13 11.09.14.826402 AM
1 select task_id, policy_name, object_name,
2 selected_for_execution, job_name
3* from user_ilmevaluationdetails
SQL> /
TASK_ID POLICY_NAME OBJECT_NAME SELECTED_FOR_EXECUTION
------------------------------------------
JOB_NAME
-------------------------------------------------------------------------------------------------
2 P1 SALES_ADO
ILMJOB2
Step 12: Check if ADO triggered compression on SH.SALES_ADO segment.
SQL> show user
USER is "SH"
SQL> analyze table SH.SALES_ADO compute statistics;
Table analyzed.
SQL> select compression, compress_for FROM user_tables where table_name =
'SALES_ADO';
COMPRESS COMPRESS_FOR
-------- ------------------------------
ENABLED ADVANCED
SQL>
select object_name, nrows_uncmp, nrows_oltp, nrows_ehcc from
user_compression_stats;
OBJECT_NAME NROWS_UNCMP NROWS_OLTP NROWS_EHCC
----------- ---------- ----------
SALES_ADO 0 918843 0
So finish testing compression policy.

9. +++++++++++++++++++++++++
TESTING: TIER POLICY
+++++++++++++++++++++++++
CLEANUP Existing Policy
Check Current ILM policies on the SH.SALES_ADO table.
SQL> select * from user_ilmpolicies;
SQL> select * from user_ilmdatamovementpolicies;
Delete all ILM Policies on the SH.SALES_ADO table.
SQL> alter table SH.SALES_ADO ilm delete_all;
NOW CHECK the threshold value
-------------------------------------------
SQL> col name format A21
SQL> col value format 9999
SQL> select * from dba_ilmparameters;
NAME VALUE
--------------------- -----
ENABLED 1
JOB LIMIT 10
EXECUTION MODE 3
EXECUTION INTERVAL 15
TBS PERCENT USED 15
TBS PERCENT FREE 85
6 rows selected.
Tablespace Usage Check
----------------------------------
SQL> set linesize 300;
SQL> SELECT a.tablespace_name TBS_NAME, ROUND (a.bytes_alloc / 1024 /
1024, 0) MEGS_ALLOC,
2 ROUND (NVL (b.bytes_free, 0) / 1024 / 1024, 0) MEGS_FREE,
3 ROUND ((a.bytes_alloc - NVL (b.bytes_free, 0)) / 1024 / 1024,0)
MEGS_USED,
4 ROUND ((NVL (b.bytes_free, 0) / a.bytes_alloc) * 100, 2) PCT_FREE,
5 100 - ROUND ((NVL (b.bytes_free, 0) / a.bytes_alloc) * 100, 2) PCT_USED,
6 ROUND (maxbytes / 1048576, 2) MAX_MEGS_ALLOC,
7 100 - ROUND ((a.bytes_alloc - NVL (b.bytes_free, 0)) / maxbytes * 100, 2)
MAX_PCT_FREE,

10. 8 ROUND ((a.bytes_alloc - NVL (b.bytes_free, 0)) / maxbytes * 100, 2)
MAX_PCT_USED
9 FROM (SELECT f.tablespace_name, SUM (f.BYTES) bytes_alloc,
10 SUM (DECODE (f.autoextensible,'YES',f.maxbytes,'NO', f.BYTES))
maxbytes
11 FROM dba_data_files f
12 GROUP BY tablespace_name) a,
13 (SELECT f.tablespace_name, SUM (f.BYTES) bytes_free
14 FROM dba_free_space f
15 GROUP BY tablespace_name) b
16 WHERE a.tablespace_name = b.tablespace_name(+)
17 ;
TBS_NAME MEGS_ALLOC MEGS_FREE MEGS_USED PCT_FREE PCT_USED MAX_MEGS_ALLOC
MAX_PCT_FREE MAX_PCT_USED
------------------------------ ---------- ---------- ---------- ---------- ---------- -------------- ------------ ------------
SYSTEM 970 4 966 .37 99.63 32767.98 97.05 2.95
MGMT_AD4J_TS 200 199 1 99.5 .5 32767.98 100 0
TSTADO 400 385 15 96.25 3.75 400 96.25 3.75
SPACE_PRESS 150 149 1 99.33 .67 150 99.33 .67
USERS 40 38 2 95.78 4.22 32767.98 99.99 .01
MGMT_ECM_DEPOT_TS 40 20 20 49.38 50.62 32767.98 99.94 .06
MGMT_TABLESPACE 500 43 457 8.51 91.49 32767.98 98.6 1.4
SYSAUX 830 46 784 5.59 94.41 32767.98 97.61 2.39
EXAMPLE 323 0 323 .02 99.98 32767.98 99.01 .99
UNDOTBS1 330 72 258 21.69 78.31 32767.98 99.21 .79
10 rows selected.
Create Policy
-----------------------
CREATE A STORAGE TIER POLICY ON TABLE SH.SALES_ADO
SQL> ALTER TABLE SH.SALES_ADO ilm ADD POLICY TIER TO
SPACE_PRESS;
Table altered.
SQL> col policy_name format a10
SQL> col tier_tbs format a20
SQL> select POLICY_NAME, ACTION_TYPE, SCOPE, TIER_TABLESPACE
2 from user_ilmdatamovementpolicies
3 order by policy_name;
POLICY_NAME ACTION_TYPE SCOPE TIER_TABLESPACE
---------------------------------------------------------------------------------
P21 STORAGE SEGMENT SPACE_PRESS

11. For testing: Now set the threshold value for TBS_PERCENT_FREE to (97%) and
TBS_PERCENT_USED to 3%
SQL> conn sys/*** as sysdba
Connected.
SQL> exec dbms_ilm_admin.customize_ilm(dbms_ilm_admin.tbs_percent_free,97)
PL/SQL procedure successfully completed.
SQL> exec dbms_ilm_admin.customize_ilm(dbms_ilm_admin.tbs_percent_used,3)
PL/SQL procedure successfully completed.
SQL> col name format A21
SQL> col value format 9999
SQL> select * from dba_ilmparameters;
NAME VALUE
--------------------- -----
ENABLED 1
JOB LIMIT 10
EXECUTION MODE 3
EXECUTION INTERVAL 15
TBS PERCENT USED 3
TBS PERCENT FREE 97
6 rows selected.
maintenance window to open and trigger the ADO policies jobs [Note: Not
waiting for the maintenance window for testing]
================================================================
SQL> conn sh/sh
Connected.
SQL> declare
v_executionid number;
begin
dbms_ilm.execute_ILM (ILM_SCOPE => dbms_ilm.SCOPE_SCHEMA,
execution_mode => dbms_ilm.ilm_execution_offline,
task_id => v_executionid);
end;
/
Recheck Tablespace Status
------------------------------------
SYSTEM 970 4 966 .37 99.63 32767.98 97.05 2.95
MGMT_AD4J_TS 200 199 1 99.5 .5 32767.98 100 0
TSTADO 400 399 1 99.75 .25 400 99.75 .25
SPACE_PRESS 150 135 15 90 10 150 90 10

12. USERS 40 38 2 95.78 4.22 32767.98 99.99 .01
MGMT_ECM_DEPOT_TS 40 20 20 49.38 50.62 32767.98 99.94 .06
MGMT_TABLESPACE 500 43 457 8.51 91.49 32767.98 98.6 1.4
SYSAUX 830 36 794 4.34 95.66 32767.98 97.58 2.42
EXAMPLE 323 0 323 .02 99.98 32767.98 99.01 .99
UNDOTBS1 330 85 245 25.83 74.17 32767.98 99.25 .75
10 rows selected.
So free space is more on TSTADO tablespace and SPACE_PRESS is showing
value for column %used.
Check the ADO task Status
--------------------------------------
SQL> select task_id, state , COMPLETION_TIME from user_ilmtasks;
2 COMPLETED 10-OCT-13 11.09.14.826402 AM
14 COMPLETED 10-OCT-13 01.53.09.689013 PM
17 COMPLETED 10-OCT-13 02.23.14.231013 PM
SQL> select table_name, tablespace_name from user_tables
2 where table_name='SALES_ADO';
TABLE_NAME TABLESPACE_NAME
-------------------------------------------
SALES_ADO SPACE_PRESS
CleanUP
----------------
Delete all ILM Policies on the SH.SALES_ADO table.
SQL> alter table SH.SALES_ADO ilm delete_all;
Table altered.
Verify that there are no ILM policies on the SH.SALES_ADO table.
SQL> select * from user_ilmdatamovementpolicies;
no rows selected
SQL> select * from user_ilmpolicies;
no rows selected
SQL> alter system set heat_map=off scope=both;
System altered.

13. Clean up Heat Map Statistics
-----------------------------------
SQL> exec dbms_ilm_admin.clear_heat_map_all;
PL/SQL procedure successfully completed.
SQL> select obj#, ts#, track_time from sys.heat_map_stat$;
OBJ# TS# TRACK_TIM
---------- ---------- ---------
-1 -1 10-OCT-13
Note: The above value is for the dummy stat.
So TIER Policy test has been completed for ADO.