My presentation from COLLABORATE 16 conference

1. Session ID:
Prepared by:
E-Business Suite Rapid
Provisioning Using New
Features Of Oracle Database 12c
10619
@ AndrejsKarpovs
Andrejs Karpovs,
Lead Oracle Applications DBA

2. About me
• Lead Oracle Apps DBA / Technical infrastructure
architect at Tieto [Latvia]
• Oracle Certified Master
• Oracle ACE
• Masters Degree in Computer Science
• Speaker: Collaborate, UKOUG, OUGH, Oracle
• RAC Attack Ninja
• Blog: adbaday.wordpress.com
2

3. Coming from Latvia
3

4. Tieto
• Tieto is the largest Nordic IT services company
4

5. Agenda
• ASM – Oracle Cloud Storage
• ACFS – Oracle Cloud file system
– Background
– EBS on ACFS
• Appl Tier
• DB Tier (12c new features)
– Features
– Use cases

6. ACFS definition (I)
• ASM Cluster file system or Automatic Storage
Management Cluster file system
• ASM appeared with 10g
• ACFS extension introduced in 11gR2
• In January 2011 ACFS has been rebranded to
Oracle Cloud file system
– Oracle Cloud Storage = ASM + ADVM + ACFS
6

7. Cloud Storage Architecture
7

8. ACFS definition (II)
• Introduced in Grid Infrastructure 11gR2
• Built from the traditional ASM disk groups
• General purpose file system
• Uses standard file related system calls like others
file systems in the market
• Disk creation for ACFS is performed with a typical
workflow process
• Supported on RHEL/OEL/SLES/AIX/WINDOWS
– ACFS Support On OS Platforms (Certification Matrix).
(Doc ID 1369107.1)
8

9. From ASM to ACFS
9

10. ACFS Deployment
• Two types of ACFS file systems:
– CRS Managed ACFS (for example, ORACLE_HOME)
• Clusterwide by default
• Automatically creates a dependency with placed
resource (DB)
– General Purpose ACFS
• No structural differences
• Starting with Oracle Database 12c
– Database files
– Snapshots
– NFS HAVIP
10

11. ACFS Disk Space Usage
• Preallocates large user files to improve performance
when writing data
• Keeps local bitmaps available to reduce contention
on the global storage bitmap when searching for
free space
– This disk space is reported as in use by tools such as
the UNIX df command even though some of it may
not actually be allocated as of yet
11

12. Application Tier
12

13. EBS Appl Tier on ACFS
• https://blogs.oracle.com/stevenChan/entry/choosing
_an_ebs_shared_file_system
– Does Oracle Certify Storage Systems? - No. Not any
more.
– Does the E-Business Suite Division Certify Storage
Systems? – No. Oracle generally suggest that
customers either perform their own product testing or
consult a trusted consultancy that compares the
relative merits of each product against a consistent
set of criteria.
– What are the EBS Requirements for a Shared
Filesystem? - Shared filesystems must be transparent
to the calling application, in this case, the E-Business
Suite. In other words, no modifications to the E-
Business Suite should be necessary to ensure
compatibility with the shared filesystem.
13

14. Is EBS 11i/R12 certified with ACFS?
• No. Because Oracle Application Server 10g is not
certified with ACFS.
• “Customers interested in running Oracle Application
Server 10g (and the E-Business Suite) on ACFS are
encouraged to log SRs against the Fusion
Middleware product to request this certification. If
sufficient numbers of customers request it, they may
consider it”
• Can you still use it for, for example,
$APPLCSF/$APPLLOG, $APPLCSF/$APPLOUT,
$PATCH_TOP? – Yes.
14

15. What about R12.2/FMW/Weblogic?
• http://www.oracle.com/technetwork/database/availa
bility/maa-fmwsharedstoragebestpractices-
402094.pdf
• ACFS was found to be a highly performant solution
15

16. EBS running on ACFS (my experience) (I)
• Since 2009 – not faced a single issue (we are doing
our own product testing, right?)
– 11g
– 12c
– Unless there is a clusterware issue
• Multiple audits and performance benchmarking
made by Oracle
– Oracle never told us we are running on unsupported
configuration
– Oracle never told us to migrate from ACFS based
solution
– Oracle worked on SR’s related to EBS and ACFS
• Didn’t find a single justification why ACFS cannot be
certified with EBS
16

17. EBS on ACFS (from experience) (II)
• Two clusters: Database and Applications cluster
• Both having Grid Infrastructure installed and ASM
used
• On Applications cluster EBS application stack is
installed and concurrent managers running
• Other nodes are mounting ACFS using NFS
• Different mount points (ACFS filesystems) created
for log files, out files, etc.
17

18. You have options...
• Run EBS Applications stack on two node cluster
and utilize NFS for multi-node installation
• Run EBS on three or four (or more) node cluster
bypassing NFS
• Do not fully run the applications stack on ACFS, use
it for log file, out files, etc.
18

19. Architecture diagram
19

20. Fast Provisioning (I)
20

21. Fast Provisioning (II)
21

22. Fast Provisioning (III)
22

23. Fast Provisioning (IV)
• Online resizing, relocation
• 256 mounts supported
23

24. Fast Provisioning (V)
• Or use CLI
– scripting
– automatic provisioning
• mkdir -p /mnt/acfs
• ASMCMD> volcreate -G ACFS -s 5g --column 1 VOL1
• mkfs -t acfs <volume_device>
• mount -t acfs <volume_device> /mnt/acfs
24

25. Snapshots (I)
• First Copy-on-Write (FCOW) methodology
• Available under .ACFS
• Point-in-time view of a file
– ACFS snapshots can be very useful for file-based
recovery or for file system backups
• Dynamic and space efficient
• Recovery using copy and replace
• Use case – ORACLE_HOME patching
• Use case – EBS TEST/DEV environments cloning
• 1023 snapshots are supported
25

26. Snapshots (II)
• acfsutil snap create snap_ro /mnt/acfs
• acfsutil snap create -w -p snap_rw /mnt/acfs
• acfsutil snap info /mnt/acfs
• snapshot name: snap_rw
• snapshot location: /mnt/acfs/.ACFS/snaps/snap_rw
• parent name: /mnt/acfs
• snapshot creation time: Tue Apr 5 11:14:20 2016
26

27. Replication (I)
• “Data Guard” for general purpose files
• Enabled replication of ACFS file systems across the
network to a remote site
• Replication logs are used to replicate from primary
to standby asynchronously
• Supports ACFS Tagging, needs specific
configuration
27

28. Replication (II)
28

29. Tagging
• Assigns a common naming attribute to a group of
files
• Selects files with tag for replication
– /sbin/acfsutil tag set logfiles -r /u02/logfiles/*.log
– /sbin/acfsutil repl init primary -s
standby/standby@standby_site tagname logfiles ...
– Avoids entire file system replication
• Requires some additional rpms and configuration
29

30. 12c New Features (I)
• Highly Available NFS (HANFS)
30

31. 12c New Features (II)
• Snapshot enhancements: Oracle ACFS read/write
snapshots have been enhanced to support
snapshots created from an existing snapshot in the
same Oracle ACFS file system (snaps of snaps)
and snapshot conversions (read-only to read/write).
(use it in cloning/restoring)
• Flex ASM
31

32. Database tier
32

33. Background
• Starting with Oracle 12c, ACFS supports Oracle
Database data files
• Oracle Database Appliance - databases are hosted
on ACFS

34. Requirements (I)
• Database version >= 11.2.0.4
• Grid infrastructure 12c
• ASM/ACFS configured
• Snapshots require DB EE license
• Database parameter filesystemio_options=setall
34

35. Requirements (II)
• ACFS doesn’t necessarily require a multi-node
cluster installation (ASM instance can be created on
standalone setup too)
• However, dbca doesn’t like it:
• Workaround: create database manually using
CREATE DATABASE statement J
35

36. EBS installation on ASM
36

37. ...or ACFS
37

38. Database Rapid Provisioning using
Snapshots
• Same setup routine as described before with Appl
Tier
• Snapshot for consistent copy
• Datafile copy for fast media recovery
• Database cloning / rapid provisioning
• Some scenarios:
http://allthingsoracle.com/?s=ACFS
38

39. Some drawbacks
• As of now no way to natively restore a snapshot
– manual data file copy from the snapshot
• Not recommended to use Production instance as a
direct snapshot provider due to performance penalty
– Use standby DB
– Use secondary standby DB – replicated to test
– Production Database copy as a provider
39

40. Write performance /w snapshots
• http://www.ludovicocaldara.net/dba/oracle-
database-on-acfs-a-perfect-marriage/
40

41. The recommended approach
41
• Rapid DB copies, space-efficient
• Source should not be performance critical

42. Provision new DB from standby
42

43. gDBClone (I)
• Script that leverages Oracle ACFS snapshot
functionality
– Space-efficient copies for Test and Dev
– Full provisioning cycle
43

44. gDBClone (II)
• Automatically converts DBs from any type to any
type
• http://www.oracle.com/technetwork/indexes/sample
code/index.html
44

45. gDBClone (III)
• See the WP
• Script is for Test/Dev/Educational purposes only
• Still being developed
• One might use it as a base and enhance/customize
45

46. General Performance
• Almost the same
– slight penalty on ACFS
• SLOB tests:
– http://www.ludovicocaldara.net/dba/oracle-database-
on-acfs-a-perfect-marriage/
• Oracle response was the same
• Under different workloads there might be some
changes in benchmarks
• ACFS cannot be faster than ASM
46

47. Licensing
• Starting with Oracle 12c, no extra license required
for ACFS features
• DB Snapshots requires Enterprise Edition

48. Conclusion: ACFS Benefits
• A better (faster and more reliable) alternative to
traditional NFS
• Dynamic management
• No need to separate storage administrators and
ASM administrators
• Mount it anywhere with NFS
• Features
• Performance
• Database fast and efficient copies
– Rapid cloning and DB provisioning
• If configured and setup properly maybe a very
powerful and efficient solution
48

49. Some last minute tips on ACFS
• Limit of 4 resizes (need to retest in 12c)
– There is a limit of 5 extents for the file system's
internal storage bitmap. This causes any attempts to
increase the file system to fail after it has been
increased four or more times
– Solution - /sbin/fsck -a -v -y -t acfs /dev/asm/acfs-368
• When mounting application tier file system to other
node using NFS, use the following nfs parameters:
– rw,intr,bg,hard,timeo=600,wsize=32768,rsize=32768,
nfsvers=3,tcp,nolock,acregmin=0,acregmax=0
– The above are recommendations based on Oracle
Internal Testing
49

50. Thank you!
Questions?

51. My other EBS sessions at #C16LV