In this presentation, Oracle ACE Brian Marshall focuses on making sure that your transition to a virtualized environment is as smooth as possible. He discusses specific settings in the virtual environment, the operating system, and Essbase to get the most out of your server. Download the presentation to learn: Why IT took your Essbase server An explanation of why your performance sucks now that your server is gone How to fix performance How to prove your performance is better once you've fixed it And more!

2. 2
AGENDA Introductions1
IT Took My Essbase Server…Why?2
Performance is Awful…Why?3
How Do We Fix It?4
EssBench5
Show Me the Benchmarks!6
Q&A7

3. 3
About Brian Marshall
 VP of Delivery
 19+ years IT and EPM/BI Experience
 14+ years with US-Analytics
 100+ projects with US-Analytics
 Presented at Kscope every year since 2010
 Frequent blogger at HyperionEPM.com EPMMarshall.com

4. 4
About US-Analytics
Dallas-based, Hyperion-focused for over 15 years with continuous business growth
 We are nimble and respond quickly to customers’ needs
 Over 500 clients and over 1,000 successful Hyperion engagements
 Seasoned business and technical acumen with EPM and BI initiatives
 Over 65 professionals with 12+ years each of Hyperion experience and certifications
 Active leaders in the Oracle community
 Founder of Hyperion Professional Women's Forum, advisory board leadership, conference
presentations, webinars, EPM Speaker of the Year at Kaleidoscope in 2015 and 2014
 Corporate culture of integrity with 100% customer commitment
 Managed services
 Managed services team is Dallas based, each with 10+ years of experience
 Proven processes for all aspects of managed services

5. ABOUTUS-ANALYTICS
Managed Services
Upgrades & Migrations
Implementations
Infrastructure
Process & Advisory
Services
Big Data
Data Governance
Business Intelligence
Financial Close & Consolidation
Planning & Forecasting
Solutions
Data IntegrationTraining
Accolades
– Original Oracle Hyperion and Pillar Partner
– Oracle Hyperion Financial Management 11
– Oracle Hyperion Planning 11
– Oracle Essbase 11
– Oracle Data Relationship Management 112013, 2014, 2015
2015 Oracle TOLA
EPM Partner of the Year

6. 6
IT Took My Essbase Server…Why?
 More physical servers are more difficult and costly to support
▶ Power requirements
▶ Various hardware configurations requiring various levels of support
▶ Physical servers are naturally inefficient due to the amount of overhead compared to
a virtualized environment
 More difficult to rapidly backup and restore
▶ Snapshots are handy
▶ High availability can be handled entirely inside of the host environment while the
guest is completely unaware, but still fault tolerant
 If you are at this point, your server is likely due to be replaced anyway
 Essbase servers, unlike relational databases, are more likely to only serve one
business group

7. 7
Performance is Awful…Why?
 Some reasons are pretty simple
▶ Your old server had a lot of processors…and now you don’t
▶ Your old server had a lot of memory…and now you don’t
▶ Your old server had local storage…and now you don’t
 Some reasons are more complex
▶ Your IT group has put you on a host that is overprovisioned to the point of
performance degradation
▶ The storage sub-system on many hosts don’t play nice with Essbase

8. 8
How Do We Fix It?
 First we have to identify the problem
▶ Processors
▶ Memory
▶ Storage It’s probably this one…
 Next we have to prove it
▶ Often times when we tell IT that the problem is on their side, they automatically
come back and blame the application
▶ Clearly the problem is Essbase, right?
▶ Wrong…but we still have to prove it

9. 9
Processors
Identifying Bottlenecks
 When going to a virtual platform, CPU issues will usually present in one of three
ways:
▶ A different number of logical cores than your original physical server
‒ IE: Your physical server has 16 logical cores and your new virtual machine only has 8…or 4
▶ A different speed or generation of process
‒ You may be transitioned to a virtual host that has older hardware than your Essbase server
‒ You may be transitioned to a virtual host with much slower processors
‒ The emphasis on a hypervisor (virtual host) is the number of cores, not the speed of those cores
‒ In most instances, Essbase will benefit from speed over cores, so this presents a problem
▶ Overprovisioned Host
‒ This one is a little trickier
‒ If everything else has been eliminated, ask IT to provide a graph of the utilization of the host

10. 10
Processors
Identifying Bottlenecks
 When going to a virtual platform, memory issues will usually present in one
of three ways:
▶ A lower amount of memory than the physical server
‒ If you transition to a virtualized system that has half the RAM of your physical Essbase server but keep the
settings the same, you may max our your RAM
‒ Once you run out of RAM, things start to page, errors start to occur, things grind to a hault
▶ Overprovisioned Host
‒ Like processor issues on the host, this one is a little trickier
‒ Again, let’s ask IT to provide a graph of the utilization of the host

11. 11
Storage
Identifying Bottlenecks
 There’s a high probability that this is the problem
 Many physical Essbase servers are running direct attached storage
 It is also common for that direct attached storage to be a solid state disk
(SSD)
 Essbase is sensitive to three performance characteristics of a storage
system:
▶ Random Read/Write Performance (Most Important)
▶ Latency (Directly impacts Random Read/Write Performance)
▶ Sequential Read/Write Performance (Least Important)

12. 12
Storage (cont.)
Identifying Bottlenecks
 Many virtualized environments run on clustered file systems
▶ Hugely beneficial for things like high availability and backups
▶ Very high latency and often challenged at random disk I/O
 Single Essbase applications are not capable of operating a high queue
depths
 High queue depths are generally the only way to get decent random I/O
performance out of a clustered file system
 When IT tells you that the SAN is “soooo fast”, it very well may be for
certain things
 Most of those things don’t matter to Essbase

13. 13
Storage (cont.)
Identifying Bottlenecks
 Run your own non-Essbase benchmarks
 Specifically focus on random (4K) performance
P3605 Locally Attached P3605 via iSCSI
Uh oh…

14. 14
Storage (cont.)
Identifying Bottlenecks
 Other things to look for:
▶ Watch your CPU utilization
‒ If you have CALCPARALLEL at 8 and you are using less than 8 cores, you either need better cache settings, or more
than likely you need better disk performance
▶ Watch you disk throughput
‒ Don’t expect disk throughput to be very, very high
‒ Most Essbase operations are done using random reads and random writes
‒ Random reads and random writes at a very low queue depth
‒ On traditional spinning disks, even with RAID, these numbers look more like 1-5MB/s
‒ On SSD’s, these will be much higher, but still lower than you might expect

15. 15
How do we prove it?
 Option 1: Test your application on your old system and compare it to your new
system
▶ This presents a problem as the old system has a variety of differences, including the
version of Essbase most likely
 Option 2: Test your application on an independent system and compare it to
your new system
▶ This can be done, but it sounds expensive to either borrow another companies
instance or have your consultant do this
 Option 3: Test a standardized benchmark application and compare it to other
tested configurations
▶ If only something like this existed, this would be the best option, right?

16. 16
Introduction
EssBench
 But wait…something like this should exist.
 This presentation serves as the launch of a new standardized benchmarking
application.
 Nothing helps you help IT more than identifying the problem for them.
 This application will help.
 This is an Essbase model with data and processes designed to make your server
say uncle.
 Additionally, the EssBench.com website will contain a full database of tested
configurations for you to compare.
 Again, the more information we can provide to IT, the more likely we are to
figure out a good solution.

17. 17
Introducing:

18. 18
Essbase Application
EssBench
 BSO Application
 Dimensions
▶ Account (1025 members, 838 stored)
▶ Period (19 members, 14 stored)
▶ Years (6 members)
▶ Scenario (3 members)
▶ Version (4 members)
▶ Currency (3 members)
▶ Entity (8767 members, 8709 stored)
▶ Product (8639 members, 8639 stored)
 Data
▶ Millions of rows of data
▶ 10+GB of .pag and .ind files

19. 19
The Benchmark
EssBench
 PowerShell Scripts
▶ Creates Log File
▶ Executes MaxL Commands
 MaxL Scripts
▶ Resets the cube
▶ Loads data (several rules)
▶ Aggs the cube
▶ Executes allocation
▶ Aggs the allocated data
▶ Executes currency conversion
▶ Restructures the database
 Executes three times…average the results

20. 20
Introduction
Show Me the Benchmarks!
 Wait, wait, wait…what are we benchmarking with?
 Physical Server Specifications:
▶ 2x Intel E5-2670 Processors
‒ 8 cores, 16 threads, 2.6GHz
‒ 16 cores, 32 threads total
▶ 128GB DDR3 Memory
‒ 16x8GB DIMMs
▶ Various storage options
‒ Single Samsung 850 EVO
‒ Four Samsung 850 EVO’s in RAID 0
‒ Twelve 15,000RPM SAS Drives in RAID 1+0
‒ Single Intel P3605 NVMe SSD
‒ Network Attached Storage

21. 21
Show Me the Benchmarks!
 And where do you keep all of this?
▶ In my garage of course, where do you
keep your servers?
 And how are you still married?
▶ Blind luck…
▶ Oh, and an understanding wife

22. 22
Physical vs. Virtual
Show Me the Benchmarks!
 Same physical server for all tests
 Boot to Windows Server 2012 R2,
execute benchmark with NVMe storage
 Boot to ESXi 6.5, start virtual machine,
execute benchmark with NVMe storage
Physical Virtual
Performance
Difference
Parallel Native Load 97 112 -15%
CSV Data Load Rule 296 391 -32%
Aggregation 542 654 -21%
Allocation 483 516 -7%
Aggregation 523 662 -27%
Currency Conversion 266 353 -33%
Dense Restructure 407 464 -14%
Total 2613 3151 -21%

23. 23
Storage Provisioning
Show Me the Benchmarks!
 Same storage
device (NVMe)
 Three separate
drives, each
provisioned
differently
Physical Virtual (Thin)
Performance
Difference
Virtual
(Thick Lazy)
Performance
Difference2
Virtual
(Thick Eager)
Performance
Difference3
Parallel Native Load 97 109 -13% 111 -15% 112 -15%
CSV Data Load Rule 296 397 -34% 404 -36% 391 -32%
Aggregation 542 643 -19% 644 -19% 654 -21%
Allocation 483 515 -7% 515 -7% 516 -7%
Aggregation 523 661 -26% 664 -27% 662 -27%
Currency Conversion 266 344 -29% 356 -34% 353 -33%
Dense Restructure 407 478 -17% 449 -10% 464 -14%
Total 2613 3147 -21% 3142 -21% 3151 -21%

24. 24
Essbase Native Load
Physical vs. Virtual
 Eight native Essbase load files
 No rule necessary
 Loading in parallel
Physical Virtual
Performance
Difference
Intel P3605 97 117 -21%
4x Samsung EVO 850 in RAID 0 102 118 -16%
Samsung EVO 850 on LSI SATA 6g 107 113 -5%
iSCSI NVMe 124 165 -34%
iSCSI HDD w/ SSD Cache 113 145 -28%
12x 15,000RPM HDD in RAID 10 342 380 -11%

25. 25
Physical Virtual
Performance
Difference
Intel P3605 295 391 -33%
4x Samsung EVO 850 in RAID 0 419 559 -33%
Samsung EVO 850 on LSI SATA 6g 410 546 -33%
iSCSI NVMe 910 1392 -53%
iSCSI HDD w/ SSD Cache 954 1480 -55%
12x 15,000RPM HDD in RAID 10 0 0 0%
Essbase Load Rule
Physical vs. Virtual
 Single file, single threaded
 Roughly 1GB of data
 Basic load rule with no
manipulation

26. 26
Physical Virtual
Performance
Difference
Intel P3605 404 561 -39%
4x Samsung EVO 850 in RAID 0 448 606 -35%
Samsung EVO 850 on LSI SATA 6g 476 616 -29%
iSCSI NVMe 523 1056 -102%
iSCSI HDD w/ SSD Cache 521 766 -47%
12x 15,000RPM HDD in RAID 10 1907 3290 -72%
Aggregation
Physical vs. Virtual
 Aggregation of two sparse
dimensions (~8000 members
each)
 CALCPARALLEL set to 16

27. 27
Physical Virtual
Performance
Difference
Intel P3605 478 513 -7%
4x Samsung EVO 850 in RAID 0 490 527 -8%
Samsung EVO 850 on LSI SATA 6g 493 529 -7%
iSCSI NVMe 518 653 -26%
iSCSI HDD w/ SSD Cache 533 594 -12%
12x 15,000RPM HDD in RAID 10 996 1101 -11%
Allocation
Physical vs. Virtual
 Simple allocation
 FIXPARALLEL set to 8

28. 28
Physical Virtual
Performance
Difference
Intel P3605 420 600 -43%
4x Samsung EVO 850 in RAID 0 493 666 -35%
Samsung EVO 850 on LSI SATA 6g 515 678 -32%
iSCSI NVMe 745 1706 -129%
iSCSI HDD w/ SSD Cache 654 1064 -63%
12x 15,000RPM HDD in RAID 10 4042 16224 -301%
Targeted Aggregation
Physical vs. Virtual
 Aggregation of the allocated
account for both sparse
dimensions
 FIXPARALLEL set to 16

29. 29
Physical Virtual
Performance
Difference
Intel P3605 420 600 -43%
4x Samsung EVO 850 in RAID 0 493 666 -35%
Samsung EVO 850 on LSI SATA 6g 515 678 -32%
iSCSI NVMe 745 1706 -129%
iSCSI HDD w/ SSD Cache 654 1064 -63%
12x 15,000RPM HDD in RAID 10 4042 16224 -301%
Targeted Aggregation
Physical vs. Virtual
 Let’s see if the graph makes a
little more sense without our
hard drive option

30. 30
Currency Conversion
Physical vs. Virtual
 Simple currency conversion
 FIXPARALLEL set to 16
Physical Virtual
Performance
Difference
Intel P3605 306 422 -38%
4x Samsung EVO 850 in RAID 0 375 445 -18%
Samsung EVO 850 on LSI SATA 6g 375 448 -20%
iSCSI NVMe 1315 2036 -55%
iSCSI HDD w/ SSD Cache 597 823 -38%
12x 15,000RPM HDD in RAID 10 3814 6876 -80%

31. 31
Currency Conversion
Physical vs. Virtual
 One more time without our
hard drive option
Physical Virtual
Performance
Difference
Intel P3605 306 422 -38%
4x Samsung EVO 850 in RAID 0 375 445 -18%
Samsung EVO 850 on LSI SATA 6g 375 448 -20%
iSCSI NVMe 1315 2036 -55%
iSCSI HDD w/ SSD Cache 597 823 -38%
12x 15,000RPM HDD in RAID 10 3814 6876 -80%

32. 32
Restructure
Physical vs. Virtual
 Force restructure
Physical Virtual
Performance
Difference
Intel P3605 369 433 -17%
4x Samsung EVO 850 in RAID 0 384 446 -16%
Samsung EVO 850 on LSI SATA 6g 382 449 -17%
iSCSI NVMe 380 528 -39%
iSCSI HDD w/ SSD Cache 385 461 -20%
12x 15,000RPM HDD in RAID 10 421 729 -73%

33. 33
Total Time
Physical vs. Virtual
 One last time without our
spinning rust for comparison
Physical Virtual
Performance
Difference
Intel P3605 2368 3037 -28%
4x Samsung EVO 850 in RAID 0 2711 3368 -24%
Samsung EVO 850 on LSI SATA 6g 2758 3378 -22%
iSCSI NVMe 4514 7537 -67%
iSCSI HDD w/ SSD Cache 3757 5333 -42%
12x 15,000RPM HDD in RAID 10 11523 28601 -148%

34. 34
Everything But a Hard Drive
Physical vs. Virtual

35. 35
Delete a few…
But I Have More Than One App
 Or…with the right storage
and server, it doesn’t
matter
 Essbase may not have a
high enough queue depth
to take advantage of an
NVMe drive the right
way…with one app
 But with more than one,
we can force the issue
 You likely have more than
one anyway
EssBch12 EssBch13 EssBch12 (P) EssBch13 (P)
Parallel Native Load 111 112 131 126
CSV Data Load Rule 404 391 418 415
Aggregation 644 654 706 698
Allocation 515 516 525 524
Aggregation 664 662 761 751
Currency Conversion 356 353 520 490
Dense Restructure 449 464 546 542
Total 3142 3151 3607 3546

36. 36
EssBch19 EssBch20 EssBch19 (P) EssBch20 (P)
Parallel Native Load 145 145 167 168
CSV Data Load Rule 1480 1480 1645 1606
Aggregation 766 766 880 864
Allocation 594 594 552 540
Aggregation 1064 1064 1234 1198
Currency Conversion 823 823 957 898
Dense Restructure 461 461 517 520
Total 5333 5333 5953 5793
An a SAN
But I Have More Than One App
 Network attached storage
suffered from a very high
amount of latency…more
on that in a second
 But in general, they can
offer great throughput
even on random reads and
write if we give it more
queue depth
 Two applications on iSCSI:

37. 37
 Let’s take a benchmark break
 iSCSI has a lot of additional overhead, especially in an virtualized environment
 Let’s first take a look at direct attached storage:
Physical
Direct Attached Storage
iSCSI
Virtual

38. 38
 Network attached storage bring with it more complexity and as a result…more
latency
 Latency killed random read and write performance in low queue depths
(which Essbase operates on)
 Let’s take a look at network based storage:
Physical
Network Based Storage
iSCSI
Virtual

39. 39
Fixing the Problem with IT
Now What?
 This is all great, but now what?
 First, take your information to IT
 Tell them which pieces are slow and see what options they have:
▶ Ask if they can provide direct attached storage to your guest
▶ Ask for more processors (if needed)
▶ Ask for more memory (if needed)

40. 40
Fixing the Problem without IT
Now What?
 If you are going virtual, hopefully you are also upgrading
 Take advantage of new functionality:
▶ RESTRUCTURETHREADS
▶ FIXPARALLEL
 Do as much in memory as possible
 Disk IO is your enemy

41. 41
IT Still Can’t Help Me
 So you took your findings to IT, and they basically can’t help you
 This is not all that uncommon, why?
▶ They have a finite number of resources to spend time investigating things
▶ They have a finite amount of hardware that only gets refreshed once it becomes fully
depreciated
▶ They are fighting a turf war and you might be losing (just kidding…I hope)
 So what then?

42. 42
Let’s Check Out The Cloud
 Let’s start with the cloud leader
(I’m not looking at you Oracle):
Amazon
 Amazon released their new I3
series of servers with NVMe storage
and very fast processors with lots
of RAM:

43. 43
Cloud Options
 Amazon has the I3 series among others
 Microsoft and its Azure cloud has the L series of servers with direct attached
SSD storage
 Google has their database-tuned servers as well
 Oracle has their DenseIO series with NVMe storage
 Watch out for any cloud-based application using “Block Storage”
 This is shared, storage that is generally network attached…aka slow for
Essbase
 But will they actually perform well compared to physical boxes?

44. 44
AWS Essbase Performance
 I was fortunate enough to have a client ask me that question just in time
for Kscope17.
 In a word…YES
Physical AWS
Performance
Difference
Parallel Native Load 97 85 13%
CSV Data Load Rule 295 332 -13%
Aggregation 404 383 5%
Allocation 478 422 12%
Aggregation 420 446 -6%
Currency Conversion 306 273 11%
Dense Restructure 369 282 24%
Total 2368 2220 6%

45. 45
Sample Client Performance
 Fastest non-Exalytics
servers I’ve worked with
 But connected to a SAN
▶ EMC Symmetrix
Mine (NVMe) Mine (iSCSI) Client (SAN)
Parallel Native Load 97 113 97
CSV Data Load Rule 295 954 814
Aggregation 404 521 554
Allocation 478 533 394
Aggregation 420 654 715
Currency Conversion 306 597 410
Dense Restructure 369 385 292
Total 2368 3757 3277

46. 46
What about PBCS?
 Things I shouldn’t present at an Oracle conference…

47. 47
A Few Thank You’s
 My Wife for letting me build a datacenter at home
 My Company (US-Analytics) for helping me out with some of the hardware
 A couple of colleagues who I probably would have stalled and not completed
this project:
▶ Jake Turrell
▶ Tim German
 My clients for letting me play with their hardware endlessly

48. 48
Shameless Plug
 Visit my blog:
▶ EPMMarshall.com
▶ Formerly HyperionEPM.com
 Visit my benchmark:
▶ EssBench.com
 Connect to #orclepm on twitter:

49. 49
Q&A