KSCOPE17 presentation

2. FDMEE Black Belt Techniques
Francisco Amores
Kscope17

3. Safe Harbor
• The following is intended for information purposes only on
behalf of Infratects, and may not be incorporated into any
contract. It is not a commitment to deliver any services or
functionality, and should not be relied upon in making
purchasing decisions. This provides no warranties with respect
to the offerings described within. The development, release,
and timing of any features or functionality described for
Oracle’s products remains at the sole discretion of Oracle. Any
third-party names, trademarks and copyrights contained in this
document are the property of their respective owners

4. My Safe Harbor
• This presentation is the result of many years sitting in front of
customers and squeezing my brain to meet their most complex
integration requirements
• This presentation contains custom code and solutions
• You can use it, but it’s at your own risk

5. About Infratects - Sucaba
DEEP EXPERTISE
HOSTING
CLOUD
DESIGN
CONFIGURATION
INSTALLATION
STRESS TESTING
UPGRADES
MIGRATIONS
PATCHING
FUNCTIONAL
INTEGRATION
SUPPORT
TRAINING
INFRASTRUCTURE
FOLLOW THE SUN
SUPPORT
80+ CONSULTANTS
15+ YEARS
EXPERIENCE
5 DATA
CENTERS
NA EMEA APAC
DEDICATED
MANAGED SERVICES
& SUPPPORT
TEAM
FOCUSED
17
IN BUSINESS
YEARS
WWW.INFRATECTS.COM
INFRATECTS FI
INFRATECTS UK
INFRATECTS NL2000
INFRATECTS AUS2008
2009
2012
INFRATECTS US2014
INFRATECTS SP2012
SUCABA JV2017
WWW.SUCABA.EU

6. A little bit about me
• Data Integration Specialist
• 10 years in Hyperion World
• 4th Kscope as a Presenter
• Link functional & technical
• FDMEE Blogger and Trainer
– fishing with FDMEE

7. My Goal for today…
I’d like you to be that kid
and learn something new today

8. What Black Belt means?
• What guides don’t show
• Building solutions
– Creativeness
– Extended Product Knowledge

9. • Data Extract (4)
• Mappings (4)
• Jython Scripting (7)
• Integration with EPM Apps (2)
• Automation & Scheduling (3)
• Bonus Track (5+1)
25 + 1 techniques

10. Data Extract

11. Source Filters behind the scenes
• Definition time  AIF_BAL_RULE_PARAMS
• Run-time  AIF_BAL_RULE_LOAD_PARAMS

12. HFM Member Lists
• Member Lists cannot be selected in the UI
• But can be typed!
• Example: use list {ProfitAndLoss} to filter Account dimension

13. Essbase Member Set Functions
• Some functions are not available in the UI
• But can be also typed 
• Example: use @UDA to extract additional accounts for Budget (BSO)

14. Essbase Member Set Functions
• Example: use @ATTRIBUTE to extract all products with class A100

15. Universal Data Adapter (UDA)
• Override filter values at run-time with BefImport event script
• Example: build list of companies as parameter for UDA

16. Extracting data through Web Services
• Good solution for
– ERP Cloud not natively supported (DBCS, OACS, Dynamics 365…)
– On prem ERP network ≠ EPM network
– ERPs not allowing data extract through DB layer
• Secured Web Service (SOAP/REST) exposed by Source System
• Seamless integration
– FDMEE calls a WS when user clicks Import
– FDMEE custom extractor (BefImport)
– File-based behind the scenes

17. Extracting data through Web Services (SAP)
• The WS executes an ABAP
program to extract data
– Parameters
• Data is included in a JSON
response
• Hybrid Extract
– Only Period and Prior Period
– Merge with last file = YTD

18. Mappings

19. Mappings based on sign
• Mapping script needed
– AMOUNT for source amount
– May need to adjust the query to
check sign for totals
Cloud
Friendly

20. Mappings based on Target Dimension Values
• Maps can be also driven by target values
• Bridge-Lookup dimensions
– No impact on Target Application
– #SQL map to copy target values into other source dim values
– Map Lookup just after dimension to be copied
Cloud
Friendly

21. Mappings based on Target Dimension Values
• Example: Map HFM Product based on HFM Account and Source Product
– Copy HFM account to source lookup
– Multi-dim map using the lookup dimension
Copied thru SQL Map

22. Mappings based on Target Dimension Values
• Example: IFRS mapping based on Target Account
1) Account is mapped
2) Lookup dimension copies target account into source IFRS
3) IFRS is mapped

23. Explicit Maps as Like Maps
• Processing Order
– Mapping Type = Explicit, Between, In, Multi-dim and Like
– Mapping Rule = alphabetical of rule name for any but Explicit
• Common Use
– Exceptions = Explicit
– Generic = Like
• What if?
– Execute MD before Explicit maps
– Conditional maps before Explicit maps
• Create Explicit as Like maps 
• Drawback: like maps are executed 1 by 1 so don’t abuse
Cloud
Friendly

24. Explicit Maps as Like Maps
1) Conditional mapping (SQL)
2) Explicit map as Like

25. Explicit Maps as Like Maps
• Everything works fine until we export/import maps
– FDMEE uses */? to identify mapping type
– Maps are imported as Explicit 
• Workaround
– Define your Explicit maps as Like maps having * as trailing character

26. #SQL Maps using HFM Metadata
• Leverage HFM Metadata in FDMEE maps
• We can create mappings based on
– Is ICP, Is Calculated, etc.
– Top Custom Members
– HFM Parent
– Account Type
– User Defined attributes
– Entity Currency
– Anything stored in the HFM DB

27. #SQL Maps using HFM Metadata

28. Scripting

29. Comprehension Lists
• Construct lists in a very natural way from any iterable object
• Alternative to build lists by appending elements matching conditions

30. Function isBeforeFirst() to check records
• Clean way to check if Select
statement returns records
• isBeforeFirst() returns
True if the cursor is before the
first record
• If no records, it returns False

31. Debugging Scripts (variables)
• Using locals() and
globals() to debug
variable’s value
– locals() returns a
dictionary of the current
namespace
– globals() returns the
dictionary of the module
namespace

32. Sharing values between Event Scripts
• AIF_PROCESS_LOGS table as temporary repository (APIs)
• Can also use Python modules like pickle (persistency)

33. Environment based Code
• Avoid different scripts across
environments (hardcoding)
• Map object (Dictionary) for code
based on FDMEE server
• Python module socket
• Same values as typing hostname
in the command line

34. Merging Source Files
• Python module shutil
• Write chunks to avoid big files into memory

35. Create HTML document for your Scripts
• Python module pydoc
– Nice look and feel HTML
– Command line

36. EPM Apps

37. Java and Web Service APIs (11.1.2.4)
Product JAVA Web Service
Planning No REST
Essbase/Provider Services Yes SOAP
LCM Yes No
FDMEE Yes* REST (PSU210)
HFM Yes REST*
DRM/DRG No SOAP/REST
HPCM No REST
Others CSS, OLU… NA
Cloud Services No REST

38. Using Essbase JAPI from FDMEE
• Library ess_japi.jar
– Validate target members
– Run calculation scripts before data is extracted
– Run Maxl scripts
– Get/Set substitution variables
– Refresh outline
– Among others

39. Running Maxl with Essbase JAPI
1) Import ess_japi.jar into Jython sys.path
2) Import Essbase JAPI Classes
3) Initialize JAPI instance
4) Sign-on to the Provider
5) Open Maxl Session
6) Execute Maxl Statement
7) Parse results

40. Using HFM Adapter API and HFM JAPI
• What can it be used for?
– Run Force Translate before data is extracted
– Run custom intersection check process
– Run custom consolidation processes (custom logic, period range…)
– Extract Cell Texts
• The HFM adapter comes with set of .py files (HFM_VALIDATE, …)
– Wrapper for HFM JAPI
– Can use functions in your custom code (not supported)

41. Translate HFM data before extracting
• Translate data for base entities with non AUD currency to level AUD Total
• Getting the Base Entities
– Function to get base entities with their currencies
– Get entities with currency other than AUD

42. Translate HFM data before extracting
• Getting Periods to be translated
– Periods being extracted from HFM

43. Translate HFM data before extracting
• Translating data with HFM Adapter API functions
– Have a glance at HFM_CONSOLIDATE.py
– Optionally insert a new entry in AIF_PROCESS_DETAILS

44. Automation

45. File Watcher
• FDMEE doesn’t have file watching capabilities yet
• Run Open Batch to capture files
• Execute the open batch only when there are files?
– Build your own File Watcher (Batch, Power Shell, Control-M, etc.)
– REST API (210+) or batch utility (runbatch)
– Can be scheduled with no impact on FDMEE
FDMEE
Open batchFile(s) found
Wait for
File(s)

46. File Watcher
1) Check files with open batch
naming convention
2) Move them to open batch
folder
3) Execute the open batch
through REST
• REST API for FDMEE is not
supported in On-premises

47. Enhancing the Standard Scheduler
• Running a DLR/batch/script every 5 minutes during x days?
• ODI offers more options that can be leveraged
– Period range
– Repeating cycle
– Etc.

48. Enhancing the standard Scheduler
• Create custom scheduling

49. Enhancing the standard Scheduler
• Deleting individual scheduling?
• Custom script to delete individual
schedule created

50. Email Notifications
• Not coming OOTB
• Automated Process
– Batch Results
– Unmapped Members
– Invalid Intersections
– Load Errors

51. Email Notifications – Intersection Check Report
• FDMEE  PDF Report OOTB
• Custom FDMEE  Send an HTML report with FDM Classic look & feel

52. Bonus Track

53. Multi Period/Scenario loads through single POV
• Multi-Period
– Requires process data through multiple POVs
– Ex: Anaplan Forecast File (5 Years) to HFM
• Multi-Scenario
– Requires multiple DLRs
– Ex: Planning (2 Scenarios) to Planning
• There is a way in FDMEE …if Drill-through is not required

54. Multi Period/Scenario loads through Single POV
• Lookup Dimension
– Import Period/Scenario
into Lookup
– Map for Lookup
• View AIF_HS_BALANCES
– Export Period/Scenario
from Lookup rather
than POV mapping
tables

55. Dynamic Validation Entity Groups
• Consolidate might fail due to non
existing HFM entities in the
Validation Entity Group
• Populate the group dynamically
based on Entities being loaded
• Can be enhanced with Parent
Entity taken from HFM hierarchy

56. Non-Blocking Intersection Check Report
• Standard Intersection Check Report
– Optional validation for HFM data loads
– PDF report
– Validate fails if there are invalid intersections
– Cannot load data until all errors are fixed (source/maps/targets)
– Partial loads if disabled (Errors are logged)
• Non-Blocking Intersection Check Report
– Validate doesn’t fail (only for mapping errors)
– Invalid intersections are flagged as IGNORE
– No errors when loading (only technical)
– Log invalid intersections and related source data

57. Non-Blocking Intersection Check Report
• Validate Passes
• Filter Ignored Data
• Can be easily filtered (INV_)
• Mappings applied can be logged

58. Replacing Source Filename on the fly
• How can I manually import an XML file?
• You select the file, the BefImport event script will do the rest
– Transform XML to CSV
– Update FDMEE DB Tables with new filename
• Import Format for CSV file

59. Custom Applications for Admin Purposes
• Standard functionality  generate a flat file for external systems
– Loading Exchange Rates into Planning/PBCS (Hybrid)
– Create write-back files for SAP BPC

60. Custom Applications for Admin Purposes
• Putting some creativeness on it, we can store
– Location owner’s email
– Output format for other custom target applications
– Parameters for our custom process
– …

61. Custom Applications for Admin Purposes
• Example: output format to export FDMEE data as SAP report files

62. REST API for FDMEE
• Introduced for DM in the Cloud
– Run DLRs/Batches
– Import/Export Maps
– Run Custom Scripts/Reports
– Refresh Application Dimensions
– Etc.
• PSU210+ introduced REST APIs for on-prem (not supported)
• REST allows external systems to launch FDMEE jobs
– EPM Automate/Groovy/PowerShell…
Cloud
Friendly

63. Contact Info
Email: francisco.amores@infratects.com
Mobile: +34674209000
LinkedIn: http://lnkd.in/dgpfvMW
Twitter: @akafdmee
Blog: http://fishingwithfdmee.blogspot.com