Improve the efficiency and accelerate job execution by moving traditional SAS workloads into Hadoop to modernize and optimize SAS analytics. How can we run traditional SAS® jobs, including SAS® Workspace Servers, on Hadoop worker nodes? The answer is SAS® Grid Manager for Hadoop, which is integrated with the Hadoop ecosystem to provide resource management, high availability and enterprise scheduling for SAS customers. By moving SAS workloads inside the Hadoop cluster, efficiency is improved and job execution is accelerated. We will also cover the role of Hadoop YARN, Hadoop Distributed File System (HDFS) storage, and Hadoop client services. We review SAS metadata definitions for SAS Grid Manager, SAS® Object Spawner, and SAS® Workspace Servers. Audio broadcast: https://hortonworks.com/webinar/configuring-sas-grid-manager-hadoop/

1. 1 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Analy>cs
Moderniza>on:
Configuring SAS® Grid
Manager for Hadoop
Mark Lochbihler, Principle Architect
Hortonworks
April 21, 2017

2. 2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Presenter
Presenter: Mark Lochbihler - Hortonworks, Inc.
Mark is a Principal Architect with 27 years of SAS
experience, having spent 17 years within Financial
Services. He is currently in his fourth year at
Hortonworks and is focused on integra>ng the Hadoop
ecosystem with strategic partner ecosystem products
and solu>ons. Mark has a BS in Computer Science from
North Carolina State University and also holds a Six Sigma
Black Belt.

3. 3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Agenda
à Why
à Architecture
à Real world Sizing
à Configura>on Details
à Demo - User Perspec>ve
à Migra>on Considera>ons
à Call to Ac>on

4. 4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Why Move SAS Workloads to Hadoop?
à Lower infrastructure and storage costs
à Op>mize performance
à Minimize administra>ve overhead

5. 5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Clickstream Web
& Social
Geolocation Sensor
& Machine
Server
Logs
Unstructured
SOURCES
Existing Systems
ERP CRM SCM
ANALYTICS
Data
Marts
Business
Analytics
Visualization
& Dashboards
ANALYTICS
Applications
Business
Analytics
Visualization
& Dashboards
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
°
HDFS
(Hadoop Distributed File System)
YARN: Data Operating System
Interactive Real-TimeBatch
SAS Grid
Manager
Batch BatchMP
P
EDW
Hadoop and YARN 101
Figure 1: A Hadoop Cluster running Batch, Interac>ve and Real Time Engines,
including SAS Grid Manager.

6. 6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Figure 2: SAS Grid Manager for Hadoop Conceptual Architecture
(Reference: SAS Grid Manager for Hadoop)

7. 7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
KEY SAS GRID ARCHITECTURE COMPONENTS
à SAS Metadata Server – A SAS service suppor>ng, among other objects, the logical to
physical mapping of SAS Logical Servers to YARN. In our examples, we will be using
the local SAS Server, SASGrid.
à SAS Grid Control Server – A SAS service running on the YARN Resource Manager
node. It is called by SAS clients to communicate with YARN Resource Manager to
nego>ate resources for SAS jobs.
à SAS Object Spawner – A SAS Service running on the YARN Resource Manager node.
It is used to launch SAS contains with YARN.
à SAS Clients – includes SAS batch jobs, SASGSUB (a batch grid u>lity), and interac>ve
Clients like SAS Enterprise Guide. SAS Clients will “Connect” and “Disconnect” from
a SAS Logical Server, like SASGrid, defined in SAS Metadata and shown above in
Figure 4.

8. 8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
KEY HADOOP INTEGRATION POINTS FOR SAS GRID
à YARN Resource Manager – A Hadoop YARN Master Service responsible for
controlling global Hadoop cluster resource usage. Resource Manager enables mul>-
tenancy and SLAs. It is also responsible for monitoring Node Manager State,
submieng Applica>on Master requests, verifying container launch and monitoring
Applica>on Master state.
à YARN Node Manager – This Hadoop YARN Worker Node Service manages local
resources on behalf of the reques>ng service. It also tracks node health and
communicates status to the Resource Manager.
à YARN Capacity Scheduler – A Hadoop YARN service, which can be configured to
provide Job Scheduling policies for SLAs, Users, Groups, and Resources.
à Hadoop Data Nodes – Hadoop Distributed File System (HDFS) storage nodes.
à Kerberos Service – The Hadoop cluster must be Kerborized.

9. 9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
à A SAS Grid Control Server and SAS Object Spawner must be deployed on the same
Hadoop Master Node as the YARN Resource Manager.
HADOOP MASTER NODE DECISIONS

10. 10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
YARN NodeManager YARN NodeManager YARN NodeManager YARN NodeManager
Job1 Container 1.1
YARN NodeManager YARN NodeManager YARN NodeManager YARN NodeManager
YARN NodeManager YARN NodeManager YARN NodeManager YARN NodeManager
Job1 Container 1.2
Job1 Container 1.3
Job 1 AM 1 SAS AM 2
SAS Grid Manager w YARN Architecture Overview
SAS Client
• SASGSUB
• SAS Batch
• SAS EG
YARN ResourceManager
YARN
Capacity
Scheduler
SAS Grid
Control Server
SAS Object
Spawner
SAS Metadata
Server
SAS Container 2.1
Figure 3: SAS Grid Manager for Hadoop w YARN Architecture Overview

11. 11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
HADOOP WORKER NODE DECISIONS
à SASHOME and SASCONFIG
For each Hadoop Worker Node which is a candidate to run SAS jobs must be configured so that
SASHOME and SASCONFIG are available.
à SASWORK and SASUTIL
It is cri>cal that each Hadoop Worker node which is a candidate to run SAS jobs is configured
correctly. A large part of the I/O required when running SAS analy>cs is to the scratch or temporary
loca>ons of SASWORK and SASUTIL. SAS required I/O throughput for these file systems, to provide
the necessary performance to a heavily loaded system, is 100MB/sec/core. Adequate sizing for
SASWORK is also necessary.
à TradiRonal Storage and Compute Verse Compute Only Worker Nodes
Within Hadoop, it is a common prac>ce to have dual purpose worker nodes which run math or programs near on the
same nodes where the Hadoop data resides. With Hadoop 2.x, the concept of dedicated Compute Only Hadoop Worker
Nodes is an op>on. For SAS Grid Manager for Hadoop, both op>ons are an op>on. For Compute Only, these Hadoop
Worker Nodes will no longer host the required services and data for HDFS, giving more compu>ng resources dedicated to
the programs running on these nodes. The tradeoff for Compute Only Hadoop Nodes is the loss of HDFS data locality.
Your sites SAS workload requirements will determine which type of Worker Nodes to deploy for SAS Grid Manager for
Hadoop.

12. 12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Figure 4: View of SAS Management Console, with expanded SASGrid Logical Server.

13. 13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
REAL WORLD CONFIGURATION EXAMPLE
Total RAM
Per Cluster
Node
Available
Container RAM
Per Node
# Worker
Nodes in
Cluster
Total
Container
RAM
Available
Amount of Cluster
RAM Allocated to
SAS Queue
Total Container
RAM for SAS
Queue
256GB 192GB 28 5.376TB 50% 2.688TB
Average # of Batch Jobs or
InteracRve Sessions per
SAS User
# Containers Per
Job or Session
Average # of AddiRonal Hadoop
Containers Spawned from iniRal
SAS Job Container
Average Total # of
Containers per SAS
user
2 2 4 8
Table 1: Total Cluster YARN Container RAM Available for SAS Users
Table 2: Average Number of Containers per SAS Users

14. 14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
REAL WORLD CONFIGURATION EXAMPLE (ConRnued)
SAS AppType
(User Type)
Container
Size
AnRcipated % of
Users Type on Server
Available Cluster
Memory for
SAS jobs
Max # of
Containers
Average #
Containers Per
SAS User
Total # of
SAS Users
Low (General/Analyst)
2GB 70% 1.881TB 940 8 117
Medium 4GB 20% 537GB 134 8 16
High 8GB 10% 268GB 33 8 4
Totals 2.688TB 1107 134
Table 3: Breakdown of SAS Applica>on Types to be configured for SAS Users

15. 15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
YARN Capacity Scheduler
Example: 50% of Cluster RAM allocated to SAS Queue
ResourceManager
Scheduler
root
Adhoc
30%
SAS
50%
Mrkting
20%
Dev
10%
Reserved
20%
Prod
70%
Prod
80%
Dev
20%
P0
70%
P1
30%
Capacity Scheduler
Hierarchical
Queues
REAL WORLD CONFIGURATION EXAMPLE (ConRnued)
Figure 5: YARN Capacity Scheduler Logical View - SAS Queue - 50% Hadoop Cluster RAM

16. 16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
REAL WORLD CONFIGURATION EXAMPLE (ConRnued)
Figure 6: YARN Capacity Scheduler Admin View - SAS Queue - 50% Hadoop Cluster RAM

17. 17 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
(SAS Grid Policy File - Grid Applica>on Type “Low” sec>on)
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<GridPolicy defaultAppType="low">
<GridApplica+onType name="low">
<jobname>SAS Low</jobname>
<priority>10</priority>
<nice>0</nice>
<memory>2048</memory>
<vcores>1</vcores>
<runlimit>480</runlimit>
<queue>sas94_queue</queue>
<hosts>
<hostGroup>sas94_work</hostGroup>
</hosts>
</GridApplicaUonType>
………….…………. ConUnued in paper ………………………………..
REAL WORLD CONFIGURATION EXAMPLE (ConRnued)

18. 18 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
REAL WORLD CONFIGURATION EXAMPLE (ConRnued)
Figure 7: Configuring SAS Metadata Groups to SAS Grid Applica>on Types

19. 19 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
A DAY IN THE LIFE OF A SAS USER LEVERAGE SAS GRID
DEMO

20. 20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
SAS WORKLOAD MIGRATION CONSIDERATIONS
à Compliment your exis>ng SAS Infrastructure - its not a forklio migra>on
à Iden>fy SAS Storage Cost Saving Opportuni>es
• libname to hive
• libname to hdfs
• filename to hdfs
à Iden>fy SAS Workload Compute Migra>on Opportuni>es
• SAS Jobs that will be using large datasets stored in Hadoop are ideal
candidates
• SAS Jobs that would benefit from SAS In Database Push Down to Hive

21. 21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Call to AcRon
à Learn More: Click apachments tab in Brighpalk for links
– Or search Hortonworks.com for Analy>cs Moderniza>on
à Get Started:
– Download the Hortonworks Sandbox:
– click Get Started on hortonworks.com
à Contact Us: Call or fill out the contact form
– click Get Started on hortonworks.com

22. 22 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Thank You