The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting tips

#ATM15 |
The Aruba Tech Support Top 10 Tips
Tarun George & Gowri Amujuri
March 2015
@ArubaNetworks

CONFIDENTIAL © Copyright 2015. Aruba Networks, Inc. All rights reserved2#ATM15 |
WLAN Design, Configuration and Troubleshooting
Tips by TAC
@ArubaNetworks

3 CONFIDENTIAL © Copyright 2015. Aruba Networks, Inc. All rights reserved#ATM15 |
• Segmental Troubleshooting
• AP Stability and System profile Optimization
• Optimize load on processes
• Datapath Debugging
• Deployment Tips
Aruba OS
@ArubaNetworks

Transition Content
#1
Segmental Troubleshooting
@ArubaNetworks

Transition Content
Segmental Troubleshooting gains.
# Faster root cause analysis.
# One time Data Collection
# Bring focus on the smallest segment in the network within our control.
@ArubaNetworks

Transition Content
Where do we start if
we are unsure of the
exact cause of the
current issue being
faced?
@ArubaNetworks

Transition Content
User
show tech-support user mac <Mac Address>
tar logs user mac <User Mac > tech-support
User Debugging
Logging Level debugging user-debug <Mac
Address>
@ArubaNetworks

Transition Content
AP
show ap tech-support ap-name <Name of AP>
show ap debug counters
show ap bss-table ap-name
show ap debug system-status ap-name
@ArubaNetworks

Transition Content
Controller
show tech-support
tar log tech-support
Outside world
Debugging for Specific process/Sub-cat.. (Explained)
Pcap
show interface gigabitethernet <slot/module/port>
Network Diagram
Note: Show tech-support <filename> Store output in file.
@ArubaNetworks

Transition Content
Segmental Troubleshooting.. Processes
show process monitor statistics
Process Monitor Statistics
Name State Restarts Allowed Restarts Timeout Value Timeout Chances Time Started
/mswitch/bin/dbstart PROCESS_RUNNING 8 0 240 3 Sat Feb 28 21:31:55 2015
/mswitch/bin/packet_filter PROCESS_RUNNING - 0 240 3 Sat Feb 28 21:31:56 2015
Mdns , httpd_wrap , Authmgr ,STM , WMS , cfgm , dhcp
@ArubaNetworks

#2
AP Stability and System Profile Optimization
@ArubaNetworks

AP System Status.
Campus and Remote APs have similar challenges to stay connected to the
controller.
Health Check of the AP is vital, since it can trigger
client and controller anomalies.
Show AP debug system-status ap-name <Name of AP>
@ArubaNetworks

• Reboot Information
DHCP/Controller/Keep Alive miss
• Rebootstrap Information
Date Time Reason (Latest 10)
LMS Change/Heartbeat Miss
• HA Failover Information
Date Time Reason (Latest 10)
@ArubaNetworks

• Recent Control Messages from AP to Controller
Date Time Message Description
Sun Mar 1 12:29:49 2015(164 secs ago): SENT REQ
type=KEEPALIVE len=45 peer=10.163.196.72 seq_num=4567
num_attempts=1 rtt=0 secs
• Rebootstrap LMS
• Crash Information
@ArubaNetworks

• CPU and Memory Usage
Timestamp CPU Util(%) Memory Util(%)
2015-03-01 12:32:27 2 24
• Peak CPU Util in the last one hour
Timestamp CPU Util(%) Memory Util(%)
2015-03-01 12:19:25 3 24
@ArubaNetworks

Heartbeat Stats of Serving Controller
Heartbeats Sent Sent Seqnum Heartbeats Received Rcvd Seqnum MTUs sent Misc sent Measurement Duration
2690183 25824 2667575 25824 22607 0 since last rebootstrap
2690193 n/a 2667575 n/a 22607 0 total since bootup
Interface counters
Interface Rx_pkts Rx_errors Rx drops Tx_pkts Tx_errors Tx_drops Resets
wifi0 3209433 16822381 2230363 236918 61 0 0
wifi1 4096977 2070224 4095468 2242763 58 0 11
@ArubaNetworks

MTU Discovery
Probes Responses Last Sent Last Rcvd
45214 22607 2712890 2712890
Switch MTU, 1500
Ethernet bonding
SlaveId Name Link State #LinkFails Ethernet Duplex/Speed Settings
Autoneg Speed (Mbps) Duplex Iface
0 eth0 UP ACTIVE 0 on 1000 Full eth0
eth1 DOWN STANDBY 0 on 10 Half eth1
@ArubaNetworks

Controller Information
Item Value
Primary LMS 10.163.196.72
Backup LMS 10.163.196.71
AP to Active Controller Message Information
Item Value
AP state REGISTERED
Power Status
Operational State : Unknown
Current HW State POE-AT: No restrictions
@ArubaNetworks

MTU Discovery
Probes Responses Last Sent Last Rcvd
45214 22607 2712890 2712890
Switch MTU, 1500
Ethernet bonding
SlaveId Name Link State #LinkFails Ethernet Duplex/Speed Settings
Autoneg Speed (Mbps) Duplex Iface
0 eth0 UP ACTIVE 0 on 1000 Full eth0
eth1 DOWN STANDBY 0 on 10 Half eth1
@ArubaNetworks

AP Stability Optimizations
@ArubaNetworks

AP Stability Optimizations
Heartbeat DSCP: Assign a DSCP value to AP heartbeats to prioritize heartbeats traveling over low-speed links. The
supported range is 0-63, and the default value is 0.
Bootstrap threshold: Number of consecutive missed heartbeats on a GRE tunnel (heartbeats are sent once per second on each
tunnel) before an AP rebootstraps. On the controller, the GRE tunnel timeout is 1.5 x bootstrap-threshold; the tunnel is torn
down after this number of seconds of inactivity on the tunnel.
SAP MTU: Maximum Transmission Unit, in bytes, on the wired link for the AP.
Spanning Tree: Select this checkbox to enable the Spanning Tree protocol.
@ArubaNetworks

# 3
Optimize load on processes
@ArubaNetworks

HTTPD
Stress on the webserver can be because of number of sessions in the initial role for
guest access. This is either because of large certificate (Key Length 2048 or 4096) that
are used by the server or a large number of devices (phones/Tablets with APPs) that
generate HTTP/HTTPS sessions and get re-directed to the web-server.
show web-server profile
Web Server Configuration
Parameter Value
SSL/TLS Protocol Config tlsv1
Captive Portal Certificate GUEST-AUTH
User session timeout <30-3600> (seconds) 3600
Maximum supported concurrent clients <25-320> 75
Enable WebUI access on HTTPS port (443) true
Enable bypass captive portal landing page false
@ArubaNetworks

HTTPD
show web-server statistics
Web Server Statistics:
Current Request Rate: 1 Req/Sec
Current Traffic Rate: 1 KB/Sec
Busy Connection Slots: 7
Available Connection Slots: 68
Total Requests Since Up Time: 284
Total Traffic Since Up Time: 1122 KB Avg.
Request Rate Since Up Time: 1 Req/Sec Avg.
Traffic Rate Since Up Time: 6144 Bytes/Sec
Server Scoreboard: _____________KKKKKK_W_____________
Scoreboard Key: _ - Waiting for Connection, s -
Starting up R - Reading Request, W - Sending
Reply K - Keepalive, D - DNS Lookup C -
Closing connection, L - Logging G - Gracefully
finishing, I - Idle cleanup of worker . - Open slot
with no current process
@ArubaNetworks

STM and WMS
STM
Station Management is responsible for all AP information and Station information. This process can be over
run if there is,
# Aggressive polling from Airwave/SNMP servers for Wlan tables.
# Network wide AP reboot and bootstraps
# AP debug scripts run from the controller.
WMS
IDS/IPS events and frequent AP bootstraps could lead to WMS being busy. WMS is actively looking for RF
information of WiFi devices(Rogue/Valid/Interfering).
@ArubaNetworks

Mitigation
Use AMON
WMS Offload to Airwave.
Reduce SNMP polling or increase the polling period
Disable WMS functionality if you do not require IDS/IPS functionality.
@ArubaNetworks

# 4
Datapath Debugging
@ArubaNetworks

Datapath Monitoring
Show datapath utilization
show datapath utilization
Datapath Network Processor Utilization
| Cpu utilization during past |
Cpu | 1 Sec 4 Secs 64 Secs |
10 | 99% | 99% | 99% |
11 | 0% | 0% | 0% |
12 | 0% | 0% | 0% |
13 | 0% | 0% | 0% |
14 | 0% | 0% | 0% |
15 | 0% | 0% | 0% |
16 | 0% | 0% | 0% |
show datapath frame 10
|SUM/| | | |
|CPU | Addr | Description Value |
+----+------+-----------------------------------------------------+
| 10 | [00] | Allocated Frames 1040|
| 10 | [01] | Max Allocated Frames 2208 |
| 10 | [03] | Unknown Unicast 147074970|
| 10 | [34] | Flood Frames 1506164167|
+----+------+-----------------------------------------------------+
| 10 | [00] | Rx Frames 635394472|
| 10 | [01] | Rx Bytes 1864525959|
| 10 | [02] | Tx Frames 1240985989|
+----+------+-----------------------------------------------------+
@ArubaNetworks

Datapath Bandwidth Management
show datapath bwm
Datapath Bandwidth Management Table Entries
Type Id Bits/sec Policed Bytes Bytes Flags CPU Status
---- ---- --------- ---------- ------- ----------- ------- ------- ------
0 1 20000000 0 78125 0/0 9 ALLOCATED
0 2 4000000 0 15625 0/0 9 ALLOCATED
0 3 160000000 0 624890 0/0 9 ALLOCATED
0 4 4000000 0 15625 0/0 9 ALLOCATED
0 5 2000128 0 7813 0/0 9 ALLOCATED
0 6 2000128 0 7813 0/0 9 ALLOCATED
0 7 2000128 0 7813 0/0 9 ALLOCATED
Firewall:
Rate limit CP untrusted ucast traffic Enabled 20 Mbps
Rate limit CP untrusted mcast traffic Enabled 4 Mbps
Rate limit CP trusted ucast traffic Enabled 160 Mbps
Rate limit CP trusted mcast traffic Enabled 4 Mbps
Rate limit CP route traffic Enabled 2 Mbps
Rate limit CP session mirror traffic Enabled 2 Mbps
Rate limit CP auth process traffic Enabled 2 Mbps
@ArubaNetworks

CP and DP Packet Capture
• Wifi -Client
packet-capture datapath wifi-client aa:aa:aa:aa:aa:aa all
• VIA client/RAP
packet-capture datapath ipsec <peer-ip>
• Generic traffic to controller
packet-capture controlpath tcp/udp 4343
@ArubaNetworks

# 5
@ArubaNetworks
Deployment Tips
Missing optimizations

Deployment Tips
Honey Comb Pattern
Wireless
Local Probe Threshold = 25
Transmit Power of AP 5Ghz Min Tx – 12 Max Tx – 15
2.4 Ghz Min Tx – 6 Max Tx – 9
Avoid Asymmetric RF
The difference between minimum and maximum Tx power on
the same radio should not be more than 6dbm
DMO Enable
Basic and Beacon rate
802.11a 5Ghz – 24
802.11g 2.4Ghz – 12
80 Mhz Channel bonding - DFS Channels
@ArubaNetworks

Deployment Tips.. Contd
GRE Stripping IP - VRRP for LMS and Stripping IP
Jumbo Frames - Enabled
802.3at
Airgroup
Dot1x
OKC
Validate PMK ID
802.11r/k/v
EAPOL Rate Optimization
@ArubaNetworks

Deployment Tips… ASE
https://ase.arubanetworks.com/
@ArubaNetworks

Deployment Tips..
ASE for troubleshooting
@ArubaNetworks

36#ATM15 |
Network Services
AirWave
ClearPass
@ArubaNetworks

Transition Content
# 6
ClearPass Platform: System Cleanup Options
@ArubaNetworks

• Free disk space threshold is a config in Cluster Wide Service Parameter. Default 30%
• A system cron job runs every hour and checks the disk utilization. If the free space falls below the
configured threshold, an alert is logged into the system. NOW in addition, the following aggressive
cron cleans up anything more than 1 day old in version 6.5 of CPPM
• Log database records
• Core files
• System load monitor files
• Application and system log files
• Auto and manual backup files
• Stored reports
• Expired guest accounts
• Audit records

We also introduced some new CLI commands
– Check on disk-space and memory usage - “show sysinfo”…
– system cleanup [# of days to retain] **This is an on-demand task

• Same command function also exist in the GUI
– Remember this is an on-demand task

# 7
@ArubaNetworks
ClearPass Platform : Graphite

• Graphite is a new reporting tool to compliment Insight in CPPM from 6.3 version.
• Graphite runs on every node irrespective of standalone or cluster and statistics can be viewed
from any node.
• Performance monitoring Display is disabled by default and should be enabled manually and set
access permission levels accordingly.
• To access Graphite data, use the URL https://<CPPM IP Address>/graphite

• Make sure Performance monitoring is enabled from GUI

• Setting up access to Graphite from CPPM UI

• We can allow or deny any networks to access Graphite for a node or cluster.
• Make sure stats collection is set true True under Service parameters.

Transition Content
# 8
ClearPass : Upgrade Utility Tool
@ArubaNetworks

• The Cluster Upgrade Tool is a simple user interface that automates the upgrade procedure
for a ClearPass cluster.
• When the upgrade is initiated, no manual actions are required until all selected nodes have
been upgraded.
• The Upgrade Tool is not available while the publisher is rebooted and migrating the
Configuration Database.
• The Upgrade Tool will not detect nodes that were upgraded manually without the tool.
• If a configured standby publisher node was manually upgraded without the tool, the Upgrade
Tool will not restore the state of the standby publisher configuration.

• The Cluster Upgrade Tool is released as a patch update. It can be downloaded and installed
either through Policy Manager’s Software Updates portal, or from the Aruba Support portal.

• Log in to Policy Manager on the publisher node and go to Administration > Agents and Software
>Updates > Software Updates.
• When the installation is complete, the Admin service will be restarted. You do not need to reboot.

• Before you begin the upgrade, the upgrade image must be present on the publisher node of the
cluster.
• Download the upgrade image to the publisher under Software updates

• To monitor the progress of the other nodes in the cluster, wait until the database migration is
complete and then log in to the tool again.
• Change the url to https://CPPM IP Address/upgrade
• We should see all the subscribers status that are in sync.
• The list of subscribers will be present and subscriber upgrades will go in parallel.
• ‘Start Upgrade’ to start the upgrade on the servers.

• Check the logs for each node by ‘View Logs’ next to each node and we can see the progress of
patches and upgrades from publisher.

ClearPass : Upgrade Utility tool

# 9
@ArubaNetworks
AirWave – VisualRF Performance Tips

• Sometimes we see VisualRF takes long time to show up new AP’s to deploy on the floor plans.
• We can manually force VisualRF to poll the AP’s to get new AP or existing AP’s updated details.
• NO need to restart VisualRF to show up new AP’s.
• Change the url in AMP to https://<Airwave IP Address>/visualrf/poll_aps_now.xml

• In 8.x moved to HTML5 for VisualRF UI for faster UI interaction with backend.
• Whilst we are improving the features on the new UI, there are some features which were present
in flash and not in HTML5.
• In VisualRF > Setup page, we can switch between HTML5 and flash so that we can take
advantage of options present in both version
• Switching between HTML5 and Flash version is easy with below URL without refreshing
VisualRF.
• Change the URL to
https://Airwave IP Address/site?campus_id=6c56c239-bfba-4d19-aeca-8ec5af68b725
from
https://Airwave IP Address/vrf?campus_id=6c56c239-bfba-4d19-aeca-8ec5af68b725

AirWave – VisualRF Performance tips

• We can change ‘vrf’ to ‘site’ on any page for VisualRF URL’s to switch to flash mode from HTML5
mode.
At times we see Heat maps not showing/updating properly in VisualRF
• We can resize the floor plan to same size so that the grid calculation happens and heat maps will
be re-drawn.
• No need to restart VisualRF for heat maps to update.

• Unlock the floor plan and go to properties and ‘Measure’
Airwave – VisualRF Performance Tips

• Select the distance, click ‘OK’ and ‘Save’ without changing the distance, this will trigger floor plan
to recalculate the heatmpas.
Airwave – VisualRF Performance Tips

Transition Content
# 10
Airwave – Tips for Data Retention Settings
@ArubaNetworks

• Data in AirWave is primarily stored in 2 formats:
• Postgres - an open source, relational SQL database. Usually, when you see data in tables, that
data is stored in Postgres.
• RRD Files - used for storing data that's displayed in time-sequence graphs (i.e, client count over
the last year, bandwidth used over the last month). There can be many thousands of RRD files
on a single AirWave server. One benefit of RRD is that its files have a fixed size. As data is
inserted to an RRD file (like by an AirWave monitoring process), it does not grow. A downside of
this is that the file starts using storage space as soon as it is created.

• We can set data retention settings under AMP Setup > General page under the section ‘Historical
Data Retention’.
• Client Association and VPN Session History. This setting has a bearing on how much history we
can show in the association history on the client historical table and how much data can be
included in the user session data.
• Its recommend to keep high because the data is useful

Airwave – Tips for Data Retention settings
• Inactive Client and VPN User Data. This setting determines how long we keep the information on
every client that has ever connected to the network.
• This impacts how long we keep RRD files. Keeping it low can save disk space.

• Client data retention Interval : This influences how much historical data you can see for each
client in the graphs, for example the signal quality, usage graphs on the client detail/diagnostic
page.
• It's very important to keep this low, like in the 14-31 days range.
• This is especially important in public wi-fi deployments that will have lots of unique users.
• This setting controls what size RRD files are created to store per-user historical signal, usage,
goodput, health and other metrics.
• Keeping it low doesn't impact device, group and folder-level monitoring, and it doesn't have any
negative impact on reports. It only impacts the graphs on the client Detail and Diagnostic pages.

• By default Rogues are kept forever which will impact the overall
system performance and for RAPIDS page load.
• It also impacts VisualRF for Rogue calculation if it has thousands
of Rogue devices.
• This is especially important in public wi-fi deployments that have
open SSID and lot of nearby devices are detected as Rogues
• Setting the value to low as 14 days will greatly help.

• Airwave by default has 20+ Reports which runs daily.
• Keep Reports that are needed for the environment and delete/disable the default reports.
• Report retention setting can be costly in high dense environments especially for disk space.
• This increase the nightly backup file size, nightly maintenance time and report generation time.
• Exporting the reports via .csv or .pdf or emailing them is a good option.
• Keeping the retention value will have the pickled client tables not to grow huge in size and makes
report generation faster.

Questions
@ArubaNetworks

THANK YOU
75#ATM15 | @ArubaNetworks

The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting tips

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (15)

Similar to The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting tips

Similar to The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting tips (20)

More from Aruba, a Hewlett Packard Enterprise company

More from Aruba, a Hewlett Packard Enterprise company (20)

Recently uploaded

Recently uploaded (20)

The Aruba Tech Support Top 10: WLAN design, configuration and troubleshooting tips

Editor's Notes