eduroam diagnostics in NTLR, IdPs and SPs Karri Huhtanen Arch Red Oy 13.3.2013 v1.1
Warning• This presentation contains detailed explanations and long sentences with rather small font.• Because of this it is already available also at SlideShare• http://www.slideshare.net/khuhtanen/ eduroam-diagnostics-in-ntlr-idps-and-sps
Background• the amount of failed authentications in eduroam seems big when organisations are looking into NTLR statistics• CSC/Funet asked if Arch Red could list and explain 10 most common reasons while authentication failed for organisations• So we (Arch Red) wrote ﬁrst few long emails, but decided then to publish our ﬁndings in two blog posts, ﬁrst in Finnish: • http://blog.archred.ﬁ/2013/01/eduroamin-vianselvitys-osa-1.html • http://blog.archred.ﬁ/2013/01/eduroamin-vianselvitys-osa-2-yleisimmat.html• This presentation is about describing those ﬁndings in English and even developing those blog posts further
Background• Arch Red runs National Top Level RADIUS (NTLR) for CSC/ Funet and has connected and even provided turn-key eduroam solutions for several Finnish universities• Via our cooperation agreement with Open System Consultants (Radiator RADIUS server), we have helped organisations around the world to join to eduroam• Arch Red also runs the Top Level RADIUS service and authentication services (IdP) for Wireless Tampere community network and several neighboring cities.• This has given us the experience of diagnosing eduroam from the perspective of Top Level RADIUS (TLR), Service Provider (SP) and Identity Provider (IdP)
Top Level RADIUS diagnostics• Do we accept RADIUS requests from source (IdP/SP)?• Do we know where to send them next? (routing) Is there a proper realm in requests?• Does the next hop (ETLR / IdP) accept the proxied RADIUS request, reject or drop them? Is it alive?• Do the RADIUS responses travel properly to opposite direction?• And that’s about it with EAP-packets. All the remaining information about connections is at the IdP.• TLR may not even register those eduroam authentications which hang or are not otherwise responded with accept or reject.
Service Provider Diagnostics• Do we accept RADIUS requests from source (NAS)?• Do we know where to send them next? (routing) Is there a proper realm in requests?• Does the next hop (TLR) accept the proxied RADIUS request, reject or drop them? Is it alive?• Do the RADIUS responses travel properly to opposite direction?• And that’s it with EAP messages. The service provider does not know anything else. It can however in some cases diagnose what is the roaming terminal’s connection quality, which is more than TLR knows.
Identity Provider Diagnostics• So now we know that Identity Provider has all the information to solve major part of its users connection problems.• Unfortunately Identity Provider such as University may not have the experience, knowledge and time needed to utilise that information.• Because of that we collected a short list of issues, which may lead to failed authentications and bad eduroam user experience.• The following problems are not in a particular order, but numbered to help referral.
1) conﬁguration problems in terminals• user terminals are not conﬁgured correctly to work in home eduroam network, they are conﬁgured hastily/incompletely in foreign network or there may be typos in usernames/realms• the complex conﬁguration is then not ﬁnished or removed but hoped magically to correct itself in some other network• this kind of partially conﬁgured client then bombards IdP server with failing authentication requests increasing failed authentication in statistics• even bigger problem are for example failing SSL/TLS tunneling, PEAP and this kind of interrupted or hanging sessions which may not be registered or seen without running IdP servers in debug mode
2) older devices and supplicant implementations• Old Nokia and Symbian based devices are good bad examples. WPA/WPA2 in them used by default EAP-SIM, EAP-AKA authentication and had to be manually changed to use PEAP/EAP-TTLS etc. If you do not know this problem, be happy that Symbian is dead.• This problem can be seen in SP, IdP and TLR logs with the usual 3gpp realm• Luckily this is going away with better defaults for supplicants, Android, Windows Phone and iPhone use by default username password authentication.
3) certiﬁcate checking problems• Proper certiﬁcate installation is hard to do correctly, iOS kinda works, Android is hell and does anyone know how to conﬁgure certiﬁcate check to Windows Phone?• EAP-TTLS/PEAP+MSCHAPv2 somewhat protects users and IdPs from man-in-the-middle attacks even if server certiﬁcate is not checked• Still some supplicants do not allow for example server name to be checked from certiﬁcate• In a better world this would be done via SSH style ﬁngerprint checking and user alerted only if RADIUS server host key would be changed.• Incorrectly or half conﬁgured certiﬁcate checks cause devices to bombard once again IdP servers and may not even show in logs.
4) periodical password change requirements in IdPs• So IdP requires all its employees and students to change password every 6-9 months?• What happens to all those device conﬁgurations with old passwords? What will those devices do?• You see the point?• Our pursuit for single-sign-on has led us to problem where password needs to be so secure that it needs to be changed often. Changing it often then breaks eduroam conﬁgurations.• How about if we would just get back to multiple passwords or have a separate eduroam password? Maybe randomized password by device in a style of Google’s passwords per device? Or Certs in the year 2000 maybe? :)
5) IdPs incorrect/invalid conﬁguration instructions• 5.1) No realm in conﬁguration instructions either in the inner or outer authentication• 5.2) Weaseling out of certiﬁcate problems by instructing users not to check server certiﬁcate or server hostname• Just google eduroam instructions, you will ﬁnd a lot of this kind of incorrect instructions• Solution for this might be to shame those organisations to correct instructions or offer only correct ofﬁcial instructions and require IdPs to conﬁgure their AAA accordingly.
6) Bugs in terminal UI or certiﬁcate management• Some devices do not allow e.g. conﬁguring preinstalled CA certiﬁcates to be allowed to be used for RADIUS server certiﬁcate veriﬁcation (Nokia N900 was one of these)• Some devices make it really hard to install CA certiﬁcates (Android)• And some just do not have UI settings for managing certiﬁcates, conﬁguring server names checks etc.• Could we shame vendors to also do these properly? Pretty please?
7) eduroam WiFi network conﬁguration differences• The actual WiFi radio network conﬁguration differences can also cause problems.• Example: Organisation provides eduroam in WPA1/WPA2 compatibility mode. Some devices assume then that eduroam always supports WPA1 and do not accept WPA2 network as eduroam. Then when user tries to conﬁgure new WPA2 eduroam network, it cannot be conﬁgured as it has the same SSID as the earlier WPA1 proﬁle. (Microsoft Windows 7)• So let’s just decide that eduroam is only pure WPA2, no WPA1, no 802.1X+WEP, no TKIP, no compatibility modes etc. The recommendation is already WPA2, can it be also obligation for pure WPA2?
Conclusions, solutions, suggestions• eduroam CAT solves a lot of these issues and in Stefan Winter’s presentation in this GN3 Workshop there were a lot of nice tests to ensure IdPs work properly• However installing an additional supplicant may interfere with terminal operating systems or other 3rd party clients.• A better approach might be to use operating system speciﬁc conﬁguration and provisioning interfaces.• For some mobile devices a generic eduroam / other conﬁguration provisioning app could be more generic solution. The app would then conﬁgure or correct the actual operating system settings based on the information received from the eduroam CAT provisioning service. App could also be used for eduroam quality and conformance monitoring as well as problem reporting.