Novell Identity Manager Troubleshooting

  • 9,848 views
Uploaded on

You have an integrated, business-focused identity and access management environment with Microsoft Active Directory, Novell eDirectory and other business related processes—and they are all integrated …

You have an integrated, business-focused identity and access management environment with Microsoft Active Directory, Novell eDirectory and other business related processes—and they are all integrated and synchronized into one harmonious implementation. And just when you think everything is going fine, that's when chaos erupts. With many business processes now reliant on your harmonious, integrated environment, you also need to factor in dealing with unexpected adversity that might arise. Whether that adversity is caused by applications, undetected bugs or by user mistakes, you need to know how to quickly isolate and resolve it.

This session will discuss techniques and tools for identifying, isolating, troubleshooting and resolving identity asset issues affecting Novell Identity Manager environments.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
9,848
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
400
Comments
0
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Novell Identity Manager ® Troubleshooting Reed Harrison Rajiv Kumar GTS Identity Services Engineer IDM developer rdharrison@novell.com krajiv@novell.com
  • 2. Agenda • IDM information sources • IDM trace definition • IDM trace capture • IDM trace validation • IDM trace reading • Appendix 2 © Novell, Inc. All rights reserved.
  • 3. Information Sources
  • 4. Where do I find product resources? Where to find information? – Novell Support Forums ® http://forums.novell.com/ – Novell Support Knowledgebase http://support.novell.com – Novell Documentation http://www.novell.com/documentation – Google http://www.google.com/ – 3rd Party Vendor website > Microsoft, Oracle, IBM, SAP, MySQL, etc 4 © Novell, Inc. All rights reserved.
  • 5. What information do I need to troubleshoot my issue? – Issue description as detailed as possible – Identify the environment - is it production? Lab? include software versions and where each piece is installed > OS Type, version and patch level for all servers > Are those real machines or VMs? If VMs, which virtualization solution? » Virtualization product name, version and patches > eDirectory , Security Services and IDM versions for all relevant servers ™ > 3rd Party applications relevant to the issue, their versions and patch levels > eDirectory replicas present on the IDM server and their types > Location of the servers and connectivity between them » Are the servers local, or across Wan links? » Are there firewalls/routers/other network devices between them? 5 © Novell, Inc. All rights reserved.
  • 6. What information do I need to begin troubleshooting? What information should we gather for troubleshooting? – Driver exports and/or Designer project exports (preferred) – OS-Related information > supportconfig on Linux OS http://www.novell.com/communities/node/2332 > config.txt on Netware OS® type LOAD CONFIG /ALL on the server's console > for Windows/Solaris/AIX find out the version and if it is 32 or 64 bit. Also, on Windows, find out what domain functional level they are running. Note that 2008 and 2008 R2 are separate products. – IDM traces, J2EE App server logs – (Optional) DSTRACE & LAN trace files, ndsd.log (Linux/Unix), Event Viewer logs (Windows), logger.txt & console.log (Netware) 6 © Novell, Inc. All rights reserved.
  • 7. Novell Identity Manager Trace ® or Now we have information What do we look at first?
  • 8. How IDM works review Local Configuration: Connected eDirectory ™ Application IDM Engine + Driver Shim 8 © Novell, Inc. All rights reserved.
  • 9. How IDM works review Remote Loader Configuration: Remote Loader Connected eDirectory ™ Application IDM Engine + Driver Shim 9 © Novell, Inc. All rights reserved.
  • 10. Engine Flow Diagram - Subscriber IDM Engine flow (simplified) – Subscriber only Merge Processor Matching Create YES TAO NO File Match Translation Processor Subscriber Filter Found? Placement Subscriber Filter Notify & Reset Sync & Ignore YES Association Sequencer Processor ADD Processor Event Cache Event NO Event Add? Command Transform Transform Not part of the channel Thread 10 © Novell, Inc. All rights reserved.
  • 11. Engine Flow Diagram - Publisher IDM Engine flow (simplified) – Publisher only Optimize Merge Modify Processor Create Match YES NO Match Translation Publisher Filter Publisher Filter Sync & Ignore Placement Notify & Reset Processor YES YES Found? Association Association Sequencer Processor Processor Post-filter Pre-filter ADD Processor Event NO NO Modify? Add? Command Event Transform Transform 11 © Novell, Inc. All rights reserved.
  • 12. IDM Trace Capture and Validation
  • 13. What is the most effective way to troubleshoot? IDM traces • In IDM, traces are a way of following step by step how the events are processed and executed • Reading an IDM trace is akin to debugging a program, since most of what IDM does is execute DirXML-Script commands on an event's XML • As with any programming language, you need to know the language well if you intend on debugging it • DirXML-Script language is explained at: – http://www.novell.com/documentation/idm36/policy/data/policytypesoverview.html – http://www.novell.com/documentation/idm36/policy_dtd/data/dtddirxmloverview.html – http://www.novell.com/documentation/idm36/policy_designer/data/bookinfo.html 13 © Novell, Inc. All rights reserved.
  • 14. When to use IDM Traces • Traces should be used only for troubleshooting, not for auditing events • Tracing can have a huge impact on driver performance (tenfold or more, depending on trace level) • IDM debug traces can be configured in iManager, Designer, or at the Remote Loader configuration file 14 © Novell, Inc. All rights reserved.
  • 15. IDM Trace Types and How to Capture • There are 2 types of traces - Engine or Remote Loader – IDM Engine trace: can be seen in 3 different ways > DSTRACE screen / DSTRACE file > iMonitor Trace Screen > IDM Trace file (also known as Java trace file) – Remote Loader trace: can be traced only to file > On Windows there is a live trace screen that can be seen if certain criteria is met (criteria varies per Windows version) 15 © Novell, Inc. All rights reserved.
  • 16. IDM Trace Levels • Engine trace levels go from 0 to 4. Each trace level shows all the status messages from previous levels – Level 0: Status Messages Only – Level 1: Current location in the Driver Logic flow – Level 2: Events (XML format) – Level 3: Driver Logic Execution Details – Level 4: Cache-related information about the event coming from eDirectory (Subscriber channel) ™ • Shim trace levels go from 3 to 10 – Information provided changes per driver, check driver docs for description of what each trace level provides for its shim 16 © Novell, Inc. All rights reserved.
  • 17. Capturing IDM Traces • Step by Step instructions on setting IDM traces – http://www.novell.com/documentation/idm36/idm_common_ driver/data/b1rc1vm.html • More information on how to read IDM traces – http://www.novell.com/communities/node/5681/capturing- and-reading-novell-identity-manager-traces • Best Information on trace reading – Trace reading cool solution: http://www.novell.com/communities/node/9677/comprehend ing-idm-traces-part-1 17 © Novell, Inc. All rights reserved.
  • 18. Basic validation of IDM traces Some things to check in the trace – Does the test user show in the trace file? Look into the src-dn and dest-dn XML attributes of the operation – Is the operation in the trace the same one performed during testing? – If you are getting an error, is it in the trace.? – Were the files taken with the proper trace level? 18 © Novell, Inc. All rights reserved.
  • 19. Basic validation of IDM Engine traces Quick Trace Parsing – To find an event coming from eDirectory , search for ™ > Start transaction – To find an even coming from the Application, search for > Receiving DOM document from application – Any actions performed in eDirectory are preceded by > Pumping XDS to eDirectory – The result of all status messages shows after > DirXML Log Event – Driver initialization starts with > Reading named passwords list 19 © Novell, Inc. All rights reserved.
  • 20. Basic validation of IDM traces grep is your friend! – grep is a tool that allows to search several files quickly, and returns one or more lines matching what you searched – grep accepts command line parameters like -A (after) and -B (before) that can be extremely useful. Some examples: > Case-insensitive search grep -i 'my text here in any case' trace.log > List all Status Log Messages in a trace grep -B 1 -A 5 'DirXML Log Event' trace.log > List the first piece of all events coming from eDirectory (might need a bigger number for the A parameter if the trace level is 4 or above) grep -A 9 'Start transaction' trace.log > Counts how many times the driver got restarted in this trace file grep 'Reading named passwords list' trace.log | wc -l 20 © Novell, Inc. All rights reserved.
  • 21. IDM Trace Reading Basics The Engine Side
  • 22. Trace Reading - Basic • Again, traces should be used only for troubleshooting, not for auditing events • An IDM trace (level 3 and above) will show all the steps done by the engine while processing an Event • We will trace Reed Harrison as he is added to OpenLdap from the Identity Vault 22 © Novell, Inc. All rights reserved.
  • 23. Trace Reading - Basic Add Reed Harrison 23 © Novell, Inc. All rights reserved.
  • 24. Trace Reading - Basic Add Reed Harrison 24 © Novell, Inc. All rights reserved.
  • 25. Trace Reading - Basic Add Reed Harrison 25 © Novell, Inc. All rights reserved.
  • 26. Trace Reading - Basic Add Reed Harrison 26 © Novell, Inc. All rights reserved.
  • 27. Trace Reading - Basic Add Reed Harrison 27 © Novell, Inc. All rights reserved.
  • 28. Trace Reading - Basic Add Reed Harrison 28 © Novell, Inc. All rights reserved.
  • 29. Trace Reading - Basic Add Reed Harrison 29 © Novell, Inc. All rights reserved.
  • 30. Trace Reading - Basic Add Reed Harrison 30 © Novell, Inc. All rights reserved.
  • 31. Trace Reading - Basic Add Reed Harrison 31 © Novell, Inc. All rights reserved.
  • 32. Trace Reading - Basic Summary – Reading an IDM trace means following events from beginning to end, and seeing how the driver logic affected them before the event's XML is handed to the destination system – An IDM engine trace level 3 or above will show all steps done while a driver processes an event – Both iManager & Designer show simplified views of the logic processing, don't let them sidetrack you 32 © Novell, Inc. All rights reserved.
  • 33. Appendix
  • 34. IDM Quick Reference Cards
  • 35. Types of Cards • Installation Troubleshooting • Engine does not load • Driver does not start • Password Synchronization Issues • Other driver issues 35 © Novell, Inc. All rights reserved.
  • 36. Installation Troubleshooting
  • 37. Installation Troubleshooting • Obtain OS name & patch level • Identify eDirectory version & patch level (if installing the IDM engine) • Identify the IDM version being installed. Double-check if the OS / eDir / IDM combination is supported in the Novell Documentation • Obtain the Install logs following the Install troubleshooting steps in the docs. • Use the cool solution “Identity Manager 3.6 Install Troubleshooting Tips” - This is the best reference for install issues. 37 © Novell, Inc. All rights reserved.
  • 38. Engine Does Not Load
  • 39. IDM Engine Does Not Load • Obtain OS name & patch level • Identify eDirectory version & patch level • Identify the IDM version • With the above information, see next page for Windows Instructions, and the one after for Linux Instructions • The best TID for this is Troubleshooting errors -641 or -783 Starting an IDM driver. TID 7002449 39 © Novell, Inc. All rights reserved.
  • 40. IDM Engine Does Not Load • Windows: – IDM is installed in the same directory where eDirectory's dlms are (by default, C:NovellNDS) – Stop the eDirectory service – Move the file “dirxml.dlm” from that directory – Start the eDirectory service – After eDirectory finishes loading, start DSTRACE.dlm, set the flags 'DirXML', 'DirXML Drivers', 'Misc Other' and start tracing to file – Move the file “dirxml.dlm” back to its original location – Close/reopen the eDirectory services console, select dirxml.dlm and hit the start button 40 © Novell, Inc. All rights reserved.
  • 41. IDM Engine Does Not Load • Linux: – Stop ndsd ( /etc/init.d/ndsd stop ) – Move the libvrdim.* files from their original directory to a different directory > eDir 8.7.3.x: /usr/lib/nds-modules/ > eDir 8.8.x: /opt/novell/eDirectory/lib/nds-modules/ – Start ndsd ( /etc/init.d/ndsd start ) – Start ndstrace with only the flags 'time', 'tags',' misc', 'dxml', ' dvrs' and save the trace to a file. Leave it running on screen – Move the libvrdim.* files back to their original location – Back on the ndstrace screen, type 'load vrdim' – After you see the error, stop ndstrace and grab the file 41 © Novell, Inc. All rights reserved.
  • 42. Driver Does Not Start
  • 43. Driver Does Not Start • If you are receiving the following error codes, this is an engine problem, not driver problem: > -783 VRDIM Not Initialized > -641 Invalid Request • For all other errors starting a driver – (optional) Set Remote Loader trace level to 5 and make sure he starts normally before attempting to start the driver – Set engine trace level to 3, and set trace to file – Try to start the driver again to capture the error in the trace file. After the attempt to start fails, get the trace file 43 © Novell, Inc. All rights reserved.
  • 44. Password Synchronization Issues
  • 45. Password Synchronization Issues •Obtain OS name & patch level •Identify eDirectory version & patch level •Obtain NMAS version & patch level •Identify the IDM version •Which drivers & connected applications are involved? Take note of their versions and where they are running •Check in the Matrix if that driver/application combination can sync passwords. IDM 3.6 docs: http://www.novell.com/documentation/idm36/idm_password_management/data/bo1o7xz.html 45 © Novell, Inc. All rights reserved.
  • 46. Password Synchronization Issues •Check which direction passwords do not synchronize – If the problem is coming from eDirectory, make sure Universal Password is configured properly and Tree keys are fine – If the problem is coming from the connected application, we need to check different things based on the application > LDAP (SunONE only): Check the password plugin on SunONE > AD: Password Synchronization filters must be installed and running http://www.novell.com/documentation/idm36drivers/ad/data/bow0k51.html > Linux&Unix: Check the platform's PAM (or LAM) configuration • Drivers have GCVs that control password flow http://www.novell.com/documentation/idm36/idm_password_management/data/bnwjt01.html 46 © Novell, Inc. All rights reserved.
  • 47. Other Driver Issues
  • 48. For ALL Other Driver Issues • ALWAYS obtain a current driver export OR designer project export • Take note of IDM version, eDirectory version on the IDM server, OS (including version and patch level) • Take note of 3rd party Application name, patch level and OS where it is running • Identify if a Remote Loader is in use. – If there is, the reference to Shim trace levels will be applied in the remote loader – If not, the Shim trace levels will be applied in the engine and the recommendation for engine trace levels can be ignored 48 © Novell, Inc. All rights reserved.
  • 49. Active Directory Driver • Users do not synchronize – Engine trace level 3, Shim trace level 3 – Take note of the test user name, location and system where he was created • Users synchronize in a single direction – Check the driver filters – Check the placement policies in the appropriate channel – Engine trace level 3, Shim trace level 3 • Passwords are not synchronizing – See section on password sync on this document 49 © Novell, Inc. All rights reserved.
  • 50. Avaya PBX Driver • Extensions are not created – Engine trace level 3, Shim trace level 3 50 © Novell, Inc. All rights reserved.
  • 51. Delimited Text Driver • Users do not get created in eDirectory – Check if the input directory exists and is properly entered in the driver configuration – Check filesystem rights and quotas on input directory&files – Engine trace level 3, Shim trace level 3 – Input csv file used to create the users • Driver does not write output files – Check if the output directory exists and is properly entered in the driver configuration – Check filesystem rights and quotas on output directory – Engine trace level 3, Shim trace level 3 51 © Novell, Inc. All rights reserved.
  • 52. eDirectory Driver ™ • eDirectory drivers work in pairs – Engine trace level 3 on both trees being connected, on the proper pair of eDirectory drivers – This driver does not support remote loader – For the Driver exports, make sure you get both eDirectory driver exports (there is one driver per tree). – If you get a Designer project, make sure that both eDirectory drivers are imported in the project 52 © Novell, Inc. All rights reserved.
  • 53. Entitlements Service Driver • This driver enables/disables entitlements on objects – Engine trace level 5 for the entitlements driver itself – LDAP Export of the Entitlement Policies used in the Driverset (they reside bellow the Driverset object) – Since this driver only changes the DirXML-EntitlementRef attribute on a user, we need to get the appropriated traces on the other drivers being affected by that change 53 © Novell, Inc. All rights reserved.
  • 54. GroupWise Driver ® • Mail accounts are not created in GroupWise – Engine trace level 3, Shim trace level 5 54 © Novell, Inc. All rights reserved.
  • 55. ID Provider Driver •This driver troubleshooting is unique in the sense it is also a service an can be accessed by external clients – Traces can be enabled in the driver & client parameters, aside from the regular IDM tracing. The driver docs go into more details here: – http://www.novell.com/documentation/idm36drivers/idprovider/data/bookinfo.html – If a customer calls in with an ID provider call, do this: > document the issue in detail > get the ID driver export > get a LDAP export of their ID Policy objects > ask the customer to provide the XSLT / Java call made to the ID Provider service 55 © Novell, Inc. All rights reserved.
  • 56. JDBC Driver •For ALL JDBC driver issues request – Database name, vendor and patch level – OS & patch level where the database in running at – Check if its a supported IDM/Database combination. Docs http://www.novell.com/documentation/idm36drivers/jdbc/data/bw17kgf.html – Driver connection mode > direct or indirect > triggered or triggerless – Customer's database schema (SQL file for the tables/views that the driver connects to) – Engine trace level 3, Shim trace level 3 (only request a higher trace level for this driver if oriented by Backline) 56 © Novell, Inc. All rights reserved.
  • 57. JMS Driver • Messages are not being sent or received from the JMS Queue/application – Engine trace level 3, Shim trace level 5 57 © Novell, Inc. All rights reserved.
  • 58. LDAP Driver • Users are not synchronizing between systems – Engine trace level 3, Shim trace level 5 – (Optional) LAN trace between the driver shim and the 3rd party LDAP system • Passwords are not synchronizing from the LDAP system into eDirectory – Password synchronization from the LDAP system is only supported currently when the LDAP system is SunONE 5.2 on certain platforms. Check the LDAP driver documentation for steps on how to configure the password plugin for SunONE 58 © Novell, Inc. All rights reserved.
  • 59. Linux and Unix Settings Driver • Attributes are not added to new users – Engine trace level 10 59 © Novell, Inc. All rights reserved.
  • 60. Linux and Unix Bi-directional Driver • User is not created on the platform, or data is not synchronizing correctly after creation – Engine trace level 3, Shim trace level 4 – from the connected Linux/Unix platform, get the file: /usr/local/nxdrv/logs/script-trace.log • Passwords are not syncing from the Linux/Unix platform – Information above plus the platform's PAM (or LAM) configuration files. Since those change per platform, there is no standard location to get them, but the customer's Linux/Unix admin should know where they are located 60 © Novell, Inc. All rights reserved.
  • 61. Linux and Unix Fan-out driver Driver has 2 parts: core driver and platform agents • Core Driver – IDM Driver connects to the Core Driver – Usually runs on the IDM server, but can run on a remote loader. When running on a Remote Loader, the logs referenced bellow will be in the Remote Loader server – Get the core driver Audit log and Operational log files > On Linux/Unix they are found at /usr/local/ASAM/data/CoreDriver/logs > On Windows they are found at C:NovellASAMdataCoreDriverlogs 61 © Novell, Inc. All rights reserved.
  • 62. Linux and Unix Fan-out driver Driver has 2 parts: core driver and platform agents • Platform Agents – Run on the connected system (1 platform agent per system) – Execute its action locally via shell scripts – Get the asamplat.conf file at /usr/local/ASAM/data/asamplat.conf – Get the platform's log files > On Linux/Unix the files reside at /usr/local/ASAM/data/PlatformServices/logs/ > On Midrange and Mainframe platforms, contact Novell Support for assistance with the call 62 © Novell, Inc. All rights reserved.
  • 63. Lotus Notes Driver • For any issues, obtain – Engine trace level 3, Shim trace level 5 • Check the Documentation about a Notes driver issue. The troubleshooting section in the docs will solve most problems. Many of the problems can be traced to a rights issue. 63 © Novell, Inc. All rights reserved.
  • 64. Manual Task Service Driver • For any issues – Engine trace level 5 64 © Novell, Inc. All rights reserved.
  • 65. PeopleSoft 5.2 Driver •For connectivity issues with PeopleSoft – Output of the CITester application http://www.novell.com/documentation/idm36drivers/peoplesoft_52/data/ah79lgj.html#ajn78pl •For any other issues – Engine trace level 3, Shim trace level 5 – Version of the PeopleTools (NOT the application, this is the API we connect to) that the customer is using 65 © Novell, Inc. All rights reserved.
  • 66. SAP HR Driver • Cannot synchronize objects to SAP – Engine trace level 3, Shim trace level 5 • Cannot synchronize objects from SAP – Engine trace level 3, Shim trace level 5 – Copy of the iDoc file processed by the driver > iDoc file location can be seen in the driver's properties, as the value of the parameter “iDoc File Directory” 66 © Novell, Inc. All rights reserved.
  • 67. SAP User Management Driver •For connectivity issues with SAP – Output of the SAP JCO test utility http://www.novell.com/documentation/idm36drivers/sap_user/data/alvws18.html •For any other issues – Engine trace level 3, Shim trace level 5 67 © Novell, Inc. All rights reserved.
  • 68. Scripting Driver • NTS does not support customizations to the scripts of the scripting driver. • We can help the customer with driver installation issues, but any custom code can only be reviewed by either Consulting or a Novell Partner (both cases for a fee, not included in any Novell Support contract) 68 © Novell, Inc. All rights reserved.
  • 69. SOAP Driver • For connectivity issues with the SOAP system – LAN trace between the driver shim and the SOAP system – Engine trace level 3, Shim trace level 5 • For any other issues – Engine trace level 3 69 © Novell, Inc. All rights reserved.
  • 70. Workorder Driver • For any issues – Engine trace level 3, Shim trace level 5 70 © Novell, Inc. All rights reserved.
  • 71. SIF Driver • Only supported on IDM 3.5.1 and 3.0.1 • NOT SUPPORTED on IDM 3.6 • For any issues – Engine trace level 3, Shim trace level 5 71 © Novell, Inc. All rights reserved.
  • 72. Windows NT Driver • Only supported on IDM 3.5.1 and 3.0.1 • NOT SUPPORTED on IDM 3.6 • For any issues – Engine trace level 3, Shim trace level 5 72 © Novell, Inc. All rights reserved.
  • 73. Microsoft Exchange 5.5 Driver • Only supported on IDM 3.5.1 and 3.0.1 • NOT SUPPORTED on IDM 3.6 • For any issues – Engine trace level 3, Shim trace level 5 73 © Novell, Inc. All rights reserved.
  • 74. Loopback Driver • Also known as “move-proxy driver” (old IDM 2.x nomenclature) or “Null” driver • For any issues – Engine trace level 3 74 © Novell, Inc. All rights reserved.
  • 75. Issues With Jobs • A driver export does not contain the Jobs information, so we absolutely need an Designer project export • There are currently 4 types of pre-defined Jobs, take a note of the job being used and the issue description. What will be required to troubleshoot the Jobs varies per Job and issue. 75 © Novell, Inc. All rights reserved.
  • 76. Unpublished Work of Novell, Inc. All Rights Reserved. This work is an unpublished work and contains confidential, proprietary, and trade secret information of Novell, Inc. Access to this work is restricted to Novell employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability. General Disclaimer This document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. Novell, Inc. makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for Novell products remains at the sole discretion of Novell. Further, Novell, Inc. reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.