1. Neumont University
Dan Taylor -- IS Capstone
PROJECT HORUS
Troubleshooting
Version 3.0

2. P a g e | 1
Contents
1.0 Introduction..................................................................................................................................2
1.1 Change Log...................................................................................................................................2
2.0 Network Topology.........................................................................................................................3
2.01 Management IPs .....................................................................................................................3
2.02 Production – VLAN X, 10.0.X.0/24.............................................................................................3
2.03 Workstation IPs.......................................................................................................................3
2.1 Power on Order............................................................................................................................3
2.11 Physical Machines ...................................................................................................................3
2.12 Virtual Machines (The next step)..............................................................................................4
2.2 Power off Order............................................................................................................................4
2.21 Virtual Machines .....................................................................................................................4
2.22 Physical Machines ...................................................................................................................4
2.3 Network Connectivity Issues..........................................................................................................4
2.31 No Internet Connectivity on VMs..............................................................................................4
3.0 SAN Connectivity Errors.................................................................................................................5
3.01 ERROR: Virtual Machines Become Inaccessible..........................................................................5
3.02 ERROR: Cannot connect to vCenter Single Sign-On server(vSphere Web Client)..........................6
3.03 ERROR: Empty Inventory..........................................................................................................6
4.0 Spector 360 Server Configuration...................................................................................................7
4.01 ERROR: Test email failedwiththe followingmessage:Failure tosendmail.Unable toreaddata
from the transport connection: net_io_connectionclosed.................................................................7
5.0 Spector 360 Server Troubleshooting...............................................................................................7
5.01 Unable to Start Spector Control Center Server Service: Logon Failure .........................................7
5.02 Unable To Retrieve Server Settings – Unable to communicate....................................................8
6.0 Spector 360 End-User Configuration...............................................................................................9
6.01 Start Windows Remote Registry Service....................................................................................9
6.02 ERROR: Cannot Open Service Control Manager on [Computer Name].........................................9
6.03 Hosts cannot ping one another...............................................................................................10

3. P a g e | 2
1.0 Introduction
The contentsof thisdocumentare onlytobe viewedbythose withsole authorizationfromthe creator
of thisdocument.Anyunauthorizedindividuals viewingthisdocumentcouldpotentiallyface charges
under18 U.S.Code§ 1905. This documentissubjecttochange and will be maintainedthroughachange
logdocumentingwhathadchangedpereach instance.
1.1 Change Log
Version # Name Changes Made SectionsAdded/Modified
1.0 Daniel Taylor Initial implementationof Change Log.All
changesmade at thispointwill be
documentedhere.
Reference Documents
Introduction
Change Log
2.0 Daniel Taylor AddeddocumentationonActive Directory
configurationsaswell assettingupand
connectingtoa SAN usingsoftware iSCSI
adapters.
3.0 (andsubsections)
4.0 (andsubsections)
3.0 Daniel Taylor Splitdocumentintotwodifferent
documents:“Installationand
Configuration”and“Troubleshooting”.

4. P a g e | 3
2.0 Network Topology
2.01 Management IPs
Object IP Configuration (All Subnets are /21)
SAN Management 10.0.0.9
Firewall 10.0.0.10
DHCP for IS Wireless Network 10.0.7.0 to 10.0.7.254
Wireless Management 10.0.0.27
Chassis 1 Blade 8 10.0.1.8
Chassis 2 Blade 8 10.0.2.8
Chassis 3 Blade 8 10.0.3.8
ESXi; Chassis 1 Blade 8 10.0.1.24
ESXi; Chassis 2 Blade 8 10.0.2.24
ESXi; Chassis 3 Blade 8 10.0.3.24
vSphere/vCenter; Chassis 1 Blade 8 10.0.1.40
vSphere/vCenter; Chassis 2 Blade 8 10.0.2.40
vSphere/vCenter; Chassis 3 Blade 8 10.0.3.40
Web Client 10.0.2.40:9443
2.02 Production – VLAN X, 10.0.X.0/24
VLAN 84, 10.0.84.XXX/24
PublicIP:69.27.22.30
PublicGateway:69.27.22.1
2.03 Workstation IPs
User Category Project Horus IPs
Workers (two VMs) 10.0.84.20; 10.0.84.21
Exploders (two VMs) 10.0.84.30; 10.0.84.31
Slackers (two VMs) 10.0.84.40; 40.0.84.41
Active Directory (one VM) 10.0.84.10; 10.0.7.253
Spector 360 Server (one VM) 10.0.84.11; 10.0.7.254
2.1 Power on Order
The order inwhichto turn onthese systemsis hierarchical andistobe followedexplicitlyasitislisted,
withno deviationorvariationforthe systemtooperate:
2.11 Physical Machines
1. Chassis2, Blade 8
2. Chassis1, Blade 8
3. Chassis3, Blade 8
Allowaperiodof 5 minutesbeforecontinuingtothe nextstep.

5. P a g e | 4
2.12 Virtual Machines (The next step)
1. On Chassis2, Blade 8, turn onthe vCenterServerVM.
a. Log in tothe vCenterServerVMandwait5 minutesbefore continuingtothe nextstep
for all of the servicestofullyactivate.
b. AccessthisVM by usingthe vSphere Clientandconnectingto10.0.2.24.
2. On Chassis1, Blade 8, turn onthe ADServerVM.
a. Log in tothe AD ServerVMand wait5 minutesbefore continuingtothe nextstepforall
of the servicestofullyactivate.
b. AccessthisVMby usingthe vSphere WEBClientandconnectingto
https://10.0.2.40:9443
3. On Chassis1, 2, and 3, turn on the twoEnd-UserVMs.
Afterthis,turnon all the End-UserVMs, of whichthere are 2 (TWO) on Blade 8 of Chassis1, 2, and 3.
2.2 Power off Order
The order inwhichto turn off these systemsishierarchical andisto be followedexplicitlyasitislisted,
withno deviationorvariationforthe systemtopoweroff inthe correct order:
2.21 Virtual Machines
1. On Chassis1, 2, and 3, all of Blade 8, turn off the End-UserVMs.
2. On Chassis1, Blade 8, turn off the AD ServerVM.
a. Thiscan be done simplybyright-clickingthe VMandclicking“PowerOff”.
3. On Chassis2, Blade 8, turn off the vCenterVM.
2.22 Physical Machines
Nowthat all of the VMs have beenturnedoff,the physical machinescanbe poweredoff inanyorder
throughtheirassociatediDRACcontrols(10.0.1.8, 10.0.2.8, 10.0.3.8 respectively).
DO NOT TURN THE SERVERS OFF BY HOLDING THE POWER BUTTON DOWN ON THE PHYSICAL
MACHINES.
2.3 Network Connectivity Issues
2.31 No Internet Connectivity on VMs
If there is an issue where aVMisunable toconnectto the internet,firstmake sure theyhave twoNICs
(NetworkInterface Cards) viathe vSphere WebClient.NIC1MUST connectto DPortGroupVLAN10and
NIC2 MUST connectto the Test.
Note:DPortGroupVLAN10isthe Internet,while Productionisthe Intranet.
Your VMSettingsshouldlooksomethinglike this:

6. P a g e | 5
3.0 SAN Connectivity Errors
3.01 ERROR: Virtual Machines Become Inaccessible
Thisissue ismostlikelycausedfromthe DataSource on whichyour VMsrun has come across some
problem,typicallydue tothe Data Source beingshutoff or the serversbeingdetachedfromthe Data
Source.To remediate thisissue,typically youcanrescanthe adapterthat isconnectedtothe Data
Source for any“newlydiscovered”storage devices,asshownbelow.
If that doesnot resolve yourissue, the issueliesinthe SAN configuration.YourSAN interface,upon
selectingyourservercluster,willmostlikelyshow some resultcomparabletoDell Compellent’s
“PartiallyConnected”status,asshownbelow:
For thisproject,the IPof the Compellent ManagementInterface is https://10.0.0.9.The solutiontothis
isverysimple.First,navigate tothe Mappingtabon any of the serverswithinyourclusterinCompellent
storage.Clickonthe Volume andthenclickRemove Mapping.Thiswill removethe Volume fromall of
your serverssimultaneously,providedyoucorrectlyimplementedthemintoaServerCluster. Now,to
add the Volume backtothe ServerCluster,navigate toStorageVolumes Big Brother Mapping
and click on “Map Volume toServer”.You’ll be presentedwithadialogue box onwhichserverorserver
clusterto selectformapping.Clickon1984.

7. P a g e | 6
Once the Volume hasbeenmappedbackto the ServerCluster,goaheadandrescanthe Storage
Adaptersasdemonstratedinthe firststep,andyou’ll see thatthe iSCSISoftware Adaptershave
connectedbackto the SAN and the Virtual Machineshave beendiscovered. The HBAsmaystill show
partiallyconnectedstatus,thoughthatisnota glaringissue tobe concernedabout.
3.02 ERROR: Cannot connect to vCenter Single Sign-On server (vSphere Web Client)
Thisissue iscausedby the vCenterServerService(locatedonthe vCenterServer) havingdifficultieson
starting.Resolvingthisissue mayvaryindifficultydependingonthe state of the vCenterServerandif
the vCenterServerService isinthe midstof startingornot. The easiestsolution,of course,istorestart
the vCenterServerandnotforce start anyprocesses;letVMWare andvSphere organize the startorder
for all of these processes.
3.03 ERROR: Empty Inventory
In the eventof an EmptyInventoryupontryingtoaccess hostson the vSphere WebClient, there isa
service thatdoesn’talwaysautomaticallystart,thoughitshould.Firstly,navigatetoLocal Serviceson
the vCenterServerandlookforVMware vCenterInventoryService andensure thatitisrunning.If itis,
simplywaitfive totenminutesforthe service toinformthe webserverof itsinventory.

8. P a g e | 7
4.0 Spector 360 Server Configuration
4.01 ERROR: Test email failed with the following message: Failure to send mail. Unable to
read data from the transport connection: net_io_connectionclosed
Shouldthiserroroccur, it isa firewall issuewiththe network-wide firewall. Toresolvethis,youmustbe
able to gainadministrativeaccesstoyourinternal Firewall. The followingtwocommandswill workfora
CiscoASA Firewall.
access-list smtp extended permit tcp any host 10.0.7.254 eq smtp
access-group smtp in interface outside
5.0 Spector 360 Server Troubleshooting
5.01 Unable to Start Spector Control Center Server Service: Logon Failure
In the eventthe belowerrorisreceived, the issue liesinwithSingle SignOncapabilities.
To resolve thisissue,openthe Spector360 Control Center(ignoreall warnings/errors)andnavigate to
“Database”.From there,clickon“Manage Database Logins”and finallyclick“Create anew Login
Account”,as shownbelow.

9. P a g e | 8
From thisscreen,clickthe “Use SQL ServerAuthentication”radial button,thenfill inusercredentialsas
seenbelow:
Before savingthis,make sure thatunderthe “SelectEvents”and“SelectTools”tabsthat all optionsare
selectedexceptfor“Auditing”under“SelectTools”.Once thatisconfirmed,click“Save andClose”,close
the Spector360 Control Centerapplicationandlogoutof the current Administratoraccount.Logback in
to the serveras an account that doesnothave single signonprivilegesanduse the newlycreatedlogon
credentialsviaDatabase Login.Alternatively,logginginas“sa” will alsoworkprovidedthe credentials
are known.
5.02 Unable To Retrieve Server Settings – Unable to communicate.
If the stepsinthe previoussubsection(6.01) donot resolve thisissue, thenthe issue liesinincorrect
logincredentialsforthe SpectorControl CenterServerService itself.Toresolve thisissue,navigateto
Local Services,locate the “SpectorControl CenterServer”service,rightclickitandclickProperties.Now
clickon the Log-Ontab. The followingwindow will appear:
There are nowtwopossibilitiesonhowtoremediate thisissue.The firstis(andeasiest)istoclickthe
“Local Systemaccount”radial buttonto allow the service tostart bythe “SYSTEM” userthan an
Administratoruseraccount.The secondchoice forfixingthisissueistochange the account credentials
that activate the service.The mostcommonproblemhere isthatthe passwordhaschangedon the
associatedaccount.Update the passwordand start the service.

10. P a g e | 9
6.0 Spector 360 End-User Configuration
6.01 Start Windows Remote Registry Service
If the WRRS can’t be reached,loginto the endusermachine as an Administratorand openservices.msi.
Navigate tothe Remote Registryservice,rightclickitandclickon Properties.Inthe dropdownbox,click
on Automatic,thenclickonstart.It will looklike thisif done right:
In the eventmanuallystartingthe service isprovingimpossible,clickthe LogOntab of the Remote
Registryservice.Make the Username fieldsay“NTAuthorityLocalService”andclearout bothpassword
fields.Hitapplyandtryrestartingthe service.
6.02 ERROR: Cannot Open Service Control Manager on [Computer Name]
If such an error occurs while runningdiagnosticsonanymachine,logintothe endusermachine as an
Administratorandnavigate tothe AdvancedFirewall Options.Make anew InboundRule forPort-based
access onTCP and UDP ports 135. If done right,there will be twoentriesinthe InboundRulessection
for WindowsFirewall,asseenonthe nextpage.

11. P a g e | 10
6.03 Hosts cannot ping one another
Resolvingthisissueisextremelysimple.First,openupaCommandPrompton the VMyouwant to ping
to and the VMyou wantto pingfrom.Next,type thiscommandall onone line:
netsh advfirewall firewall add rule name="ICMP Allow incoming V4 echo
request"protocol=icmpv4:8,any dir=in action=allow
Thisadjuststhe firewall toallowICMPpacketstobe sentto it.