Your SlideShare is downloading. ×
0
After the BreachAfter the Breach
Dennis SchmidtDennis Schmidt
Director, Office of Information SystemsDirector, Office of I...
OMG, We have a breach!OMG, We have a breach!
 In late July, 2009, UNC Information Technology
employees discovered that a ...
Incident DiscoveryIncident Discovery
 OIS receives call from departmental serverOIS receives call from departmental serve...
Forensic AnalysisForensic Analysis
-- A Long, Painful Process ---- A Long, Painful Process --
 Verification – Verify the ...
Forensic Analysis (cont.)Forensic Analysis (cont.)
 Evidence Collection
 All available computer information (volatile an...
Forensic Analysis (cont.)Forensic Analysis (cont.)
 Timeline Creation & Analysis – Use time-stamps
from internal and exte...
Forensic Analysis (cont.)Forensic Analysis (cont.)
 Data Recovery – extracting unallocated data in
order to recover any d...
Houston, We have a problem!Houston, We have a problem!
 Virus/worm/trojan infection for 2 yearsVirus/worm/trojan infectio...
Qualys Scan ResultsQualys Scan Results
But, did they get anything?But, did they get anything?
 When did compromise occur? Is it still active?When did compromise...
The Antivirus DilemmaThe Antivirus Dilemma
 Full virus scan changes the last accessed time onFull virus scan changes the ...
No Smoking GunNo Smoking Gun
 There was no way to prove that data on theThere was no way to prove that data on the
server...
Second OpinionSecond Opinion
 Magnitude of potential breach warrantedMagnitude of potential breach warranted
additional o...
Notification is not an IT DecisionNotification is not an IT Decision
 University Counsel makes final recommendationUniver...
How do we notify 180,000 people?How do we notify 180,000 people?
 Is their address current? Do we have anIs their address...
The Notification ProcessThe Notification Process
 UNC Hired Rust Consulting to assistUNC Hired Rust Consulting to assist
...
Technical ResponseTechnical Response
 Major concern: Uncontrolled serverMajor concern: Uncontrolled server
proliferationp...
The Scope of the ProblemThe Scope of the Problem
 500+ machines with server OS’s on SOM500+ machines with server OS’s on ...
Manual Data CollectionManual Data Collection
 Mandatory self reporting of serversMandatory self reporting of servers
 43...
Long Range StrategyLong Range Strategy
 IT Simplification and Security RFP (Dell)IT Simplification and Security RFP (Dell...
Recovery from the breachRecovery from the breach
 Moved data to centrally managed serversMoved data to centrally managed ...
How much did it cost?How much did it cost?
 Average breach reportedly costs $204 per nameAverage breach reportedly costs ...
Lessons LearnedLessons Learned
 Implementation of IT Governance is criticalImplementation of IT Governance is critical
 ...
Questions?Questions?
Upcoming SlideShare
Loading in...5
×

After the Breach

366

Published on

The UNC School of Medicine suffered a security breach last summer that required notification of over 100,000 patients that their information had been exposed. This presentation will talk about the scope of damage that is caused by a breach of this
magnitude and the many steps that are necessary for damage control and recovery.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
366
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "After the Breach"

  1. 1. After the BreachAfter the Breach Dennis SchmidtDennis Schmidt Director, Office of Information SystemsDirector, Office of Information Systems HIPAA Security OfficerHIPAA Security Officer UNC School of MedicineUNC School of Medicine
  2. 2. OMG, We have a breach!OMG, We have a breach!  In late July, 2009, UNC Information Technology employees discovered that a server which contained sensitive information on 180,000 research subjects, including 114,000 Social Security Numbers, had been the target of a computer hack in 2007. The compromised server was taken down and the data on the server were removed.
  3. 3. Incident DiscoveryIncident Discovery  OIS receives call from departmental serverOIS receives call from departmental server admin reporting that a server would not rebootadmin reporting that a server would not reboot after power failure.after power failure.  OIS technician suspects virus and performs fullOIS technician suspects virus and performs full virus scan on machine. Virus detected.virus scan on machine. Virus detected.  Technician is told by department that server mayTechnician is told by department that server may contain sensitive information.contain sensitive information.  Server turned over to OIS Information SecurityServer turned over to OIS Information Security for forensic analysis.for forensic analysis.
  4. 4. Forensic AnalysisForensic Analysis -- A Long, Painful Process ---- A Long, Painful Process --  Verification – Verify the incident occurred  Interview the SysAdmins and other users involved  Examine system and application logs (Snort, Tipping Point, etc.)  Check volatile information using forensic tools  System Description  Physical observation, forensic tools  Interview SysAdmins and users, determine use  Hardware and software system characteristics  Hard disk geometry
  5. 5. Forensic Analysis (cont.)Forensic Analysis (cont.)  Evidence Collection  All available computer information (volatile and non-volatile) is collected and transferred to external media or forensic workstation to perform analysis tasks.  Data must be collected in order of volatility and data integrity safeguarded by hash signature, MD5
  6. 6. Forensic Analysis (cont.)Forensic Analysis (cont.)  Timeline Creation & Analysis – Use time-stamps from internal and external sources to correlate into timeline that traces back the system activity.  Media Analysis – Thorough examination of the media layers (physical, data, metadata, file system and file name) searching for evidence.
  7. 7. Forensic Analysis (cont.)Forensic Analysis (cont.)  Data Recovery – extracting unallocated data in order to recover any deleted files. File fragments could represent a critical piece of information relevant to the case  String Search – searching for specific strings or keywords contained inside files to reveal useful information relevant to the case.  Reporting -- detailed report(s) of the forensic process explaining the evidence found, together with the techniques and methodology used.
  8. 8. Houston, We have a problem!Houston, We have a problem!  Virus/worm/trojan infection for 2 yearsVirus/worm/trojan infection for 2 years  26 files containing over 500,000 records26 files containing over 500,000 records  180,000 unique research subjects180,000 unique research subjects  114,000 Social Security Numbers114,000 Social Security Numbers
  9. 9. Qualys Scan ResultsQualys Scan Results
  10. 10. But, did they get anything?But, did they get anything?  When did compromise occur? Is it still active?When did compromise occur? Is it still active?  When were the sensitive files put on theWhen were the sensitive files put on the machine? When were they last accessed?machine? When were they last accessed?  Was it during the compromise window?Was it during the compromise window?  Is there any corroborating evidence on theIs there any corroborating evidence on the network of file downloads from the server?network of file downloads from the server?
  11. 11. The Antivirus DilemmaThe Antivirus Dilemma  Full virus scan changes the last accessed time onFull virus scan changes the last accessed time on everyevery file.file.  It now becomes impossible to determine if theIt now becomes impossible to determine if the malware actually accessed specific files.malware actually accessed specific files.  e.g., If compromise occurred one week ago, and laste.g., If compromise occurred one week ago, and last access of sensitive file was one month ago, you knowaccess of sensitive file was one month ago, you know the data was not likely accessed by the malware.the data was not likely accessed by the malware.  If virus scan was done yesterday, you no longerIf virus scan was done yesterday, you no longer know when the file was last accessed.know when the file was last accessed.
  12. 12. No Smoking GunNo Smoking Gun  There was no way to prove that data on theThere was no way to prove that data on the server was accessed inappropriately.server was accessed inappropriately.  And… there was no way to prove that data onAnd… there was no way to prove that data on the server wasthe server was notnot accessed inappropriately.accessed inappropriately.  The doors were unlocked and people were in theThe doors were unlocked and people were in the house, but we couldn’t prove that they stolehouse, but we couldn’t prove that they stole anything.anything.
  13. 13. Second OpinionSecond Opinion  Magnitude of potential breach warrantedMagnitude of potential breach warranted additional opinionsadditional opinions  ITS Security conducted parallel investigation toITS Security conducted parallel investigation to verify or refute initial findingsverify or refute initial findings  Additional corroborating data searchedAdditional corroborating data searched  Network traffic logs (only last 90 days)Network traffic logs (only last 90 days)
  14. 14. Notification is not an IT DecisionNotification is not an IT Decision  University Counsel makes final recommendationUniversity Counsel makes final recommendation based on inputs from:based on inputs from:  IT Security (OIS & ITS)IT Security (OIS & ITS)  University RelationsUniversity Relations  UNC Health Care Communications/MarketingUNC Health Care Communications/Marketing  UNC Health Care CounselUNC Health Care Counsel  HIPAA Privacy and HIPAA Security OfficersHIPAA Privacy and HIPAA Security Officers
  15. 15. How do we notify 180,000 people?How do we notify 180,000 people?  Is their address current? Do we have anIs their address current? Do we have an address?address?  Are they still alive?Are they still alive?  Who writes the letters?Who writes the letters?  Who addresses the envelopes? Licks theWho addresses the envelopes? Licks the stamps?stamps?  Who handles phone calls from concernedWho handles phone calls from concerned recipients?recipients?
  16. 16. The Notification ProcessThe Notification Process  UNC Hired Rust Consulting to assistUNC Hired Rust Consulting to assist  Consultation servicesConsultation services  Mailed notification lettersMailed notification letters  Established and staffed Call CenterEstablished and staffed Call Center  Responded to calls; referred problem calls to UNCResponded to calls; referred problem calls to UNC  Received 4,144 callsReceived 4,144 calls  450 calls referred to UNC450 calls referred to UNC
  17. 17. Technical ResponseTechnical Response  Major concern: Uncontrolled serverMajor concern: Uncontrolled server proliferationproliferation  Determine scope of problemDetermine scope of problem  Protect high risk machines firstProtect high risk machines first  Develop long term strategy to mitigate riskDevelop long term strategy to mitigate risk
  18. 18. The Scope of the ProblemThe Scope of the Problem  500+ machines with server OS’s on SOM500+ machines with server OS’s on SOM networknetwork  2200 machines running a service2200 machines running a service  2068 File Server / File Services2068 File Server / File Services  1989 Remote Access / Remote Management1989 Remote Access / Remote Management  762 Web Servers762 Web Servers  194 Database Servers194 Database Servers
  19. 19. Manual Data CollectionManual Data Collection  Mandatory self reporting of serversMandatory self reporting of servers  433 servers reported433 servers reported  98 server admins98 server admins  47 different OS flavors and versions47 different OS flavors and versions  Qualys scans on all servers reporting sensitiveQualys scans on all servers reporting sensitive information (200 machines)information (200 machines)
  20. 20. Long Range StrategyLong Range Strategy  IT Simplification and Security RFP (Dell)IT Simplification and Security RFP (Dell)  Develop Plan for streamlining IT resources in SOMDevelop Plan for streamlining IT resources in SOM  Develop strategic virtualization architectureDevelop strategic virtualization architecture  Develop enterprise storage architectureDevelop enterprise storage architecture  Develop security umbrella to cover centralizedDevelop security umbrella to cover centralized operationoperation  Goal: Provide robust centralGoal: Provide robust central servicesservices that willthat will get end users out ofget end users out of serverserver businessbusiness
  21. 21. Recovery from the breachRecovery from the breach  Moved data to centrally managed serversMoved data to centrally managed servers  Database encrypted behind hardware firewallDatabase encrypted behind hardware firewall  All working files encrypted with PGP Net ShareAll working files encrypted with PGP Net Share  All machines, including desktops, scanned with QualysAll machines, including desktops, scanned with Qualys  Well defined procedures documented, approved by IRBWell defined procedures documented, approved by IRB  Two person rule for manual movement of data filesTwo person rule for manual movement of data files  Update software to automate processesUpdate software to automate processes
  22. 22. How much did it cost?How much did it cost?  Average breach reportedly costs $204 per nameAverage breach reportedly costs $204 per name  $204 X 180,000 = $36.7 Million!$204 X 180,000 = $36.7 Million!  Other references state that a major breach costsOther references state that a major breach costs an organization aan organization a minimumminimum of $1 Million.of $1 Million.  Postage alone cost $75,000.Postage alone cost $75,000.  Rust Consulting cost $260,000Rust Consulting cost $260,000  Thousands of person hours spent on the projectThousands of person hours spent on the project  OIS Security, ITS Security, OUC, P&A, HIPAAOIS Security, ITS Security, OUC, P&A, HIPAA Privacy, senior leadership, etc. etc. etc.Privacy, senior leadership, etc. etc. etc.
  23. 23. Lessons LearnedLessons Learned  Implementation of IT Governance is criticalImplementation of IT Governance is critical  Decentralized server environment is high riskDecentralized server environment is high risk  New procedures for virus investigationsNew procedures for virus investigations involving sensitive datainvolving sensitive data  Disconnect from networkDisconnect from network  Do not shut downDo not shut down  Do not perform virus scanDo not perform virus scan  Notify IT SecurityNotify IT Security
  24. 24. Questions?Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×