“ Is your site ready?” Disaster planning, preparation and recovery for Joomla! TM Sites Tom Canavan JoomlaRescue.com ™

Disasters DO happen Disaster preparedness is what you do before, not after, a disaster hits. Crackers/hackers are only part of your concern. Disaster—the meaning of the word has its root in early Italian, from the word disastro (Meaning away from star). It was thought that an unfavorable position of a star or planet was the cause of mishaps and calamities 1 1 Robert K. Barnhart, “The Barnhart concise Dictionary of Etymology – The origins of American English words”, (New York: HarperCollins books, 1995) 208

Disaster preparedness is what you do before, not after, a disaster hits.

Crackers/hackers are only part of your concern.

Disaster—the meaning of the word has its root in early Italian, from the word disastro (Meaning away from star).

It was thought that an unfavorable position of a star or planet

was the cause of mishaps and calamities 1

1 Robert K. Barnhart, “The Barnhart concise Dictionary of Etymology – The origins of American English words”, (New York: HarperCollins books, 1995) 208

What do you consider a disaster? 4-19-1995 Murrah Bldg Okla City 9-11-2001 Ground Zero 8/28/2005 Hurricane Katrina

I’ll take Disaster Recovery Planning for $500.00 -QUIZ- Who has a working DR Plan? If your site was offline for 7 to 10 days, would your company go bankrupt?

-QUIZ-

Who has a working DR Plan?

If your site was offline for 7 to 10 days,

would your company go bankrupt?

404: Page Not found A 1978 Study by the University of Minnesota showed that if a business could not recover their systems within a week, will be out of business in a year. That’s only four to six days of interruption of services in 1978 Aasgaard, D.O. et al., “An evaluation of Data processing ‘Machine room’ Loss and Selected Recovery Strategies,” MISRC Working Papers (Minneapolis, MN: University of Minnesota, 1978) 1 1-

A 1978 Study by the University of Minnesota showed that if a business could not recover their systems within a week,

will be out of business in a year.

That’s only four to six days

of interruption of services in 1978

Disaster Planning Life Cycle 1 2 3 4 Determine Risks Document Your Business Build Your plan Test & document

Worst Practices for DR/DP Failure to get management support No risk assessment No written plan Lack of ‘good’ backup’s You put the tapes where??

Failure to get management support

No risk assessment

No written plan

Lack of ‘good’ backup’s

You put the tapes where??

Today’s agenda Planning Determine risks Fortify Test/Document The elements, issues and challenges with planning Hackers are only one concern – there’s more Chances are GOOD you are exposed somewhere to attack Test and Documentation is vital to a healthy plan Communications Who needs to be informed, how to inform, Media/Press Ω

Determine Risks What ‘could’ go wrong? Hardware/Software Failure, DNS, Hackers What can you do to mitigate it? Hot site, backups, planned recovery

What ‘could’ go wrong?

Hardware/Software Failure, DNS, Hackers

What can you do to mitigate it?

Hot site, backups, planned recovery

Determine Risks People Safety (of staff) Where will they work? Do they KNOW procedures (fire drill much?) Telephones, Pagers, Cell Phones, Email Hosting Co-Location (shared, dedicated, VPS) Workstations

People

Safety (of staff)

Where will they work?

Do they KNOW procedures (fire drill much?)

Telephones, Pagers, Cell Phones, Email

Hosting

Co-Location (shared, dedicated, VPS)

Workstations

Determine Risk Restoration costs BY host ($$$) Backups, Yes but.. License keys Copies of source/apps – do they exist? Safe place to keep digital media Identify ‘stakeholders’ Insurance – Do you have any? Your own computers – virus free? What about your ‘backup server’ itself?

Restoration costs BY host ($$$)

Backups, Yes but..

License keys

Copies of source/apps – do they exist?

Safe place to keep digital media

Identify ‘stakeholders’

Insurance – Do you have any?

Your own computers – virus free?

What about your ‘backup server’ itself?

Affordability of a Risk Elements to consider How much $$$ are you willing to spend Does management buy into your plan? Are they willing to commit to it financially? Does your site “justify” a DR plan Determine if risks JUSTIFY cost At the end of the day, if you have a blog site, then perhaps its not worth it. If you have an ecommerce site, then it WILL be.

Elements to consider

How much $$$ are you willing to spend

Does management buy into your plan?

Are they willing to commit to it financially?

Does your site “justify” a DR plan

Determine if risks JUSTIFY cost

At the end of the day, if you have a blog site,

then perhaps its not worth it. If you have an

ecommerce site, then it WILL be.

Key Points Know your risks Know your what the costs are Cost of experiencing the risk Cost of restoration from downtime Have a plan to mitigate and recover

Know your risks

Know your what the costs are

Cost of experiencing the risk

Cost of restoration from downtime

Have a plan to mitigate and recover

Why do need a plan? Recognize that trouble WILL come Mr. Murphy on line one for you… Your plan should be : SMART based Specific, Measurable, Attainable, Realistic, and Time-sensitive " A good plan, violently executed now, is better than a perfect plan next week .“ General George Patton

Recognize that trouble WILL come

Mr. Murphy on line one for you…

Your plan should be : SMART based

Specific, Measurable, Attainable, Realistic, and Time-sensitive

" A good plan, violently executed now, is better than a perfect plan next week .“

General George Patton

Preparing to Plan Recognize the following A hard to execute plan will likely fail Avoid ‘conforming’ to multiple opinions Staff members will fight the plan A plan untested is no good Plans take time to build A solid “one-page” plan is better than none

Recognize the following

A hard to execute plan will likely fail

Avoid ‘conforming’ to multiple opinions

Staff members will fight the plan

A plan untested is no good

Plans take time to build

A solid “one-page” plan is better than none

Planning Elements RTO/RPO – what is yours? Recovery Time Objective Recovery Point Objective Who is in charge? Who else is in charge Moving parts of your plan Where to store media, labeling, media type

RTO/RPO – what is yours?

Recovery Time Objective

Recovery Point Objective

Who is in charge?

Who else is in charge

Moving parts of your plan

Where to store media, labeling, media type

Planning Elements Do you have a ‘fall-back’ When will you ‘activate’ you plan? Define a communications strategy Which ‘systems’ have priority? Develop a schedule to plan Can you afford your plan?

Do you have a ‘fall-back’

When will you ‘activate’ you plan?

Define a communications strategy

Which ‘systems’ have priority?

Develop a schedule to plan

Can you afford your plan?

Key Points Keep your planning team small Involve Sr. Mgmt, CAREFULLY Keep strong focus, for short bursts Planning takes ‘time’ – and comfort Your Plan WILL fail the first time you use it Your staff will not buy in at first Setup a start, middle and end for plan

Keep your planning team small

Involve Sr. Mgmt, CAREFULLY

Keep strong focus, for short bursts

Planning takes ‘time’ – and comfort

Your Plan WILL fail the first time you use it

Your staff will not buy in at first

Setup a start, middle and end for plan

Fortification Preparation of your site is key – check: Extensions, hosting, root kits, open ports Set permissions correctly Files and directories (644 / 755) Latest version of Joomla (1.0.xx and 1.5) Check your HOST’s setup Ports, Versions of apache, etc.

Preparation of your site is key – check:

Extensions, hosting, root kits, open ports

Set permissions correctly

Files and directories (644 / 755)

Latest version of Joomla (1.0.xx and 1.5)

Check your HOST’s setup

Ports, Versions of apache, etc.

Fortify at risk code Can you find the problem?

Vulnerable Code It’s missing the critical code: // no direct access defined( '_VALID_MOS' ) or die( 'Restricted access‘); While this problem is less prevalent - It still exists and can trip you up Note: the previous code snip was purposely modified for demonstration purposes only !

It’s missing the critical code:

// no direct access

defined( '_VALID_MOS' ) or die( 'Restricted access‘);

While this problem is less prevalent - It still exists and can trip you up

Note: the previous code snip was purposely modified for demonstration purposes only !

Fortify - .htaccess .htaccess – your first line of defense

Fortify - Permissions Permissions Very common problem Check files and Dirs FILES: 644 DIR : 755

Permissions

Very common problem

Check files and Dirs

FILES: 644

DIR : 755

Fortify – PHP.INI Safe Mode: OFF Open basedir: none Display Errors: ON Short Open Tags: ON File Uploads: ON Magic Quotes: ON Register Globals: OFF

Safe Mode: OFF

Open basedir: none

Display Errors: ON

Short Open Tags: ON

File Uploads: ON

Magic Quotes: ON

Register Globals: OFF

Fortify - Versions Using 1.0.xx Make sure you are at least at 1.0.15 Using 1.5 Make sure you are at least at 1.5.3 Older versions are exploitable

Using 1.0.xx

Make sure you are at least at 1.0.15

Using 1.5

Make sure you are at least at 1.5.3

Older versions are exploitable

Fortify – Common Trip Ups Common issues Admin still named ADMIN Easy to guess passwords like P@ssw0rd Permissions set wrong Lack of .htaccess or php.ini Vulnerable components Hosts not setup properly

Common issues

Admin still named ADMIN

Easy to guess passwords like P@ssw0rd

Permissions set wrong

Lack of .htaccess or php.ini

Vulnerable components

Hosts not setup properly

Fortify - Poor Host Security Example: Ports open that need not be Real case from JoomlaRescue.com Client The host had 1,700 ports open. Port 53 – Allows for Zone Transfers Port 23 – Telnet – Allowed “Banner Grabbing” Port 21 – Allowed me (shouldn’t have) to FTP in Port 6667 (note BackOrfice) – Cult of the Dead Cow And 1,677 more – (HUN???) Host told client: “ That’s ok you have a Virtual private Server (VPS) setup”

Example: Ports open that need not be

Real case from JoomlaRescue.com Client

The host had 1,700 ports open.

Port 53 – Allows for Zone Transfers

Port 23 – Telnet – Allowed “Banner Grabbing”

Port 21 – Allowed me (shouldn’t have) to FTP in

Port 6667 (note BackOrfice) – Cult of the Dead Cow

And 1,677 more – (HUN???)

Host told client:

“ That’s ok you have a Virtual private Server (VPS) setup”

Fortification Tools Tools to check host out: NMAP ( only with host’s permission ) Tools from http://centralops.net/co Domain Dossier Joomla Health Check (available from J!) Google Google Hacks (again permission please) Hire JoomlaRescue.com

Tools to check host out:

NMAP ( only with host’s permission )

Tools from http://centralops.net/co

Domain Dossier

Joomla Health Check (available from J!)

Google

Google Hacks (again permission please)

Hire JoomlaRescue.com

Documentation Documentation is a product of your risk assessment, goals, planning and fortification. It’s the chief cornerstone of your DR plan.

Documentation is a product of your risk assessment, goals, planning and fortification.

It’s the chief cornerstone of your DR plan.

Documentation Documentation considerations First recognize its not the Holy Bible It CAN be changed as needed to fit Establish a review process It will change from time to time Make sure the Date is on it Keep it in a safe place Key DR team members must have it Don’t let it fall into competitors hands

Documentation considerations

First recognize its not the Holy Bible

It CAN be changed as needed to fit

Establish a review process

It will change from time to time

Make sure the Date is on it

Keep it in a safe place

Key DR team members must have it

Don’t let it fall into competitors hands

Maintaining your plan Test your plan Accomplished through drills Document the results Change documentation as needed Collect old docs, distribute new Tracking changes Why did you change it? Always ask WHY changes will increase survivability

Test your plan

Accomplished through drills

Document the results

Change documentation as needed

Collect old docs, distribute new

Tracking changes

Why did you change it?

Always ask WHY changes

will increase survivability

Drill for results Establish a ‘failure’ test Purpose: To shake down your documentation To train your staff To learn where your plan works and fails Establish a ‘regular’ drill time Key members should be present at each test

Establish a ‘failure’ test

Purpose:

To shake down your documentation

To train your staff

To learn where your plan works and fails

Establish a ‘regular’ drill time

Key members should be present at each test

Some things your plan should have Team member contact information Plan initiation instructions ‘when’ we activate the plan Location of backup media Passwords and other security information Contact for host Technical support, escalation procedures Instructions on HOW to restore

Team member contact information

Plan initiation instructions

‘when’ we activate the plan

Location of backup media

Passwords and other security information

Contact for host

Technical support, escalation procedures

Instructions on HOW to restore

Documentation Example

A few words on drilling Conducting a live test helps increase your site’s survivability by proving your plan works, and ensuring your staff knows their job

About your plan "No plan survives first engagement with the enemy" Von Clausewitz.—Prussian Military Thinker

Key Points Your Plan/Docs is a living document Care and feed for it Test it once you develop Conduct regular drills Change it if its not working Establish a process for distribution Keep it safe

Your Plan/Docs is a living document

Care and feed for it

Test it once you develop

Conduct regular drills

Change it if its not working

Establish a process for distribution

Keep it safe

Communications Understanding crisis communication Preparing media kits in advance Communicating with your team

Understanding crisis communication

Preparing media kits in advance

Communicating with your team

Crisis Communication Internal with team Coordinates efforts for recovery Internal with employees other staff Helps to control rumors Communications with media / customers Prepare plan in advance This helps you control the message Helps retain customer base

Internal with team

Coordinates efforts for recovery

Internal with employees other staff

Helps to control rumors

Communications with media / customers

Prepare plan in advance

This helps you control the message

Helps retain customer base

Media Communications Media contact Baseline communication regarding the event. Reestablishes trust and ensure facts not conjecture. The message should drive the behavior you want Accomplish this through advanced preparation Talking points for employees. A template for developing a news release. A list of reporters, media outlets or blog sites you want your message directed to. A fact sheet for media, both downloadable PDF and paper based.

Media contact

Baseline communication regarding the event.

Reestablishes trust and ensure facts not conjecture.

The message should drive the behavior you want

Accomplish this through advanced preparation

Talking points for employees.

A template for developing a news release.

A list of reporters, media outlets or blog sites you want your message directed to.

A fact sheet for media, both downloadable PDF and paper based.

Staff Communications Establish a communications tree Assign a Communications person or team Make sure you do two things Communicate openly and often with DR team Carefully distribute information to rest of staff Keep in mind what you say, may end up on a blog or in the paper.

Establish a communications tree

Assign a Communications person or team

Make sure you do two things

Communicate openly and often with DR team

Carefully distribute information to rest of staff

Keep in mind what you say, may end up

on a blog or in the paper.

Tools for communication www.freeconferencecall.com Establish a media checklist Establish a Priority system Be as ‘open’ as you can If you’re hacked and had credit card data stolen, it may not be the best time to discuss it DURING the crisis

www.freeconferencecall.com

Establish a media checklist

Establish a Priority system

Be as ‘open’ as you can

If you’re hacked and had credit card data stolen, it may not be the best time to discuss it DURING the crisis

Key Points Be sure you have a plan to communicate Keep in mind nothing is “off the record” Internal/External communications is vital Keeps speculation down

Be sure you have a plan to communicate

Keep in mind nothing is “off the record”

Internal/External communications is vital

Keeps speculation down