SlideShare a Scribd company logo
1 of 14
Storing & Restoring Data
                             General Introduction To Simpler Methods



                                            Gowtham

                                       sgowtham@mtu.edu




                                                                       2008.09.04
Tuesday, February 24, 2009
Care?
                     Shoul
                      why


                             Be Paranoid?

                        dI
                   Worry?
            What’s the worst that can happen ?
Tuesday, February 24, 2009
Hard Drive WILL                      Shortage of Time
   NOT live
                                          Schedule

                             YE
    forever
                                          Conflicts
                                               Loss of
      Power                                  Time Wall
      Outage
                                      because CPU
           Stealth             Hardware Failure




                              S
                             Virus Attacks
                                             ACCIDENTS
Hackers
                                        friday 5 pm syndrome
           Natural Disasters
Tuesday, February 24, 2009
Backup Solutions
         Floppy, CD, Flash Drives, DVD
              Economical & Portable
                                         Quality HDs
              Easy to loose / corrupt
                                         ~$0.20 / GB
         Tape Backup
                                         Seagate
              Not all of us can afford
                                         Western Digital
                                         Maxtor
         Online Storage
              Costs (a lot of) $$
                                         Enclosure: ~$20
              Data Peepers
                                         External Drives with
         In-House Methods                Push Button
                                         Backup Solutions
              Takes some effort
              FREE (almost)
Tuesday, February 24, 2009
WHAT, WHERE and WHEN


    FAILING TO
       PLAN
         IS

   PLANNING TO
       FAIL
Tuesday, February 24, 2009
WHAT, WHERE and WHEN
                                                     bin
           /home/$USER
                                                     doc
              Usually contains most of user’s data
                                                     lib
                                                     src
           /usr/local and /opt
                                                     tmp
              Locally installed/compiled programs
                                                     research
                                                     .files
           /var                                      .folders
              Websites
              Mail


         Databases
              MySQL
              PostgreSQL

Tuesday, February 24, 2009
WHAT, WHERE and WHEN
         A Reliable Destination
              Not too old of a machine
              Enough free space
              Preferably same version of OS

                                                How many
                                              copies should
                                                one keep?



Tuesday, February 24, 2009
How
    Secure Copy

    mkdir -p /home/$USER/research/rama
    cd /home/$USER/research/rama
    scp -r rama.phy.mtu.edu:/home/$USER/research/ .


                             What if the source
                              grows by few GB
                              every day & the
                             network is slower?

Tuesday, February 24, 2009
How
    rsync over ssh

    rsync -ave ssh -H rama:/home/$USER/research/ 
     /home/$USER/research/rama/



                             Trailing slashes are
                                    crucial!
                             Can SSH not prompt
                                for password?

Tuesday, February 24, 2009
SSH With No Password
    ssh-keygen -t rsa -b 1024

    scp .ssh/id_rsa.pub rama:id_rsa_ganesh.pub

    ssh $USER@rama

    cat id_ssh_ganesh.pub >> .ssh/authorized_keys


                             SOURCE will NOT ask for
                                 password when
                             connecting from CLIENT
                               ssh, scp, rsync+ssh
Tuesday, February 24, 2009
Backing Up At Odd Hours
       # cron job listing - save this file as
       # /home/$USER/bin/cron_jobs
       #
       # Submit to cron via the command
       # crontab /home/$USER/bin/cron_jobs
       #
       09 02 * * * /home/$USER/bin/rama_backup.sh > /dev/null



                             Command

                             Day of the Week (00-06; 0 = Sunday)

                             Month (01-12)

                             Day of the Month (01-31)

                             Hour (0-23)

                             Minute (0-59)
Tuesday, February 24, 2009
BackupPC
         Open source project
         Good Documentation and Support
         Very elegant and effective
         Clients machines can have any operating system
         Works equally well for desktops and laptops
         Sends emails when a client is not backed up
         Requires a working installation of Apache, PHP, PERL




Tuesday, February 24, 2009
BackupPC



                                                                  Open Source
                                                                  Apache, PERL
                                                                  GUI (TUI)

                                                                  Flexible Scheduling
                                                                  Add / remove clients
                                                                  Linux, Mac, Win
                                                                  Email Alerts
                                                                  Effective Disk Usage
           http://sgowtham.net/blog/archives/
                                                                  Initiate backups
               Search for ‘BackupPC’                              Restore files/folders
               Steps to configure Red Hat Enterprise Linux as server
Tuesday, February 24, 2009
Acknowledgments
         Dr. Ravi Pandey

         Center for Experimental Computation @ MTU

         Tim Obermann

         Linux Users’ Group @ MTU




Tuesday, February 24, 2009

More Related Content

Similar to Back Up 101

Linux Performance Analysis and Tools
Linux Performance Analysis and ToolsLinux Performance Analysis and Tools
Linux Performance Analysis and ToolsBrendan Gregg
 
ACN South Florida Chapter Meeting 37 Preso
ACN South Florida Chapter Meeting 37 PresoACN South Florida Chapter Meeting 37 Preso
ACN South Florida Chapter Meeting 37 Presojsazout
 
Protecting TYPO3 With Suhosin And Modsecurity
Protecting TYPO3 With Suhosin And ModsecurityProtecting TYPO3 With Suhosin And Modsecurity
Protecting TYPO3 With Suhosin And ModsecurityXavier Perseguers
 
intl me this, intl me that
intl me this, intl me thatintl me this, intl me that
intl me this, intl me thatAndrei Zmievski
 
Using filesystem capabilities with rsync
Using filesystem capabilities with rsyncUsing filesystem capabilities with rsync
Using filesystem capabilities with rsyncHazel Smith
 
Writing malware while the blue team is staring at you
Writing malware while the blue team is staring at youWriting malware while the blue team is staring at you
Writing malware while the blue team is staring at youRob Fuller
 
Farms, Fabrics and Clouds
Farms, Fabrics and CloudsFarms, Fabrics and Clouds
Farms, Fabrics and CloudsSteve Loughran
 
Advanced Level Training on Koha / TLS (ToT)
Advanced Level Training on Koha / TLS (ToT)Advanced Level Training on Koha / TLS (ToT)
Advanced Level Training on Koha / TLS (ToT)Ata Rehman
 
Tech comm presentation 2014 01-16
Tech comm presentation 2014 01-16Tech comm presentation 2014 01-16
Tech comm presentation 2014 01-16Ken Tucker
 
Online backup with netwise hosting
Online backup with netwise hostingOnline backup with netwise hosting
Online backup with netwise hostingNetwise Hosting
 
Accelerate Your Rails Site with Automatic Generation-Based Action Caching
Accelerate Your Rails Site with Automatic Generation-Based Action CachingAccelerate Your Rails Site with Automatic Generation-Based Action Caching
Accelerate Your Rails Site with Automatic Generation-Based Action Cachingelliando dias
 
Cloudify - Scalability On Demand
Cloudify - Scalability On DemandCloudify - Scalability On Demand
Cloudify - Scalability On DemandFederico Feroldi
 
Computer_Troubleshooting.ppt
Computer_Troubleshooting.pptComputer_Troubleshooting.ppt
Computer_Troubleshooting.pptneil448324
 
App cap2956v2-121001194956-phpapp01 (1)
App cap2956v2-121001194956-phpapp01 (1)App cap2956v2-121001194956-phpapp01 (1)
App cap2956v2-121001194956-phpapp01 (1)outstanding59
 

Similar to Back Up 101 (20)

Performance Strategies
Performance StrategiesPerformance Strategies
Performance Strategies
 
Linux Performance Analysis and Tools
Linux Performance Analysis and ToolsLinux Performance Analysis and Tools
Linux Performance Analysis and Tools
 
ACN South Florida Chapter Meeting 37 Preso
ACN South Florida Chapter Meeting 37 PresoACN South Florida Chapter Meeting 37 Preso
ACN South Florida Chapter Meeting 37 Preso
 
Protecting TYPO3 With Suhosin And Modsecurity
Protecting TYPO3 With Suhosin And ModsecurityProtecting TYPO3 With Suhosin And Modsecurity
Protecting TYPO3 With Suhosin And Modsecurity
 
intl me this, intl me that
intl me this, intl me thatintl me this, intl me that
intl me this, intl me that
 
Succor's MyMedCloud& MyMedBackup
Succor's MyMedCloud& MyMedBackupSuccor's MyMedCloud& MyMedBackup
Succor's MyMedCloud& MyMedBackup
 
Using filesystem capabilities with rsync
Using filesystem capabilities with rsyncUsing filesystem capabilities with rsync
Using filesystem capabilities with rsync
 
Writing malware while the blue team is staring at you
Writing malware while the blue team is staring at youWriting malware while the blue team is staring at you
Writing malware while the blue team is staring at you
 
Epidemic Failures
Epidemic FailuresEpidemic Failures
Epidemic Failures
 
Farms, Fabrics and Clouds
Farms, Fabrics and CloudsFarms, Fabrics and Clouds
Farms, Fabrics and Clouds
 
Advanced Level Training on Koha / TLS (ToT)
Advanced Level Training on Koha / TLS (ToT)Advanced Level Training on Koha / TLS (ToT)
Advanced Level Training on Koha / TLS (ToT)
 
Debian on EC2
Debian on EC2Debian on EC2
Debian on EC2
 
Tech comm presentation 2014 01-16
Tech comm presentation 2014 01-16Tech comm presentation 2014 01-16
Tech comm presentation 2014 01-16
 
Online backup with netwise hosting
Online backup with netwise hostingOnline backup with netwise hosting
Online backup with netwise hosting
 
Accelerate Your Rails Site with Automatic Generation-Based Action Caching
Accelerate Your Rails Site with Automatic Generation-Based Action CachingAccelerate Your Rails Site with Automatic Generation-Based Action Caching
Accelerate Your Rails Site with Automatic Generation-Based Action Caching
 
MySQL Sandbox 3
MySQL Sandbox 3MySQL Sandbox 3
MySQL Sandbox 3
 
How to compile a kernel suse 2
How to compile a kernel   suse 2How to compile a kernel   suse 2
How to compile a kernel suse 2
 
Cloudify - Scalability On Demand
Cloudify - Scalability On DemandCloudify - Scalability On Demand
Cloudify - Scalability On Demand
 
Computer_Troubleshooting.ppt
Computer_Troubleshooting.pptComputer_Troubleshooting.ppt
Computer_Troubleshooting.ppt
 
App cap2956v2-121001194956-phpapp01 (1)
App cap2956v2-121001194956-phpapp01 (1)App cap2956v2-121001194956-phpapp01 (1)
App cap2956v2-121001194956-phpapp01 (1)
 

Recently uploaded

Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDGMarianaLemus7
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsAndrey Dotsenko
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Neo4j
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024BookNet Canada
 
Bluetooth Controlled Car with Arduino.pdf
Bluetooth Controlled Car with Arduino.pdfBluetooth Controlled Car with Arduino.pdf
Bluetooth Controlled Car with Arduino.pdfngoud9212
 
Unlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsUnlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsPrecisely
 
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsHyundai Motor Group
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 

Recently uploaded (20)

Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping ElbowsPigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
 
APIForce Zurich 5 April Automation LPDG
APIForce Zurich 5 April  Automation LPDGAPIForce Zurich 5 April  Automation LPDG
APIForce Zurich 5 April Automation LPDG
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024Build your next Gen AI Breakthrough - April 2024
Build your next Gen AI Breakthrough - April 2024
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
New from BookNet Canada for 2024: BNC BiblioShare - Tech Forum 2024
 
Bluetooth Controlled Car with Arduino.pdf
Bluetooth Controlled Car with Arduino.pdfBluetooth Controlled Car with Arduino.pdf
Bluetooth Controlled Car with Arduino.pdf
 
Unlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsUnlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power Systems
 
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter RoadsSnow Chain-Integrated Tire for a Safe Drive on Winter Roads
Snow Chain-Integrated Tire for a Safe Drive on Winter Roads
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
The transition to renewables in India.pdf
The transition to renewables in India.pdfThe transition to renewables in India.pdf
The transition to renewables in India.pdf
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 

Back Up 101

  • 1. Storing & Restoring Data General Introduction To Simpler Methods Gowtham sgowtham@mtu.edu 2008.09.04 Tuesday, February 24, 2009
  • 2. Care? Shoul why Be Paranoid? dI Worry? What’s the worst that can happen ? Tuesday, February 24, 2009
  • 3. Hard Drive WILL Shortage of Time NOT live Schedule YE forever Conflicts Loss of Power Time Wall Outage because CPU Stealth Hardware Failure S Virus Attacks ACCIDENTS Hackers friday 5 pm syndrome Natural Disasters Tuesday, February 24, 2009
  • 4. Backup Solutions Floppy, CD, Flash Drives, DVD Economical & Portable Quality HDs Easy to loose / corrupt ~$0.20 / GB Tape Backup Seagate Not all of us can afford Western Digital Maxtor Online Storage Costs (a lot of) $$ Enclosure: ~$20 Data Peepers External Drives with In-House Methods Push Button Backup Solutions Takes some effort FREE (almost) Tuesday, February 24, 2009
  • 5. WHAT, WHERE and WHEN FAILING TO PLAN IS PLANNING TO FAIL Tuesday, February 24, 2009
  • 6. WHAT, WHERE and WHEN bin /home/$USER doc Usually contains most of user’s data lib src /usr/local and /opt tmp Locally installed/compiled programs research .files /var .folders Websites Mail Databases MySQL PostgreSQL Tuesday, February 24, 2009
  • 7. WHAT, WHERE and WHEN A Reliable Destination Not too old of a machine Enough free space Preferably same version of OS How many copies should one keep? Tuesday, February 24, 2009
  • 8. How Secure Copy mkdir -p /home/$USER/research/rama cd /home/$USER/research/rama scp -r rama.phy.mtu.edu:/home/$USER/research/ . What if the source grows by few GB every day & the network is slower? Tuesday, February 24, 2009
  • 9. How rsync over ssh rsync -ave ssh -H rama:/home/$USER/research/ /home/$USER/research/rama/ Trailing slashes are crucial! Can SSH not prompt for password? Tuesday, February 24, 2009
  • 10. SSH With No Password ssh-keygen -t rsa -b 1024 scp .ssh/id_rsa.pub rama:id_rsa_ganesh.pub ssh $USER@rama cat id_ssh_ganesh.pub >> .ssh/authorized_keys SOURCE will NOT ask for password when connecting from CLIENT ssh, scp, rsync+ssh Tuesday, February 24, 2009
  • 11. Backing Up At Odd Hours # cron job listing - save this file as # /home/$USER/bin/cron_jobs # # Submit to cron via the command # crontab /home/$USER/bin/cron_jobs # 09 02 * * * /home/$USER/bin/rama_backup.sh > /dev/null Command Day of the Week (00-06; 0 = Sunday) Month (01-12) Day of the Month (01-31) Hour (0-23) Minute (0-59) Tuesday, February 24, 2009
  • 12. BackupPC Open source project Good Documentation and Support Very elegant and effective Clients machines can have any operating system Works equally well for desktops and laptops Sends emails when a client is not backed up Requires a working installation of Apache, PHP, PERL Tuesday, February 24, 2009
  • 13. BackupPC Open Source Apache, PERL GUI (TUI) Flexible Scheduling Add / remove clients Linux, Mac, Win Email Alerts Effective Disk Usage http://sgowtham.net/blog/archives/ Initiate backups Search for ‘BackupPC’ Restore files/folders Steps to configure Red Hat Enterprise Linux as server Tuesday, February 24, 2009
  • 14. Acknowledgments Dr. Ravi Pandey Center for Experimental Computation @ MTU Tim Obermann Linux Users’ Group @ MTU Tuesday, February 24, 2009

Editor's Notes