Shell Script
Upcoming SlideShare
Loading in...5
×
 

Shell Script

on

  • 530 views

Shell scripts useful for MySQL DBA

Shell scripts useful for MySQL DBA

Statistics

Views

Total Views
530
Views on SlideShare
529
Embed Views
1

Actions

Likes
0
Downloads
10
Comments
0

1 Embed 1

http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Shell Script Shell Script Presentation Transcript

  • Database Backup
    • # cat /usr/backup/backup-dumpdaily
    • mysqldump -uroot -pPassWord DBName --routines --flush-logs --single-transaction --master-data=2 > /backup/dumpmysqldaily/MyApp`date +'%d-%b-%Y-%H-%M'`.sql 2> /backup/dumpmysqldaily/MyApp_err.txt
    • `(ls /backup/dumpmysqldaily/MyApp* -t | head -n 6; ls /backup/dumpmysqldaily/MyApp*)| sort | uniq -u | xargs rm`
  • Ftp to remote location once in a week on every Thursday
    • DISPLAY=:0 notify-send "Backup started and copying on desktop $standby" -t 60000
    • Myfile=”/backup/dumpmysqldaily/MyApp`date +'%d-%b-%Y-%H-%M'`.sql”
    • mydate=`date | awk '{print $1}'`
    • if [[ $mydate = 'Thu' && -s /backup/mysqldump/$myfile ]];then
    • myvar1="SQL data dump file created and today is Wednesday"
    • ftp -in << mytransfer
    • open 110.212.191.62
    • user UserName PassWord
    • bin
    • mput $myfile
    • quit
    • exit 0
    • mytransfer
    • fi
  • Exit Code status
    • if [[ $? -eq 0 ]];then
    • mysql -h110.212.191.62 -uroot -pPassWord -e&quot;insert into backup.report_log values(NULL, now(), '$mydb', '$myvar1', 'success')&quot;
    • else
    • mysql -h110.212.191.62 -uroot -pPassWord -e&quot;insert into backup.report_log values(NULL, now(), '$mydb', '$myvar1', 'fail')&quot;
    • fi
    • DISPLAY=:0 notify-send &quot;Backup process complete!&quot;
  • Disk space shell script
    • #!/bin/sh
    • # Shell script to monitor or watch the disk space
    • # set alert level 60% is default
    • ALERT=60
    • df -HP | grep -vE '^Filesystem|tmpfs|cdrom' | awk '{ print $5 &quot; &quot; $1 }' | while read output;
    • do
    • usep=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
    • partition=$(echo $output | awk '{ print $2 }' )
    • if [ $usep -ge $ALERT ]; then
    • # do something
    • fi
    • done
  • 5 ways to log the messages
    • # echo &quot;Running out of space &quot;$partition ($usep%)&quot; on $(hostname) as on $(date)&quot; >> /home/develop/disk.txt 2>> /home/develop/disk_err.txt
    • # logger &quot;Running out of space &quot;$partition ($usep%)&quot; on $(hostname) as on $(date)&quot;
    • # echo &quot;Running out of space &quot;$partition ($usep%)&quot; on $(hostname) as on $(date)&quot; | mail -s &quot;Alert: Almost out of disk space $usep&quot; $ADMIN
    • # curl --basic --user username:passWord --data status=&quot;Running out of space &quot;$partition ($usep%)&quot; on $(hostname) as on $(date)&quot; http://twitter.com/statuses/update.xml > /home/develop/loadsuccess.txt 2> /home/develop/loaderr.txt
    • # curl -Ld'username=soak&password=43417010&source=oksoft&dmobile=9702977470&message=disk+is+running+out+on+master+server' http://64.120.23.38/smsclient//api.php
  • Watch key stats
    • #!/bin/sh
    • # watch key stats
    • newTicket=$(mysql -uroot -pPassWord DBName -Bse&quot;select count(*) as mycount from tbl_Response where Status='success' AND TxnDate > date_sub(now(), interval 1 hour)&quot;)
    • newCount=$(mysql -uroot -pPassWord DBName -Bse&quot;select count(*) as mycount from tbl_UserMaster where Created_dt > date_sub(now(), interval 1 hour)&quot;)
    • if [[ $newTicket == 0 ]] || [[ $newCount == 0 ]];then
    • echo &quot;some problem with tickets or new registrations'
    • fi
  • Find Orphan Pages
    • #!/bin/sh
    • > /home/develop/orphan1.txt
    • for myfile in `ls -R /var/www/html/Working_latest/ | tr &quot; &quot; &quot; &quot;`
    • do
    • grep -R -m1 $myfile /var/www/html/Working_latest/* > /dev/null 2>&1
    • if [[ $? -ne 0 ]];then
    • find /var/www/html/Working_latest/ -name $myfile >> /home/develop/orphan1.txt
    • fi
    • Done
    • # sort, remove duplicates and remove path
    • # sort -u /home/develop/orphan1.txt | sed 's/^/var/www/html//g'
  • Import files to MySQL
    • #!/bin/sh
    • # I have abc.sql xyz.sql and pqr.sql files
    • # to be imported in mysql
    • for file in `ls /home/develop/report/*.sql | tr &quot; &quot; &quot; &quot;`
    • do
    • mysql -uroot -pPassWord dbName -f < $file 1>/dev/null 2>&1
    • done
  • Copy file to servers Part I
    • #!/bin/sh
    • #copy file to 4 servers
    • while read IP
    • do
    • scp -p /home/newindex.php root@$IP:/var/www/html/ > /dev/null
    • done < myIPlist
    • 111.222.333.444
    • 555.666.777.888
    • 999.111.222.333
    • 444.555.666.777
    • myIPlist
  • Copy file to servers Part II
    • #!/bin/sh
    • #copy file to 4 servers
    • myfile=&quot; /home/newindex.php&quot;
    • destination=&quot;/var/www/html/&quot;
    • while read IP
    • do
    • scp -p $myfile root@$IP:$destination > /dev/null
    • done < /root/myIPlist.txt
    cat /root/myIPlist.txt 111.222.333.444 555.666.777.888 999.111.222.333 444.555.666.777
  • Execute SQL statement
    • while read IP
    • do
    • 0</dev/null ssh root@$IP &quot;mysql -uroot -pPassWord -e'select User, Host from mysql.user;'&quot;
    • done < /root/myIPlist.txt
  • Copy table to remote servers
    • while read DBName IP
    • do
    • mysqldump -uroot -pPassWord test reports | ssh root@$IP &quot;mysql -uroot -pPassWd $DBName&quot;
    • done < myIPlist
    • mumbai_report 111.222.333.444
    • delhi_report 555.666.777.888
    • chennai 999.111.222.333
    • bangalore 444.555.666.777
    • myIPlist
  • Run sql query or dump
    • while read DBName IP
    • do
    • 0</dev/null ssh root@$IP mysql -uroot -pPassWord $DBName < myquery.txt
    • done < /var/www/html/IPlist.txt
  • Search for a word in the database
    • #!/bin/bash
    • > tosave.txt
    • mysql -uroot -pPassWord -Be&quot;show databases&quot; | egrep -v &quot;(backup|Database|information_schema|mysql|test)&quot; | while read mydb
    • do
    • mysqldump -uroot -pPassWord $mydb -d --routines | grep 'machine_problem'
    • if [[ $? -ne 0 ]];then
    • echo &quot;$mydb did not has the word&quot; >> tosave.txt
    • fi
    • done
  • Start server if not already active
    • service ntpd status | grep 'stopped'
    • if [[ $? -eq 0 ]];then
    • service ntpd start
    • fi
  • High Process Alert
    • # count the processes and alert the user if the processes are more than 30
    • myvar2=$(mysqladmin -uroot -pPassWord processlist | wc -l | awk '{print int($1)}');
    • if [[ $myvar2 -gt 30 ]];then
    • echo &quot;The number of processes is too high&quot;
    • fi
  • Check if MySQL is alive
    • # ping mysql service
    • islive=`mysqladmin -uroot -pPassWord ping | grep alive | wc -l`
    • if [[ $islive -ne 1 ]];then
    • echo &quot;Mysql is dead&quot;
    • fi
  • status_success file output
    • mysqladmin -uroot -pPassWord status >> /home/develop/status_success.txt
    • Uptime: 2914371 Threads: 5 Questions: 181438411 Slow queries: 9351 Opens: 124111 Flush tables: 81 Open tables: 150 Queries per second avg: 62.256
    • Uptime: 2914431 Threads: 5 Questions: 181443669 Slow queries: 9355 Opens: 124111 Flush tables: 81 Open tables: 150 Queries per second avg: 62.257
    • Uptime: 2914491 Threads: 5 Questions: 181448454 Slow queries: 9358 Opens: 124115 Flush tables: 81 Open tables: 150 Queries per second avg: 62.257
    • Uptime: 2914551 Threads: 5 Questions: 181452947 Slow queries: 9359 Opens: 124115 Flush tables: 81 Open tables: 150 Queries per second avg: 62.258
    • Uptime: 2914611 Threads: 5 Questions: 181459383 Slow queries: 9359 Opens: 124124 Flush tables: 81 Open tables: 150 Queries per second avg: 62.259
    • Uptime: 2914671 Threads: 7 Questions: 181467155 Slow queries: 9359 Opens: 124129 Flush tables: 81 Open tables: 150 Queries per second avg: 62.260
    • Uptime: 2914731 Threads: 5 Questions: 181481907 Slow queries: 9360 Opens: 124159 Flush tables: 81 Open tables: 149 Queries per second avg: 62.264
    • Uptime: 2914791 Threads: 5 Questions: 181485640 Slow queries: 9360 Opens: 124160 Flush tables: 81 Open tables: 150 Queries per second avg: 62.264
  • Negative uptime – MySQL restarted
    • #$uptime $threads $questions $slow $opens $flush $otables $aqueries
    • uptime=$(tail -2 /home/develop/status_success.txt | awk ' {
    • gsub(/[a-zA-Z: ]+/,&quot; &quot;)
    • m=split($0,a,&quot; &quot;);
    • for (i=1;i<=m;i++)
    • if (NR==1) b[i]=a[i]; else print a[i] - b[i]
    • } ' | head -1)
    • if [[ $uptime -lt 0 ]];then
    • echo &quot;mysql restarted&quot;
    • fi
  • High processes alert
    • #$uptime $threads $questions $slow $opens $flush $otables $aqueries
    • questions=$(tail -2 /home/develop/status_success.txt | awk ' {
    • gsub(/[a-zA-Z: ]+/,&quot; &quot;)
    • m=split($0,a,&quot; &quot;);
    • for (i=1;i<=m;i++)
    • if (NR==1) b[i]=a[i]; else print a[i] - b[i]
    • } ' | head -3 | tail -1)
    • if [[ $questions -gt 20000 ]];then
    • echo &quot;too many queries hitting MySQL every minute&quot;
    • fi
  • Slow queries are too high
    • #$uptime $threads $questions $slow $opens $flush $otables $aqueries
    • slow=$(tail -2 /home/develop/status_success.txt | awk ' {
    • gsub(/[a-zA-Z: ]+/,&quot; &quot;)
    • m=split($0,a,&quot; &quot;);
    • for (i=1;i<=m;i++)
    • if (NR==1) b[i]=a[i]; else print a[i] - b[i]
    • } ' | head -4 | tail -1)
    • if [[ $slow -gt 10 ]];then
    • echo &quot;too many slow queries&quot;
    • fi
  • Email new offers to your customers
    • #!/bin/sh
    • for customerMail in `mysql -uroot -pPassWord dbName -BNe&quot;select emailid from UserMaster&quot;`
    • do
    • mail -s &quot;New discounts offered&quot; $customerMail < mbox.txt -- -f customercare@CompanyName.com -F 'Customer Care'
    • done
  • Slave Check Script
    • #!/bin/sh
    • # Slave Check status
    • mystatus=`mysql -uroot -pPassWord -e&quot;show slave statusG&quot; | grep 'Slave_SQL_Running' | awk '{print $2}'`
    • if [[ $mystatus = 'No' ]];then
    • echo &quot;Slave is not working&quot;
    • else
    • echo &quot;Slave is working OK&quot;
    • fi
  • Change Master position script
    • #!/bin/sh
    • # change the master position after reading error log
    • binfile=`grep &quot;Slave I/O thread exiting, read up to log&quot; /var/log/mysqld.log | tail -1 | awk '{print $12}'`
    • position=`grep &quot;Slave I/O thread exiting, read up to log&quot; /var/log/mysqld.log | tail -1 | awk '{print $14}'`
    • echo &quot;CHANGE MASTER TO MASTER_HOST='192.168.50.31', MASTER_USER='slave_user', MASTER_PASSWORD='slave_user', MASTER_LOG_FILE=$binfile MASTER_LOG_POS=$position;&quot;
  • Show database, tables and count # mysqlshow trigger --count Database: trigger +--------------+----------+------------+ | Tables | Columns | Total Rows | +--------------+----------+------------+ | Response | 9 | 2 | | mytime | 1 | 1 | | todel | 11 | 2 | +--------------+----------+------------+ 3 rows in set. [root@localhost ~]# mysqlshow +--------------------+ | Databases | +--------------------+ | information_schema | | amaravati | | dropme | | lost+found | | mysql | | trigger | +--------------------+ # mysqlshow trigger Database: trigger +--------------+ | Tables | +--------------+ | Response | | mytime | | todel | +--------------+
  • Show database, tables and count II [root@localhost ~]# mysqlshow +--------------------+ | Databases | +--------------------+ | information_schema | | amaravati | | dropme | | lost+found | | mysql | | trigger | +--------------------+ # mysqlshow | awk '{print $2}' Databases information_schema amaravati dropme lost+found mysql trigger # mysqlshow | awk '{print &quot;myslqshow --count&quot;, $2}' myslqshow --count myslqshow --count Databases myslqshow --count myslqshow --count information_schema myslqshow --count amaravati myslqshow --count dropme myslqshow --count lost+found myslqshow --count mysql myslqshow --count trigger myslqshow --count mysqlshow | awk '{print &quot;myslqshow --count&quot;, $2}' | sh
  • Kill runaway processes #!/bin/sh # kill runaway processes SEC=$1 IFS=&quot;|&quot; if [[ $SEC -lt 1 ]]; then echo &quot;Usage: $0 seconds&quot; exit 1 fi mysqladmin proc -v | grep Query | grep -Evi &quot;delete|update|insert|alter table&quot; | while read dummy qid qusr qhost qdb qstat qsec qstat2 query do if [ $qsec -gt $SEC ];then echo &quot;Killing query $qid...&quot; mysqladmin kill $qid fi done
  • Table count in general log
    • for mytable in `mysql -uroot -pPassWord dbName -BNe&quot;show tables&quot;`
    • do
    • mycount=`grep -c &quot;$mytable&quot; /root/mysql-gen.log`
    • echo &quot;$mytable&quot; &quot;$mycount&quot; >> tableReport.txt
    • done
  • Delete files listed in a text file
    • #!/bin/sh
    • # remove files listed in the unwanted.txt file
    • cnt=0
    • while read fileName
    • do
    • rm -f $fileName
    • cnt=$(($cnt+1))
    • done < unwanted.txt
    • echo &quot;total files dropped $cnt&quot;