• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
L hammonds adventure
 

L hammonds adventure

on

  • 585 views

 

Statistics

Views

Total Views
585
Views on SlideShare
585
Embed Views
0

Actions

Likes
0
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft Word

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    L hammonds adventure L hammonds adventure Document Transcript

    • Posts: 138 LHammonds Special Member My Notes for Installing Zimbra OSE on Ubuntu Server 10.04 LTS Greetings and salutations, I hope this thread will be helpful to those who follow in my foot steps as well as getting any advice based on what I have done / documented. I am a documentation freak so this will probably grow into a very LARGE step-by-step process. If useful, it will probably be best to convert to a wiki once completed. This is a Work-In-Progress topic so I will be updating this thread as I go through this transition from MS Exchange to Zimbra. High-level overview This thread will cover installation of Ubuntu server inside a virtual machine vSphere 4.1 running on ESXi 4.1 servers. Although there is some VMware-specific steps, they are very few and the majority of this documentation will work for other VMs such as VirtualBox or even directly installed onto a physical machine. It will also cover the installation and configuration of Zimbra OSE and the Zimbra desktop client. This is a merge of several other guides, comments and my own experience of putting it all together in one place in a format that is easy for me to follow and reproduce the same results (yes, I create documentation that I will have to re-read one day because of my bad memory problems!). Please excuse my ignorance since I am fairly new to Linux and Zimbra...if you have any advice on doing things better, please let me know. I love feedback and learning better ways of doing things! Company background and environment I have finished evaluating different mail options for my company and we decided to make the switch to Zimbra. We are a small shop of about 200 employees and are currently using Microsoft Exchange 2003 on a physical server. We are going to switch from Exchange to Zimbra. We are not fully utilizing Exchange features and are mainly using it for email, contacts, calendars and shared calendars (for scheduling meeting rooms). We use Active Directory (Windows 2003 domain) to manage users and distribution lists. The desktops are mainly Windows XP and some newer boxes are coming in with Windows 7. We are using Microsoft Outlook with personal folders since we have mailbox size restrictions of 200 MB per mailbox as well as the web interface for people that constantly move from desk-to-desk. iPhones and iPads are also being used as portable devices. The email server will be sitting behind a firewall on our LAN. Here is a graphic representation of the network layout:
    • About the IT staff We are primarily a Microsoft shop and as such, most of us (3) have Microsoft certifications such as MCSE, MCDBA and MCP. None of us are Unix administrators but we are fairly competent in the use of the system from having to support an AIX server so we are comfortable in the VI editor but know very little in terms of Unix hardware and software administration (just basic user stuff, enough to be dangerous) The deployment process
    • Here are the high-level milestones for this project. Phase 1 - Setup a fully-functional system for testing, training and migration. Phase 2 - Create Backup / Restore process and documentation Phase 3 - Install client software on all devices Phase 4 - Create and deploy training materials (web docs / videos) Phase 5 - Migrate users (over a weekend) Phase 6 - Decommission old system (with a flamethrower) Tools utilized in this process Ubuntu Server 10.04 LTS, 64-bit Zimbra OSE 7.2.0, 64-bit Zimbra Desktop Client 7.1.4 Portable PuTTY 0.62 VMware vSphere 4.1.0 Helpful links The list below are sources of information that helped me configure this system as well as some places that might be helpful to me later on as this process continues. Ubuntu Documentation Zimbra Network Edition Documentation (PDF files) Zimbra Open Source Edition Documentation (PDF files) Zimbra Desktop User Guide Ubuntu Firewall Basics Firewall Configuration for Zimbra Ubuntu AppArmor Basics Ubuntu 8 and Zimbra Install Guide Making Zimbra and BIND Work Together Split DNS Guide Z-Push (Over-The-Air ActiveSync) Funambol (Over-The-Air iPhone contact sync) djigzo (email encryption @ the gateway) Mondo Rescue (Ubuntu OS backup) MailArchiva (email Journaling and archival) TurnKeyLinux Zimbra Appliance (fast-n-easy to get running even if you are 100% unfamiliar with Linux/Zimbra) ZeXtras Migration Tool (upgrade OS or change from dedicated to multi-domain server) Location of Zimbra Log Files Zimbra Online Quote Tool
    • Assumptions This documentation will need to make use of some very-specific information that will most-likely be different for each person / location. And as such, I will note some of these in this section. They will be highlighted in red throughout the document as a reminder that you should plug-in your own value rather than actually using my "place-holder" value. Under no circumstance should you use that actual values I list below. They are placeholders for the real thing. This is just a checklist template you need to have answered before you start the install process. Wherever you see RED in this document, you need to substitute it for what your company uses. Use the list below as a template you need to have answered before you continue. Ubuntu Server name: mail Internet domain: mydomain.com Server domain alias (Windows): work Server fully-qualified windows domain: work.mydomain.com Ubuntu Server IP address: 192.168.107.25 (my attempt at geek humor...raise your hand if you get it) Ubuntu Server IP subnet mask: 255.255.255.0 Ubuntu Server IP gateway: 192.168.107.1 Internal DNS Server 1: 192.168.107.23 Internal DNS Server 2: 192.168.107.212 Internal Active Directory: 192.168.107.212 External ISP DNS Server 1: 200.100.10.1 External ISP DNS Server 2: 200.100.10.2 External IP to Domain: 198.100.100.100 Ubuntu Admin ID: administrator Ubuntu Admin Password: myadminpass Ubuntu Root Password: myrootpass Zimbra Admin Password: myzimbrapass I also assume the reader knows how to use the VI editor. If not, you will need to beef up your skill set or use a different editor in place of it. __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) Last edited by LHammonds; 05-07-2012 at 03:03 PM.. #2 (permalink) 04-26-2012, 07:07 PM LHammonds Special Member Posts: 138
    • Analysis and Design Analysis and Design The user base will probably not exceed 300 any time soon based on the company size over the last 80 years. A single server design is chosen for this deployement based on the amount of current users (~200) The Ubuntu Server Long-Term Support (LTS) is free but we have the option of buy support and that is the main reason this server was selected. Zimbra OSE is also free but we have the option of buying support by upgrading to the Network Edition. The Ubuntu LTS / Zimbra OSE combination makes for an excellent choice to evaluate the products (without incurring licensing fees) as well as having an upgrade path to obtain support. Much of how the system will run is based on how you install / configure Ubuntu before you ever install Zimbra. The largest decision over the configuration of Ubuntu seems to be how the hard drive space is sliced up (partitioned). I lot of my time was spent researching and learning the various options when it comes to the design layout of the storage. The Zimbra documentation's recommendation for Ubuntu in this area basically says to put everything on one LVM/Volume. I can see how tech support would want to favor a setup like this because it makes it easier to troubleshoot. However, I would rather make use of a design that allows for dynamic growth and fine-tuning if need be. I also do not like being caught offguard with a scenario where space is filled up with no immediate option other than deleting files. Long-term life and growth of the system as well as budgeting concerns have to be taken into consideration. Based on my observations during the first Zimbra install, I decided that a single volume wasn't a good idea, especially having continual-growth of a folder that is on the root volume. I would rather contain the root volume to mainly just static data that won't grow much and push the other folders that are dynamic into their own volumes. In case you don't know, filling up the root volume on a *nix system is a very bad thing and should be avoided at all costs. I also want to configure it where the file systems are not taking up 100% of the logical volume. This will allow me (through automated scripts) to grow the file system as needed and give me some time to add more drives if necessary or shrink other volumes to get more space. Here are my thoughts on how to slice up the space. Boot - This will remain static in size. It is also the only space residing outside the Logical Volume Manager (LVM)
    • Root volume - Operating system and everything else which should remain fairly static. Swap volume - This will remain static in size. However, if amount of RAM is adjusted, this should be adjusted as well. Opt volume - This Zimbra storage will continue to grow over time. Backup volume - This will contain a local backup of the Opt volume and individually compressed mailbox archives. So space needs to be around double /opt/zimbra. Temp volume - This location will be used to hold data during backups, archives and restores and should be about double the size of opt/zimbra. Offsite Storage - This will be handled elsewhere but will be mounted on this server. My estimated mailbox store will be about 25 GB. However, we will start this server off with a much lower number. We will then pretend some event happened such as acquiring an additional company / employees. This will increase our storage needs and we will go through adjusting the entire system to accommodate a larger storage capacity. This will be a good exercise / documentation for what will eventually need to be done later to handle growth. To get a good idea of the hard drive layout and to understand the process better, here is a graphical representation of the initial design for the server:
    • These numbers will be used for the initial build of the system: boot = 200 MB root = 4 GB swap = 8 GB opt = 10 GB temp = 20 GB
    • backup = 20 GB Virtual Machine Settings Virtual Manager: VMware vSphere Client 4.1 Virtual Host: VMware ESXi Server 4.1 Configuration: Custom Name: ZimbraMail Datastore: MainOffice-LUN0 Virtual Machine Version: 7 Guest Operating System: Linux, Version: Ubuntu Linux (64-bit) Number of virtual processors: 1 Memory Size: 2048 MB Number of NICs: 1 NIC 1: VM Network Adapter: E1000, Connect at Power On: Checked SCSI controller: LSI Logic Parallel Select a Disk: Create a new virtual disk Create a Disk: 63 GB, No thin provisioning, No cluster features, Store with the virtual machine Advanced Options: Virtual Device Node = SCSI (0:0) Remove Floppy Drive Mount CD/DVD Drive to Ubuntu ISO (ubuntu-10.04.4-server-amd64.iso). Make sure CD/DVD is set to Connect at power on Set boot options to Force BIOS Setup so you can set CDROM to boot before the Hard Disk Install PuTTY The 1st time I installed Ubuntu and Zimbra, I used the console for everything but it was painfully slow to view man (manual) pages and navigating in VI (text editor). However, I found out that PuTTY is a far better solution for your Ubuntu console because it handles the screen draws much faster and allows for scrolling and copying text. Download the portable edition and run the install...except it doesn't really "install" like a normal program, it simply extracts to a specified folder and will run from that folder even if you put it on a USB stick and carry over to a new computer (requires no install to run and thus leaves a very small footprint on your system) 1. Start PuTTY 2. Type the following and click the Save button: Host Name: mail (or the IP such as 192.168.107.25) Port: 22 Connection type: SSH Saved Sessions: zimbra 3. Now all you have to do is double-click on the zimbra session and it will connect to
    • your server (when online). __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #3 (permalink) 04-26-2012, 07:14 PM LHammonds Special Member Posts: 138 Install Ubuntu Server Install Ubuntu Server NOTE: During the setup process, we are going to use the root account to make things easier. Later on, it will be best to use the "sudo" command to only temporarily grant root access rather than logging in with root access. 1. Power on the Virtual Machine (VM) 2. Press {ENTER} to accept English 3. Select Install Ubuntu Server {ENTER} 4. Press {ENTER} to accept English 5. Press {ENTER} to accept United States 6. Select No to not detect keyboard layout 7. Press {ENTER} to accept USA 8. Press {ENTER} to accept USA 9. Type mail {ENTER} (this is your hostname) 10. Press {ENTER} to accept detected time zone (America/Chicago) 11. Select Manual {ENTER} 12. Select SCSI3 (0,0,0) (sda) - 67.6 GB VMware Virtual disk {ENTER} 13. Select Yes to create new empty partition table, {ENTER} 14. Select pri/log 67.6 GB FREE SPACE {ENTER} 15. Select Create a new partition {ENTER} 16. Type 200 MB, {ENTER} (NOTE: This will be the /boot partition) 17. Select Primary {ENTER} 18. Select Beginning {ENTER} 19. Select Use as: Ext4 journaling file system {ENTER} 20. Select Ext2 file system {ENTER} 21. Select Mount point: / {ENTER} 22. Select /boot - static files of the boot loader {ENTER} 23. Select Bootable flag: off {ENTER} (NOTE: This toggles it on) 24. Select Done setting up the partition {ENTER} 25. Select Configure the Logical Volume Manager {ENTER} 26. Select Yes to write change to disks and configure LVM, {ENTER} 27. Select Create volume group {ENTER} 28. Type LVG {ENTER}
    • 29. Select /dev/sda free #1 (67445; FREE SPACE), {SPACEBAR}, {ENTER} 30. Select Yes to write change to disks and configure LVM, {ENTER} 31. Select Create logical volume {ENTER} 32. Select LVG (67440MB) {ENTER} 33. Type swap {ENTER} 34. Type 8G {ENTER} (NOTE: This is double the amount of RAM) 35. Select Create logical volume {ENTER} 36. Select LVG (59441MB) {ENTER} 37. Type bak {ENTER} 38. Type 20G {ENTER} 39. Select Create logical volume {ENTER} 40. Select LVG (39443MB) {ENTER} 41. Type opt {ENTER} 42. Type 10G {ENTER} 43. Select Create logical volume {ENTER} 44. Select LVG (29444MB) {ENTER} 45. Type temp {ENTER} 46. Type 20G {ENTER} 47. Select Create logical volume {ENTER} 48. Select LVG (9445MB) {ENTER} 49. Type root {ENTER} 50. Type 9445MB {ENTER} (we are letting it take what is left over) 51. Select Finish {ENTER} 52. Select #1 8.0 GB directly under LVM VG LVG, LV swap, {ENTER} 53. Select Use as: do not use {ENTER} 54. Select swap area {ENTER} 55. Select Done setting up the partition {ENTER} 56. Select #1 9.4 GB directly under LVM VG LVG, LV root, {ENTER} 57. Select Use as: do not use {ENTER} 58. Select Ext4 journaling file system {ENTER} 59. Select Mount point: none {ENTER} 60. Select / - the root file system {ENTER} 61. Select Done setting up the partition {ENTER} 62. Select #1 20.0 GB directly under LVM VG LVG, LV bak, {ENTER} 63. Select Use as: do not use {ENTER} 64. Select Ext4 journaling file system {ENTER} 65. Select Mount point: none {ENTER} 66. Select Enter manually {ENTER} 67. Type /var/backup {ENTER} 68. Select Label: none {ENTER} 69. Type backup {ENTER} 70. Select Done setting up the partition {ENTER} 71. Select #1 10.0 GB directly under LVM VG LVG, LV opt, {ENTER} 72. Select Use as: do not use {ENTER} 73. Select Ext4 journaling file system {ENTER} 74. Select Mount point: none {ENTER} 75. Select /opt - add-on application software packages {ENTER} 76. Select Label: none {ENTER} 77. Type zimbra {ENTER}
    • 78. Select Done setting up the partition {ENTER} 79. Select #1 20.0 GB directly under LVM VG LVG, LV temp, {ENTER} 80. Select Use as: do not use {ENTER} 81. Select Ext4 journaling file system {ENTER} 82. Select Mount point: none {ENTER} 83. Select Enter manually {ENTER} 84. Type /var/temp {ENTER} 85. Select Label: temp {ENTER} 86. Select Done setting up the partition {ENTER} 87. Here is what the screen looks like at this point: Partitions 88. Select Finish partitioning and write changes to disk {ENTER} 89. Select Yes to write changes to disk, {ENTER} 90. Type Administrator, {ENTER} for the full name (NOTE: Do not use "zimbra" as a choice here) 91. Press {ENTER} to accept the default of the lowercase name of administrator 92. Type myadminpass, {ENTER}, myadminpass, {ENTER} 93. Select No, {ENTER} to not encrypt your home directory 94. Press {ENTER} to accept a blank line for the HTTP proxy 95. Select Install security updates automatically, {ENTER} 96. Highlight only OpenSSH server and press {SPACEBAR} to enable, {ENTER} to continue. NOTE: This allows us to use PuTTY after installation to connect to the server. 97. Select Yes, {ENTER} to install GRUB boot loader to the master boot record 98. Installation Complete - from the VM menu, select VM --> Edit Settings and select CD/DVD Drive 1 and change to "Client Device" which will effectively remove the ISO. Now press {ENTER} to reboot. Initial Configurations 1. At the console login prompt, login with your administrator account (administrator / myadminpass) 2. At the $ prompt, type sudo passwd root {ENTER}, type your administrator password (myadminpass), it will ask for a new password, type myrootpass and verify. NOTE: This enables the root account. 3. At the $ prompt, change to the super user root account by typing su {ENTER} and then provide the new root password you just configured. 4. Type vi /etc/network/interfaces {ENTER} and change the following: (We need to change the network interface card (NIC) from using DHCP to a static IP) From: Code: iface eth0 inet dhcp To:
    • Code: iface eth0 inet static address 192.168.107.25 netmask 255.255.255.0 gateway 192.168.107.1 network 192.168.107.0 broadcast 192.168.107.255 NOTE #1: You may need to manually remove the DHCP record (lease) associated to this Ubuntu server from your DHCP server so the correct IP can be found by other machines on the network. This can be avoided by temporarily configuring the VM Network Adapter connection to be "Host Only Network" instead of "VM Network" so the server is isolated during setup...at least until you reach the testing of the static IP below. NOTE #2: You might also need to manually add a HOST(A) record to your Windows DNS server (for mail.mydomain.com and mail.work.mydomain.com) 5. Restart the network by typing /etc/init.d/networking restart 6. Sanity check! Type ifconfig and make sure the settings are correct. Then type ping Google or similar and see if ping works. 7. Shutdown and power off the server by typing shutdown -P now {ENTER} 8. At this point forward, you can use PuTTY to access the console rather than the console itself for better performance, ability to scroll, etc. 9. In VM menu, select VM --> Snapshot --> Take Snapshot. Give it a name like STEP 1 and description of Ubuntu Server 10.04.4 LTS, clean install, Static IP: 192.168.107.25 and click OK __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #4 (permalink) 04-26-2012, 07:22 PM LHammonds Special Member Posts: 138 Volume / Disk Management Volume / Disk Management At this point, let's assume something has changed and we need additional storage (e.g. acquired new company, added more people, etc.) For this scenario, we are expecting storage space to require 25 GB for all user mailboxes. If
    • they need 25 GB for storage, let's double that amount for the actual size. Here is the planned adjustments for each file system: opt = 50 GB (double the expected/actual mailbox storage) temp = 100 GB (double the opt size) backup = 100 GB (double the opt size) In the analysis and design section, we wanted to have some file systems smaller than the logical volume on which they sit. This will allow us to allocate additional space when needed. However, when we created the volumes, Ubuntu automatically expanded the file systems to the maximum size of the volume. Normally, this is OK...but we want a system that will allow growth when needed and ensure that we will have time to add additional hard drives BEFORE they are needed which will keep us from being stuck between a rock and a hard place! The design calls for the backup, opt and temp file systems to be slightly smaller than the maximum space available for the logical volume. So, let's make the logical volumes an extra 25 GB larger. This means that we will need an additional 275 GB of storage {(75 + 125 + 125 ) - 50}. If we add two drives that are 140 GB each, that will be just enough to cover our needs. (NOTE: This was an arbitrary number because I wanted to show you how to add 2 drives to the system) If we were to type df -h right now, we should see something like this: Code: Filesystem /dev/mapper/LVG-root none none none none none /dev/mapper/LVG-bak /dev/mapper/LVG-opt /dev/mapper/LVG-temp /dev/sda1 Size 8.7G 1000M 1005M 1005M 1005M 1005M 19G 9.2G 19G 179M Used 824M 184K 0 32K 0 0 172M 149M 172M 17M Avail Use% Mounted on 7.5G 10% / 1000M 1% /dev 1005M 0% /dev/shm 1005M 1% /var/run 1005M 0% /var/lock 1005M 0% /lib/init/rw 18G 1% /var/backup 8.6G 2% /opt 18G 1% /var/temp 152M 10% /boot Since I am running VMware, adding additional space is a snap. However, I will add it in such a way that Ubuntu will see 2 drives added to the system just as if we were to add 2 physical drives to the server. 1. Shutdown and power off the server by typing shutdown -P now {ENTER} 2. In the vSphere client, right-click the Virtual Machine and choose Edit Settings. 3. On the hardware tab, click the Add button and select Hard Disk. Click Next, choose
    • "Create a new virtual disk", click Next, set the size to 140 GB, click Next, Next, Finish. 4. Add another 140 GB disk using the same steps above and click OK to close the settings and allow VMware to process the changes. Collect information about the newly added drives. 1. Start the Ubuntu server and connect using PuTTY. 2. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 3. Type pvdisplay which should show something similar to this: Code: PV Name VG Name PV Size Allocatable PE Size Total PE Free PE Allocated PE PV UUID /dev/sda5 LVG 62.81 GiB / not usable 3.00 MiB yes (but full) 4.00 MiB 16079 0 16079 XjUHyI-kloS-kicz-jLdd-E01t-7RYP-K3YqvM The important bits of info here are the PV Name and VG Name for our existing configuration. 4. Type fdisk -l which should show something similar to this (however I abbreviated it to show just the important parts): Code: Disk /dev/sda: 67.6 GB, 67645734912 bytes Device Boot Start End Blocks Id /dev/sda1 * 1 25 194560 83 Partition 1 does not end on cylinder boundary. /dev/sda2 25 8224 65862657 5 /dev/sda5 25 8224 65862656 8e Disk /dev/sdb: 150.3 GB, 150323855360 bytes Disk /dev/sdb doesn't contain a valid partition table Disk /dev/sdc: 150.3 GB, 150323855360 bytes Disk /dev/sdc doesn't contain a valid partition table System Linux Extended Linux LVM The important bits of info here are the device paths for the new drives which are
    • highlighted in red. Prepare the first drive (/dev/sdb) to be used by the LVM Type the following: Code: fdisk /dev/sdb n (Create New Partition) p (Primary Partition) 1 (Partition Number) {ENTER} (use default for first cylinder) {ENTER} (use default for last cylinder) t (Change partition type) 8e (Set to Linux LVM) p (Preview how the drive will look) w (Write changes) Prepare the second drive (/dev/sdc) to be used by the LVM Do the exact same steps as above but start with fdisk /dev/sdc Create physical volumes using the new drives If we type fdisk -l, we now see /dev/sdb1 and /dev/sdc1 which are Linux LVM partitions. Type the following to create physical volumes: Code: pvcreate /dev/sdb1 pvcreate /dev/sdc1 Now add the physical volumes to the volume group (LVG) by typing the following: Code: vgextend LVG /dev/sdb1 vgextend LVG /dev/sdc1 Now that the space of both drives have been added to the logical volume group called LVG, we can now allocate that space to grow the logical volume. To get a list of volume paths to use in the next commands, type lvscan to show your current volumes and their sizes. Type the following to grow each volume by a specified amount (the number after the plus sign):
    • Code: lvextend -L+65G /dev/LVG/opt lvextend -L+105G /dev/LVG/bak lvextend -L+105G /dev/LVG/temp or you can specify the exact size by excluding the plus sign and specifying the end-result size you want: Code: lvextend -L75G /dev/LVG/opt lvextend -L125G /dev/LVG/bak lvextend -L125G /dev/LVG/temp To see the new sizes, type lvscan The last thing to do now is the actual growth of the file systems. We want to grow the existing file systems but only to a certain amount so we do not take up all the space in the volume...we want room to grow in the future so we have time to order and install new drives when needed. Code: resize2fs /dev/LVG/opt 50G resize2fs /dev/LVG/bak 100G resize2fs /dev/LVG/temp 100G If we need to increase space in /opt at a later point, we can issue the following command (without any downtime): Code: resize2fs /dev/LVG/opt 51G We could continue to increase this particular file system all the way until we reach the limit of the volume which is 75 GB at the moment. Remember, df -h will tell you the size of the file system and lvscan will tell you the size of the volumes where the file systems live in. If we were to type df -h right now, we should see something like this: Code: Filesystem /dev/mapper/LVG-root none none none none Size 8.7G 1000M 1005M 1005M 1005M Used 824M 200K 0 32K 0 Avail Use% Mounted on 7.5G 10% / 1000M 1% /dev 1005M 0% /dev/shm 1005M 1% /var/run 1005M 0% /var/lock
    • none 1005M /dev/mapper/LVG-bak 99G /dev/mapper/LVG-opt 50G /dev/mapper/LVG-temp 99G /dev/sda1 179M 0 1005M 188M 94G 159M 47G 188M 94G 17M 152M 0% 1% 1% 1% 10% /lib/init/rw /var/backup /opt /var/temp /boot EDIT (2012-05-03): In favor of manage by exception, I wrote a script that can be scheduled to run daily to check the file systems to see if they are getting close to filling up and will automatically expand them a little bit and give you an email notice. Everything is done on the gigabyte level. If you do not want the script to perform the increase, simply add a pound sign in front of the resize2fs command on line 64 to comment it out. Might also want to modify the log and email messages so it does not look like it actually "performed" the resize but instead is telling YOU how to perform the resize. Here are the lines I added to my crontab schedule which will check each file system I expect will grow on a daily basis @ 1am, 2am and 3am. I have them checking to see if we have less than 5GB available and if so, it will try to increase by 1GB: Code: 0 1 * * * /var/scripts/prod/check-storage.sh opt 5 1 > /dev/null 2>&1 0 2 * * * /var/scripts/prod/check-storage.sh bak 5 1 > /dev/null 2>&1 0 3 * * * /var/scripts/prod/check-storage.sh temp 5 1 > /dev/null 2>&1 /var/scripts/prod/check-storage.sh Code: #!/bin/bash ############################################# ## Name : check-storage.sh ## Version : 1.2 ## Date : 2012-05-01 ## Author : LHammonds ## Purpose : Check available space for a file system and expand if necessary. ## Compatibility : Verified on Ubuntu Server 10.04.4 LTS ## Requirements : None ## Run Frequency : Recommend once per day for each FS to monitor. ## Parameters : ## 1 = (Required) File System name (e.g. opt) ## 2 = (Required) File System Threshold (e.g. 2G) ## 3 = (Required) Amount to increase File System (e.g. 1G) ## Exit Codes : ## 0 = Success (either nothing was done or FS expanded without error) ## 1 = ERROR: Missing or incorrect parameter(s) ## 2 = ERROR: Invalid parameter value(s) ## 4 = ERROR: Lock file detected ## 8 = ERROR: Resize2fs error ## 16 = SEVERE: No room to expand ## 32 = ERROR: Script not run by root user ################ CHANGE LOG #################
    • ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2012-05-01 LTH Created script. ## 2012-05-02 LTH Improved log messages. ## 2012-05-02 LTH Improved email messages. ############################################# ## Import standard variables and functions. ## source /var/scripts/common/standard.conf ## Define local variables. LOGFILE="${TEMPDIR}/check-storage.log" LOCKFILE="${TEMPDIR}/check-storage.lock" ErrorFlag=0 ReturnCode=0 ####################################### ## FUNCTIONS ## ####################################### function f_cleanup() { if [ -f ${LOCKFILE} ];then ## Remove lock file so other check space jobs can run. rm ${LOCKFILE} 1>/dev/null 2>&1 fi exit ${ErrorFlag} } function f_showhelp() { echo -e "nUsage : ${SCRIPTNAME} FileSystemName ThresholdSizeInGB AmountToIncreaseByInGBn" echo -e "nExample: ${SCRIPTNAME} opt 2 1n" } function f_auto-increment() { let RoomInLV=${LVSize}-${FSSize} if [[ ${RoomInLV} -gt ${FSIncreaseBy} ]]; then ## There is room in the LV to increase space to the FS. resize2fs ${FSVol} ${NewFSSize}G ReturnCode=$? echo "`date +%Y-%m-%d_%H:%M:%S` --- resize2fs ${FSVol} ${NewFSSize}G, ReturnCode=${ReturnCode}" | tee -a ${LOGFILE} if [[ ${ReturnCode} -ne 0 ]]; then ## There was an error in resize2fs. return ${ReturnCode} fi else ## There is not enough room in the LV to increase space in the FS. return 50
    • fi return 0 } ####################################### ## MAIN PROGRAM ## ####################################### if [ -f ${LOCKFILE} ]; then # Lock file detected. Abort script. echo "Check space script aborted" echo "This script tried to run but detected the lock file: ${LOCKFILE}" echo "Please check to make sure the file does not remain when check space is not actually running." f_sendmail "ERROR: check storage script aborted" "This script tried to run but detected the lock file: ${LOCKFILE}nnPlease check to make sure the file does not remain when check space is not actually running.nnIf you find that the script is not running/hung, you can remove it by typing 'rm ${LOCKFILE}'" ErrorFlag=4 f_cleanup else echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE} fi ## Requirement Check: Script must run as root user. if [ "$(id -u)" != "0" ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. echo "ERROR: Root user required to run this script." echo "" ErrorFlag=32 f_cleanup fi ## Check existance of required command-line parameters. case "$1" in "") f_showhelp ErrorFlag=1 f_cleanup ;; --help|-h|-?) f_showhelp ErrorFlag=1 f_cleanup ;; *) FSName=$1 ;; esac case "$2" in "")
    • f_showhelp ErrorFlag=1 f_cleanup ;; --help|-h|-?) f_showhelp ErrorFlag=1 f_cleanup ;; *) FSThreshold=$2 ;; esac case "$3" in "") f_showhelp ErrorFlag=1 f_cleanup ;; --help|-h|-?) f_showhelp ErrorFlag=1 f_cleanup ;; *) FSIncreaseBy=$3 ;; esac ## Check validity of File System name. case "${FSName}" in "opt") FSVol="/dev/LVG/opt" FSMap="/dev/mapper/LVG-opt" ;; "bak") FSVol="/dev/LVG/bak" FSMap="/dev/mapper/LVG-bak" ;; "temp") FSVol="/dev/LVG/temp" FSMap="/dev/mapper/LVG-temp" ;; *) echo "ERROR: ${FSName} does not match a known file system defined in this script." f_showhelp ErrorFlag=2 f_cleanup ;; esac
    • ## Check validity of threshold value. test ${FSThreshold} -eq 0 1>/dev/null 2>&1 if [[ $? -eq 2 ]]; then ## Threshold parameter is not an integer. echo "ERROR: ${FSThreshold} is not an integer." f_showhelp ErrorFlag=2 f_cleanup fi ## Check validity of increment value. test ${FSIncreaseBy} -eq 0 1>/dev/null 2>&1 if [[ $? -eq 2 ]]; then ## FSIncreaseBy parameter is not an integer. echo "ERROR: ${FSIncreaseBy} is not an integer." f_showhelp ErrorFlag=2 f_cleanup fi ## Get available space for the file system. FSAvailable="`df --block-size=g ${FSMap} | awk '{ print $4 }' | tail -n 1 | sed 's/G//'`" ## Get the current size of the File System. FSSize="`df --block-size=g ${FSMap} | awk '{ print $2 }' | tail -n 1 | sed 's/G//'`" ## Get the current size of the Logical Volume for the File System LVSize="`lvs --noheadings --nosuffix --units=g ${FSMap} | awk '{ print $4}' | sed 's/[.].*//'`" ## Calculate the new size of the FS in case we need it. let NewFSSize=${FSSize}+${FSIncreaseBy} if [[ ${FSAvailable} -lt ${FSThreshold} ]]; then echo "`date +%Y-%m-%d_%H:%M:%S` - Starting expansion of ${FSVol}" | tee a ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` --- LVSize=${LVSize}GB, FSSize=${FSSize}GB, FSAvail=${FSAvailable}GB, FSThreshold=${FSThreshold}GB, FSIncreaseBy=${FSIncreaseBy}GB" | tee -a ${LOGFILE} ## Run the auto-expansion function. f_auto-increment ReturnCode=$? case ${ReturnCode} in 0) f_sendmail "NOTICE: File System Expanded" "${FSVol} was expanded because it was nearing max capacity. Please review disk space usage and plan appropriately. LVSize=${LVSize}GB, FSSize=${FSSize}GB, FSAvailable=${FSAvailable}GB, FSThreshold=${FSThreshold}GB, FSIncreaseBy=${FSIncreaseBy}GB" ;;
    • 50) echo "`date +%Y-%m-%d_%H:%M:%S` - SEVERE: No room to expand ${FSVol}" | tee -a ${LOGFILE} ErrorFlag=16 f_sendmail "SEVERE: No room to expand ${FSVol}" "There is not enough room in the Logical Volume to expand the ${FSVol} File System. Immediate action is required. Make sure there is free space in the Volume Group 'LVG' and then expand the Logical Volume...then expand the File System.nnLVSize=${LVSize}GB, FSSize=${FSSize}GB, FSAvailable=${FSAvailable}GB, FSThreshold=${FSThreshold}GB, FSIncreaseBy=${FSIncreaseBy}GB.nnType 'vgs' to see if there is any free space in the Volume Group which can be given to the Logical Volume.nnType 'lvs' to see the current sizes of the LVs.nnType 'lvdisplay' to see a list of Logical Volumes so you can get the LV Name which is used in the lvextend and resize2fs commands.nnType 'lvextend -L+1G /dev/LVG/opt' if you want to extend the opt Logical Volume by 1 gigabyte (assuming there is 1GB available in the Volume Group).nnType 'df --block-size=g' to see a list of file systems and their associated size and available space.nnType 'resize2fs /dev/LVG/opt ${NewFSSize}G' to set the size of opt to ${NewFSSize} gigabytes. Make sure you set the size to the desired endresult which should be LARGER than the current FS size so you do not lose data." ;; *) echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Expansion failure for ${FSVol}" | tee -a ${LOGFILE} ErrorFlag=8 f_sendmail "ERROR: File System Expansion Failed" "${FSVol} Expansion failed with return code of ${ReturnCode}. LVSize=${LVSize}GB, FSSize=${FSSize}GB, FSAvailable=${FSAvailable}GB, FSThreshold=${FSThreshold}GB, FSIncreaseBy=${FSIncreaseBy}GB" ;; esac echo "`date +%Y-%m-%d_%H:%M:%S` - Finished expansion of ${FSVol}" | tee a ${LOGFILE} else echo "`date +%Y-%m-%d_%H:%M:%S` - ${FSVol} ${FSAvailable}G>${FSThreshold}G No action required." | tee -a ${LOGFILE} fi ## Perform cleanup routine. f_cleanup Here is the typical output when it does not have to increase the FS: /var/temp/check-storage.log Code: 2012-05-01_01:00:00 2012-05-01_02:00:00 2012-05-01_03:00:00 2012-05-02_01:00:00 - /dev/LVG/opt 44G>5G No action required. /dev/LVG/bak 91G>5G No action required. /dev/LVG/temp 93G>5G No action required. /dev/LVG/opt 44G>5G No action required.
    • 2012-05-02_02:00:00 2012-05-02_03:00:00 2012-05-03_01:00:00 2012-05-03_02:00:00 2012-05-03_03:00:00 - /dev/LVG/bak 91G>5G No action required. /dev/LVG/temp 93G>5G No action required. /dev/LVG/opt 44G>5G No action required. /dev/LVG/bak 91G>5G No action required. /dev/LVG/temp 93G>5G No action required. Here is a sample of what the log will look like when it perform increases...notice how I had to increase the threshold in order for it to fire off the increase (but it still just issued a single gigabyte increase): /var/temp/check-storage.log Code: 2012-05-02_01:00:00 - Starting expansion of /dev/LVG/opt 2012-05-02_01:00:00 --- LVSize=75GB, FSSize=50GB, FSAvail=44GB, FSThreshold=55GB, IncreaseBy=1GB 2012-05-02_01:00:00 --- resize2fs /dev/LVG/opt 51, ReturnCode=0 2012-05-02_01:00:00 - Finished expansion of /dev/LVG/opt 2012-05-02_02:00:00 - Starting expansion of /dev/LVG/bak 2012-05-02_02:00:00 --- LVSize=125GB, FSSize=99GB, FSAvail=91GB, FSThreshold=100GB, IncreaseBy=1GB 2012-05-02_02:00:00 --- resize2fs /dev/LVG/bak 100, ReturnCode=0 2012-05-02_02:00:00 - Finished expansion of /dev/LVG/bak 2012-05-02_03:00:00 - Starting expansion of /dev/LVG/temp 2012-05-02_03:00:00 --- LVSize=125GB, FSSize=99GB, FSAvail=93GB, FSThreshold=100GB, IncreaseBy=1GB 2012-05-02_03:00:00 --- resize2fs /dev/LVG/temp 100, ReturnCode=0 2012-05-02_03:00:00 - Finished expansion of /dev/LVG/temp __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) Last edited by LHammonds; 05-03-2012 at 06:58 AM.. #5 (permalink) 04-26-2012, 07:26 PM LHammonds Special Member Software Configurations Software Configurations 1. Start the Ubuntu server and connect using PuTTY. 2. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) Posts: 138
    • 3. At the $ prompt, type aptitude -y install vim-nox for use instead of the built-in VI editor. more info 4. At the $ prompt, type aptitude -y install bind9 to install a local DNS server. 5. At the $ prompt, type aptitude -y install p7zip-full to install 7-zip archive utility. 6. Type dpkg-reconfigure dash and answer No 7. Remove AppArmor to avoid problems by typing the following: Code: /etc/init.d/apparmor stop update-rc.d -f apparmor remove aptitude remove apparmor apparmor-utils 8. Reboot the server by typing reboot VMware Tools If you are installing under VMware like me, be sure to install the tools to insure maximum performance in a virtual environment. 1. In the VM toolbar for the VM, click Guest, Install VMware Tools 2. At the Ubuntu prompt, type the following: Code: mkdir /mnt/cdrom mount /dev/cdrom /mnt/cdrom ls /mnt/cdrom tar zxvf /mnt/cdrom/VMwareTools-x.x.x-xxxx.tar.gz -C /tmp/ (use the correct numbers version discovered from the list command in the prior step) cd /tmp/vmware-tools-distrib/ ./vmware-install.pl -d reboot Configure Ubuntu for File Sharing This file sharing section is optional but can be handy if you need to swap files between the Linux server and a Windows machine. This documentation will utilize this share for passing pre-configured files (configs, scripts, etc.) to make it faster/easier during installation. 1. Start the Ubuntu server and connect using PuTTY. 2. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass)
    • 3. Install Samba by typing aptitude -y install samba smbfs (NOTE: To share a folder with Windows, you just need the samba package, to connect to a Windows share, you need both samba and smbfs) 4. Type the following commands: Code: cp /etc/samba/smb.conf /etc/samba/smb.conf.bak mkdir -p /srv/samba/share chown nobody:nogroup /srv/samba/share/ chmod 0777 /srv/samba/share 5. Edit the configuration file by typing vi /etc/samba/smb.conf 6. Change workgroup = WORKGROUP to workgroup = work (you are using the domain alias) 7. Change: Code: # security = user to: Code: security = user 8. Add the following section to the end of the file: Code: [share] comment = Ubuntu File Server Share path = /srv/samba/share browsable = yes guest ok = yes read only = no create mask = 0755 9. Save and exit the file. 10. Restart the samba services to utilize the new configuration by typing: Code: restart smbd restart nmbd
    • 11. You should now be able to click Start --> Run and type mail or 192.168.107.25 {ENTER} and see an explorer window with a Share folder. Drag-n-drop a file into the Share folder. If it worked, it will not display an error message and you should be able to view it from the server by typing ls -l /srv/samba/share/ 12. Shutdown and power off the server by typing shutdown -P now {ENTER} 13. In VM menu, select VM --> Snapshot --> Take Snapshot. Give it a name like STEP 2 and description of Ubuntu Server 10.04.4 LTS, File share configured, Static IP: 192.168.107.25. The Snapshot Manager should now have a nice hierarchy of snapshots (STEP 1 --> STEP 2 --> You are here) Configure Windows Remote Mount Point The remote offsite location is a Windows 2008 server that has a vast amount of hard drive space and currently backs up other servers. Part of the backup process will be to write files to a mounted folder to a Windows share. First, let's document the variables for this solution below and highlight them in red throughout the document for easy identification. The values below are merely samples which you need to change in order to match your environment: Windows AD Domain Name: work Windows Server Name: SRV-Backup Windows Server IP: 192.168.107.218 Windows Share Name: zimbra Windows Physical Share Location: D:Zimbra Windows Share ID: zimbrashare Windows Share Password: zimbrapassword Create a share on a Windows 2008 server 1. In Windows Explorer, right-click on the D:Zimbra folder and select Properties 2. Click the Sharing tab 3. Click the Advanced Sharing button 4. Place a checkmark beside Share this folder 5. Change the Share name to zimbra 6. Set the Comment to Zimbra Backup 7. Click the Permissions button 8. Select Everyone and click the Remove button 9. Click the Add button 10. Type in your zimbra share account: workzimbrashare and click the Check Names button, click OK 11. Place a checkmark for Allow Full Control and click OK, click OK, click OK 12. Create a text file in the root of the shared folder called "online.txt" and you might want to add some text inside saying to never delete this file because it is used (will be)
    • by a Zimbra backup script. Probably a good idea to make it read-only as well. Example: [B]D:zimbraonline.txt[B] Create an NFS mount to the Windows 2008 server Connecting to a Windows share requires the samba and smbfs packages to be installed. If you did not install them (from a prior section), type aptitude -y install samba smbfs 1. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 2. Type the following commands: Code: mkdir -p /mnt/backup chown root:root /mnt/backup chmod 0755 /mnt/backup echo "This file is used to tell if the mount is active or not" > /mnt/backup/offline.txt chown root:root /mnt/backup/offline.txt chmod 0444 /mnt/backup/offline.txt touch /etc/cifspw chmod 0600 /etc/cifspw 3. Type vi /etc/cifspw and add the following text, save and exit the file: Code: username=workzimbrashare password=zimbrapassword 4. Type vi /etc/hosts and add the following line anywhere in the file: Code: 192.168.107.218 srv-backup 5. At this point, you might want to type ping srv-backup to make sure you typed the right IP address as well as seeing a good response. 6. To mount this system for backups, type the following command: Code: mount -t cifs //srv-backup/zimbra /mnt/backup --options
    • nouser,rw,nofail,noatime,noexec,credentials=/etc/cifspw 7. To test it, type cp /etc/hosts /mnt/backup/hosts.txt and look on the Windows server and see if the file shows up. Then type rm /mnt/backup/hosts.txt and verify that the file was deleted on the windows server. 8. This would also be a good time to verify that you can see the "online.txt" file that will be used by the backup script. Type ls -l /mnt/backup/*.txt 9. To dismount the windows share, type the following command: Code: umount /mnt/backup The scripts will call a common mount and unmount function to connect to this share only when needed. However, if you would rather have it mounted all the time (even after a reboot), do the following (but remember to not use the mount/umount functions in the scripts later): 1. Type vi /etc/fstab and add the following line at the bottom of the file: Code: //srv-backup/zimbra /mnt/backup cifs nouser,rw,nofail,noatime,noexec,credentials=/etc/cifspw 0 2. Type mount -a and if it does not spew out any error messages, it will quietly mount the share. 3. To test it, type cp /etc/hosts /mnt/backup/hosts.txt and look on the Windows server and see if the file shows up. Then type rm /mnt/backup/hosts.txt and verify that the file was deleted on the windows server. 4. If you need to unmount it, simply type umount /mnt/backup and it will remain unmounted until you reboot. To make it permanent, you need to remove the line you added in the /etc/fstab file. Sometimes it is helpful during tests to toggle the mount on or off so here is a script you might find helpful. You can find the contents of "standard.conf" file further down the thread. togglemount.sh Code: #!/bin/bash ############################################# ## Name : togglemount.sh ## Version : 1.0 ## Date : 2011-11-05 ## Author : LHammonds 0
    • ## Compatibility : Ubuntu Server 10.04.4 LTS ## Purpose : Toggle the mount status of a pre-configured backup mount. ## Run Frequency : Manual as needed. ## Exit Codes : ## 0 = success ## 1 = failure ################ CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2011-11-05 LTH Created script. ############################################# ## Import common variables and functions. ## source /var/scripts/common/standard.conf ERRORFLAG=0 if [ -f ${OFFSITEDIR}/offline.txt ]; then echo "Windows share is not mounted. Mounting share now..." f_mount sleep 2 if [ -f ${OFFSITEDIR}/online.txt ]; then echo "Mount successful. Listing contents:" else echo "Mount failed. Listing contents:" ERRORFLAG=1 fi else echo "Windows share is mounted. Dismounting share now..." f_umount sleep 2 if [ -f ${OFFSITEDIR}/offline.txt ]; then echo "Dismount successful. Listing contents:" else echo "Dismount failed. Listing contents:" ERRORFLAG=1 fi fi ls -l ${OFFSITEDIR} exit ${ERRORFLAG} __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #6 (permalink) 04-26-2012, 07:27 PM LHammonds Special Member Configure DNS Configure Domain Resolution (DNS) Posts: 138
    • 1. Connect to the server using PuTTY 2. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 3. Sanity check! Make sure BIND is running. Type /etc/init.d/bind9 status 4. At this point, you can save yourself a lot of typing if you have a copy of the configuration files on your PC and simply copy them to the share. Then you can set the appropriate file permissions and copy them where they need to go. But if not and this is your 1st time installing the server, here are the details for configuring by hand. 5. Edit the hosts by typing vi /etc/hosts and change the following: From: Code: 127.0.0.1 127.0.1.1 localhost mail.mydomain.com mail To: Code: 127.0.0.1 localhost.localdomain localhost 192.168.107.25 mail.mydomain.com mail 6. Edit the DNS servers by typing vi /etc/resolv.conf and set it to the following: Code: nameserver 127.0.0.1 nameserver 192.168.107.23 nameserver 192.168.107.212 nameserver 200.100.10.1 nameserver 200.100.10.2 domain mydomain.com search mydomain.com 7. Type the following commands: Code: mkdir /var/cache/bind/internal mkdir /var/cache/bind/external touch /var/cache/bind/internal/db.mydomain.com touch /var/cache/bind/internal/db.107.168.192.in-addr.arpa touch /var/cache/bind/external/db.mydomain.com cp /etc/bind/named.conf.options /etc/bind/named.conf.options.bak cp /etc/bind/named.conf.local /etc/bind/named.conf.local.bak cp /etc/bind/named.conf.default-zones /etc/bind/named.conf.default-zones.bak
    • 8. Stop the DNS server by typing /etc/init.d/bind9 stop 9. Edit the DNS options by typing vi /etc/bind/named.conf.options and set the following: Code: options { directory "/var/cache/bind"; query-source address * port 53; forwarders { 200.100.10.1; 200.100.10.2; }; auth-nxdomain no; # conform to RFC1035 listen-on-v6 { any; }; }; NOTE: The IP addresses are public IP addresses of the DNS servers you use in the outside world. The query-source address line is to allow your machine to hit the DNS if oddball DNS ports are blocked. You can leave it commented if you do not need it. 10. Restart the network by typing /etc/init.d/networking restart 11. Type the following vi /etc/hostname and add the following text, save and exit: Code: mail.mydomain.com 12. To check results, type hostname and then hostname -f which should result with: Code: mail mail.mydomain.com 13. Edit the local DNS file by typing vi /etc/bind/named.conf.local and set the following: Code: acl internals { 127.0.0.0/8; 192.168.107.0/24; }; view "internal" { match-clients { internals; }; recursion yes; zone "mydomain.com" { type master; file "/var/cache/bind/internal/db.mydomain.com";
    • }; zone "107.168.192.in-addr.arpa" { type master; file "/var/cache/bind/internal/db.107.168.192.inaddr.arpa"; }; }; view "external" { match-clients { any; }; recursion no; zone "mydomain.com" { type master; file "/var/cache/bind/external/db.mydomain.com"; }; }; 14. Edit the internal reverse zone file by typing vi /var/cache/bind/internal/db.107.168.192.in-addr.arpa and set the following: Code: $TTL 86400 @ IN 1 NS PTR SOA mail.mydomain.com. root.mydomain.com. ( 201109231448 ; Serial (increment after edit) 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 86400) ; Negative Cache TTL mail.mydomain.com. mail.mydomain.com. 15. Edit the internal zone file by typing vi /var/cache/bind/internal/db.mydomain.com and set the following: Code: ; mydomain.com $TTL 86400 @ IN SOA root.mydomain.com. ( mail.mydomain.com. 201109231335 ; Serial 604800 86400 2419200 604800) ; ; ; ; (increment after edit) Refresh Retry Expire Negative Cache TTL ; Define the nameservers and the mail servers @ IN NS 192.168.107.25. IN MX 10 mail.mydomain.com. IN A 192.168.107.25 mail IN A 192.168.107.25 16. Edit the external zone file by typing vi /var/cache/bind/external/db.mydomain.com
    • and set the following: Code: ; mydomain.com $TTL 86400 @ IN SOA root.mydomain.com. ( mail.mydomain.com. 201109231335 ; Serial 604800 86400 2419200 604800) ; ; ; ; (increment after edit) Refresh Retry Expire Negative Cache TTL ; Define the nameservers and the mail servers @ IN NS 198.100.100.100. IN MX 10 mail.mydomain.com. IN A 198.100.100.100 mail IN A 198.100.100.100 17. Since views were used, they also need to be used in the default zone. Type vi /etc/bind/named.conf.default-zones and set the following: Code: // prime the server with knowledge of the root servers acl internals-default { 127.0.0.0/8; 192.168.107.0/24; 192.168.106.0/24; here. }; // Keep adding all your internal subnets view "internal-default" { match-clients { internals-default; }; recursion yes; zone "." { type hint; file "/etc/bind/db.root"; }; // be authoritative for the localhost forward and reverse zones, and for // broadcast zones as per RFC 1912 zone "localhost" { type master; file "/etc/bind/db.local"; }; zone "127.in-addr.arpa" { type master; file "/etc/bind/db.127";
    • }; zone "0.in-addr.arpa" { type master; file "/etc/bind/db.0"; }; zone "255.in-addr.arpa" { type master; file "/etc/bind/db.255"; }; }; 18. Ensure the correct ownership and permissions of all config files: Code: chown chown chmod chmod root:bind /var/cache/bind/internal/* root:bind /var/cache/bind/external/* 0644 /var/cache/bind/internal/* 0644 /var/cache/bind/external/* 19. Start the DNS server by typing /etc/init.d/bind9 start 20. That was a lot of typing and room for error so check the log for any errors: vi /var/log/daemon.log 21. Sanity check! At this point, if you type nslookup mail.mydomain.com, you should see that your internal DNS server (127.0.0.1) returned the result of your internal IP address (192.168.107.25 for your FQDN of mail.mydomain.com) 22. Sanity check! Type dig mydomain.com mx and make sure you see a status of NOERROR along with an MX record to your FQDN, NS record for your internal IP and an A record your FQDN to your internal IP. 23. Shutdown and power off the server by typing shutdown -P now {ENTER} 24. In VM menu, select VM --> Snapshot --> Take Snapshot. Give it a name like STEP 3 and description of Ubuntu Server 10.04.4 LTS, Split-DNS configured, Static IP: 192.168.107.25. Click OK. The Snapshot Manager should now have a nice hierarchy of snapshots (STEP 1 --> STEP 2 --> STEP 3 --> You are here) Final Ubuntu Changes 1. Start the Ubuntu server and connect using PuTTY. 2. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 3. Update the package database by typing aptitude update 4. Install the latest updates by typing aptitude -y safe-upgrade 5. Shutdown and power off the server by typing shutdown -P now {ENTER} 6. In VM menu, select VM --> Snapshot --> Take Snapshot. Give it a name like STEP 4 and description of Ubuntu Server 10.04.4 LTS, all patches applied, Static IP: 192.168.107.25. The Snapshot Manager should now have a nice hierarchy of snapshots (STEP 1 --> STEP 2 --> STEP 3 --> STEP 4 --> You are here)
    • 7. NOTE: Over time, upgrades to the kernel can start taking up space in /etc/src. I found this out because my root partition started to get full. When I typed this command: du sh /usr/src/* I found out there was over a gigabyte of old files. To cleanly clear out old an unused headers, type the following commands: Code: apt-get -f install apt-get autoremove __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #7 (permalink) 04-26-2012, 07:32 PM LHammonds Special Member Posts: 138 Install Zimbra Install Zimbra NOTE: Install Zimbra by typing "./install.sh" and uninstall by typing "./install.sh -u" Prerequisite packages: NPTL sudo-1.7.2p1-1ubuntu5.3 libidn11-1.15-2 libpcre3-7.8-3build1 libgmp3c2-2:4.3.2+dfsg-1ubuntu1 libexpat1-2.0.1-7ubuntu1 libstdc++6-4.4.3-4ubuntu5 libperl5.10-5.10.1-8ubuntu2.1 Suggested prerequisites: perl-5.10.1 sysstat sqlite3 Install Procedure:
    • 1. Start the Ubuntu server and connect using PuTTY. 2. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 3. Download the Open Source Edition from Open Source Edition Downloads: Enterprise Messaging and Collaboration Software by Zimbra Method #1 - Download from your PC A. Download the .tgz file and copy it to the Ubuntu share --> mailshare or 192.168.107.25share B. Once copied to the server, access the console and type the following: Code: mv /srv/samba/share/zcs* /var/temp Method #2 - Download directly from the server A. Obtain the download URL from the Zimbra OSE page B. Type the following commands on the server console (using the correct URL) Code: cd /var/temp wget http://files.zimbra.com/downloads/7.2.0_GA/zcs7.2.0_GA_2669.UBUNTU10_64.20120410002303.tgz 4. Extract the archive by typing the following commands: Code: mkdir /var/temp/install tar -C /var/temp/install --strip-components=1 -xzf /var/temp/zcs* 5. Start the installation program by typing the following: Code: cd /var/temp/install ./install.sh 6. Agree to license agreement? Press y {ENTER} 7. If you are missing any prerequisites, exit the installation and install them. You can install all at one time by separating each name with a space after aptitude install such as the following:
    • Code: aptitude -y install sysstat sqlite3 8. Install zimbra-ldap? Press y {ENTER} 9. Install zimbra-logger? Press y {ENTER} 10. Install zimbra-mta? Press y {ENTER} 11. Install zimbra-snmp? Press y {ENTER} 12. Install zimbra-store? Press y {ENTER} 13. Install zimbra-apache? Press y {ENTER} 14. Install zimbra-spell? Press y {ENTER} 15. Install zimbra-memcached? Press n {ENTER} (if multi-server, press y instead) 16. Install zimbra-proxy? Press n {ENTER} (if multi-server, press y instead) 17. The system will be modified. Continue? Press y {ENTER} 18. DNS ERROR resolving MX for mail.mydomain.com. Change domain name? Press y {ENTER} 19. Now we put the correct domain name in so type mydomain.com {ENTER} 20. You are presented a menu for resolving unconfigured options, press 3 {ENTER} 21. Select Admin password, press 4 {ENTER}. Then type your new password (myzimbrapass) {ENTER} 22. Press r {ENTER} for previous menu 23. Press a {ENTER} to apply changes. 24. Press y {ENTER}, {ENTER} to save configuration (/opt/zimbra/config.9066) 25. The system will be modified - continue? Press y {ENTER} 26. Notify Zimbra of your installation? Press n {ENTER} (feel free to answer yes if this is not a test system) 27. Press {ENTER} to exit setup. NOTE: Log files placed in /opt/zimbra/log 28. Shutdown and power off the server by typing the following: Code: /etc/init.d/zimbra stop shutdown -P now 29. In VM menu, select VM --> Snapshot --> Take Snapshot. Give it a name like STEP 5 and description of Ubuntu Server 10.04.4 LTS, Zimbra 7.2.0 OSE installed, Static IP: 192.168.107.25. The Snapshot Manager should now have a nice hierarchy of snapshots (STEP 1 --> STEP 2 --> STEP 3 --> STEP 4 --> STEP 5 --> You are here) Admin Console and Initial Test 1. Start the Ubuntu server. 2. Access the webmail control panel by opening a web browser and typing this in the address bar: https://192.168.107.25:7071 3. If your browser barks about an untrusted connection, tell it to continue anyway and add an exception for it so it will not have a problem with it next time. 4. Login as admin@mydomain.com / myrootpassword
    • 5. Create a new user account. 6. When done creating a new user, open a new browser tab and access: http://192.168.107.25/ and login with your new user account. See if you can send an email! If it does not work, well, what can I say...you followed the advice of a Linux newbie. Scripted sendmail My scripts heavily reply upon the ability to send email notifications and this step will enable that ability. 1. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 2. To configure the sendmail command to work with Zimbra, type the following: Code: update-alternatives --install /usr/sbin/sendmail mta-send /opt/zimbra/postfix/sbin/sendmail 25 3. Test sendmail by typing the following and then check your admin email to see if it is there: Code: sendmail -t{ENTER} to:admin@mydomain.com{ENTER} from:webmaster@mydomain.com{ENTER} subject: Test from Ubuntu sendmail{ENTER} {ENTER} Testing 1-2-3{ENTER} .{ENTER} Delete Snapshots Be sure to delete all snapshots when you are satisfied that you will no longer need to revert your installation to any of the prior snapshots. Zimbra and SSL Encryption I do not see a reason not to use SSL, especially if your user accounts and passwords match other systems such as Active Directory. A compromised email password can also
    • compromise your network in that case. With that in mind, this is what I have done to make sure it is used at my location. If you just installed Zimbra, the installation process created a self-signed certificate. Step #1 - Enable SSL as the primary method of logging onto your server: 1. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 2. At the server console, type the following to convert webmail to use encrypted https instead of http: Code: su - zimbra -c "zmtlsctl https" 3. For additional documentation, here is how you would switch back to non-SSL mode: Code: su - zimbra -c "zmtlsctl http" 4. For additional documentation, here is how you could use either SSL or non-SSL mode: Code: su - zimbra -c "zmtlsctl mixed" 5. Type reboot to reboot the server for the HTTPS change to take effect. Step #2 - Export the certificate for Windows. The certificate on the Zimbra server cannot be directly used by Windows and needs to be converted into a format that can be used. Do so using the following commands: Code: openssl x509 -in /opt/zimbra/ssl/zimbra/ca/ca.pem -out /srv/samba/share/cacert.der -outform DER chmod 0644 /srv/samba/share/cacert.der Step #3 - Import the certificate on Windows. For Windows machines, you might need to install the certificate manually to avoid Internet Explorer, MS Outlook and Google Chrome issues with self-signed certificates.
    • Here are two methods for doing so. Method A - Domain Group Policy If you have a Windows domain, this is a very easy process to get all your machines to automatically accept the self-signed certificate on your Zimbra server. 1. 2. 3. 4. 5. On the domain controller, open Active Directory Users and Computers Right-click on your domain and select Properties Select the Group Policy tab Select the Default Domain Policy and click the Edit button Expand Computer Configuration --> Windows Settings --> Security Settings --> Public Key Policies 6. Right-click on Trusted Root Certification Authorities and click Import 7. Import Wizard - Click Next 8. Type the path to the cacert.der from your Zimbra server or click Browse (if you browse, you will need to change the file type to All Files in order to see the certificate file) and then click Next 9. Select Place all certificates in the following store --> Trusted Root Certification Authorities and click Next, Finish 10. The import was successful - Click OK 11. You should now see your mail server domain in the list. Example: mail.mydomain.com 12. Close Group Policy Object Editor and any other windows you opened Any Windows machine on your domain should now be able to open Internet Explorer and visit your SSL-enabled mail server without complaining about the self-signed certificate. Same goes for Outlook, Google Chrome and others. You may need to logout and log back in for the policy to take effect on the PCs. Method B - Manually Import the Certificate If you are forced to manually touch each PC by hand, here is how you can install the certificate (for Windows XP, Windows Vista or Windows 7) 1. Close Internet Explorer if it is open. 2. Click Start --> Run (or type in "Search programs and files" box on Windows 7) and type mmc to load an empty Microsoft Management Console. 3. If you receive a security warning, click the positive acknowledgement button (actual name varies all over the place). 4. From the menu, click File, Add / Remove Snap-In 5. Select Certificates and click the Add button 6. Select Computer account and click Next 7. Select Local computer and click Finish, OK 8. Expand Certificates, Trusted Root Certification Authorities and select the Certificates container 9. Right-click on the Certificates container, and choose the All Tasks --> Import 10. Import Wizard - Click Next 11. Type 192.168.107.25sharecacert.der and click Next
    • 12. Select Place all certificates in the following store --> Trusted Root Certification Authorities and click Next, Finish 13. The import was successful - Click OK 14. Look in the list of certificates and you should see your domain such as mail.mydomain.com 15. Close the console (do not bother saving the MMC when asked) Step #4 - Update Your Certificate As time goes by, the certificate that was created with your installation will expire (360 days from install) and you will need to generate a new one. 1. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 2. Type the commands below to generate a new self-signed certificate: Code: /opt/zimbra/bin/zmcertmgr /opt/zimbra/bin/zmcertmgr /opt/zimbra/bin/zmcertmgr /opt/zimbra/bin/zmcertmgr createca -new createcrt -new -days 365 deploycrt self deployca 3. Check to see if the certificate was deployed to all services by typing /opt/zimbra/bin/zmcertmgr viewdeployedcrt 4. Now go back through the above steps to export the new certificate and import into all your PCs Reference Article: Zimbra Wiki: Administration Console and CLI Certificate Tools __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #8 (permalink) 04-26-2012, 07:43 PM LHammonds Special Member Posts: 138 Scripting Scripting Much of the solutions beyond this point involve scripts (programming snippets / automated commands).
    • In particular, they are Bash Scripts. I chose this due to its popularity and the fact it comes with Ubuntu. I try to make use of what comes with the system without requiring additional software / services unless they really add to the bottom line such as decreasing the time it takes for a process to run or to conserve storage and bandwidth usage. When setting up a server and testing things out, there is typically very little concern for procedures / process since much of the activity is exploration and experimentation as well as not having an impact on production. However, once a server goes into production, processes and procedures need to be in place to ensure the availability of the services being provided. In regards to these scripts, they will be treated like any other program and will require being tested, documented and go through a promotion process. The ideal situation would involve 3 servers (for a single server setup). A test / development server, a quality assurance staging server and the production server itself. If 3 servers cannot be utilized, then it can still work well with 2 servers. Testing scripts / programs / restore on the production server is not advisable and many times impractical...how can you test your restore process / data periodically if you only have a production server? The QA Staging server would be as close to production as possible. The server should be setup in such a way that production backups are restored to this server which also tests and validates the backup / restore process as well as maintains a close representation of the production server to mitigate variable risk involved when testing new or modified programs and upgrades. The test / development server can serve as the QA server if absolutely necessary. The directory structure and how scripts can import other scripts will be configured to facilitate this process. Example: Directory path for scripts to import common variables, functions and server settings: /var/scripts/common/ Directory path for production scripts: /var/scripts/prod/ Directory path for QA staging area scripts: /var/scripts/qa/ Directory path for test / development scripts: /var/scripts/test/ With a production and test servers on physically different machines, the "common" scripts folder can be custom-tailored for that environment and allow for minimal changes to a script when running on the test, QA or production server. This is similar to "normalizing" a database. If you have a variable, path or function that is duplicated in multiple scripts, consider pulling it out and placing it in the common folder. If ever you need to change who receives the email reports, you only need to update a single script and all programs will use the new reference from that point on.
    • Most of my scripts will import a file called "standard.conf" from the common script folder. /var/scripts/common/standard.conf (contents of the file on the production server) Code: ## Global Variables ## TEMPDIR="/var/temp" ZIMBRADIR="/opt/zimbra" SHAREDIR="/srv/samba/share" MYDOMAIN="mydomain.com" ADMINEMAIL="admin@${MYDOMAIN}" REPORTEMAIL="lhammonds@${MYDOMAIN}" BACKUPDIR="/var/backup" OFFSITEDIR="/mnt/backup" OFFSITETESTFILE="${OFFSITEDIR}/online.txt" ARCHIVEMETHOD="tar.7z" ## Choices are tar.7z or tgz HOSTNAME="$(hostname -s)" SCRIPTNAME="$0" SCRIPTDIR="/var/scripts" MAILFILE="${TEMPDIR}/mailfile.$$" ## Global Functions ## function f_sendmail() { ## Purpose: Send administrative email message. ## Parameter #1 = Subject ## Parameter #2 = Body echo "From: ${ADMINEMAIL}" > ${MAILFILE} echo "To: ${REPORTEMAIL}" >> ${MAILFILE} echo "Subject: ${1}" >> ${MAILFILE} echo "" >> ${MAILFILE} echo -e "${2}" >> ${MAILFILE} echo "" >> ${MAILFILE} echo -e "nnServer: ${HOSTNAME}nProgram: ${SCRIPTNAME}nLog: ${LOGFILE}" >> ${MAILFILE} sendmail -t < ${MAILFILE} rm ${MAILFILE} } function f_sendusermail() { ## Purpose: Send end-user email message. ## Parameter #1 = To ## Parameter #2 = Subject ## Parameter #3 = Body echo "From: ${ADMINEMAIL}" > ${MAILFILE} echo "To: ${1}" >> ${MAILFILE} echo "Subject: ${2}" >> ${MAILFILE} echo "" >> ${MAILFILE} echo -e "${3}" >> ${MAILFILE} sendmail -t < ${MAILFILE} rm ${MAILFILE} } function f_mount() { ## Mount the pre-configured Windows share folder.
    • ## NOTE: The Windows share should have a file called "online.txt" mount -t cifs //srv-backup/zimbra ${OFFSITEDIR} --options nouser,rw,nofail,noatime,noexec,credentials=/etc/cifspw } function f_umount() { ## Dismount the Windows share folder. ## NOTE: The unmounted folder should have a file called "offline.txt" umount ${OFFSITEDIR} } /var/scripts/common/standard.conf (contents of the file on the test server) Code: ## Global Variables ## TEMPDIR="/tmp" ZIMBRADIR="/opt/zimbra800" SHAREDIR="/srv/samba/share" MYDOMAIN="mytestdomain.com" ADMINEMAIL="test1@${MYDOMAIN}" REPORTEMAIL="test2@${MYDOMAIN}" BACKUPDIR="/var/backup" OFFSITEDIR="/mnt/fakedir" OFFSITETESTFILE="${OFFSITEDIR}/online.txt" ARCHIVEMETHOD="tar.7z" ## Choices are tar.7z or tgz HOSTNAME="$(hostname -s)" SCRIPTNAME="$0" SCRIPTDIR="/var/scripts/" MAILFILE="${TEMPDIR}/mailfile.$$" ## Global Functions ## function f_sendmail() { ## Purpose: Send administrative email message. ## Parameter #1 = Subject ## Parameter #2 = Body echo "From: ${ADMINEMAIL}" > ${MAILFILE} echo "To: ${REPORTEMAIL}" >> ${MAILFILE} echo "Subject: ${1}" >> ${MAILFILE} echo "" >> ${MAILFILE} echo -e "${2}" >> ${MAILFILE} echo "" >> ${MAILFILE} echo -e "nnServer: ${HOSTNAME}nProgram: ${SCRIPTNAME}nLog: ${LOGFILE}" >> ${MAILFILE} sendmail -t < ${MAILFILE} rm ${MAILFILE} } function f_sendusermail() { ## Purpose: Send end-user email message. ## Parameter #1 = To ## Parameter #2 = Subject ## Parameter #3 = Body echo "From: ${ADMINEMAIL}" > ${MAILFILE} echo "To: ${1}" >> ${MAILFILE} echo "Subject: ${2}" >> ${MAILFILE}
    • echo "" >> ${MAILFILE} echo -e "${3}" >> ${MAILFILE} sendmail -t < ${MAILFILE} rm ${MAILFILE} } function f_mount() { ## Mount the pre-configured Windows share folder. ## NOTE: The Windows share should have a file called "online.txt" mount -t cifs //mypc/zimbra ${OFFSITEDIR} --options nouser,rw,nofail,noatime,noexec,credentials=/etc/cifspw } function f_umount() { ## Dismount the Windows share folder. ## NOTE: The unmounted folder should have a file called "offline.txt" umount ${OFFSITEDIR} } When receiving email notifications, the server name, script name and path will be included at the bottom of the email every time. It will be readily apparent if the email was generated from the test, qa or production server simply because of the location (even if test, qa and production are all on the same server). Here are the scripts to help automate the creation of this structure on the various servers (would run all of them if all are the same box) setup-script-prod.sh Code: #!/bin/bash if [ ! -d /var/scripts/prod ]; then mkdir -p /var/scripts/prod fi if [ ! -d /var/scripts/common ]; then mkdir -p /var/scripts/common fi chown root:root /var/scripts chown root:root /var/scripts/prod chown root:root /var/scripts/common chmod 0777 /var/scripts chmod 0755 /var/scripts/prod chmod 0755 /var/scripts/common setup-script-qa.sh Code: #!/bin/bash if [ ! -d /var/scripts/qa ]; then mkdir -p /var/scripts/qa fi if [ ! -d /var/scripts/common ]; then mkdir -p /var/scripts/common fi
    • chown chown chown chmod chmod chmod root:root /var/scripts root:root /var/scripts/qa root:root /var/scripts/common 0777 /var/scripts 0777 /var/scripts/qa 0755 /var/scripts/common setup-script-test.sh Code: #!/bin/bash if [ ! -d /var/scripts/test ]; then mkdir -p /var/scripts/test fi if [ ! -d /var/scripts/common ]; then mkdir -p /var/scripts/common fi chown root:root /var/scripts chown root:root /var/scripts/test chown root:root /var/scripts/common chmod 0777 /var/scripts chmod 0777 /var/scripts/test chmod 0777 /var/scripts/common __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) Last edited by LHammonds; 05-05-2012 at 12:07 PM.. #9 (permalink) 04-26-2012, 07:48 PM LHammonds Special Member Posts: 138 Keep Zimbra in Sync with Active Directory For my scenario, I want mailboxes created automatically when I create an Active Directory (AD) account. I do not want the mailbox to authenticate against AD though. To accomplish this, two or three scripts will be created and setup to run on a regular basis. The 1st script will be run on a Windows server and will create a comma-delimited (CSV) file for all users who belong to a security group called "GRP_ZimbraEmail" which allows me to selectively pick who is allowed to have a mailbox. The script will probably just run once per day (after hours). The CSV file will be placed on Ubuntu's Samba share. The 2nd script will be running on Ubuntu and checking for the input file very often (every minute). It will then cycle through all the AD users and see which users do not have a mailbox account and then create the mailbox using the supplied details.
    • export_zimbra_ad_users.vbs Code: Option Explicit '**************************************** '** Name: export_zimbra_ad_users.vbs '** Version: 1.1 '** Date: 2011-10-29 '** Author: LHammonds '** Purpose: Export AD users to a comma-delimited file '** that are authorized to have a Zimbra mailbox. '**************************************** '* Field #1 = LoginID '* Field #2 = First Name '* Field #3 = Middle Initial '* Field #4 = Last Name '* Field #5 = Full Name '* Field #6 = Title '* Field #7 = Description '* Field #8 = Comments '* Field #9 = Telephone '* Field #10 = Home Phone '* Field #11 = Mobile Phone '* Field #12 = Fax Number '* Field #13 = Pager '* Field #14 = Company '* Field #15 = Office '* Field #16 = Street Address '* Field #17 = PO Box '* Field #18 = City '* Field #19 = State '* Field #20 = Postal Code '* Field #21 = Country '* Field #22 = Password Replacement Value '* Field #23 = Unused (mainly to avoid the end-of-line character being read into the last value) '** NOTE: This could use a data cleanup routine that replaces all commas in a ** '** variable with something else such as a period instead to avoid CSV issues. ** Const Const Const Const ForAppending = 8 FilePath = "D:Reportsadlist.csv" ZimbraSecurityGroup = "GRP_ZimbraEmail" DefaultPassword = "ChangeThisPassword" Dim objRootDSE, strDNC, objDomain, objFSO, objFile Set objRootDSE = GetObject("LDAP://RootDSE") strDNC = objRootDSE.Get("DefaultNamingContext") Set objDomain = GetObject("LDAP://" & strDNC) '** Create / Overwrite the export file. ** Set objFSO = CreateObject("Scripting.FileSystemObject") Set objFile = objFSO.CreateTextFile(FilePath) objFile.Close Set objFile = Nothing
    • Call TrollTheFolders(objDomain) Sub TrollTheFolders(pobjDomain) '** Function: Traverse the AD structure to find users wherever they may reside. ** '** The trick is that this function is called recursively in order to ** '** inspect every sub-folder that may contain user accounts. ** Dim lobjFile, lobjFSO, lobjMember, lstrLine, lblnInZimbraGroup Dim lstrSamAccountName, lstrFirstName, lstrInitials, lstrLastName Dim lstrFullName, lstrTitle, lstrDescription, lstrComment Dim lstrTelephoneNumber, lstrHomePhone, lstrMobile, lstrFaxNumber, lstrPager Dim lstrCompany, lstrOffice, lstrStreetAddress, lstrPostOfficeBox Dim lstrCity, lstrState, lstrPostalCode, lstrCountry, lstrPassword Dim lcolGroups, lobjGroup, lstrTemp For Each lobjMember In pobjDomain '** Examine each object but process only "user" objects. ** If lobjMember.Class = "user" Then Set lobjFile = objFSO.OpenTextFile (FilePath, ForAppending, True) If Not (isempty(lobjMember.samAccountName)) Then lstrSamAccountName = lobjMember.samAccountName Else lstrSamAccountName = "" End If If Not (isempty(lobjMember.GivenName)) Then lstrFirstName = lobjMember.GivenName Else lstrFirstName = "" End If If Not (isempty(lobjMember.initials)) Then lstrInitials = lobjMember.initials Else lstrInitials = "" End If If Not (isempty(lobjMember.sn)) Then lstrLastname = lobjMember.sn Else lstrLastName = "" End If If Not (isempty(lobjMember.CN)) Then lstrFullName = lobjMember.CN Else lstrFullName = "" End If If Not (isempty(lobjMember.title)) Then lstrTitle = lobjMember.title Else lstrTitle = "" End If If Not (isempty(lobjMember.Description)) Then lstrDescription = lobjMember.Description Else lstrDescription = "" End If If Not (isempty(lobjMember.comment)) Then lstrComment = lobjMember.comment Else lstrComment = "" End If If Not (isempty(lobjMember.telephoneNumber)) Then lstrTelephoneNumber = lobjMember.telephoneNumber Else lstrTelephoneNumber = "" End If If Not (isempty(lobjMember.homePhone)) Then lstrHomePhone = lobjMember.homePhone Else lstrHomePhone = "" End If If Not (isempty(lobjMember.mobile)) Then lstrMobile = lobjMember.mobile Else lstrMobile = "" End If If Not (isempty(lobjMember.otherFacsimileTelephoneNumber)) Then lstrFaxNumber = lobjMember.otherFacsimileTelephoneNumber Else lstrFaxNumber = "" End If If Not (isempty(lobjMember.pager)) Then lstrPager = lobjMember.pager Else lstrPager = "" End If If Not (isempty(lobjMember.company)) Then lstrCompany = lobjMember.company Else lstrCompany = "" End If If Not (isempty(lobjMember.physicalDeliveryOfficeName)) Then lstrOffice = lobjMember.physicalDeliveryOfficeName Else lstrOffice = "" End If If Not (isempty(lobjMember.streetAddress)) Then lstrStreetAddress = lobjMember.streetAddress Else lstrStreetAddress = "" End If If Not (isempty(lobjMember.postOfficeBox)) Then lstrPostOfficeBox = lobjMember.postOfficeBox Else lstrPostOfficeBox = "" End If If Not (isempty(lobjMember.l)) Then lstrCity = lobjMember.l Else lstrCity = "" End If
    • If Not (isempty(lobjMember.st)) Then lstrState = lobjMember.st Else lstrState = "" End If If Not (isempty(lobjMember.postalCode)) Then lstrPostalCode = lobjMember.postalCode Else lstrPostalCode = "" End If If Not (isempty(lobjMember.countryCode)) Then lstrCountry = lobjMember.countryCode Else lstrCountry = "" End If lstrPassword = DefaultPassword lblnInZimbraGroup = 0 For Each lobjGroup in lobjMember.Groups '** See if this member belongs to the group that allows Zimbra mailboxes ** If LCase(lobjGroup.cn) = LCase(ZimbraSecurityGroup) Then lblnInZimbraGroup = 1 End If Next If lblnInZimbraGroup = 1 Then '** Member is associated to the Zimbra Email group and thus allowed to have a Zimbra mailbox. ** lstrLine = lstrSamAccountName & "," & lstrFirstName & "," & lstrInitials & "," & lstrLastName & "," & lstrFullName & "," &_ lstrTitle & "," & lstrDescription & "," & lstrComment & "," & lstrTelephoneNumber & "," & lstrHomePhone & "," & lstrMobile & "," &_ lstrFaxNumber & "," & lstrPager & "," & lstrCompany & "," & lstrOffice & "," & lstrStreetAddress & "," & lstrPostOfficeBox & "," &_ lstrCity & "," & lstrState & "," & lstrPostalCode & "," & lstrCountry & "," & lstrPassword & ",unused" lobjFile.WriteLine(lstrLine) End If lobjFile.Close Set lobjFile = Nothing End If If lobjMember.Class = "organizationalUnit" or lobjMember.Class = "container" Then '** Recurse further down to find the users. ** TrollTheFolders(lobjMember) End If Next End Sub Sample adlist.csv Code: mmouse,Mickey,,Mouse,Mickey Mouse,,Lab Assistant,,,,,,800-8675309,,,,,,,,0,ChangeThisPassword,unused jdirt,Joe,,Dirt,Joe Dirt,Chief Information Officer,Chief Information Officer,,800-555-5555,,,,,ABC,Information Systems,404 Hollywood Street,,Hollywood,CA,90210,840,ChangeThisPassword,unused ddiggler,Dirk,,Diggler,Dirk Diggler,Database Administrator,IS-Database Administrator,,800-555-0404,800-789-1234,888-111-2222,,,,Information Systems,404 Cowboy Country Road,,Hickville,TX,77501,840,ChangeThisPassword,unused jdoe,John,,Doe,John Doe,System Analyst,IS-Telco Systems Analyst,,800-9998888,,,,,,Information Systems,666 Gov Road,,Washington,WA,10110,840,ChangeThisPassword,unused import-ad.sh Code: #!/bin/bash
    • ############################################# ## Name : import-ad.sh ## Version : 1.2 ## Date : 2011-11-02 ## Author : LHammonds ## Purpose : Add mailbox accounts for AD users that have none. ## Compatibility : Verified on Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 - 7.2.0 OSE ## Requirements : Zimbra must be online, must be run as root user. ## Run Frequency : Run very frequently (every minute) ## Exit Codes : ## 0 = Success ## 1 = Failure ################ CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2011-10-11 LTH Created script. ## 2011-10-29 LTH Better logging, mail support, bug fixes. ## 2011-11-02 LTH Moved standard variables/functions to external file. ############################################# ## Import standard variables and functions. ## source /var/scripts/common/standard.conf LOGFILE="${TEMPDIR}/import-ad.log" MAILFILE="${TEMPDIR}/import-ad-mail.$$" ADLISTORG="${SHAREDIR}/adlist.csv" ADLISTNEW="${TEMPDIR}/import-ad-adlist.$$" ZMUSERS="${TEMPDIR}/import-ad-zmuserlist.$$" ZMCMD="${TEMPDIR}/import-ad-zmcmd.$$" NEWUSERS="" RETURNVALUE=0 ## Temporarily change the default field separator to a comma. # OLDIFS="${IFS}" IFS="," ## If the Active Directory user list file is not found, exit the script. # if [ ! -f ${ADLISTORG} ]; then ## No file to process. Exit script. exit 0 fi echo "`date +%Y-%m-%d_%H:%M:%S` - Active Directory file found! Import process started." >> ${LOGFILE} ## Create files and set permissions for only the root user. # touch ${ZMCMD} touch ${ZMUSERS} chmod 0600 ${ZMCMD} chmod 0600 ${ZMUSERS} ## Take ownership and set permissions for only the root user. # chown root:root ${ADLISTORG} chmod 0400 ${ADLISTORG} ## Move the file to a working folder. # mv ${ADLISTORG} ${ADLISTNEW} ## Create a list of users in Zimbra for the specified domain. # ${ZIMBRADIR}/bin/zmprov -l getAllAccounts ${MYDOMAIN} > ${ZMUSERS} ## Loop through the Active Directory user list. # while read -a ADLINE ; do
    • ## Slot 0 = UserID ## Slot 1 = First Name (GivenName) ## Slot 2 = Initials ## Slot 3 = Last Name (sn) ## Slot 4 = Full Name (CN) ## Slot 5 = Title ## Slot 6 = Description ## Slot 7 = Comment ## Slot 8 = Telephone / Work Number ## Slot 9 = Home Phone ## Slot 10 = Mobile Phone ## Slot 11 = Fax Number (facsimileTelephoneNumber) ## Slot 12 = Pager ## Slot 13 = Company ## Slot 14 = Office (physicalDeliveryOfficeName) ## Slot 15 = Street Address ## Slot 16 = Post Office Box ## Slot 17 = City (l) ## Slot 18 = State (st) ## Slot 19 = Postal Code ## Slot 20 = Country Code (co) ## Slot 21 = Password ## Set the "Found Match" variable to false # FOUND=0 ## Loop through the Zimbra User List. # while read -a ZMLINE ; do ## Convert Usernames to lower case. # ZMUSERID=${ZMLINE[0],,} ADUSERID=${ADLINE[0],,} ## Add domain address to the end of the AD Username # ADUSERID=${ADUSERID}"@"${MYDOMAIN} ## Compare the lowercase AD username to the lower case Zimbra username. # if [ ${ADUSERID} == ${ZMUSERID} ]; then ## Match found. Exit this while loop. # FOUND=1 break fi done < ${ZMUSERS} if [ ${FOUND} -eq 0 ]; then ## We have an AD user that does not have a Zimbra mailbox. # NEWUSERS="${NEWUSERS}${ADLINE[0]}@${MYDOMAIN}n" echo "`date +%Y-%m-%d_%H:%M:%S` --- Adding new user: ${ADLINE[0]}@${MYDOMAIN}" >> ${LOGFILE} echo "createAccount ${ADLINE[0]}@${MYDOMAIN} "${ADLINE[21]}" gn "${ADLINE[1]}" initials "${ADLINE[2]}" sn "${ADLINE[3]}" cn "${ADLINE[4]}" title "${ADLINE[5]}" description "${ADLINE[6]}" zimbraNotes "${ADLINE[7]}" telephoneNumber "${ADLINE[8]}" homePhone "${ADLINE[9]}" mobile "${ADLINE[10]}" facsimileTelephoneNumber "${ADLINE[11]}" pager "${ADLINE[12]}" company "${ADLINE[13]}" physicalDeliveryOfficeName "${ADLINE[14]}" street "${ADLINE[15]}" postOfficeBox "${ADLINE[16]}" l "${ADLINE[17]}" st "${ADLINE[18]}" postalCode "${ADLINE[19]}" co "${ADLINE[20]}"">> ${ZMCMD} fi done < ${ADLISTNEW} ## If the Zimbra command file is not empty, we need to process it. # if [ -s ${ZMCMD} ]; then ${ZIMBRADIR}/bin/zmprov < ${ZMCMD} 1>/dev/null 2>&1 RETURNVALUE=$?
    • if [ ${RETURNVALUE} -ne 0 ]; then ## Something went wrong with the mailbox creation. ## This error should notify administrators to the problem. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: zmprov reported the following error: ${RETURNVALUE}" >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` - Import aborted. EXIT CODE = 1" >> ${LOGFILE} f_sendmail "Zimbra Active Directory Import" "ERROR: zmprov reported the following error: ${RETURNVALUE}nnRemaining temporary files:n${ZMUSERS}n${ZMCMD}n${ADLISTNEW}n" exit 1 fi ## Send email notification of the new users added to Zimbra. # ## NOTE: To be 100% sure about this, we could loop through the users # ## and verify that they now have Zimbra mailboxes. # f_sendmail "Zimbra Notification - New Users" "The following users were added to Zimbra:nn${NEWUSERS}" else ## There were no new users to process. # echo "`date +%Y-%m-%d_%H:%M:%S` --- No new users to import." >> ${LOGFILE} fi echo "`date +%Y-%m-%d_%H:%M:%S` - Active Directory import completed." >> ${LOGFILE} ## Restore the default field separator value. # IFS="${OLDIFS}" ## rm rm rm Remove temporary files. # ${ZMUSERS} ${ZMCMD} ${ADLISTNEW} Sample email when new mailboxes are created: Code: Subject: Zimbra Notification - New Users The following users were added to Zimbra: ddiggler@mydomain.com jdoe@mydomain.com jsmith@mydomain.com lhammonds@mydomain.com mmouse@mydomain.com Server: mail Program: /var/scripts/prod/import-ad.sh Log: /var/temp/import-ad.log Here is a sample log file (NOTE: There are no entries if no file is found): Code: 2011-10-29_17:06:49 2011-10-29_17:06:52 2011-10-29_17:06:53 2011-10-29_17:06:53 2011-10-29_17:06:53 2011-10-29_17:06:53 - Active Directory file found! Import process started. --- Adding new user: ddiggler@mydomain.com --- Adding new user: jdoe@mydomain.com --- Adding new user: jsmith@mydomain.com --- Adding new user: lhammonds@mydomain.com --- Adding new user: mmouse@mydomain.com
    • 2011-10-29_17:06:56 2011-10-29_17:07:29 2011-10-29_17:07:32 2011-10-29_17:07:32 - Active Directory import completed. - Active Directory file found! Import process started. --- No new users to import. - Active Directory import completed. If you keep a database of all your user's IDs and passwords, you could export that information to the Linux box and have it safely tucked away and used in-between the AD Export and AD Import process. It could run through the AD user list and replace the default password with their existing password making it operate seamlessly within the environment. If user passwords are unknown, the process would be to let the individual know their mailbox was created with a standard password and that they need to login and change it ASAP. Reference Articles: zmprov command-line documentation Bash Guide NOTE: If you like this solution, you might also want to review the modified version by digidt in my older thread. __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #10 (permalink) 04-26-2012, 07:50 PM LHammonds Special Member Posts: 138 Global Address List for iPhones Global Address List for iPhones I have a working solution in place for our iPhone users to access the Internal Global Address List (GAL) when they start a new email message and click the "To" button to show a list of all contacts in our company (not just personal contacts). Normally, you will only get a list of contacts in your personal folder. I have tried various ways to get this to work such as sharing the _InternalGAL folder but could not get anything to work well. I certainly do not want to give my users the credentials to the galsync account (which is my admin account). This method uses a service account (dummy mailbox) to hold a copy of the GAL. Each iPhone user then adds an additional CardDAV account and enters the shared account's details. For this example, I created a mailbox called "addresslist@mydomain.com" and then logged
    • into that account and shared the contacts folder as "Public" for anyone to access. Next, login to the admin console, expand Configuration --> Domains and click on mydomain.com. Click the Configure GAL button. Set the following and click Next, Finish: GAL mode: Internal Most results returned by GAL search: 250 Use GAL Sync account (recommended): Check GAL sync account name: admin@mydomain.com (or whatever galsync account you want) Datasource name for internal GAL: InternalGAL Internal GAL polling interval: 1 days Next, force the initial sync by typing the following at the Ubuntu console as the root user: Code: su - zimbra -c "zmgsautil forceSync -a admin@mydomain.com -n InternalGAL" If you login to the web mail client and login with the galsync (admin) account, you should see the "_InternalGAL" folder under the Address Book tab and it should be populated with your internal users. Before the account can be backed up with the script below, the services need to be restarted. Code: /etc/init.d/zimbra stop /etc/init.d/zimbra start Now setup the script below to run on a regular schedule which will backup the GAL Sync's "_InternalGAL" folder to an archive. It will then extract the archive and change references of "_InternalGAL" to "Contacts" in all files so it can be restored in a folder that the iPhone will recognize. Then it is re-packed into an archive and restored to the "addresslist" account which completely replaces whatever is in the "Contacts" folder with the updated list. As an additional step, it will delete specific addresses which are not desired to show up in the global list. Also note that this is not necessary on the Pro Network Edition which has an iPhone sync utility. gal-sync.sh Code: #!/bin/bash ############################################# ## Name : gal-sync.sh ## Version : 1.1 ## Date : 2011-11-02 ## Author : LHammonds ## Compatibility : Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.3 7.2.0 OSE ## Requirements : Zimbra must be online, must be run as root user. ## Purpose : Expose Internal GAL (Global Address List) for use on
    • iPhones ## Run Frequency : Once per day (after hours preferable but not necessary) ## Exit Codes : (if multiple errors, value is the addition of codes) ## 0 = success ## 1 = security failure ## 2 = zmmailbox getRestURL failure ## 4 = zmmailbox postRestURL failure ## 8 = zmmailbox getAllContacts failure ## 16 = tar extraction failure ## 32 = tar creation failure ################# CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2011-10-28 LTH Created script. ## 2011-10-31 LTH Added check for root ID. ## 2011-11-02 LTH Moved standard variables / functions to external file. ## NOTE: Should add detection of address list account. ############################################## ## Import common variables and functions. ## source /var/scripts/common/standard.conf GALSYNCACCT="admin@${MYDOMAIN}" PUBLICGALACCT="addresslist@${MYDOMAIN}" EXCEPTIONS="addresslist@${MYDOMAIN};admin@${MYDOMAIN};test@${MYDOMAIN};test 2@${MYDOMAIN}" GALSYNCDIR="${TEMPDIR}/galsync" LOGFILE="${TEMPDIR}/gal-sync.log" CONTACTFILE="${TEMPDIR}/gal-sync-contact.$$" GALARCHIVE1="galsync1.tgz" GALARCHIVE2="galsync2.tgz" CONTACTNUMDEL=0 CONTACTNUMALL=0 RETURNVALUE=0 ERRORFLAG=0 ####################################### ## FUNCTIONS ## ####################################### function f_cleanup() { ## Purpose: Cleanup files and folders used by the script. if [ -d ${GALSYNCDIR} ]; then rm -rf ${GALSYNCDIR} fi if [ -f ${TEMPDIR}/${GALARCHIVE1} ]; then rm ${TEMPDIR}/${GALARCHIVE1} fi if [ -f ${TEMPDIR}/${GALARCHIVE2} ]; then rm ${TEMPDIR}/${GALARCHIVE2} fi if [ -f ${CONTACTFILE} ]; then rm ${CONTACTFILE} fi } function f_emergencyexit()
    • { ## Purpose: Exit script as cleanly as possible. ## Parameter #1 = Error Code. f_cleanup f_sendmail "Zimbra iPhone GAL Sync Error" "EXIT CODE = ${1}" echo "`date +%Y-%m-%d_%H:%M:%S` - GAL Sync aborted. EXIT CODE: ${1}" >> ${LOGFILE} ## Write script name and error code to the system log. logger "${SCRIPTNAME}: ERROR CODE = ${1}" exit $1 } ####################################### ## MAIN PROGRAM ## ####################################### echo "`date +%Y-%m-%d_%H:%M:%S` - GAL Sync started." >> ${LOGFILE} ## Requirement Check: Script must run as root user. if [ "$(id -u)" != "0" ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. echo "`date +%Y-%m-%d_%H:%M:%S` - GAL Sync aborted. ERROR: Root user required." >> ${LOGFILE} ERRORFLAG=$((${ERRORFLAG} + 1)) f_emergencyexit ${ERRORFLAG} fi ## Backup the global address listing. ${ZIMBRADIR}/bin/zmmailbox -z -m ${GALSYNCACCT} getRestURL "/_InternalGAL?fmt=tgz" > ${TEMPDIR}/${GALARCHIVE1} RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: ${ZIMBRADIR}/bin/zmmailbox -z -m ${GALSYNCACCT} getRestURL '/_InternalGAL?fmt=tgz'" >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` --- Return Value: ${RETURNVALUE}" >> ${LOGFILE} ERRORFLAG=$((${ERRORFLAG} + 2)) f_emergencyexit ${ERRORFLAG} fi ## Prepare the extraction folder and set appropriate permissions. mkdir -p ${GALSYNCDIR} chmod 0600 ${GALSYNCDIR} ## Extract the archive to modify its contents. tar -xf ${TEMPDIR}/${GALARCHIVE1} -C ${GALSYNCDIR} 1>/dev/null 2>&1 RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: tar -xf ${TEMPDIR}/${GALARCHIVE1} -C ${GALSYNCDIR}" >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` --- Return Value: ${RETURNVALUE}" >> ${LOGFILE} ERRORFLAG=$((${ERRORFLAG} + 16)) f_emergencyexit ${ERRORFLAG} fi ## Search and replace all meta data occurances of "_InternalGAL" with "Contacts" ## This will allow the contacts to be restored to the "Contacts" folder.
    • sed -i 's/_InternalGAL/Contacts/g' ${GALSYNCDIR}/*.meta sed -i 's/_InternalGAL/Contacts/g' ${GALSYNCDIR}/_InternalGAL/*.meta ## Count the number of meta files (contacts) in the GAL folder. CONTACTNUMALL=`ls -1 ${GALSYNCDIR}/_InternalGAL/*.meta | wc -l` ## Create a new archive with the modified data. tar -czf ${TEMPDIR}/${GALARCHIVE2} ${GALSYNCDIR} 1>/dev/null 2>&1 RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: tar -czf ${TEMPDIR}/${GALARCHIVE2} ${GALSYNCDIR}" >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` --- Return Value: ${RETURNVALUE}" >> ${LOGFILE} ERRORFLAG=$((${ERRORFLAG} + 32)) f_emergencyexit ${ERRORFLAG} fi ## Replace contacts in the iPhone GAL account. ${ZIMBRADIR}/bin/zmmailbox -z -m ${PUBLICGALACCT} postRestURL "//?fmt=tgz&resolve=reset" ${TEMPDIR}/${GALARCHIVE2} RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: ${ZIMBRADIR}/bin/zmmailbox -z -m ${PUBLICGALACCT} postRestURL '//?fmt=tgz&resolve=reset'" >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` --- Return Value: ${RETURNVALUE}" >> ${LOGFILE} ERRORFLAG=$((${ERRORFLAG} + 4)) f_emergencyexit ${ERRORFLAG} fi ## Export all contacts to a working file to determine which contacts to remove. ${ZIMBRADIR}/bin/zmmailbox -z -m ${PUBLICGALACCT} getAllContacts > ${CONTACTFILE} RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: ${ZIMBRADIR}/bin/zmmailbox -z -m ${PUBLICGALACCT} getAllContacts" >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` --- Return Value: ${RETURNVALUE}" >> ${LOGFILE} ERRORFLAG=$((${ERRORFLAG} + 8)) f_emergencyexit ${ERRORFLAG} fi ## Obtain list of Contact IDs with matching email addresses in the exception list. while read LINEID LINEVALUE; do if [ "${LINEID}" = "Id:" ]; then CURRID=${LINEVALUE} fi if [ "${LINEID}" = "email:" ]; then ## Determine if this is an email address that we are looking for. if `echo ${EXCEPTIONS} | grep -q "${LINEVALUE}"` then ## Found a contact to remove from the address book. ## Verify that the contact ID is an integer.
    • case ${CURRID} in *[!0-9]*) echo "`date +%Y-%m-%d_%H:%M:%S` --- WARNING: Invalid integer for contact ID detected: ${CURRID}" >> ${LOGFILE} ;; *) CONTACTNUMDEL=$((${CONTACTNUMDEL} + 1)) if [ "${CONTACTIDS}" != "" ]; then ## This is not the 1st contact added to the list. CONTACTIDS="${CONTACTIDS},${CURRID}" else ## This is the 1st contact added to the list (do not add a comma). CONTACTIDS="${CURRID}" fi ;; esac fi fi done < ${CONTACTFILE} if [ "${CONTACTIDS}" != "" ]; then ## Remove any addresses in the address book that match the exceptions. ${ZIMBRADIR}/bin/zmmailbox -z -m ${PUBLICGALACCT} deleteContact ${CONTACTIDS} RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## WARNING DETECTED: Send email notification. echo "`date +%Y-%m-%d_%H:%M:%S` --- WARNING: ${ZIMBRADIR}/bin/zmmailbox -z -m ${PUBLICGALACCT} DeleteContact" >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` --- Return Value: ${RETURNVALUE}" >> ${LOGFILE} f_sendmail "Zimbra iPhone GAL Sync Warning" "WARNING: ${ZIMBRADIR}/bin/zmmailbox -z -m ${PUBLICGALACCT} getAllContacts, RETURN VALUE = ${RETURNVALUE}" fi fi echo "`date +%Y-%m-%d_%H:%M:%S` - GAL Sync completed. Processed = ${CONTACTNUMALL}, removed = ${CONTACTNUMDEL}." >> ${LOGFILE} ## Perform cleanup routine. f_cleanup ## Exit with the combined return code value. exit ${ERRORFLAG} Sample gal-sync.log showing all possible errors and good exits at the end. Code: 2011-10-31_13:40:20 - GAL Sync started. 2011-10-31_13:40:20 - GAL Sync aborted. ERROR: Root user required. 2011-10-31_13:40:20 - GAL Sync aborted. EXIT CODE: 1 2011-10-31_13:41:30 - GAL Sync started. 2011-10-31_13:41:33 --- ERROR: /opt/zimbra/bin/zmmailbox -z -m administrator@mydomain.com getRestURL '/_InternalGAL?fmt=tgz' 2011-10-31_13:41:33 --- Return Value: 444 2011-10-31_13:41:33 - GAL Sync aborted. EXIT CODE: 2 2011-10-31_13:41:43 - GAL Sync started. 2011-10-31_13:41:46 --- ERROR: tar -xf /temp/galsync1.tgz -C /temp/galsync 2011-10-31_13:41:46 --- Return Value: 555
    • 2011-10-31_13:41:46 - GAL Sync aborted. EXIT CODE: 16 2011-10-31_13:41:56 - GAL Sync started. 2011-10-31_13:41:59 --- ERROR: tar -czf /temp/galsync2.tgz /temp/galsync 2011-10-31_13:41:59 --- Return Value: 666 2011-10-31_13:41:59 - GAL Sync aborted. EXIT CODE: 32 2011-10-31_13:42:10 - GAL Sync started. 2011-10-31_13:42:15 --- ERROR: /opt/zimbra/bin/zmmailbox -z -m addresslist@mydomain.com postRestURL '//?fmt=tgz&resolve=reset' 2011-10-31_13:42:15 --- Return Value: 777 2011-10-31_13:42:15 - GAL Sync aborted. EXIT CODE: 4 2011-10-31_13:42:21 - GAL Sync started. 2011-10-31_13:42:28 --- ERROR: /opt/zimbra/bin/zmmailbox -z -m addresslist@mydomain.com getAllContacts 2011-10-31_13:42:28 --- Return Value: 888 2011-10-31_13:42:28 - GAL Sync aborted. EXIT CODE: 8 2011-10-31_13:42:36 - GAL Sync started. 2011-10-31_13:42:45 --- WARNING: /opt/zimbra/bin/zmmailbox -z -m addresslist@mydomain.com DeleteContact 2011-10-31_13:42:45 --- Return Value: 999 2011-10-31_13:42:46 - GAL Sync completed. Processed = 19, removed = 4. 2011-10-31_13:42:51 - GAL Sync started. 2011-10-31_13:43:00 - GAL Sync completed. Processed = 19, removed = 4. 2011-10-31_13:47:05 - GAL Sync started. 2011-10-31_13:47:14 - GAL Sync completed. Processed = 19, removed = 4. Sample error email notification: Code: Subject: Zimbra iPhone GAL Sync Error EXIT CODE = 32 Server: mail Program: /var/scripts/prod/gal-sync.sh Log: /var/temp/gal-sync.log Sample warning email notification: Code: Subject: Zimbra iPhone GAL Sync Warning WARNING: /opt/zimbra/bin/zmmailbox -z -m addresslist@[COLOR="red"]mydomain.com[/URL] getAllContacts, RETURN VALUE = 999 Server: mail Program: /var/scripts/prod/gal-sync.sh Log: /var/temp/gal-sync.log Sample of /var/log/messages: Code: Oct 1 Oct Oct Oct Oct 31 13:40:20 mail lhammonds: /var/scripts/prod/gal-sync.sh: ERROR CODE = 31 31 31 31 13:41:33 13:41:46 13:41:59 13:42:15 mail mail mail mail root: root: root: root: /var/scripts/prod/gal-sync.sh: /var/scripts/prod/gal-sync.sh: /var/scripts/prod/gal-sync.sh: /var/scripts/prod/gal-sync.sh: ERROR ERROR ERROR ERROR CODE CODE CODE CODE = = = = 2 16 32 4
    • Oct 31 13:42:28 mail root: /var/scripts/prod/gal-sync.sh: ERROR CODE = 8 This is how I setup an iPhone to access personal emails: 1. Click the Settings icon 2. Select Mail, Contacts, Calendars, Add Account..., Other, Add Mail Account 3. Set the following and click Next: Name: John Doe Address: JDoe@mydomain.com Password: ****** Description: JDoe @ mydomain 4. Select IMAP, set the following and click Next, Finish Incoming Hostname: mail.mydomain.com Incoming User Name: JDoe Incoming Password: ***** Outgoing Hostname: mail.mydomain.com Outgoing User Name: JDoe Outgoing Password: ***** to access personal calendars: 1. Click the Settings icon 2. Select Mail, Contacts, Calendars, Add Account..., Other, Add CalDAV Account 3. Set the following and click Next: Server: mail.mydomain.com User Name: JDoe Password: ****** Description: Personal Calendars @ mydomain to access personal contacts: 1. Click the Settings icon 2. Select Mail, Contacts, Calendars, Add Account..., Other, Add CardDAV Account 3. Set the following and click Next: Server: mail.mydomain.com User Name: JDoe Password: ****** Description: Personal Contacts @ mydomain to access the Global Address List: 1. Click the Settings icon 2. Select Mail, Contacts, Calendars, Add Account..., Other, Add CardDAV Account 3. Set the following and click Next:
    • Server: mail.mydomain.com User Name: addresslist Password: ****** Description: Global Address List @ mydomain References: GAL Sync Account (wiki) LHammonds Special Member Posts: 138 Backup Zimbra Backup There are various methods of backup that can be utilized but these are the ones I will be using: 1. Backup to local folder using RSync 2. Backup individual mailboxes 3. Backup using LVM snapshots If I have time, I will also explore the setup of an additional server to serve as a hot-spare soto-speak. I'd like to see a backup server configured to receive backups from the production server and in the event the production server goes dark, spin up the backup server to be its replacement. For the 1st method of using RSync, there is one major catch using RSync alone in that you cannot backup files currently in use. That means you have to shutdown the services in order to get a complete backup. With this in mind, my solution is to utiliize two scripts. The 1st script will backup what it can while the server is "online" and the 2nd script will run during off-hours to shutdown the services (make it offline) and perform a complete backup.
    • The online script will be scheduled to run multiple times throughout the day and will not cause any downtime. The offline script will be scheduled once per day at a time that will cause the least amount of impact because the server will go offline for about 3 to 5 minutes. It will then compress the backup into a single archive and move it to an offsite server. My offsite storage will be on a Windows 2008 server. The 1st thing to do will be the configuration of the Windows server to allow the Ubuntu server to connect and manage files. Backup - RSync while online rsync-online.sh Code: #!/bin/bash ############################################# ## Name : rsync-online.sh ## Version : 1.1 ## Date : 2011-11-04 ## Author : LHammonds ## Compatibility : Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 7.2.0 OSE ## Purpose : Partial backup while Zimbra is online. ## Run Frequency : Several times throughout the day. ## Exit Codes : (if multiple errors, value is the addition of codes) ## 0 = success ## 1 = rsync failure ################ CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2011-10-21 LTH Created script. ## 2011-11-04 LTH Moved common variables and functions to external file. ############################################# ## Import common variables and functions. ## source /var/scripts/common/standard.conf LOGFILE="${TEMPDIR}/rsync.log" TARGET="${BACKUPDIR}/zimbra" LOCKFILE="${TEMPDIR}/rsync.lock" ERRORFLAG=0 RETURNVALUE=0 ####################################### ## FUNCTIONS ## ####################################### function f_cleanup() {
    • if [ -f ${LOCKFILE} ];then ## Remove lock file so other rsync jobs can run. rm ${LOCKFILE} 1>/dev/null 2>&1 fi echo "`date +%Y-%m-%d_%H:%M:%S` - Online RSync backup exit code: ${ERRORFLAG}" >> ${LOGFILE} } ####################################### ## MAIN PROGRAM ## ####################################### if [ -f ${LOCKFILE} ]; then ## rsync lock file detected. Abort script. f_sendmail "Online RSync Backup Aborted - Lock File" "This script tried to run but detected the lock file: ${LOCKFILE}nnPlease check to make sure the file does not remain when rsync is not actually running." exit 1 else echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE} fi echo "`date +%Y-%m-%d_%H:%M:%S` - Online RSync backup mode started." >> ${LOGFILE} ## Output the current version to a text file which will be included in the backup. ## NOTE: You can also view /opt/zimbra/.install_history su - zimbra -c "zmcontrol -v > ${ZIMBRADIR}/conf/zimbra_version.txt" ## rsync will copy what it can to the target location. rsync -apogHK --delete --exclude=*.pid ${ZIMBRADIR} ${TARGET} 1>/dev/null 2>&1 RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## ERROR: Send email notification. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Online RSync failed. ${ZIMBRADIR} -> ${TARGET}" >> ${LOGFILE} f_sendmail "Online RSync Backup Failure - rsync" "ERROR: Online rsync failed. ${ZIMBRADIR} -> ${TARGET}, RETURN VALUE = ${RETURNVALUE}" ERRORFLAG=${ERRORFLAG} + 1 fi ## Document statistics. echo "`date +%Y-%m-%d_%H:%M:%S` - Disk usage for both source and target:" >> ${LOGFILE} echo "`du -sh ${ZIMBRADIR}`" >> ${LOGFILE} echo "`du -sh ${TARGET}`" >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` - Online RSync backup mode completed." >> ${LOGFILE} ## Perform cleanup routine. f_cleanup ## Exit with the combined return code value. exit ${ERRORFLAG} __________________
    • Full Offline Backup Backup - RSync while offline Although this backup takes the services down, the RSync job just prior to taking the services down ensures the vast majority of the data is already backed up. The only thing that has to be done after the services are down is to copy the changes in the last minute and the files that could not be copied while the services were running. The actual downtime is typically between 3 to 5 minutes of my server and I doubt it will require longer downtime time even as the data store increases. rsync-offline.sh Code: #!/bin/bash ############################################# ## Name : rsync-offline.sh ## Version : 1.4 ## Date : 2012-01-09 ## Author : LHammonds ## Purpose : Complete backup while Zimbra is offline. ## Compatibility : Verified on Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 - 7.2.0 OSE ## Requirements : p7zip-full (if ARCHIVEMETHOD=tar.7z) ## Run Frequency : Once per day after hours. ## Exit Codes : (if multiple errors, value is the addition of codes) ## 0 = success ## 1 = rsync failure ## 2 = archive failure ## 4 = archive purge failure ## 8 = configuration error ## 16 = crontab backup error ## 32 = crontab restore warning ## 64 = mount warning ################ CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2011-10-21 LTH Created script. ## 2011-10-28 LTH Added 7-Zip compression (gunzip optional) ## 2011-11-04 LTH Move common variables and funtions to external file. ## 2011-11-08 LTH Added zimbra subfolder to remote site for organization. ## 2012-01-09 LTH Bugfix for f_PurgeOldestArchive ############################################# ## Import common variables and functions. ## source /var/scripts/common/standard.conf LOGFILE="${TEMPDIR}/rsync.log" TEMPLOG="${TEMPDIR}/rsync-offline-temp.$$" LOCKFILE="${TEMPDIR}/rsync.lock" TARGETDIR="${BACKUPDIR}/zimbra" OFFSITEBACKDIR="${OFFSITEDIR}/zimbra" ARCHIVEFILE="`date +%Y-%m-%d-%H-%M`_zimbra-backup.${ARCHIVEMETHOD}" ZMCRONFILE="${TEMPDIR}/zimbra-crontab.$$" ERRORFLAG=0
    • ####################################### ## FUNCTIONS ## ####################################### function f_PurgeOldestArchive() { ## Purpose: Delete the oldest archive on the remote site. ## Return values: ## 0 = Success ## 1 = Cannot delete file ## 9 = Configuration error, path empty ## Variable Error Check. * if [ ${OFFSITEBACKDIR} = "" ]; then ## Make darn sure the path is not empty since we do NOT ## want to start purging files from a random location. echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OFFSITEBACKDIR site variable is empty!" >> ${TEMPLOG} return 9 fi ## Get the name of the oldest file. OLDESTFILE=`ls -1t ${OFFSITEBACKDIR} | tail -1` FILESIZE=`ls -la ${OFFSITEDIR}/${OLDFILE} | awk '{print $5}' | sed -e :a -e 's/(.*[0-9])([0-9]{3})/1,2/;ta'` if [ "${OLDESTFILE}" = "" ]; then ## Error. Filename variable empty. echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OLDESTFILE variable is empty." >> ${TEMPLOG} return 9 else FILESIZE=`ls -lak "${OFFSITEBACKDIR}/${OLDESTFILE}" | awk '{ print $5 }' | sed -e :a -e 's/(.*[0-9])([0-9]{3})/1,2/;ta'` echo "`date +%Y-%m-%d_%H:%M:%S` --- Purging old file: ${OFFSITEBACKDIR}/${OLDESTFILE}, Size = ${FILESIZE} kb" >> ${TEMPLOG} rm "${OFFSITEBACKDIR}/${OLDESTFILE}" if [ -f "${OFFSITEBACKDIR}/${OLDESTFILE}" ]; then ## File still exists. Return error. return 1 else return 0 fi fi } function f_cleanup() { echo "`date +%Y-%m-%d_%H:%M:%S` - Offline RSync backup exit code: ${ERRORFLAG}" >> ${TEMPLOG} ## Copy temporary log file into main log file. cat ${TEMPLOG} >> ${LOGFILE} if [ -f ${LOCKFILE} ];then ## Remove lock file so other rsync jobs can run. rm ${LOCKFILE} 1>/dev/null 2>&1 fi ## Email the log to the administrator. EMAILBODY="" while read FILELINE; do EMAILBODY=${EMAILBODY}${FILELINE}"n" done < ${TEMPLOG} f_sendmail "Offline RSync Backup Log" "${EMAILBODY}"
    • if [ -f ${TEMPLOG} ]; then rm ${TEMPLOG} fi } function f_emergencyexit() { ## Purpose: Exit script as cleanly as possible. ## Parameter #1 = Error Code f_cleanup exit $1 } ####################################### ## MAIN PROGRAM ## ####################################### if [ -f ${LOCKFILE} ]; then # rsync lock file detected. Abort script. f_sendmail "Offline RSync Backup Aborted - Lock File" "This script tried to run but detected the lock file: ${LOCKFILE}nnPlease check to make sure the file does not remain when rsync is not actually running." exit 1 else echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE} fi echo "`date +%Y-%m-%d_%H:%M:%S` - Offline RSync backup started." >> ${TEMPLOG} ## If the 7-Zip archive method is specified, make sure the package is installed. if [ "${ARCHIVEMETHOD}" = "tar.7z" ]; then if [ ! -f "/usr/bin/7za" ]; then ## Required package (7-Zip) not installed. echo "`date +%Y-%m-%d_%H:%M:%S` - CRITICAL ERROR: 7-Zip package not installed. Please install by typing 'aptitude -y install p7zip-full'" >> ${TEMPLOG} ERRORFLAG=2 f_emergencyexit ${ERRORFLAG} fi fi echo "`date +%Y-%m-%d_%H:%M:%S` --- Partition status:" >> ${TEMPLOG} df -h >> ${TEMPLOG} ## Output the current version to a text file to be included in the backup. ## NOTE: You can also view /opt/zimbra/.install_history su - zimbra -c "zmcontrol -v > ${ZIMBRADIR}/conf/zimbra_version.txt" echo "`date +%Y-%m-%d_%H:%M:%S` --- Begin pre-shutdown rsync command." >> ${TEMPLOG} rsync -apogHK --delete --exclude=*.pid ${ZIMBRADIR} ${TARGETDIR} 1>/dev/null 2>&1 ## Backup the zimbra crontab schedule. echo "`date +%Y-%m-%d_%H:%M:%S` --- Backup and disable Zimbra crontab schedule." >> ${TEMPLOG} crontab -u zimbra -l > ${ZMCRONFILE} RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then
    • ## Something went wrong with the backup of the crontab file. ## This error will be considered fatal and terminate the script. f_sendmail "RSync Offline Backup Error - Crontab" "Could not backup crontab for the zimbra user account. RETURN VALUE = ${RETURNVALUE}" ERRORFLAG=$((${ERRORFLAG} + 16)) f_emergencyexit ${ERRORFLAG} else ## Backup successful, now disable the zimbra crontab schedule. echo "" > ${TEMPDIR}emptyfile.$$ crontab -u zimbra ${TEMPDIR}emptyfile.$$ rm ${TEMPDIR}emptyfile.$$ 1>/dev/null 2>&1 ## Add a copy of the crontab file to the Zimbra folder to be backed up. cp ${ZMCRONFILE} ${ZIMBRADIR} 1>/dev/null 2>&1 fi StartTime="$(date +%s)" ## Stop Zimbra Services. echo "`date +%Y-%m-%d_%H:%M:%S` --- Stopping Zimbra services." >> ${TEMPLOG} /etc/init.d/zimbra stop 1>/dev/null 2>&1 ## Kill any orphaned Zimbra processes. pkill -9 -u zimbra 1>/dev/null 2>&1 echo "`date +%Y-%m-%d_%H:%M:%S` --- Space consumed in ${ZIMBRADIR} = `du sh ${ZIMBRADIR} | awk '{ print $1 }'`" >> ${TEMPLOG} ## Now that Zimbra is offline, rsync can quickly copy all remaining files to the backup target. echo "`date +%Y-%m-%d_%H:%M:%S` --- Begin post-shutdown rsync command." >> ${TEMPLOG} rsync -apogHK --delete --exclude=*.pid ${ZIMBRADIR} ${TARGETDIR} 1>/dev/null 2>&1 RETURNVALUE=$? ## Quickly start the Zimbra services. echo "`date +%Y-%m-%d_%H:%M:%S` --- Starting Zimbra services." >> ${TEMPLOG} /etc/init.d/zimbra start 1>/dev/null 2>&1 if [ ${RETURNVALUE} -ne 0 ]; then ## rsync command failed. Send warning email. f_sendmail "RSync Offline Backup Failure - rsync" "rsync failed with return value of ${RETURNVALUE}" ERRORFLAG=$((${ERRORFLAG} + 1)) fi ## Calculates downtime of Zimbra services. FinishTime="$(date +%s)" ElapsedTime="$(expr ${FinishTime} - ${StartTime})" Hours=$((${ElapsedTime} / 3600)) ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600)) Minutes=$((${ElapsedTime} / 60)) Seconds=$((${ElapsedTime} - ${Minutes} * 60)) echo "`date +%Y-%m-%d_%H:%M:%S` --- Zimbra downtime: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" >> ${TEMPLOG} ## Restore the zimbra crontab schedule.
    • echo "`date +%Y-%m-%d_%H:%M:%S` --- Restore Zimbra crontab schedule." >> ${TEMPLOG} crontab -u zimbra ${ZMCRONFILE} RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## Could not restore the zimbra crontab schedule. ## This is considered a non-fatal error and the script will continue. f_sendmail "Offline RSync Backup Warning - Crontab" "Could not restore the zimbra crontab schedule using ${ZMCRONFILE} RETURN VALUE = ${RETURNVALUE}" ERRORFLAG=$((${ERRORFLAG} + 32)) else ## Zimbra crontab schedule restored. Remove temp file. rm ${ZMCRONFILE} fi echo "`date +%Y-%m-%d_%H:%M:%S` --- Space consumed in ${TARGETDIR} = `du sh ${TARGETDIR} | awk '{ print $1 }'`" >> ${TEMPLOG} ## Show status of Zimbra services. echo "`date +%Y-%m-%d_%H:%M:%S` --- Status of Zimbra services:" >> ${TEMPLOG} su - zimbra -c "zmcontrol status" >> ${TEMPLOG} ## Compress the backup into a single file based on archive method specified. echo "`date +%Y-%m-%d_%H:%M:%S` --- Compressing archive: ${TEMPDIR}/${ARCHIVEFILE}" >> ${TEMPLOG} case "${ARCHIVEMETHOD}" in tar.7z) tar -cpf - ${TARGETDIR} | 7za a -si -mx=9 -w${TEMPDIR} ${TEMPDIR}/${ARCHIVEFILE} 1>/dev/null 2>&1 ## Restore using one of the following commands (do not uncomment, only for notation): ## 7za x -so -w/tmp ${TEMPDIR}/${ARCHIVEFILE} | tar -C / -xf ## 7za x -so -w/tmp ${TEMPDIR}/${ARCHIVEFILE} | tar -C ${TEMPDIR}/restore --strip-components=1 -xf ;; tgz) tar -cpzf ${TEMPDIR}/${ARCHIVEFILE} ${TARGETDIR} 1>/dev/null 2>&1 ## Restore using one of the following commands (do not uncomment, only for notation): ## tar -C / -xzf ${TEMPDIR}/${ARCHIVEFILE} ## tar -C ${TEMPDIR}/restore --strip-components=1 -xzf ${TEMPDIR}/${ARCHIVEFILE} ;; *) tar -cpzf ${TEMPDIR}/${ARCHIVEFILE} ${TARGETDIR} 1>/dev/null 2>&1 ;; esac RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## tar command failed. Send warning email. f_sendmail "Offline RSync Backup Failure - tar" "tar failed with return value of ${RETURNVALUE}" ERRORFLAG=$((${ERRORFLAG} + 2)) fi ## Mount the Windows share folder. ##
    • f_mount sleep 2 if [ ! -f ${OFFSITETESTFILE} ]; then ## Could not find expected file on remote site. Assuming failed mount. ERRORFLAG=$((${ERRORFLAG} + 64)) f_sendmail "Offline RSync Backup - Win Mount Error" "${HOSTNAME} could not detect ${OFFSITETESTFILE}" else ## Remote site is available. FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'` BACKUPSIZE=`ls -lak "${TEMPDIR}/${ARCHIVEFILE}" | awk '{ print $5 }'` ## Make sure space is available on the remote server to copy the file. if [ ${FREESPACE} -lt ${BACKUPSIZE} ]; then ## Not enough free space available. Purge existing backups until there is room. ENOUGHSPACE=0 while [ ${ENOUGHSPACE} -eq 0 ] do f_PurgeOldestArchive RETURNVALUE=$? case ${RETURNVALUE} in 1) ## Cannot purge archives to free up space. End program gracefully. echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Not enough free space on ${OFFSITEDIR} and cannot purge old archives. Script aborted." >> ${TEMPLOG} ## Stop and exit the script with an error code. ERRORFLAG=$((${ERRORFLAG} + 4)) f_emergencyexit ${ERRORFLAG} ;; 9) ## Configuration error, end program gracefully. echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Configuration problem. Script aborted." >> ${TEMPLOG} ## Stop and exit the script with an error code. ERRORFLAG=$((${ERRORFLAG} + 8)) f_emergencyexit ${ERRORFLAG} ;; esac FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'` if [ ${FREESPACE} -gt ${BACKUPSIZE} ]; then ## Enough space is now available. ENOUGHSPACE=1 else ## Not enough space is available yet. ENOUGHSPACE=0 fi done fi ## Copy the backup to an offsite storage location. echo "`date +%Y-%m-%d_%H:%M:%S` --- Copying archive file to offsite location." >> ${TEMPLOG} cp ${TEMPDIR}/${ARCHIVEFILE} ${OFFSITEBACKDIR}/${ARCHIVEFILE} 1>/dev/null 2>&1 if [ ! -f ${OFFSITEBACKDIR}/${ARCHIVEFILE} ]; then ## NON-FATAL ERROR: Copy command did not work. Send email notification. echo "`date +%Y-%m-%d_%H:%M:%S` --- WARNING: Remote copy failed.
    • ${OFFSITEBACKDIR}/${ARCHIVEFILE} does not exist!" >> ${TEMPLOG} f_sendmail "Offline RSync Backup Failure - Remote Copy" "Remote copy failed. ${OFFSITEBACKDIR}/${ARCHIVEFILE} does not existnnBackup file still remains in this location: ${HOSTNAME}:${TEMPDIR}/${ARCHIVEFILE}" else ## Remove local copy of the compressed backup file rm ${TEMPDIR}/${ARCHIVEFILE} fi fi ## Unmount the Windows shared folder. f_umount ## Calculate total time for backup. FinishTime="$(date +%s)" ElapsedTime="$(expr ${FinishTime} - ${StartTime})" Hours=$((${ElapsedTime} / 3600)) ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600)) Minutes=$((${ElapsedTime} / 60)) Seconds=$((${ElapsedTime} - ${Minutes} * 60)) echo "`date +%Y-%m-%d_%H:%M:%S` --- Total backup time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" >> ${TEMPLOG} echo "`date +%Y-%m-%d_%H:%M:%S` - Offline RSync backup completed." >> ${TEMPLOG} ## Perform cleanup routine. f_cleanup ## Exit with the combined return code value. exit ${ERRORFLAG} Example output in the log file: Code: 2011-10-28_17:43:21 - Offline RSync backup started. 2011-10-28_17:43:21 --- Partition status: Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/LVG-root 3842104 1467924 2179008 41% / none 2025164 188 2024976 1% /dev none 2030020 0 2030020 0% /dev/shm none 2030020 296 2029724 1% /var/run none 2030020 0 2030020 0% /var/lock none 2030020 0 2030020 0% /lib/init/rw none 3842104 1467924 2179008 41% /var/lib/ureadahead/debugfs /dev/sda1 182331 33053 139550 20% /boot /dev/mapper/LVG-bak 240306696 3333320 224766480 2% /var/backup /dev/mapper/LVG-opt 120152260 3287176 110761636 3% /opt /dev/mapper/LVG-temp 240306696 799184 227300616 1% /var/temp 2011-10-28_17:43:22 --- Begin pre-shutdown rsync command. 2011-10-28_17:45:10 --- Backup and disable Zimbra crontab schedule. 2011-10-28_17:45:10 --- Stopping Zimbra services. 2011-10-28_17:45:54 --- Space consumed in /opt/zimbra = 3.3G 2011-10-28_17:45:58 --- Begin post-shutdown rsync command. 2011-10-28_17:47:05 --- Starting Zimbra services. 2011-10-28_17:48:32 --- Zimbra downtime: 0 hour(s) 3 minute(s) 22 second(s) 2011-10-28_17:48:32 --- Restore Zimbra crontab schedule. 2011-10-28_17:48:33 --- Space consumed in /temp/zimbra = 3.4G 2011-10-28_17:48:35 --- Status of Zimbra services: Host mail.mydomain.com antispam Running
    • antivirus ldap logger mailbox mta snmp spell stats zmconfigd 2011-10-28_17:48:50 28_zmbackup.tar.7z 2011-10-28_18:10:09 2011-10-28_18:23:38 second(s) 2011-10-28_18:23:39 2011-10-28_18:23:39 Running Running Running Running Running Running Running Running Running --- Compressing archive: /var/temp/2011-10--- Copying archive file to offsite location. --- Total backup time: 0 hour(s) 38 minute(s) 28 - Offline RSync backup completed. - Offline RSync backup exit code: 0 To test the online script, make sure you are root and type the following commands: Code: nohup ./rsync-online.sh & tail -f /var/temp/rsync-online*.log The command above will start the script in the background and will allow it to run even if you log off the system. The tail command will display the last 10 lines of the log. The -f option will keep the command active and continuously show the end of the log...thus you can sit there and just watch the entries show up in the log as they are written (works with any log file). Also note that this is the temporary log file which everything is placed until the very end...which is then copied to the main log file shared between both online and offline scripts. Press CTRL+C to stop watching the log file. Reference: OSE Backup Procedures (multiple) NOTE: Synchronize files with rsync __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #13 (permalink) 04-26-2012, 07:55 PM LHammonds Special Member Individual Mailbox Backup Backup - Individual Mailbox Posts: 138
    • The 2nd method involves backing up individual mailboxes. This process can occur while the server is online and thus will not cause any downtime. It can be run as often as you want but you should consider the performance impact on the server as well as how long each run will take. I plan to schedule it once per day during non-peak hours. mailbox-backup.sh Code: #!/bin/bash ############################################# ## Name : mailbox-backup.sh ## Version : 1.1 ## Date : 2012-01-09 ## Author : LHammonds ## Purpose : Backup individual mailbox accounts. ## Compatibility : Verified on Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 - 7.2.0 OSE ## Requirements : Zimbra must be online, must be run as root user. ## Run Frequency : Once or more per day. ## Exit Codes : (if multiple errors, value is the addition of codes) ## 0 = Success (or non-critical failure) ## 1 = Root access failure ## 2 = Archive creation failure ## 4 = Archive purge error ## 8 = configuration error ## 16 = Offsite mount failure ## 32 = Offsite copy failure ################ CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2011-10-23 LTH Created script. ## 2011-11-05 LTH Move common variables and functions to external file. ## 2011-11-06 LTH Additional error checks and better log info. ## 2012-01-09 LTH Bugfix f_PurgeOldestArchive ############################################# ## Import common variables and functions. source /var/scripts/common/standard.conf ## Define local variables. LOGFILE="${TEMPDIR}/mailbox-backup.log" HISTORYFILE="${TEMPDIR}/mailbox-backup-size-history.log" TARGETDIR="${BACKUPDIR}/mailbox" OFFSITEBACKDIR="${OFFSITEDIR}/mailbox" ARCHIVEFILE="`date +%Y-%m-%d-%H-%M`_mailbox-all.tar" LOCKFILE="${TEMPDIR}/mailbox-backup.lock" EXCEPTIONS="spam.ppnvqogp0@${MYDOMAIN};ham.iki6sotcy@${MYDOMAIN};virusquarantine.qvj6nc_jl@${MYDOMAIN}" RETURNVALUE=0 UCOUNT=0 ERRORFLAG=0 ####################################### ## FUNCTIONS ## ####################################### function f_PurgeOldestArchive() { ## Purpose: Delete the oldest archive on the remote site.
    • ## Return values: ## 0 = Success ## 1 = Cannot delete file ## 9 = Configuration error, path empty ## Variable Error Check. * if [ ${OFFSITEBACKDIR} = "" ]; then ## Make darn sure the path is not empty since we do NOT ## want to start purging files from a random location. echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OFFSITEBACKDIR site variable is empty!" >> ${LOGFILE} return 9 fi ## Get the name of the oldest file. OLDESTFILE=`ls -1t ${OFFSITEBACKDIR} | tail -1` FILESIZE=`ls -la ${OFFSITEDIR}/${OLDFILE} | awk '{print $5}' | sed -e :a -e 's/(.*[0-9])([0-9]{3})/1,2/;ta'` if [ "${OLDESTFILE}" = "" ]; then ## Error. Filename variable empty. echo "`date +%Y-%m-%d_%H:%M:%S` --- Purge error: OLDESTFILE variable is empty." >> ${LOGFILE} return 9 else FILESIZE=`ls -lak "${OFFSITEBACKDIR}/${OLDESTFILE}" | awk '{ print $5 }' | sed -e :a -e 's/(.*[0-9])([0-9]{3})/1,2/;ta'` echo "`date +%Y-%m-%d_%H:%M:%S` --- Purging old file: ${OFFSITEBACKDIR}/${OLDESTFILE}, Size = ${FILESIZE} kb" >> ${LOGFILE} rm "${OFFSITEBACKDIR}/${OLDESTFILE}" if [ -f "${OFFSITEBACKDIR}/${OLDESTFILE}" ]; then ## File still exists. Return error. return 1 else return 0 fi fi } function f_cleanup() { if [ -f "${LOCKFILE}" ]; then ## Remove run file since this script is complete. rm ${LOCKFILE} fi } function f_emergencyexit() { ## Purpose: Exit script as cleanly as possible. ## Parameter #1 = Error Code. f_cleanup f_sendmail "Zimbra Mailbox Backup Error" "EXIT CODE = ${1}" echo "`date +%Y-%m-%d_%H:%M:%S` - Mailbox backup aborted. EXIT CODE: ${1}" >> ${LOGFILE} ## Write script name and error code to the system log. logger "${SCRIPTNAME}: ERROR CODE = ${1}" exit $1 } ####################################### ## MAIN PROGRAM ##
    • ####################################### ## Requirement Check: Script must run as root user. if [ "$(id -u)" != "0" ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. ERRORFLAG=$((${ERRORFLAG} + 1)) echo "`date +%Y-%m-%d_%H:%M:%S` - Mailbox backup aborted. ERROR CODE ${ERRORFLAG}: Script must be run as root user instead of $(whoami)." | tee -a ${LOGFILE} f_emergencyexit ${ERRORFLAG} fi if [ -f "${LOCKFILE}" ]; then ## Last call to this script is still running or failed to remove its lock file. ## As an additional check, see if the file is older than today...if so, we should ## probably send an email notification of a problem that may need manual interention. FILEDATE=$(stat -c %y ${LOCKFILE}) FILEDATE=${FILEDATE%% *} if [ "${FILEDATE}" != "$(date +%Y-%m-%d)" ]; then ## Lock file not created today, might need to be manually deleted. Send email notification. f_sendmail "Zimbra Mailbox Backup Warning" "Warning: This script cannot run if it detects this lock file: ${LOCKFILE}nnThis file should only exist while this script is running which should not take more than a day.nnSystem Date: $(date +%Y-%m-%d)nLock File Date: ${FILEDATE}nnIf you determine that the file should be removed, do so by typing this command on the server's console: rm ${LOCKFILE}" fi exit 0 else ## Create the "script is running" lock file and process the script. echo "`date +%Y-%m-%d_%H:%M:%S` - ${HOSTNAME}:${SCRIPTNAME} is currently running." > ${LOCKFILE} fi ## Record the start time of the backup process. STARTTIME="$(date +%s)" echo "`date +%Y-%m-%d_%H:%M:%S` - Individual mailbox backup started." >> ${LOGFILE} if [ -d "${TARGETDIR}" ]; then ## Purge existing archives. rm ${TARGETDIR}/*.tgz 1>/dev/null 2>&1 else ## Make the folder since it does not exist. mkdir -p ${TARGETDIR} 1>/dev/null 2>&1 fi for ACCT in `su - zimbra -c "zmprov -l gaa"` do ## Check to see if current account should be skipped. if echo "${EXCEPTIONS}" | grep -q ${ACCT} then ## Exception found, skip this account. echo "" > /dev/null else ## Backup user account. UCOUNT=$((UCOUNT+1))
    • ${ZIMBRADIR}/bin/zmmailbox -z -m ${ACCT} getRestURL "//?fmt=tgz" > ${TARGETDIR}/${ACCT}.tgz RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## Something went wrong. echo "`date +%Y-%m-%d_%H:%M:%S` --- Error on ${ACCT}, RETURN VALUE = ${RETURNVALUE}" >> ${LOGFILE} ERRORFLAG=$((ERRORFLAG+1)) else ## Calculate archive size. MAILBOXSIZE=$(stat -c %s ${TARGETDIR}/${ACCT}.tgz) ## Comment out the below line if you do not want details in the log file. echo "`date +%Y-%m-%d_%H:%M:%S` --- ${ACCT}, ${MAILBOXSIZE} bytes" >> ${LOGFILE} fi fi done echo "`date +%Y-%m-%d_%H:%M:%S` --- ${UCOUNT} accounts processed." >> ${LOGFILE} ## Calculate mailbox backup time. FINISHTIME="$(date +%s)" ELAPSEDTIME="$(expr ${FINISHTIME} - ${STARTTIME})" HOURS=$((${ELAPSEDTIME} / 3600)) ELAPSEDTIME=$((${ELAPSEDTIME} - ${HOURS} * 3600)) MINUTES=$((${ELAPSEDTIME} / 60)) SECONDS=$((${ELAPSEDTIME} - ${MINUTES} * 60)) echo "`date +%Y-%m-%d_%H:%M:%S` --- Backup time for ${UCOUNT} mailboxes: ${HOURS} hour(s) ${MINUTES} minute(s) ${SECONDS} second(s)" >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` --- Setting file permissions on ${TARGETDIR}/*.tgz" >> ${LOGFILE} chmod 0600 ${TARGETDIR}/*.tgz echo "`date +%Y-%m-%d_%H:%M:%S` --- Creating a single file for archiving: ${TEMPDIR}/${ARCHIVEFILE}" >> ${LOGFILE} tar -cf ${TEMPDIR}/${ARCHIVEFILE} ${TARGETDIR} 1>/dev/null 2>&1 RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## Something went wrong. echo "`date +%Y-%m-%d_%H:%M:%S` --- Error creating ${TEMPDIR}/${ARCHIVEFILE}, Return Value: ${RETURNVALUE}" >> ${LOGFILE} ERRORFLAG=$((ERRORFLAG+2)) fi ARCHIVESIZE=$(stat -c %s ${TEMPDIR}/${ARCHIVEFILE}) if [ ! -f "${HISTORYFILE}" ]; then ## Create history log and include header description as 1st line, field separator = space character. echo "Date Archive SizeInBytes" > ${HISTORYFILE} fi ## Append archive size to the history log. echo "`date +%Y-%m-%d` ${ARCHIVEFILE} ${ARCHIVESIZE}" >> ${HISTORYFILE} ## Mount the offsite Windows share folder. echo "`date +%Y-%m-%d_%H:%M:%S` --- Mounting ${OFFSITEDIR}" >> ${LOGFILE} f_mount if [ -f ${OFFSITETESTFILE} ]; then ## Remote site is online / available. if [ ! -d ${OFFSITEBACKDIR} ]; then
    • ## Make the backup folder since it does not exist yet. mkdir -p ${OFFSITEBACKDIR} fi if [ -f ${TEMPDIR}/${ARCHIVEFILE} ]; then ## Make sure space is available on the remote server to copy the file. FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'` BACKUPSIZE=`ls -lak "${TEMPDIR}/${ARCHIVEFILE}" | awk '{ print $5 }'` if [ ${FREESPACE} -lt ${BACKUPSIZE} ]; then ## Not enough free space available. Purge existing backups until there is room. ENOUGHSPACE=0 while [ ${ENOUGHSPACE} -eq 0 ] do f_PurgeOldestArchive RETURNVALUE=$? case ${RETURNVALUE} in 1) ## Cannot purge archives to free up space. End program gracefully. echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Not enough free space on ${OFFSITEBACKDIR} and cannot purge old archives. Script aborted." >> ${TEMPLOG} ## Stop and exit the script with an error code. ERRORFLAG=$((${ERRORFLAG} + 4)) f_emergencyexit ${ERRORFLAG} ;; 9) ## Configuration error, end program gracefully. echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Configuration problem. Script aborted." >> ${TEMPLOG} ## Stop and exit the script with an error code. ERRORFLAG=$((${ERRORFLAG} + 8)) f_emergencyexit ${ERRORFLAG} ;; esac FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'` if [ ${FREESPACE} -gt ${BACKUPSIZE} ]; then ## Enough space is now available. ENOUGHSPACE=1 else ## Not enough space is available yet. ENOUGHSPACE=0 fi done fi echo "`date +%Y-%m-%d_%H:%M:%S` --- Copying ${TEMPDIR}/${ARCHIVEFILE} to ${OFFSITEBACKDIR}/" >> ${LOGFILE} cp ${TEMPDIR}/${ARCHIVEFILE} ${OFFSITEBACKDIR}/${ARCHIVEFILE} 1>/dev/null 2>&1 fi else ## Remote site is offline / unavailable. echo "`date +%Y-%m-%d_%H:%M:%S` --- Error: Remote site is unavailable: ${OFFSITEBACKDIR}" >> ${LOGFILE} ERRORFLAG=$((ERRORFLAG+16)) fi
    • if [ -f ${OFFSITEBACKDIR}/${ARCHIVEFILE} ]; then ## Remote copy worked. Remove local archive. echo "`date +%Y-%m-%d_%H:%M:%S` --- Copied ${ARCHIVESIZE} bytes." >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` --- Removing ${TEMPDIR}/${ARCHIVEFILE}" >> ${LOGFILE} rm ${TEMPDIR}/${ARCHIVEFILE} FREESPACE=`df -k ${OFFSITEDIR} | grep ${OFFSITEDIR} | awk '{ print $3 }'` ## Comment out the below line if you do not want to receive statistic emails. f_sendmail "Zimbra Individual Mailbox Backup" "${UCOUNT} accounts backed up.nnTotal archive size: $((${ARCHIVESIZE}/1024)) kbnnAvailable Backup Space: ${FREESPACE} kb" ## Uncomment the following 2 lines if you do not wish to have a local copy of individual mailboxes. # rm ${TARGETDIR}/*.tgz # rmdir ${TARGETDIR} else ## Remote copy failed. echo "`date +%Y-%m-%d_%H:%M:%S` --- Error copying to ${OFFSITEBACKDIR}/${ARCHIVEFILE}" >> ${LOGFILE} ERRORFLAG=$((ERRORFLAG+32)) fi ## Calculate total time for backup. FINISHTIME="$(date +%s)" ELAPSEDTIME="$(expr ${FINISHTIME} - ${STARTTIME})" HOURS=$((${ELAPSEDTIME} / 3600)) ELAPSEDTIME=$((${ELAPSEDTIME} - ${HOURS} * 3600)) MINUTES=$((${ELAPSEDTIME} / 60)) SECONDS=$((${ELAPSEDTIME} - ${MINUTES} * 60)) echo "`date +%Y-%m-%d_%H:%M:%S` --- Total backup time: ${HOURS} hour(s) ${MINUTES} minute(s) ${SECONDS} second(s)" >> ${LOGFILE} if [ ${ERRORFLAG} -ne 0 ]; then f_sendmail "Zimbra Individual Mailbox Backup Error" "${ERRORFLAG} errors detected while trying to backup ${UCOUNT} individual mailboxes." fi ## Unmount the offsite backup location. echo "`date +%Y-%m-%d_%H:%M:%S` --- Dismounting ${OFFSITEDIR}" >> ${LOGFILE} f_umount ## Perform cleanup routine. f_cleanup ## Exit with the combined return code value. echo "`date +%Y-%m-%d_%H:%M:%S` - Individual mailbox backup complete. Exit code: ${ERRORFLAG}" >> ${LOGFILE} exit ${ERRORFLAG} Sample output of mailbox-backup.log Code: 2011-11-06_15:48:02 2011-11-06_15:48:11 2011-11-06_15:48:13 2011-11-06_15:48:16 - Individual mailbox backup started. --- admin@mydomain.com, 6781 bytes --- lhammonds@mydomain.com, 1066 bytes --- jsmith@mydomain.com, 1067 bytes
    • 2011-11-06_15:48:18 --- jdoe@mydomain.com, 1066 bytes 2011-11-06_15:48:21 --- administrator@mydomain.com, 1069 bytes 2011-11-06_15:48:23 --- foobar@mydomain.com, 1067 bytes 2011-11-06_15:48:25 --- jvorhees@mydomain.com, 1068 bytes 2011-11-06_15:48:28 --- conferencerooms@mydomain.com, 1067 bytes 2011-11-06_15:48:30 --- test@mydomain.com, 1068 bytes 2011-11-06_15:48:48 --- 9 accounts processed. 2011-11-06_15:48:48 --- Backup time for 9 mailboxes: 0 hour(s) 0 minute(s) 46 second(s) 2011-11-06_15:48:48 --- Setting file permissions on /var/backup/mailbox/*.tgz 2011-11-06_15:48:48 --- Creating a single file for archiving: /var/temp/2011-11-06-15-48_mailbox-all.tar 2011-11-06_15:48:48 --- Mounting /mnt/backup 2011-11-06_15:48:49 --- Copying /var/temp/2011-11-06-15-48_mailbox-all.tar to /mnt/backup/mailbox/ 2011-11-06_15:48:49 --- Copied 61440 bytes. 2011-11-06_15:48:49 --- Removing /var/temp/2011-11-06-15-48_mailbox-all.tar 2011-11-06_15:48:49 --- Total backup time: 0 hour(s) 0 minute(s) 47 second(s) 2011-11-06_15:48:49 --- Dismounting /mnt/backup 2011-11-06_15:48:49 - Individual mailbox backup complete. Exit code: 0 Sample output of mailbox-backup-size-history.log Code: Date Archive SizeInBytes 2011-11-06 2011-11-06-11-21_mailbox-all.tar 2011-11-06 2011-11-06-15-17_mailbox-all.tar 2011-11-06 2011-11-06-15-21_mailbox-all.tar 2011-11-06 2011-11-06-15-22_mailbox-all.tar 2011-11-06 2011-11-06-15-32_mailbox-all.tar 2011-11-06 2011-11-06-15-48_mailbox-all.tar 61440 61440 61440 61440 61440 61440 Sample email notification Code: Subject: Zimbra Individual Mailbox Backup 9 accounts backed up. Total archive size: 60 kb Available Backup Space: 28440616 kb Server: mail Program: /var/scripts/prod/mailbox-backup.sh Log: /var/temp/mailbox-backup.log Sample error email notification Code: Subject: Zimbra Individual Mailbox Backup Error 1 errors detected while trying to backup 9 individual mailboxes. Server: mail Program: /var/scripts/prod/mailbox-backup.sh
    • Log: /var/temp/mailbox-backup.log Sample /var/log/messages Code: Nov 6 11:13:01 ERROR CODE = 1 Nov 6 11:15:02 = 2 Nov 6 11:18:03 = 4 Nov 6 11:23:04 = 8 Nov 6 11:33:05 = 16 Nov 6 11:43:06 = 32 mail administrator: /var/scripts/prod/mailbox-backup.sh: mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE mail root: /var/scripts/prod/mailbox-backup.sh: ERROR CODE __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #14 (permalink) 04-26-2012, 07:59 PM LHammonds Special Member Posts: 138 Restore Zimbra Restore I see the need to restore from at least several different scenarios. That means coming up with a restore plan and scripts to help speed up restore time and reduce errors during emergency scenarios. This also means testing each of these restore plans; otherwise you are playing Russian roulette with more than one bullet in the revolver! Always remember that the vast majority of administrators involuntarily removed from their job are typically related to data loss that cannot be recovered. Here are the scenarios I am going to cover and test: 1. Restore individual mailbox from a local copy - we will restore a mailbox account using the latest individual mailbox backup which is stored on the local server. This can be done at any time and will not impact server uptime. 2. Restore individual mailbox from an offsite location - we will restore a mailbox account using one of the archives located on the offsite storage. This can be done at any time and will not impact server uptime. 3. Restore existing server from a local copy (fastest, but potentially problematic) - we will use the local folder used for doing daily rsync backups. The caveat here is that you only want to do this if the last rsync was from the offline rsync script...meaning it is a complete snapshot and not from any of the online rsyncs which did not copy open
    • and in-use files. This will cause the server to be offline, so be sure to coordinate and plan accordingly. 4. Restore existing server from an offsite location (slower, but more reliable) - we will restore the server using one of the archives located on the offsite storage. This will cause the server to be offline, so be sure to coordinate and plan accordingly. 5. Restore to new server (production server unavailable) - we will assume the production server is offline and inaccessible. We will configure a new server and restore using one of the archives located on the offsite storage. __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #15 (permalink) 04-26-2012, 08:01 PM LHammonds Special Member Posts: 138 Disaster Scenario #1 Scenario #1 Restore individual mailbox from a local copy This process will assume you are wanting to restore a mailbox back to the way it was when last backed up. This means anything new since the backup will be wiped out. If this is not desired, a simple solution (which will be my standard protocol) will be to create a new mailbox and restore to it. If creating a new mailbox, be sure to set a password on it and test it by logging into the account...this also creates the mailbox store when accessed the 1st time. Then copy the user's mailbox archive to look like the archive for the restore account. Example #1 - Restore the admin mailbox. Code: /var/scripts/prod/mailbox-restore-local.sh admin@mydomain.com Onscreen result: Code: 2011-11-06_17:45:55 - Restored mailbox --> admin@mydomain.com Recorded event in the log: /var/temp/mailbox-restore.log Code: 2011-11-06_17:45:55 - Restored mailbox --> admin@mydomain.com Example #2 - Restore the admin mailbox to a temporary account called "restore".
    • 1. 2. 3. 4. 5. Login to the admin console: h t t p s ://mail.mydomain.com:7071 Create a new account called restore and set the password Login to the webmail using the restore account: h t t p s ://mail.mydomain.com Once logged in to verify the empty account works, logout Access the Ubuntu server. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 6. Type the following: Code: cp /var/backup/mailbox/admin@mydomain.com.tgz /var/backup/mailbox/restore@mydomain.com.tgz /var/scripts/prod/mailbox-restore-local.sh restore@mydomain.com mailbox-restore-local.sh Code: #!/bin/bash ############################################# ## Name : mailbox-restore-local.sh ## Version : 1.0 ## Date : 2011-11-06 ## Author : LHammonds ## Compatibility : Verified on Ubuntu 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 7.2.0 ## Purpose : Restore individual mailbox accounts. ## Run Frequency : Manual as needed. ## Exit Codes : (if multiple errors, value is the addition of codes) ## 0 = success ## 1 = parameter failure ## 2 = unknown mailbox account ## 4 = missing archive folder ## 8 = missing archive file ## 16 = mailbox restore failure ################ CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2011-10-24 LTH Created script. ## 2011-11-06 LTH Move common variables and functions to external file. ############################################# ## Import common variables and functions. source /var/scripts/common/standard.conf ## Define local variables. LOGFILE="${TEMPDIR}/mailbox-restore.log" ARCHIVEDIR="${BACKUPDIR}/mailbox" ACCT=$1 RETURNVALUE=0 ####################################### ## FUNCTIONS ## #######################################
    • function f_showhelp() { echo "" echo "Usage : ${SCRIPTNAME} user@${MYDOMAIN}" echo "" echo "NOTE: You could create a temporary account and copy the desired archive" echo " to be named the same as the temporary account in order to restore" echo " to a new mailbox and preserve the existing mailbox." echo " Example:" echo " cd ${ARCHIVEDIR}" echo " cp admin@${MYDOMAIN}.tgz restore@${MYDOMAIN}.tgz" echo " ${SCRIPTNAME} restore@${MYDOMAIN}" echo "" } function f_emergencyexit() { ## Purpose: Exit script as cleanly as possible. ## Parameter #1 = Error Code. echo -e "Individual mailbox restore aborted. EXIT CODE: ${1}n" exit $1 } ####################################### ## MAIN PROGRAM ## ####################################### ## Check command-line parameters. case "$1" in "") f_showhelp f_emergencyexit 1 ;; --help|-h|-?) f_showhelp f_emergencyexit 1 ;; *) ACCT=$1 ;; esac ## Check to make sure the archive folder exists. if [ ! -d "${ARCHIVEDIR}" ]; then ## ERROR: Archive folder does not exist. echo "ERROR: The archive folder does not exist! f_emergencyexit 4 fi ${ARCHIVEDIR}" ## Check to make sure the archive exists. if [ ! -f "${ARCHIVEDIR}/${ACCT}.tgz" ]; then ## ERROR: Archive does not exist. echo "ERROR: The archive does not exist! ${ARCHIVEDIR}/${ACCT}.tgz" f_emergencyexit 8 fi ## Check to make sure parameter is a valid mailbox account name. if `su - zimbra -c "zmprov -l gaa" | grep -q ${ACCT}`
    • then ## The parameter contains a valid mailbox account. echo "" > /dev/null else ## Invalid mailbox account. echo "ERROR: Invalid mailbox: ${ACCT}" f_showhelp f_emergencyexit 2 fi ${ZIMBRADIR}/bin/zmmailbox -z -m ${ACCT} postRestURL "//?fmt=tgz&resolve=reset" ${ARCHIVEDIR}/${ACCT}.tgz RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## Something went wrong. echo "`date +%Y-%m-%d_%H:%M:%S` - Error restoring ${ACCT}, RETURN VALUE = ${RETURNVALUE}" | tee -a ${LOGFILE} f_emergencyexit 16 fi echo "`date +%Y-%m-%d_%H:%M:%S` - Restored mailbox --> ${ACCT}" | tee -a ${LOGFILE} f_sendmail "Zimbra User Mailbox Restore" "${ACCT} was manually restored." exit 0 If you do not supply an account to the script as a parameter, it will display the following help text: Code: /var/scripts/prod/mailbox-restore.sh Usage : ./mailbox-restore-local.sh user@mydomain.com NOTE: You could create a temporary account and copy the desired archive to be named the same as the temporary account in order to restore to a new mailbox and preserve the existing mailbox. Example: cd /var/backup/mailbox cp admin@mydomain.com.tgz restore@mydomain.com.tgz /var/scripts/prod/mailbox-restore.sh restore@mydomain.com Individual mailbox restore aborted. EXIT CODE: 1 NOTE: The above will actually use your domain in all examples because it pulls the domain from the standard.conf file. __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #16 (permalink) 04-26-2012, 08:02 PM LHammonds Special Member Disaster Scenario #2 Posts: 138
    • Scenario #2 Restore individual mailbox from an offsite location Here is the general outline of the script: Parameter #1 (required) = Account in which we will find the archive for. Parameter #2 (optional) = Account in which we will restore the selected archive into (default = parameter 1) Step 1 - Display all offsite archives (order oldest to newest) and present list to user for selection Step 2 - Extract only the account's file from the archive to the temp folder Step 3 - Restore archive to desired account ---------------------/var/scripts/prod/mailbox-restore.sh Code: #!/bin/bash ############################################# ## Name : mailbox-restore.sh ## Version : 1.0 ## Date : 2011-11-08 ## Author : LHammonds ## Compatibility : Verified on Ubuntu 10.04.3 - 10.04.4 LTS, Zimbra 7.1.3 7.2.0 OSE ## Purpose : Restore individual mailbox accounts. ## Run Frequency : Manual as needed. ## Parameters : ## 1 = (Required) Account to search for archive (e.g. jdoe@mydomain.com) ## 2 = (Optional) Account to restore archive to (e.g. restore@mydomain.com) ## Exit Codes : ## 0 = Success or non-critical problem ## 1 = Archive folder does not exist ## 2 = Invalid mailbox account ## 3 = Mailbox restore failure ################ CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2011-11-08 LTH Created script. ############################################# ## Import common variables and functions. source /var/scripts/common/standard.conf ## Define local variables. LOGFILE="${TEMPDIR}/mailbox-restore.log" ARCHIVEDIR="${OFFSITEDIR}/mailbox" ACCT1=$1 ACCT2=$2 RETURNVALUE=0
    • ####################################### ## FUNCTIONS ## ####################################### function f_showhelp() { echo "" echo "Usage : ${SCRIPTNAME} user@${MYDOMAIN} restore@${MYDOMAIN} (optional)" echo "" echo "NOTE: If you specify a 2nd account, the 1st account data will be" echo " restored to the 2nd account." echo "" } function f_emergencyexit() { ## Purpose: Exit script as cleanly as possible. ## Parameter #1 = Error Code. if [ -f ${REMOTEDIR}/online.txt ]; then ## Dismount the remote site. f_umount fi echo -e "Individual mailbox restore aborted. exit $1 } ####################################### ## MAIN PROGRAM ## ####################################### ## Check command-line parameters. case "$1" in "") f_showhelp exit 0 ;; --help|-h|-?) f_showhelp exit 0 ;; *) ACCT1=$1 ACCT2=$2 ;; esac if [ "${ACCT1}" = "" ]; then ## Required parameter not specified. f_showhelp exit 0 fi if [ "${ACCT2}" = "" ]; then ## Restore to the same account. ACCT2=${ACCT1} fi ## Mount the remote site. EXIT CODE: ${1}n"
    • f_mount ## Check to make sure the archive folder exists. if [ ! -d "${ARCHIVEDIR}" ]; then ## ERROR: Archive folder does not exist. echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: The archive folder does not exist! ${ARCHIVEDIR}, EC=1" | tee -a ${LOGFILE} f_emergencyexit 1 fi ## Define user prompt using the special PS3 variable. PS3="Type number for the desired archive or 'q' to quit: " ## Get sorted list of all archives (newest at the bottom). FILELIST=$(find ${ARCHIVEDIR}/*.tar -maxdepth 1 -type f | sort -f) ## Prompt user to select a file to use. ## NOTE: If it is a long list, user can scroll up if ## using PuTTY to see older files. select GETFILE in ${FILELIST}; do if [ "${GETFILE}" != "" ]; then FILENAME=${GETFILE} fi break done if [ "${FILENAME}" = "" ]; then ## User opted to quit. echo -e "Exiting restore program.n" exit 0 fi ## Extract just the mailbox file from the archive. tar --extract -C ${TEMPDIR} --file=${FILENAME} --wildcards "*${ACCT1}*" -no-anchored --strip-components=3 if [ ! -f ${TEMPDIR}/${ACCT1}.tgz ]; then ## File did not exist in the archive. echo -e "No file found matching ${ACCT1}n" exit 0 fi ## Check to make sure parameter is a valid mailbox account name. if `su - zimbra -c "zmprov -l gaa" | grep -q ${ACCT2}` then ## The parameter contains a valid mailbox account. echo "" > /dev/null else ## Invalid mailbox account. echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Invalid mailbox: ${ACCT2}, EC=2" | tee -a ${LOGFILE} f_showhelp f_emergencyexit 2 fi ${ZIMBRADIR}/bin/zmmailbox -z -m ${ACCT2} postRestURL "//?fmt=tgz&resolve=reset" ${TEMPDIR}/${ACCT1}.tgz RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## Something went wrong.
    • echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: Restore failure on ${ACCT2}, RETURN VALUE=${RETURNVALUE}, EC=3" | tee -a ${LOGFILE} f_emergencyexit 3 fi echo "`date +%Y-%m-%d_%H:%M:%S` - Restored mailbox --> ${ACCT2}" | tee -a ${LOGFILE} f_sendmail "Zimbra User Mailbox Restore" "${ACCT2} was manually restored from ${ACCT1}" ## Clean up temporary files. if [ -f ${TEMPDIR}/${ACCT1}.tgz ]; then rm ${TEMPDIR}/${ACCT1}.tgz fi ## Dismount the remote site. f_umount exit 0 /var/temp/mailbox-restore.log (sample) Code: 2011-11-08_13:01:39 - ERROR: The archive folder does not exist! /mnt/backup/configtypo, EC=1 2011-11-08_15:54:34 - ERROR: Restore failure for admin@mydomain.com, RETURN VALUE=2, EC=3 2011-11-08_15:55:47 - Restored mailbox --> admin@mydomain.com 2011-11-08_15:57:50 - Restored mailbox --> lhammonds@mydomain.com 2011-11-08_16:00:14 - Restored mailbox --> ddiggler@mydomain.com 2011-11-08_16:01:16 - Restored mailbox --> restore@mydomain.com 2011-11-08_16:28:38 - ERROR: Invalid mailbox: bogus@mydomain.com, EC=2 __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #17 (permalink) 04-26-2012, 08:03 PM LHammonds Special Member Posts: 138 Disaster Scenario #3 Scenario #3 Restore existing server from a local copy We will use the local folder used for doing daily rsync backups. The caveot here is that you only want to do this if the last rsync was from the offline rsync script...meaning it is a complete snapshot and not from any of the online rsyncs which did not copy open and in-use files. If the last rsync was the online version, we will need to use a different script to access remote storage. This script is intended to be run manually as the root user.
    • Here are the instructions to restore the currently running server using the script below: 1. Before running this, make sure everyone is aware that Zimbra will be going down to be restored. 2. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 3. Run the script by typing /var/scripts/prod/restore-rsync-local.sh 4. If you missed any of the output on the screen, you can view the log by typing vi /var/temp/restore-rsync.log restore-rsync-local.sh Code: #!/bin/bash ############################################# ## Name : restore-rsync-local.sh ## Version : 1.1 ## Date : 2011-11-08 ## Author : LHammonds ## Purpose : Restore Zimbra using rsync and the local backup. ## Compatibility : Verifed on Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.3 - 7.2.0 OSE ## Requirements : Must be run as root user, last rsync should be the offline version. ## Run Frequency : Manual as needed. ## Exit Codes : Errors displayed to screen. ################ CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2011-10-18 LTH Created script. ## 2011-10-31 LTH Added better logging, error checks, user input. ## 2011-11-08 LTH Move common variables and funtions to external file. ############################################# ## Import common variables and functions. ## source /var/scripts/common/standard.conf LOGFILE="${TEMPDIR}/restore-rsync.log" LOCKFILE="${TEMPDIR}/rsync.lock" RSYNCLOGFILE="${TEMPDIR}/rsync.log" PRODROOT="/opt" OLDDIR="${BACKUPDIR}/zimbra.old" LOCALBACKUP="${BACKUPDIR}/zimbra/zimbra" LOCALBACKUPDEV="/dev/mapper/LVG-bak" NOBACKUP=0 ####################################### ## FUNCTIONS ## ####################################### function f_cleanup() { if [ -f ${LOCKFILE} ];then ## Remove lock file so other rsync jobs can run. rm ${LOCKFILE} 1>/dev/null 2>&1
    • fi } function f_checkspace() { ## Need to check and see if there is enough space available to move Zimbra to a backup location. FREESPACE=`df -k ${LOCALBACKUPDEV} | grep ${LOCALBACKUPDEV} | awk '{ print $4 }'` BACKUPSIZE=`du -sk ${ZIMBRADIR} | awk '{ print $1 }'` if [ ${BACKUPSIZE} -gt ${FREESPACE} ]; then echo "There is not enough freespace available to archive production Zimbra." echo "Zimbra size is ${BACKUPSIZE} kb, Free space is ${FREESPACE} kb" echo "" echo "To override and not make a backup 1st, use the --nobackup option." echo "" echo "Example: ${SCRIPTNAME} --nobackup" echo "" f_cleanup exit 1 fi } ####################################### ## MAIN PROGRAM ## ####################################### if [ -f ${LOCKFILE} ]; then # rsync lock file detected. Abort script. echo "Restore rsync aborted" echo "This script tried to run but detected the lock file: ${LOCKFILE}" echo "Please check to make sure the file does not remain when rsync is not actually running." f_sendmail "restore rsync aborted" "This script tried to run but detected the lock file: ${LOCKFILE}nnPlease check to make sure the file does not remain when rsync is not actually running." exit 1 else echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE} fi clear echo "" # Determine if we are going to backup existing folder or not. if [ "$1" = "--nobackup" ]; then NOBACKUP=1 else NOBACKUP=0 f_checkspace fi ## Requirement Check: Script must run as root user. if [ "$(id -u)" != "0" ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. echo "ERROR: Root user required for restore." echo ""
    • f_cleanup exit 1 fi if [ ! -f ${RSYNCLOGFILE} ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. echo "ERROR: Cannot find log: ${RSYNCLOGFILE}" echo "This log file is required to determine status of last rsync job." echo "" f_cleanup exit 1 fi echo "This restore method can only work if the last rsync backup was" echo "the 'Offline' version which allows for a complete backup." echo "" echo "The 'Online' version is only a partial backup and will not work." echo "" read -p "The next step will try to auto-detect the last backup. Continue (y/n)? " if [ "${REPLY}" != "y" ]; then echo "Script cancelled." echo "" f_cleanup exit 1 fi echo "" tail -n1 ${RSYNCLOGFILE} | grep -q "Offline RSync backup exit code" if [ $? -eq 0 ]; then echo "Last rsync seems to have been a good offline backup" else echo "WARNING: This script did not see the offline entry at the end of the log file." fi echo "" echo tail echo echo "--------------------------------------------------------" -n5 ${RSYNCLOGFILE} "--------------------------------------------------------" "" read -p "Do you see 'Offline RSync backup exit code: 0' in the log above (y/n)? " if [ "${REPLY}" != "y" ]; then echo "Script cancelled." echo "" f_cleanup exit 1 fi echo "`date +%Y-%m-%d_%H:%M:%S` - Restore started." | tee -a ${LOGFILE} StartTime="$(date +%s)" ## Stop Zimbra Services. echo "`date +%Y-%m-%d_%H:%M:%S` --- Stopping Zimbra services." | tee -a ${LOGFILE} /etc/init.d/zimbra stop
    • ## Kill any orphaned Zimbra processes. pkill -9 -u zimbra 1>/dev/null 2>&1 if [ ${NOBACKUP} -eq 1 ]; then ## Leave current production folder alone. echo "`date +%Y-%m-%d_%H:%M:%S` --- 'No Backup' override specified, skipping folder move." | tee -a ${LOGFILE} else ## Move production folder to backup location. echo "`date +%Y-%m-%d_%H:%M:%S` --- Moving Zimbra from ${ZIMBRADIR} to ${OLDDIR}." | tee -a ${LOGFILE} mv ${ZIMBRADIR} ${OLDDIR} RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## Could not move Zimbra to backup location. ## Document error and terminate script. echo "" echo "ERROR: The move was not successful. Return value = ${RETURNVALUE}" | tee -a ${LOGFILE} echo "" echo "COMMAND was 'mv ${ZIMBRADIR} ${OLDDIR}'" | tee -a ${LOGFILE} echo "" echo "The Zimbra services are currently stopped and an unknown amount of files were moved" echo "from the source. That means there some files are still in ${ZIMBRADIR}" echo "and some files were moved to ${OLDDIR}" echo "This needs to be resolved before you can start the services back up." echo "Either move the files back or delete all files and run the restore." f_cleanup exit 99 fi fi ## Now that Zimbra is offline, rsync can restore all files. echo "`date +%Y-%m-%d_%H:%M:%S` --- Restoring files with rsync." | tee -a ${LOGFILE} rsync -apogHK ${LOCALBACKUP} ${PRODROOT} RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## rsync command failed. Display warning message. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: rsync return value = {$RETURNVALUE}" | tee -a ${LOGFILE} fi if [ ! -f ${ZIMBRADIR}/.install_history ]; then ## Something is not right...at least this file should exist here. ## NOTE: This is not a guarentee all files were restored, it is just a quick check. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Missing file(s). Did not find ${ZIMBRADIR}/.install_history" | tee -a ${LOGFILE} fi ## Fix permissions. echo "`date +%Y-%m-%d_%H:%M:%S` --- Fixing file ownership and permissions." | tee -a ${LOGFILE} chown -R zimbra:zimbra ${ZIMBRADIR}
    • ${ZIMBRADIR}/libexec/zmfixperms ## Start Zimbra services. echo "`date +%Y-%m-%d_%H:%M:%S` --- Starting Zimbra services." | tee -a ${LOGFILE} /etc/init.d/zimbra start ## Calculates downtime of Zimbra services. FinishTime="$(date +%s)" ElapsedTime="$(expr ${FinishTime} - ${StartTime})" Hours=$((${ElapsedTime} / 3600)) ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600)) Minutes=$((${ElapsedTime} / 60)) Seconds=$((${ElapsedTime} - ${Minutes} * 60)) echo "`date +%Y-%m-%d_%H:%M:%S` --- Zimbra restore time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" | tee -a ${LOGFILE} ## Show status of Zimbra services. echo "`date +%Y-%m-%d_%H:%M:%S` --- Status of Zimbra services:" | tee -a ${LOGFILE} su - zimbra -c "zmcontrol status" | tee -a ${LOGFILE} if [ -d ${OLDDIR} ]; then echo -e "nnDon't forget to purge ${OLDDIR} once everything is running smoothly again.nn" fi f_cleanup echo "`date +%Y-%m-%d_%H:%M:%S` - Restore completed." | tee -a ${LOGFILE} Sample output from script: Code: 2011-10-18_16:26:40 - Restore started. 2011-10-18_16:26:40 --- Stopping Zimbra services. Host mail.mydomain.com Stopping stats...Done. Stopping mta...Done. Stopping spell...Done. Stopping snmp...Done. Stopping cbpolicyd...Done. Stopping archiving...Done. Stopping antivirus...Done. Stopping antispam...Done. Stopping imapproxy...Done. Stopping memcached...Done. Stopping mailbox...Done. Stopping logger...Done. Stopping zmconfigd...Done. Stopping ldap...Done. 2011-10-18_16:27:22 --- Moving Zimbra from /opt/zimbra to /temp/zimbra.old. 2011-10-18_16:27:22 --- Restoring files with rsync. 2011-10-18_16:31:53 --- Fixing file ownership and permissions. 2011-10-18_16:31:55 --- Starting Zimbra services. Host mail.mydomain.com Starting ldap...Done. Starting zmconfigd...Done. Starting logger...Done. Starting mailbox...Done.
    • Starting antispam...Done. Starting antivirus...Done. Starting snmp...Done. Starting spell...Done. Starting mta...Done. Starting stats...Done. 2011-10-18_16:33:25 --- Zimbra restore time: 0 hour(s) 6 minute(s) 45 second(s) 2011-10-18_16:33:25 --- Status of Zimbra services: Host mail.mydomain.com antispam Running antivirus Running ldap Running logger Running mailbox Running mta Running snmp Running spell Running stats Running zmconfigd Running 2011-10-18_16:33:55 - Restore completed. __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #18 (permalink) 04-26-2012, 08:05 PM LHammonds Special Member Posts: 138 Disaster Scenario #4 Scenario #4 Restore existing server from an offsite location /var/scripts/prod/rsync-restore.sh Code: #!/bin/bash ############################################# ## Name : rsync-restore.sh ## Version : 1.0 ## Date : 2011-11-08 ## Author : LHammonds ## Purpose : Restore Zimbra using rsync from a remote archive. ## Compatibility : Verified on Ubuntu Server 10.04.3 - 10.04.4 LTS, Zimbra 7.1.2 - 7.2.0 OSE ## Requirements : Must be root user, p7zip-full (if ARCHIVEMETHOD=tar.7z) ## Run Frequency : Manual as needed. ## Exit Codes : ## 0 = Success or non-critical problem ## 1 = Archive folder does not exist ## 2 = Archive extration failed ## 3 = Zimbra restore failure ## 99 = Move production to temp directory failure (bad)
    • ################ CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2011-11-08 LTH Created script. ############################################# ## Import common variables and functions. ## source /var/scripts/common/standard.conf LOGFILE="${TEMPDIR}/rsync-restore.log" LOCKFILE="${TEMPDIR}/rsync.lock" ARCHIVEDIR="${OFFSITEDIR}/zimbra" OLDDIR="${BACKUPDIR}/zimbra.old" NOBACKUP=0 ####################################### ## FUNCTIONS ## ####################################### function f_cleanup() { if [ -f ${LOCKFILE} ];then ## Remove lock file so other rsync jobs can run. rm ${LOCKFILE} 1>/dev/null 2>&1 fi } function f_checkspace() { ## Need to check and see if there is enough space available to move Zimbra to a backup location. FREESPACE=`df -k ${BACKUPDIR} | grep ${BACKUPDIR} | awk '{ print $4 }'` BACKUPSIZE=`du -sk ${ZIMBRADIR} | awk '{ print $1 }'` if [ ${BACKUPSIZE} -gt ${FREESPACE} ]; then echo "There is not enough freespace available to archive production Zimbra." echo "Zimbra size is ${BACKUPSIZE} kb, Free space is ${FREESPACE} kb" echo "" echo "To override and not make a backup 1st, use the --nobackup option." echo "" echo "Example: ${SCRIPTNAME} --nobackup" echo "" f_cleanup exit 1 fi } function f_emergencyexit() { ## Purpose: Exit script as cleanly as possible. ## Parameter #1 = Error Code. if [ -f ${REMOTEDIR}/online.txt ]; then ## Dismount the remote site. f_umount fi echo -e "Zimbra restore aborted. EXIT CODE: ${1}n" exit $1
    • } ####################################### ## MAIN PROGRAM ## ####################################### if [ -f ${LOCKFILE} ]; then ## Restore lock file detected. Abort script. echo "Restore aborted" echo "This script tried to run but detected the lock file: ${LOCKFILE}" echo "Please check to make sure the file does not remain when rsync or restore is not actually running." f_sendmail "Zimbra Restore aborted" "This script tried to run but detected the lock file: ${LOCKFILE}nnPlease check to make sure the file does not remain when rsync/restore is not actually running." exit 1 else echo "`date +%Y-%m-%d_%H:%M:%S` ${SCRIPTNAME}" > ${LOCKFILE} fi clear echo "" ## Determine if we are going to backup existing folder or not. if [ "$1" = "--nobackup" ]; then NOBACKUP=1 else NOBACKUP=0 ## f_checkspace fi ## Requirement Check: Script must run as root user. if [ "$(id -u)" != "0" ]; then ## FATAL ERROR DETECTED: Document problem and terminate script. echo "ERROR: Root user required for restore." echo "" f_cleanup exit 1 fi ## Mount the remote site. f_mount ## Check to make sure the archive folder exists. if [ ! -d "${ARCHIVEDIR}" ]; then ## ERROR: Archive folder does not exist. echo "`date +%Y-%m-%d_%H:%M:%S` - ERROR: The archive folder does not exist! ${ARCHIVEDIR}, EC=1" | tee -a ${LOGFILE} f_cleanup f_emergencyexit 1 fi ## Define user prompt using the special PS3 variable. PS3="Type number for the desired archive or 'q' to quit: " echo " Z I M B R A R E S T O R E" echo " ---------------------------" ## Get sorted list of all archives (newest at the bottom). FILELIST=$(find ${ARCHIVEDIR}/*.${ARCHIVEMETHOD} -maxdepth 1 -type f | sort
    • -f) ## Prompt user to select a file to use. ## NOTE: If it is a long list, user can scroll up if ## using PuTTY to see older files. select GETFILE in ${FILELIST}; do if [ "${GETFILE}" != "" ]; then FILENAME=${GETFILE} fi break done if [ "${FILENAME}" = "" ]; then ## User opted to quit. echo -e "Exiting restore program.n" f_cleanup f_umount exit 0 fi echo echo echo read -e -e -e -p "nSelected file: ${FILENAME}n" " This restore will shutdown Zimbra and make it" " unavailable while it is being restored.n" "Are you absolutely sure you wish to restore (y/n)? " if [ "${REPLY}" != "y" ]; then echo -e "nnRestore aborted.nn" f_cleanup exit 0 fi echo "`date +%Y-%m-%d_%H:%M:%S` - Restore started." | tee -a ${LOGFILE} StartTime="$(date +%s)" ## Stop Zimbra Services. echo "`date +%Y-%m-%d_%H:%M:%S` --- Stopping Zimbra services." | tee -a ${LOGFILE} /etc/init.d/zimbra stop ## Kill any orphaned Zimbra processes. pkill -9 -u zimbra 1>/dev/null 2>&1 if [ ${NOBACKUP} -eq 1 ]; then ## Leave current production folder alone. echo "`date +%Y-%m-%d_%H:%M:%S` --- 'No Backup' override specified, skipping folder move." | tee -a ${LOGFILE} else ## Move production folder to backup location. echo "`date +%Y-%m-%d_%H:%M:%S` --- Moving Zimbra from ${ZIMBRADIR} to ${OLDDIR}." | tee -a ${LOGFILE} mv ${ZIMBRADIR} ${OLDDIR} RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## Could not move Zimbra to backup location. ## Document error and terminate script. echo "" echo "ERROR: The move was not successful. Return value = ${RETURNVALUE}" | tee -a ${LOGFILE} echo ""
    • echo "COMMAND was 'mv ${ZIMBRADIR} ${OLDDIR}'" | tee -a ${LOGFILE} echo "" echo "The Zimbra services are currently stopped and an unknown amount of files were moved" echo "from the source. That means there some files are still in ${ZIMBRADIR}" echo "and some files were moved to ${OLDDIR}" echo "This needs to be resolved before you can start the services back up." echo "Either move the files back or delete all files and run the restore." f_cleanup exit 99 fi fi ## Now that Zimbra is offline, we can restore all files. echo "`date +%Y-%m-%d_%H:%M:%S` --- Restoring files from ${FILENAME}" | tee -a ${LOGFILE} if [ ! -d ${ZIMBRADIR} ]; then mkdir -p ${ZIMBRADIR} fi ## Extract the entire contents of the archive back to its original location. case "${ARCHIVEMETHOD}" in tar.7z) 7za x -so -w/${TEMPDIR} ${FILENAME} | tar -C ${ZIMBRADIR} --extract -strip-components=4 --file ;; tgz) tar --extract -C ${ZIMBRADIR} --strip-components=4 --file=${FILENAME} ;; *) tar --extract -C ${ZIMBRADIR} --strip-components=4 --file=${FILENAME} ;; esac RETURNVALUE=$? if [ ${RETURNVALUE} -ne 0 ]; then ## tar extract command failed. Display warning message. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: tar extract return value = {$RETURNVALUE}" | tee -a ${LOGFILE} f_cleanup f_emergencyexit 2 fi if [ ! -f ${ZIMBRADIR}/.install_history ]; then ## Something is not right...at least this file should exist here. ## NOTE: This is not a guarentee all files were restored, it is just a quick check. echo "`date +%Y-%m-%d_%H:%M:%S` --- ERROR: Missing file(s). Did not find ${ZIMBRADIR}/.install_history" | tee -a ${LOGFILE} fi ## Fix permissions. echo "`date +%Y-%m-%d_%H:%M:%S` --- Fixing file ownership and permissions." | tee -a ${LOGFILE} chown -R zimbra:zimbra ${ZIMBRADIR} ${ZIMBRADIR}/libexec/zmfixperms
    • ## Start Zimbra services. echo "`date +%Y-%m-%d_%H:%M:%S` --- Starting Zimbra services." | tee -a ${LOGFILE} /etc/init.d/zimbra start ## Calculates downtime of Zimbra services. FinishTime="$(date +%s)" ElapsedTime="$(expr ${FinishTime} - ${StartTime})" Hours=$((${ElapsedTime} / 3600)) ElapsedTime=$((${ElapsedTime} - ${Hours} * 3600)) Minutes=$((${ElapsedTime} / 60)) Seconds=$((${ElapsedTime} - ${Minutes} * 60)) echo "`date +%Y-%m-%d_%H:%M:%S` --- Zimbra restore time: ${Hours} hour(s) ${Minutes} minute(s) ${Seconds} second(s)" | tee -a ${LOGFILE} ## Show status of Zimbra services. echo "`date +%Y-%m-%d_%H:%M:%S` --- Status of Zimbra services:" | tee -a ${LOGFILE} su - zimbra -c "zmcontrol status" | tee -a ${LOGFILE} if [ -d ${OLDDIR} ]; then echo -e "nnDon't forget to purge ${OLDDIR} once everything is running smoothly again.nn" fi f_cleanup ## Dismount the remote site. f_umount echo "`date +%Y-%m-%d_%H:%M:%S` - Restore completed." | tee -a ${LOGFILE} __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #19 (permalink) 04-26-2012, 08:05 PM LHammonds Special Member Posts: 138 Disaster Scenario #5 Scenario #5 Restore to new server ** UNFINISHED ** __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) #20 (permalink) 04-26-2012, 08:06 PM
    • LHammonds Special Member Posts: 138 Crontab Schedule Crontab I would not advise anyone to ever "edit" a live crontab schedule by typing "crontab -e" but rather edit a saved schedule file and then load the schedule file. This will allow you to make backups of the schedule so you can always go back to a known-good schedule or at least back to the way it was before you made a change...assuming you always work with a copy of the schedule 1st. Here is my root crontab scheduling file: /var/scripts/data/crontab.root Code: ######################################## # Name: Crontab Schedule for root user # Author: LHammonds ############# Update Log ############### # 2011-10-29 - LTH - Created schedule # 2011-11-22 - LTH - Added shell and path # 2012-01-30 - LTH - Added time adjustment # 2012-05-02 - LTH - Added check-space # 2012-05-05 - LTH - Added check mailbox size ######################################## # # Import Active Directory users (run every minute) 0-59 * * * * /var/scripts/prod/import-ad.sh > /dev/null 2>&1 # # Update the Global Address List for iPhone usage. # 0 23 * * * /var/scripts/prod/gal-sync.sh > /dev/null 2>&1 # # Partial backup of Zimbra to a local folder without taking it offline. # 0 8-18 * * * /var/scripts/prod/rsync-online.sh > /dev/null 2>&1 # # Backup Zimbra to a local folder, archive and store offsite. # 0 23 * * * /var/scripts/prod/rsync-offline.sh > /dev/null 2>&1 # # Backup individual mailboxes to a local folder, archive and store offsite. # 0 6-18 * * * /var/scripts/prod/zm-user-backup.sh > /dev/null 2>&1 # # Daily check for available space on /opt # 0 1 * * * /var/scripts/prod/check-storage.sh opt 5 1 > /dev/null 2>&1 # # Daily check for available space on /var/backup # 0 2 * * * /var/scripts/prod/check-storage.sh bak 5 1 > /dev/null 2>&1
    • # # # 0 # # # 0 Daily check for available space on /var/temp 3 * * * /var/scripts/prod/check-storage.sh temp 5 1 > /dev/null 2>&1 Daily check for mailbox size notifications 6 * * * /var/scripts/prod/mailbox-size-notification.sh > /dev/null 2>&1 Once you have created the file, make sure appropriate permissions are set by typing the following: Code: chown root:root /var/scripts/data/crontab.root chmod 0600 /var/scripts/data/crontab.root To enable the root schedule using this file, type the following: Code: crontab -u root /var/scripts/data/crontab.root To disable the root schedule, type the following: Code: touch /tmp/deleteme crontab -u root /tmp/deleteme rm /tmp/deleteme If you need to modify the schedule, make a backup copy 1st. For example: Code: cp /var/scripts/data/crontab.root /var/scripts/prod/2011-11-28-crontab.root vi /var/scripts/data/crontab.root (make your changes) crontab -u root /var/scripts/data/crontab.root __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) Last edited by LHammonds; 05-05-2012 at 12:20 PM..
    • LHammonds Special Member Posts: 138 Mailbox Size Notifications My company wants end-users to be notified when their mailbox is approaching a certain size but does not want an functionality to be diminished if maximums are exceeded. So this calls for a custom script to check mailbox sizes and send notifications. However, they also want certain people to have different limitations than the masses and some people completely excluded from any warnings. This solution calls for a quota definition file to handle special cases. Anyone not in the quota file will be handled using system-wide default values. Here is a sample of the quota definition file: /var/scripts/data/quota-data.txt Code: ;AccountName, WarningSizeInMB, MaxSizeInMB, Exempt? ceo@mydomain.com, 0, 0, Y cio@mydomain.com, 0, 0, Y hr@mydomain.com, 270, 300, N webmaster@mydomain.com, 360, 400, N boss@mydomain.com, 180, 200, N In the above sample, the CEO and CIO are exempt from notifications and thus, will never be warned about mailbox size usage. The HR person will receive warning messages when the mailbox size is between 270MB and 300MB. Once the mailbox is over 300MB, the message will turn into an alert message with slightly different text. The webmaster will receive warnings between 360MB and 400MB and alerts over 400MB. The boss will receive warnings between 180MB and 200MB and alerts over 200MB.
    • Here is a sample warning message: Code: From: admin@mydomain.com To: lhammonds@mydomain.com Sent: Saturday, May 5, 2012 11:21:12 AM Subject: WARNING: Mailbox size Your mailbox size is about to reach the maximum limit. Please delete or archive messages to reduce the size of your mailbox. Mailbox size = 200MB. 250MB. You have 50MB before you reach the maximum of Here is a sample alert message: Code: From: admin@mydomain.com To: lhammonds@mydomain.com Sent: Saturday, May 5, 2012 10:48:47 AM Subject: ALERT: Mailbox full Your mailbox has reached the maximum limit. Please delete or archive messages to reduce the size of your mailbox. Mailbox size = 200MB. You are 50MB over the maximum of 150MB. Here is the code for the notification script: /var/scripts/prod/mailbox-size-notification.sh Code: #!/bin/bash ############################################# ## Name : mailbox-size-notification.sh ## Version : 1.0 ## Date : 2012-05-04 ## Author : LHammonds ## Purpose : Check mailbox size usage. ## Compatibility : Verified on Ubuntu Server 10.04.4 LTS, Zimbra 7.2.0 OSE ## Requirements : Must be run as root user ## Run Frequency : As needed. Recommend once per day. ## Parameters : None ## Exit Codes : ## 0 = Success ## 1 = Failure ################ CHANGE LOG ################# ## DATE WHO WHAT WAS CHANGED ## ---------- --- ---------------------------## 2012-05-04 LTH Created script. ############################################# ## Import common variables and functions. source /var/scripts/common/standard.conf ## Define local variables. LOGFILE="${TEMPDIR}/mailbox-size-notification.log"
    • DATAFILE="${SCRIPTDIR}/data/quota-data.txt" ACCTFILE="${TEMPDIR}/mailbox-size-notification-accts.txt" ## Default minimum threshold in MB when email notifications start getting sent. DefaultMin=180 ## Default maximum mailbox size in MB (fake limit). DefaultMax=200 DefaultExempt="N" ## Convert bytes to megabytes. ConversionMultiplier=1048576 ConversionUnit="MB" ReturnCode=0 ErrorFlag=0 ## Message to send when mailbox is over the threshold but under the max. WarningMessage="Your mailbox size is about to reach the maximum limit. Please delete or archive messages to reduce the size of your mailbox." ## Message to send when mailbox is over the max. AlertMessage="Your mailbox has reached the maximum limit. Please delete or archive messages to reduce the size of your mailbox." echo "`date +%Y-%m-%d_%H:%M:%S` - Quota check started." >> ${LOGFILE} ## Build a list of user accounts and their mailbox size. su - zimbra -c "zmprov gqu ${MYDOMAIN}" > ${ACCTFILE} ReturnCode=$? if [ ${ReturnCode} -ne 0 ]; then echo " ERROR: zmprov return code was ${ReturnCode}" >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` - Quota check aborted." >> ${LOGFILE} exit 1 fi if [ ! -s "${ACCTFILE}" ]; then echo " ERROR: Account listing file is empty." >> ${LOGFILE} echo "`date +%Y-%m-%d_%H:%M:%S` - Quota check aborted." >> ${LOGFILE} exit 1 fi while read AcctLine do ## Loop through each mailbox account. AcctName=`echo ${AcctLine} | cut -d ' ' -f1` AcctSize=`echo ${AcctLine} | cut -d ' ' -f3` AdjAcctSize=$((${AcctSize}/${ConversionMultiplier})) QuotaData=`grep ${AcctName} ${DATAFILE}` if [ -z "${QuotaData}" ]; then ## Set variable as if it were read from the file but using defaults. QuotaData="${AcctName} ${DefaultMin} ${DefaultMax} ${DefaultExempt}" fi ## Obtain quota values and exemption info. QuotaMin=`echo ${QuotaData} | cut -d ' ' -f2` QuotaMax=`echo ${QuotaData} | cut -d ' ' -f3` QuotaExempt=`echo ${QuotaData} | cut -d ' ' -f4` if [[ "${QuotaExempt}" = "N" || "${QuotaExempt}" = "n" ]]; then ## User is not exempt from notification messages. if [[ ${AdjAcctSize} > ${QuotaMin} ]]; then ## A notification email needs to be sent. if [[ ${AdjAcctSize} < ${QuotaMax} ]]; then ## Mailbox size has not gone over the maximum. SpaceAvailable=$((${QuotaMax}-${AdjAcctSize})) echo " WARNING: ${AcctName} Mailbox=${AdjAcctSize}${ConversionUnit} Min=${QuotaMin} Max=${QuotaMax}" >> ${LOGFILE} f_sendusermail "${AcctName}" "WARNING: Mailbox size"
    • "${WarningMessage}nnMailbox size = ${AdjAcctSize}${ConversionUnit}. You have ${SpaceAvailable}${ConversionUnit} before you reach the maximum of ${QuotaMax}${ConversionUnit}." else ## Mailbox size is now larger than maximum. echo " ALERT: ${AcctName} Mailbox=${AdjAcctSize}${ConversionUnit} Min=${QuotaMin} Max=${QuotaMax}" >> ${LOGFILE} SpaceOver=$((${AdjAcctSize}-${QuotaMax})) f_sendusermail "${AcctName}" "ALERT: Mailbox full" "${AlertMessage}nnMailbox size = ${AdjAcctSize}${ConversionUnit}. You are ${SpaceOver}${ConversionUnit} over the maximum of ${QuotaMax}${ConversionUnit}." fi fi fi done < ${ACCTFILE} ## Remove the temporary file. if [ -f ${ACCTFILE} ]; then rm ${ACCTFILE} fi echo "`date +%Y-%m-%d_%H:%M:%S` - Quota check finished." >> ${LOGFILE} exit 0 The default values in this script have been set to 180 (GB) and a max of 200 (GB). That means the "boss" line in the sample quota definition file is unnecessary and can be removed from that file since he matches the system-wide default thresholds. Here is a sample of the log file output: /var/temp/mailbox-size-notification.log Code: 2012-05-03_06:00:00 - Quota check started. ALERT: lhammonds@mydomain.com Mailbox=250MB Min=180 Max=200 WARNING: joedirt@mydomain.com Mailbox=190MB Min=180 Max=200 WARNING: hr@mydomain.com Mailbox=280MB Min=270 Max=300 2012-05-03_06:00:05 - Quota check finished. 2012-05-04_06:00:00 - Quota check started. ALERT: lhammonds@mydomain.com Mailbox=280MB Min=180 Max=200 WARNING: joedirt@mydomain.com Mailbox=181MB Min=180 Max=200 2012-05-04_06:00:04 - Quota check finished. 2012-05-05_06:00:00 - Quota check started. 2012-05-05_06:00:05 - Quota check finished. EDIT 2012-05-05: I have edited the Scripting post on the 1st page and updated the "standard.conf" file which includes an additional global variable and end-user email function. __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) Last edited by LHammonds; 05-05-2012 at 12:19 PM..
    • #22 (permalink) 05-07-2012, 08:44 AM LHammonds Special Member Posts: 138 ToDoList (version 1.0) I use the portable version of ToDoList to keep track of projects and things that need to get done. This is a copy of my rollout project for Zimbra. Save the text into a file such as ZimbraRollout.xml EDIT: Could not paste XML text here...too big. So I used PasteBin. Next Tasks on my List: - Manually add Distribution Groups to somewhat match Exchange groups - Manually add external contacts to the global address list (have not figured out this one just yet) - Test-run a mass mailbox conversion from Exchange to Zimbra and note amount of time required. - End-user documentation (normal use and conversion/migration info) - Admin-user documentation (for other admins that are not necessarily unix admins) __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) Last edited by LHammonds; 05-07-2012 at 03:20 PM.. #23 (permalink) 05-07-2012, 11:00 PM davidkillingsworth Active Member Wow. That's very thorough and details instructions. Thank You for taking the time to provide that to the community. Can I make a request that I don't see throughout your thread. What are the steps for upgrading Zimbra on Ubuntu 10.04 Posts: 31
    • 7.1.2 to 7.2.0 for example. Thanks, in advance. __________________ ZIMBRA VERSION: Release 7.2.0_GA_2669.UBUNTU10_64 UBUNTU10_64 FOSS edition #24 (permalink) 05-07-2012, 11:16 PM phoenix Posts: 20,310 Zimbra Consultant & Moderator Quote: Originally Posted by davidkillingsworth What are the steps for upgrading Zimbra on Ubuntu 10.04 That would be exactly the same steps as upgrading any other release of ZCS, run the install.sh script as per the information in the Release Notes with the added proviso of taking a full backup before any upgrade. __________________ Regards Bill #25 (permalink) 05-07-2012, 11:23 PM davidkillingsworth Active Member Posts: 31 Quote: Originally Posted by phoenix That would be exactly the same steps as upgrading any other release of ZCS, run the install.sh script as per the information in the Release Notes with the added proviso of taking a full backup before any upgrade. I'm aware of the release notes, but was hoping for some of LHammonds extraordinarily detailed "tell it to me like I'm 5 years old" step by step instructions. I just want to make sure that I'm thinking of all the steps.
    • __________________ ZIMBRA VERSION: Release 7.2.0_GA_2669.UBUNTU10_64 UBUNTU10_64 FOSS edition #26 (permalink) 05-07-2012, 11:33 PM phoenix Posts: 20,310 Zimbra Consultant & Moderator If you already have a backup strategy in place then take a full backup, while it's running read the Release Notes for anything that may be needed then then run the update, what else is there to know? How much more simple can it be? If my comments aren't simple enough then feel free to ignore them and wait for something more detailed. __________________ Regards Bill #27 (permalink) 05-08-2012, 07:12 AM LHammonds Special Member Posts: 138 Quote: Originally Posted by davidkillingsworth Wow. That's very thorough and details instructions. Thank You for taking the time to provide that to the community. Thank you and you are welcome. Quote: Originally Posted by davidkillingsworth Can I make a request that I don't see throughout your thread. What are the steps for upgrading Zimbra on Ubuntu 10.04 7.1.2 to 7.2.0 for example. As phoenix has already mentioned, it is fairly straight-forward to upgrade 7.1 to 7.2. I did not
    • perform an upgrade for the current 7.2 instructions but the main steps are to backup your data, read the release notes thoroughly, run the install (which detects the current installation and performs an upgrade), make another backup and test everything out. I did document my experience when I upgraded a test system from 7.1.2 to 7.1.3 in another thread that was deleted. Here is a copy of it (which includes an issue I had with the upgrade which may not be repeatable using the same versions or newer upgrades but I made note of it anyway) Also, when comparing how these old notes might apply to the current thread and release, you would obviously need to download and use the current version which is 7.2.0 and the "temp" folder in this server layout is /var/temp EDIT: And as an additional note, since this particular upgrade, I have found that the error regarding lock tables is related to temp tables and has been a "known" issue / bug in MySQL that has never been resolved. I'm not sure why the "integrity" report script was not slightly modified to ignore that particular error on temp tables...or at the very least change it from a critical show-stopper to just an FYI with a bit of info about it such as a link to the longexisting bug report. Quote: Upgrade Zimbra 7.1.2 to 7.1.3 Before Upgrading 1. Read and understand the release notes before doing anything else. 2. Make a verified offline backup and be sure to place it at an offsite location as well. 3. Since this is a virtual environment, might as well shutdown the server and create a snapshot of the server. Give it a name like Before 7.1.3 Upgrade and description of Ubuntu Server 10.04.4 LTS, Zimbra 7.1.3 OSE installed, Static IP: 192.168.107.25. 4. Once the snapshot is complete, start up the Ubuntu server. Upgrade Steps 1. At the login prompt, login with your administrator account (administrator / myadminpass) and then type su and the root password (myrootpass) 2. Update the package database by typing aptitude update 3. Install the latest updates by typing aptitude -y safe-upgrade 4. Download the Open Source Edition from Open Source Edition Downloads: Enterprise Messaging and Collaboration Software by Zimbra Method #1 - Download from your PC A. Download the .tgz file and copy it to the Ubuntu share --> mailshare or 192.168.107.25share
    • B. Once copied to the server, access the console and type the following: Code: mv /srv/samba/share/zcs* /temp Method #2 - Download directly from the server A. Obtain the download URL from the Zimbra OSE page B. Type the following commands on the server console (using the correct URL) Code: cd /temp wget http://files.zimbra.com/downloads/7.1.3_GA/zcs7.1.3_GA_3346.UBUNTU10_64.20110928134610.tgz 5. Extract the archive by typing the following commands: Code: mkdir /temp/install tar -C /temp/install --strip-components=1 -xzf /temp/zcs* 6. Start the installation program by typing the following: Code: cd /temp/install ./install.sh 7. The following is displayed: Code: Operations logged to /tmp/install.log.3865 Checking for existing installation... zimbra-ldap...FOUND zimbra-ldap-7.1.2_GA_3268.UBUNTU10_64 zimbra-logger...FOUND zimbra-logger7.1.2_GA_3268.UBUNTU10_64 zimbra-mta...FOUND zimbra-mta-7.1.2_GA_3268.UBUNTU10_64 zimbra-snmp...FOUND zimbra-snmp-7.1.2_GA_3268.UBUNTU10_64 zimbra-store...FOUND zimbra-store-7.1.2_GA_3268.UBUNTU10_64 zimbra-apache...FOUND zimbra-apache7.1.2_GA_3268.UBUNTU10_64 zimbra-spell...FOUND zimbra-spell-7.1.2_GA_3268.UBUNTU10_64 zimbra-convertd...NOT FOUND zimbra-memcached...NOT FOUND zimbra-proxy...NOT FOUND
    • zimbra-archiving...NOT FOUND zimbra-cluster...NOT FOUND zimbra-core...FOUND zimbra-core-7.1.2_GA_3268.UBUNTU10_64 ZCS upgrade from 7.1.2 to 7.1.3 will be performed. Saving existing configuration file to /opt/zimbra/.saveconfig PLEASE READ THIS AGREEMENT CAREFULLY BEFORE USING THE SOFTWARE. ZIMBRA, INC. ("ZIMBRA") WILL ONLY LICENSE THIS SOFTWARE TO YOU IF YOU FIRST ACCEPT THE TERMS OF THIS AGREEMENT. BY DOWNLOADING OR INSTALLING THE SOFTWARE, OR USING THE PRODUCT, YOU ARE CONSENTING TO BE BOUND BY THIS AGREEMENT. IF YOU DO NOT AGREE TO ALL OF THE TERMS OF THIS AGREEMENT, THEN DO NOT DOWNLOAD, INSTALL OR USE THE PRODUCT. License Terms for the Zimbra Collaboration Suite: http://www.zimbra.com/license/zimbra_public_eula_2.1.html 8. Agree to license agreement? Press y {ENTER} 9. If you are missing any prerequisites, exit the installation and install them. You can install all at one time by separating each name with a space after apt-get install such as the following: Code: aptitude install sysstat sqlite3 10. Do you want to verify message store database integrity? Press y {ENTER} 11. You will probably see the following output: Code: Verifying integrity of message store databases. This may take a while. Starting mysqld...done. mysqld is alive Database errors found. /opt/zimbra/mysql/bin/mysqlcheck --defaultsfile=/opt/zimbra/conf/my.cnf -S /opt/zimbra/db/mysql.sock -A -C -s -u root --password=gHEZvnZd323VvkK0uaUcDprGtYI --auto-repair mysql.general_log Error : You can't use locks with log tables. mysql.slow_log Error : You can't use locks with log tables. mysqld is alive Stopping mysqld... done. Checking for installable packages Found zimbra-core Found zimbra-ldap Found zimbra-logger
    • Found Found Found Found Found Found Found zimbra-mta zimbra-snmp zimbra-store zimbra-apache zimbra-spell zimbra-memcached zimbra-proxy The Zimbra Collaboration Suite appears already to be installed. It can be upgraded with no effect on existing accounts, or the current installation can be completely removed prior to installation for a clean install. 12. Do you wish to upgrade? Press y {ENTER} 13. Install zimbra-memcached? Press n {ENTER} (this is not a multi-server) 14. Install zimbra-proxy? Press n {ENTER} (this is not a multi-server) 15. The system will be modified. Continue? Press y {ENTER} 16. See notes below: I could never get past this step unless I answered "no" when asked to verify database integrity a few steps earlier. 17. Notify Zimbra of your installation? Press n {ENTER} (feel free to answer yes if this is not a test system) 18. Press {ENTER} to exit setup. NOTE: Log files placed in /opt/zimbra/log After Upgrading 1. Make a verified offline backup and be sure to place it at an offsite location as well. 2. Since this is a virtual environment, might as well shutdown the server and create a snapshot of the server. Give it a name like After 7.1.3 Upgrade and description of Ubuntu Server 10.04.4 LTS, Zimbra 7.1.3 OSE installed, Static IP: 192.168.107.25. 3. Once the snapshot is complete, start up the Ubuntu server. 4. Now go through your checklist of things to verify that your server is online and fully functional. If something is not working right, now is your opportunity to restore. 5. If you are satisfied that everything is solid, delete the snapshots. --------------------------- Start of notes after step 15 above -----------------------------EDIT: I just did a remote VPN to work to check on how things were going but after 2 hours, it has not progressed much since I left it. When I left work, the last message displayed was "Shutting down zimbra mail" Two hours later, it only added one additional message saying "Backing up the ldap database" If nothing has gone wrong, I suppose this would be a REALLY long upgrade if it actually had a bunch of users and a large mail storage.
    • EDIT: It is now 5 hours since I started the upgrade and it is still sitting on the "Backing up the ldap database" message. That is at least 3 hours just on this part alone. I suppose something is broken somewhere. hmmm....with a simple one-off install / upgrade and having a problem with the upgrade, it is not inspiring a lot of confidence at this point. I'll assume it is something stupid that I did (or failed to do). I'll do some problem analysis and resolution research tomorrow morning. EDIT: I was just too curious to go to bed and sleep on it. So I pressed CTRL+Z to send the install into the background as a paused process. I then typed bg 1 which told it to continue running in the background while I did some research. The 1st thing I checked was my drive space even though the install script checks for enough space at the beginning. I typed df -h and it does show I have quite a bit of space available so running out of room was not the issue. I did notice it took several seconds to complete the command, which was the display of the remote Windows server mount. Thinking that the mount might be causing a slowdown, I typed umount /mnt/backup to temporarily unmount my remote backup server. But that did not seem to be the problem either. I then looked at the /tmp/install.log file and sure enough, I found the problem: Code: Stopping ldap...Failed. Killing slapd with pid 874...................................................................... ......................................................................... ......................................................................... ......................................................................... ........... gave up waiting! COMMAND: /opt/zimbra/libexec/zmslapcat /opt/zimbra/data/ldap hdb_db_open: database "": unclean shutdown detected; attempting recovery. hdb_db_open: database "": recovery skipped in read-only mode. Run manual recovery if errors are encountered. For whatever reason, the ldap service failed to stop for the install script. That makes no sense to me whatsoever. I have run /etc/init.d/zimbra stop both manually and through scripts and never once had a problem stopping or starting the services. I'm at a loss for how to resolve the problem or keep it from happening again in the future. I've read several times before to simply run the install script again after an error to see if it can get past it so I'll give that a shot right now (after I manually try to control the services). Since the install is still running in the background, I'm going to type fg 1 to bring it back to the foreground so I can press CTRL+C to break out and stop the install script and try again. EDIT: Several hours later (after I woke up from falling asleep in my chair) I have finally figured out how to get past that problem. It did not matter if zimbra services were stopped or started before running install.sh, it always hung up during the ldap backup if I answered yes when asked to verify message store database integrity. Answering no to that question allowed the install to proceed past the ldap backup step. --------------------------- End of notes after step 15 above ------------------------------
    • As a side note, I was logged into my mailbox on one of my browser tabs and did not realize it before starting the upgrade. When I went back to that tab to login and see if things were working, I was greeted with the following awesome and convenient little message: Code: Warning ------The server software has been updated. It is recommended that you reload your browser so that you are up to date. Old version: 7.1.2_GA_3268 20110804130819 20110804-1312 FOSS New version: 7.1.3_GA_3346 20110928134610 20110928-1350 FOSS Would you like to reload the browser now? Warning: if you choose not to reload, there is a chance you will encounter errors. [Yes] [No] This popup box was sitting on top of another popup box that said the following (which must have popped up 1st): Code: Zimbra -----Cannot connect to server. [Show Details] [OK] __________________ Type su - zimbra -c "zmcontrol -v" to get your version and copy that into your profile (more info here) Last edited by LHammonds; 05-08-2012 at 07:19 AM.. #28 (permalink) 05-09-2012, 08:42 AM edgarosy New Member Posts: 4 Very nice post. I woul like to get similar information for a fresh installation of Zimbra on a Red-Hat 6 server and migrating from openldap server instead of Active Directory. #29 (permalink) 05-11-2012, 07:44 PM ccelis5215 Advanced Member Posts: 206
    • Quote: Very nice post. I woul like to get similar information for a fresh installation of Zimbra on a Red-Hat 6 server and migrating from openldap server instead of Active Directory. edgarosy, pls let us know! ccelis