Using filesystem capabilities with rsyncHazel Smith
As presented at the FLOSS UK Unconference 2015.
Updated 2015-02-08: added details of caveats, primarily the fact that CAP_DAC_READ_SEARCH does exactly what it says on the tin, and covering precautions like ensuring that password authentication is *never* allowed for the backuphelper user.
Linux Backup System using rsync, tar, gzipKyle Mathews
Presentation from my first project in the unclass at BYU. Link to writeup -- https://island.byu.edu/wiki/final-report-linux-backup-system-using-rsync-and-tar-and-amazon-s3
Getting instantly up and running with Docker and SymfonyAndré Rømcke
A look into how you can start to use Docker today with ready made setup with php7, nginx, redis, blackfire and so on. How you may extend it, and integrating it into your continuous integration workflow, and how you can setup a continuous deployment workflow using for instance Travis-CI.
Quicklink: https://legacy.joind.in/19070
Using filesystem capabilities with rsyncHazel Smith
As presented at the FLOSS UK Unconference 2015.
Updated 2015-02-08: added details of caveats, primarily the fact that CAP_DAC_READ_SEARCH does exactly what it says on the tin, and covering precautions like ensuring that password authentication is *never* allowed for the backuphelper user.
Linux Backup System using rsync, tar, gzipKyle Mathews
Presentation from my first project in the unclass at BYU. Link to writeup -- https://island.byu.edu/wiki/final-report-linux-backup-system-using-rsync-and-tar-and-amazon-s3
Getting instantly up and running with Docker and SymfonyAndré Rømcke
A look into how you can start to use Docker today with ready made setup with php7, nginx, redis, blackfire and so on. How you may extend it, and integrating it into your continuous integration workflow, and how you can setup a continuous deployment workflow using for instance Travis-CI.
Quicklink: https://legacy.joind.in/19070
This talk is aimed at technical people who have heard of Docker but not used it yet.
Docker is the new way to package, distribute and run your application or microservice. It's the bees-knees. It's the best thing since sliced bread. It's the kings pajamas etc, etc. In this short but packed talk I'll explain what Docker actually is and how it works. I'll show you how much of the hype is real and give a practical demo or two to illustrate the power of Docker. Is Docker the panacea for all ills? This talk will help you make up your mind.
Join us to discover how to use the PHP frameworks and tools you love in the Cloud with Heroku. We will cover best practices for deploying and scaling your PHP apps and show you how easy it can be. We will show you examples of how to deploy your code from Git and use Composer to manage dependencies during deployment. You will also discover how to maintain parity through all your environments, from development to production. If your apps are database-driven, you can also instantly create a database from the Heroku add-ons and have it automatically attached to your PHP app. Horizontal scalability has always been at the core of PHP application design, and by using Heroku for your PHP apps, you can focus on code features, not infrastructure.
Dockerize your Symfony application - Symfony Live NYC 2014André Rømcke
With the advent of docker it is now easier then ever to make sure you develop, test and deploy using the same environment, resulting in no more issues caused by differences or missing libraries. Talk will go into the basics of containers, docker, and showcase how you might setup a basic php + mysql environment for your symfony app.
https://joind.in/12188
Dockerizing Symfony2 application. Why Docker is so cool And what is Docker? And what are Containers? How they works? What are the ecosystem of Docker? And how to dockerize your web application (can be based on Symfony2 framework)?
Wordpress y Docker, de desarrollo a produccionSysdig
Docker esta revolucionando cómo desplegamos nuestras aplicaciones. Desde el entorno de desarrollo hasta la puesta en producción.
Veremos las ventajas que nos aporta Docker para el desarrollo en WordPress, las herramientas y procesos desde el punto de vista de un desarrollador.
A la hora de mover nuestra aplicación WordPress a producción, presentaremos los retos que presenta y las ventajas que aportan herramientas de orquestación como Kubernetes.
Tanto si eres un desarrollador como si también tienes que gestionar los sistemas que alojan tu WordPress, saldrás de esta charla queriendo poner todos tus WordPress en contenedores.
Docker orchestration using core os and ansible - Ansible IL 2015Leonid Mirsky
The last couple of years have seen an increasing interest in Docker and related technologies. One of these technologies is CoreOS, a new operating system built from the ground up for running Docker containers at scale.
In this talk we will learn about CoreOS main concepts and tools. We will get our hands dirty as we work together toward a goal of running a CoreOS cluster on AWS (using Ansible) and running docker containers on it.
The talk will conclude with a discussion on the place of Ansible (and configuration management tools in general) in the "next-generation" stack.
Vagrant - Version control your dev environmentbocribbz
Vagrant facilitates the creation and configuration of lightweight, reproducible, and portable development environments.
It is currently in use at companies like Disqus, BBC, Mozilla, Nokia, and O'Reilly Media. More information about Vagrant is available at: http://www.vagrantup.com/
Links:
Boxes: https://github.com/opscode/bento
Cookbooks: http://community.opscode.com/
LAMP demo: https://github.com/bocribbz/cookbook-lampdemo
Continuous Delivery helps to keep your software and Docker images updated and deploy new versions in production easily. Microservices are great reducing the attack vector and limiting the privileges or credentials access of each piece of your application. Containers provide an opportunity to implement better security, small, immutable, single process and purpose. In this session, we will discover real use case examples on how to make your CI/CD pipeline interact with Docker security tools. But security doesn’t stop where your deployment pipeline ends. How we can prepare for 0-days and policy violations that happen at run-time? Can we make it part of the CI/CD process?
While there have been many improvements around securing containers, there is still a large gap in monitoring the behaviour of containers in production. Sysdig Falco is an open source behavioural activity monitor for containerized environments.
Sysdig Falco can detect and alert on anomalous behaviour at the application, file, system, and network level. In this session get a deep dive into Falco: How does behavioural security differ from existing security solutions like image scanning, seccomp, SELinux or AppArmor? What can Sysdig Falco detect? Building and customizing rules for your Docker and Kubernetes apps. Forensics analysis with Sysdig Inspect even when the container doesn't exist anymore!
Read more on:
https://sysdig.com/blog/docker-runtime-security/
https://sysdig.com/blog/runtime-security-kubernetes-sysdig-falco/
Be a happier developer with Docker: Tricks of the tradeNicola Paolucci
The talk will teach developers to automate and streamline their development environment setups using Docker, covering awesome tricks to make the experience smooth, fast, powerful and repeatable.The topics covered will be a selection amongst:
- Sharing folders into containers
- Transparent tunnels, dynamic ports for your apps
- Tiny Core Linux - the secret horse to super fast container automation
- Dockerfile caching tricks
- Cheap orchestration tricks
From Zero to Hadoop: a tutorial for getting started writing Hadoop jobs on Am...Alexander Dean
Hadoop is everywhere these days, but it can seem like a complex, intimidating ecosystem to those who have yet to jump in.
In this hands-on workshop, Alex Dean, co-founder of Snowplow Analytics, will take you "from zero to Hadoop", showing you how to run a variety of simple (but powerful) Hadoop jobs on Elastic MapReduce, Amazon's hosted Hadoop service. Alex will start with a no-nonsense overview of what Hadoop is, explaining its strengths and weaknesses and why it's such a powerful platform for data warehouse practitioners. Then Alex will help get you setup with EMR and Amazon S3, before leading you through a very simple job in Pig, a simple language for writing Hadoop jobs. After this we will move onto writing a more advanced job in Scalding, Twitter's Scala API for writing Hadoop jobs. For our final job, we will consolidate everything we have learnt by building a more sophisticated job in Scalding.
OSBConf 2015 | Backups with rdiff backup and rsnapshot by christoph mitasch &...NETWAYS
The simpler the backup, the higher is the chance that the backups will be done by users. So let’s look for simple backup methods.
Two known command-line backup tools of this kind are rdiff-backup and rsnapshot. Both tools allow easy-to-setup backups, and impress with a direct file-level access to the last backup copy.
Although the two tools share the same main idea – “incremental forever backups” – they differ in how they reach this goal. While rdiff-backup stores old versions of a file by only saving the changes (increments) compared to the following version, rsnapshot creates a new backup file whenever a file has changed since the last backup. This fact leads to different advantages and disadvantages: while rdiff-backup is very space-efficient, a restore of an older version of a file might take longer. Rsnapshot on the other hand allows fast restores even for older versions, but as a downside it needs more backup space to store the same amount of data.
So which tool should I use? rdiff-backup or rsnapshot? Well, it depends…
In this talk we will explain how rdiff-backup and rsnapshot work in detail, and show different use-cases. Along with that knowledge, you will find the right answer for you which tool to choose.
This talk is aimed at technical people who have heard of Docker but not used it yet.
Docker is the new way to package, distribute and run your application or microservice. It's the bees-knees. It's the best thing since sliced bread. It's the kings pajamas etc, etc. In this short but packed talk I'll explain what Docker actually is and how it works. I'll show you how much of the hype is real and give a practical demo or two to illustrate the power of Docker. Is Docker the panacea for all ills? This talk will help you make up your mind.
Join us to discover how to use the PHP frameworks and tools you love in the Cloud with Heroku. We will cover best practices for deploying and scaling your PHP apps and show you how easy it can be. We will show you examples of how to deploy your code from Git and use Composer to manage dependencies during deployment. You will also discover how to maintain parity through all your environments, from development to production. If your apps are database-driven, you can also instantly create a database from the Heroku add-ons and have it automatically attached to your PHP app. Horizontal scalability has always been at the core of PHP application design, and by using Heroku for your PHP apps, you can focus on code features, not infrastructure.
Dockerize your Symfony application - Symfony Live NYC 2014André Rømcke
With the advent of docker it is now easier then ever to make sure you develop, test and deploy using the same environment, resulting in no more issues caused by differences or missing libraries. Talk will go into the basics of containers, docker, and showcase how you might setup a basic php + mysql environment for your symfony app.
https://joind.in/12188
Dockerizing Symfony2 application. Why Docker is so cool And what is Docker? And what are Containers? How they works? What are the ecosystem of Docker? And how to dockerize your web application (can be based on Symfony2 framework)?
Wordpress y Docker, de desarrollo a produccionSysdig
Docker esta revolucionando cómo desplegamos nuestras aplicaciones. Desde el entorno de desarrollo hasta la puesta en producción.
Veremos las ventajas que nos aporta Docker para el desarrollo en WordPress, las herramientas y procesos desde el punto de vista de un desarrollador.
A la hora de mover nuestra aplicación WordPress a producción, presentaremos los retos que presenta y las ventajas que aportan herramientas de orquestación como Kubernetes.
Tanto si eres un desarrollador como si también tienes que gestionar los sistemas que alojan tu WordPress, saldrás de esta charla queriendo poner todos tus WordPress en contenedores.
Docker orchestration using core os and ansible - Ansible IL 2015Leonid Mirsky
The last couple of years have seen an increasing interest in Docker and related technologies. One of these technologies is CoreOS, a new operating system built from the ground up for running Docker containers at scale.
In this talk we will learn about CoreOS main concepts and tools. We will get our hands dirty as we work together toward a goal of running a CoreOS cluster on AWS (using Ansible) and running docker containers on it.
The talk will conclude with a discussion on the place of Ansible (and configuration management tools in general) in the "next-generation" stack.
Vagrant - Version control your dev environmentbocribbz
Vagrant facilitates the creation and configuration of lightweight, reproducible, and portable development environments.
It is currently in use at companies like Disqus, BBC, Mozilla, Nokia, and O'Reilly Media. More information about Vagrant is available at: http://www.vagrantup.com/
Links:
Boxes: https://github.com/opscode/bento
Cookbooks: http://community.opscode.com/
LAMP demo: https://github.com/bocribbz/cookbook-lampdemo
Continuous Delivery helps to keep your software and Docker images updated and deploy new versions in production easily. Microservices are great reducing the attack vector and limiting the privileges or credentials access of each piece of your application. Containers provide an opportunity to implement better security, small, immutable, single process and purpose. In this session, we will discover real use case examples on how to make your CI/CD pipeline interact with Docker security tools. But security doesn’t stop where your deployment pipeline ends. How we can prepare for 0-days and policy violations that happen at run-time? Can we make it part of the CI/CD process?
While there have been many improvements around securing containers, there is still a large gap in monitoring the behaviour of containers in production. Sysdig Falco is an open source behavioural activity monitor for containerized environments.
Sysdig Falco can detect and alert on anomalous behaviour at the application, file, system, and network level. In this session get a deep dive into Falco: How does behavioural security differ from existing security solutions like image scanning, seccomp, SELinux or AppArmor? What can Sysdig Falco detect? Building and customizing rules for your Docker and Kubernetes apps. Forensics analysis with Sysdig Inspect even when the container doesn't exist anymore!
Read more on:
https://sysdig.com/blog/docker-runtime-security/
https://sysdig.com/blog/runtime-security-kubernetes-sysdig-falco/
Be a happier developer with Docker: Tricks of the tradeNicola Paolucci
The talk will teach developers to automate and streamline their development environment setups using Docker, covering awesome tricks to make the experience smooth, fast, powerful and repeatable.The topics covered will be a selection amongst:
- Sharing folders into containers
- Transparent tunnels, dynamic ports for your apps
- Tiny Core Linux - the secret horse to super fast container automation
- Dockerfile caching tricks
- Cheap orchestration tricks
From Zero to Hadoop: a tutorial for getting started writing Hadoop jobs on Am...Alexander Dean
Hadoop is everywhere these days, but it can seem like a complex, intimidating ecosystem to those who have yet to jump in.
In this hands-on workshop, Alex Dean, co-founder of Snowplow Analytics, will take you "from zero to Hadoop", showing you how to run a variety of simple (but powerful) Hadoop jobs on Elastic MapReduce, Amazon's hosted Hadoop service. Alex will start with a no-nonsense overview of what Hadoop is, explaining its strengths and weaknesses and why it's such a powerful platform for data warehouse practitioners. Then Alex will help get you setup with EMR and Amazon S3, before leading you through a very simple job in Pig, a simple language for writing Hadoop jobs. After this we will move onto writing a more advanced job in Scalding, Twitter's Scala API for writing Hadoop jobs. For our final job, we will consolidate everything we have learnt by building a more sophisticated job in Scalding.
OSBConf 2015 | Backups with rdiff backup and rsnapshot by christoph mitasch &...NETWAYS
The simpler the backup, the higher is the chance that the backups will be done by users. So let’s look for simple backup methods.
Two known command-line backup tools of this kind are rdiff-backup and rsnapshot. Both tools allow easy-to-setup backups, and impress with a direct file-level access to the last backup copy.
Although the two tools share the same main idea – “incremental forever backups” – they differ in how they reach this goal. While rdiff-backup stores old versions of a file by only saving the changes (increments) compared to the following version, rsnapshot creates a new backup file whenever a file has changed since the last backup. This fact leads to different advantages and disadvantages: while rdiff-backup is very space-efficient, a restore of an older version of a file might take longer. Rsnapshot on the other hand allows fast restores even for older versions, but as a downside it needs more backup space to store the same amount of data.
So which tool should I use? rdiff-backup or rsnapshot? Well, it depends…
In this talk we will explain how rdiff-backup and rsnapshot work in detail, and show different use-cases. Along with that knowledge, you will find the right answer for you which tool to choose.
OpenStack Tokyo Meeup - Gluster Storage DayDan Radez
November 2012 Tokyo OpenStack meetup was dedicated to using Gluster storage. This presentation showed the fuse mount method to integrating gluster into OpenStack. There are new drivers that have been developed that make mounting gluster volumes to instances more efficient. This presentation doesn't show how to use them.
How I Learned to Stop Worrying and Love the Cloud - Wesley Beary, Engine YardSV Ruby on Rails Meetup
Wesley Beary: Cloud computing scared the crap out of me - the quirks and nightmares
of provisioning computing and storage on AWS, Terremark, Rackspace,
etc - until I took the bull by the horns. Let me now show you how I
tamed that bull.
Learn how to easily get started cloud computing with fog. It gives you
the reins within any Ruby application or script. If you can control
your infrastructure choices, you can make better choices in
development and get what you need in production.
You'll get an overview of fog and concrete examples to give you a head
start on your own provisioning workflow.
More info at http://blog.carlossanchez.eu/tag/devops
Video en español: http://youtu.be/E_OE4l3t5BA
The DevOps movement aims to improve communication between developers and operations teams to solve critical issues such as fear of change and risky deployments. But the same way that Agile development would likely fail without continuous integration tools, the DevOps principles need tools to make them real, and provide the automation required to actually be implemented. Most of the so called DevOps tools focus on the operations side, and there should be more than that, the automation must cover the full process, Dev to QA to Ops and be as automated and agile as possible. Tools in each part of the workflow have evolved in their own silos, and with the support of their own target teams. But a true DevOps mentality requires a seamless process from the start of development to the end in production deployments and maintenance, and for a process to be successful there must be tools that take the burden out of humans.
Apache Maven has arguably been the most successful tool for development, project standardization and automation introduced in the last years. On the operations side we have open source tools like Puppet or Chef that are becoming increasingly popular to automate infrastructure maintenance and server provisioning.
In this presentation we will introduce an end-to-end development-to-production process that will take advantage of Maven and Puppet, each of them at their strong points, and open source tools to automate the handover between them, automating continuous build and deployment, continuous delivery, from source code to any number of application servers managed with Puppet, running either in physical hardware or the cloud, handling new continuous integration builds and releases automatically through several stages and environments such as development, QA, and production.
Open Source Backup Conference 2014: Workshop bareos introduction, by Philipp ...NETWAYS
It gives an introduction to the architecture of Bareos, and how the components of Bareos interact. The configuration of Bareos will be discussed and the main Bareos features will be shown. As a practical part of the workshop the adaption of the preconfigured standard backup scheme to the attendees’ wishes will be developed.
Attendees are kindly asked to contribute configuration tasks that they want to have solved.
Puppet for Java developers - JavaZone NO 2012Carlos Sanchez
Example code at https://github.com/carlossg/puppet-for-java-devs
More info at http://blog.carlossanchez.eu/tag/devops
Video at http://vimeo.com/49483627
Puppet is an infrastructure-as-code tool that allows easy and automated provisioning of servers, defining the packages, configuration, services,... in code. Enabling DevOps culture, tools like Puppet help drive Agile development all the way to operations and systems administration, and along with continuous integration tools like Jenkins, it is a key piece to accomplish repeatability and continuous delivery, automating the operations side during development, QA or production, and enabling testing of systems configuration.
Traditionally a field for system administrators, Puppet can empower developers, allowing both to collaborate coding the infrastructure needed for their developments, whether it runs in hardware, virtual machines or cloud. Developers and sysadmins can define what JDK version must be installed, application server, version, configuration files, war and jar files,... and easily make changes that propagate across all nodes.
Using Vagrant, a command line automation layer for VirtualBox, they can also spin off virtual machines in their local box, easily from scratch with the same configuration as production servers, do development or testing and tear them down afterwards.
We’ll show how to install and manage Puppet nodes with JDK, multiple application server instances with installed web applications, database, configuration files and all the supporting services. Including getting up and running with Vagrant and VirtualBox for quickstart and Puppet experiments, as well as setting up automated testing of the Puppet code.
Manage WordPress with Awesome using wp cliGetSource
Manage WordPress with Awesome using wp-cli talk from WordCamp Vegas 2012.
Does updating all your plugins by running `wp plugin update –all` sound too good to be true? Enter wp-cli, an open source WordPress management tool. Learn how to install it locally or globally on your host, perform common WordPress administration tasks, and expand its functionality with plugins of your own.
Presented by Mike Schroder (@GetSource/DH-Shredder)
Get hands-on with security features and best practices to protect your containerized services. Learn to push and verify signed images with Docker Content Trust, and collaborate with delegation roles. Intermediate to advanced level Docker experience recommended, participants will be building and pushing with Docker during the workshop.
Led By Docker Security Experts:
Riyaz Faizullabhoy
David Lawrence
Viktor Stanchev
Experience Level: Intermediate to advanced level Docker experience recommended
Similar to How we setup Rsync-powered Incremental Backups (20)
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
12. We needed to backup Netware servers too, so we used NCP mounting , which was very slow.
13. The Scripts echo "Here we rsync DATA volume on CASADMLX server to /bkup/casadmlx/DATA/ folder on CASARKLX server" rsync -av --link-dest= /bkup/casadmlx/DATA/current root@casadmlx:/media/nss/DATA/ /bkup/casadmlx/DATA/ $date rm /bkup/casadmlx/DATA/current ln -s /bkup/casadmlx/DATA/ $date /bkup/casadmlx/DATA/current rm -r /bkup/casadmlx/DATA/ `date --date='30 days ago' +%F`
14. The Scripts echo "Here we rsync DATA volume on CASADM server to /bkup/casadm/DATA/ folder on CASARKLX server" ncpmount -S casadm -A casadm -U .bkup.cs.users.selkirk -P provo /mnt/casadm/ rsync -av --link-dest= /bkup/casadm/DATA/current /mnt/casadm/DATA/ /bkup/casadm/DATA/ $date rm /bkup/casadm/DATA/current ln -s /bkup/casadm/DATA/ $date /bkup/casadm/DATA/current rm -r /bkup/casadm/DATA/ `date --date='30 days ago' +%F` umount /mnt/casadm/
15. The Scripts echo "Here we rsync INSTRUCTORS volume on FS1 server to /bkup/fs1/INSTRUCTORS/ folder on CASARKLX server" mount.cifs //fs1/INSTRUCTORS /mnt/fs1/ -o user=Administrator,pass=provo rsync -av -e 'ssh -p7774' --link-dest= /bkup/fs1/INSTRUCTORS/current /mnt/fs1/INSTRUCTORS/ /bkup/fs1/INSTRUCTORS/ $date rm /bkup/fs1/INSTRUCTORS/current ln -s /bkup/fs1/INSTRUCTORS/ $date /bkup/fs1/INSTRUCTORA/current rm -r /bkup/fs1/INSTRUCTORS/ `date --date='30 days ago' +%F` umount /mnt/fs1/
16.
17. We wanted to have good error checking, replication, reporting and central control.
18. We also wanted to actually understand what was going on and we wanted a system that would fit our exact needs. So we wrote our own.
19.
20. We decided to implement only the bare-minimum that we needed.
21. The result was a 100-line script that supports custom mount drivers and performs an incremental backup using rsync .
22.
23. Google Code: http://alpine-backup.googlecode.com
43. The QNAPs are NASs running some mystery meat Linux, that was the source of many headaches.
44. Initially, we added more configuration files that would copy the backups from the main backup server to the offsite one, but this grew quite complicated.
47. But, when a QNAP reads the configuration files, it would run in slave mode and pull the backups from the backup server, instead of the live server.
TTP Conference 2011 – Celebrating 25 years of supporting Novell Solutions
According to the man page, --link-dest hard links to files in the link destination when the backup is the same as the source. (ie. identical files).
We set our link dest to current, a symbolic link always pointing to the current backup, then preform the backup. Next we set our current link to the latest backup and delete backups older than 30 days.
This is how we backed up a Netware server using NCP.
..and a Windows one, using CIFS.
This is a screenshot of alpine-backup's code. As you can see, it is quite similar to our bash scripts, but adds error checking and is written in PHP. If you want to have a look at alpine-backup, the URLs are on the bottom.
This what our backup directory looks like. As you can see, only the first backup is large, the rest are quite small, only about 200MB large.
This an example of the configuration file format alpine-backup uses. It is an INI file. (I used this format because PHP had a built-in parser for it.)
This what our mount drivers look like. This particular one actually stops and restarts services for backup. (We use it for iPrint backups.)
Here is a screenshot of the mail we get from cron. As you can see from the summary, the backups are all successful!
This is a diagram of Selkirk College's organizational structure. All our servers are backed up in at least two locations. As you can see, there are two islands: the Nelson-centric one and the Castlegar-centric one. This is due to the fact that there are no fibre links between Castlegar and Nelson.
...about half. (we sometimes use different parameters for our main and offsite backups.)
At this point I'd like to show you our first demo: How no-change syncing works. QNAP doing a Live Backup
QNAP in Slave Mode
Well, this does not explain much, let's have a look at snapup.sh
The first two commands SSH into our GroupWise server and use the perl scripts to create and activate a snapshot. Then we mount the snapshot and perform the backup via SSH.
Now, I'm going to move on to our next demo. Live Post Office
We start up the POA using an a backup.
..and now: some old mail.
alp-dbrestore does not like dashes in DB Names.
Next, I'm going to show you how we restore our MySQL backups. First, we create test article.
Then, we create a new DB with an old version of the site. (Avoid dashes in database name)
We switch Drupal to the restore DB.
The article has now been rolled back.
This is a screenshot of our Oracle backups in action. Since we're running early, I'm going to show you what alpine-backup's code is like.
Any Questions? TTP Conference 2011 – Celebrating 25 years of supporting Novell Solutions
![[object Object],[object Object]](https://image.slidesharecdn.com/rsyncbackup2-110718214608-phpapp01/85/How-we-setup-Rsync-powered-Incremental-Backups-1-320.jpg)
![Rsync Backups ,[object Object],Nicholas and Nick Paun](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)