This document lists files in a directory related to documentation for an engine project. It includes documentation files for Lua, C#, C++, libraries, modules, tasks, and how to run samples. It also includes source files related to language bindings between Lua, C++ and C#.
Project collaboration between Stanford University and NIST (National Institute of Standards and Technology) to preserve the Stephen M. Cabrinety Collection in the History of Microcomputing, ca. 1975-1995
Splunk talk at the AWS Big Data Meetup in Palo Alto on Nov 17 2015stevemcpherson
Ledion Bitincka from Splunk spoke at the AWS Big Data Meetup in Palo Alto and give an overview of Splunk’s processing pipeline topology and explained their approach to indexing data at scale.
Open Source Backup Cpnference 2014: Bareos in scientific environments, by Dr....NETWAYS
To backup 110 (partly virtualized) Linux servers the Max Planck Institute for Radio Astronomy has been using Bareos for 5 years now. The full backup volume is constantly growing and has just passed the 35 TiB mark with up to 6 million files per TiB. Naturally there were problems with scalability and flexibility which needed to be addressed.
We are using 2 Spectra Logic T950 (LTO5/LTO6) tape libraries, 40 TiB of disk backup space, and a dedicated 1GbE/10GbE backup LAN.
As it may be an inspiration to other users, we would like to share our experience utilizing virtual full backups, concurrent jobs, backup of Heartbeat/DRBD Failover Clusters and integrating Bareos with REAR for disaster recovery.
Coming from TSM, passing Bacula on the way, we finally found our destination with Bareos!
The Max Planck Institute for Neurological Research operates several brain scanners for human and animal studies. Imaging techniques used here comprise magnetic resonance imaging (MRI), positron emission tomography (PET), optical imaging and microscopy.
Research is often interdisciplinary, including contributions from the fields of biology, physics, medicine, psychology, genetics, biochemistry, radiochemistry – with very heterogeneous characteristics of data and analysis methods. Backup requirements range between file systems with literally millions of very small files (DICOM raw data or FSL intermediate results) to files of 200 GB+ size (PET listmode).
“Good Scientific Practice” mandates backup/archiving primary data and “everything else needed to reproduce published results” (tools, documentation of tool chains, intermediate results) – which is a veritable challenge in a high-end, dynamic lab environment.
Until recently, we have used a HSM system from Sun/Oracle Inc (SAM-FS) to meet our requirements of backup and archiving, in particular, using HSM-type filesystems for scientific computing in order to have a fine-grained backup.
However, a significantly larger and more powerful system was needed and we are now migrating to a Quantum i6000 (LTO-6) tape library with Grau OpenArchiver as HSM frontend. With help from our colleagues in Bonn (MPI for Radio Astronomy), we were able to use Bareos for archiving some vital filesystems (backup-to-disk using a HSM file system with WORM tapes; one job per file; file archives < 5 GB; mostly unixoid backup clients).
We are very pleased with the performance, ease of handling and flexibility this approach offers, e.g. when using incremental backups of virtual machines, listing the 5 largest files can tell a lot about a system’s “health”; pre- and posthooks allow some interesting security features in an ESX-cluster environment (taking network interfaces automatically up before saving sensitive data and shutting the interfaces down afterwards); analysing backup reports reveal longterm trends for hot spots, etc.
Project collaboration between Stanford University and NIST (National Institute of Standards and Technology) to preserve the Stephen M. Cabrinety Collection in the History of Microcomputing, ca. 1975-1995
Splunk talk at the AWS Big Data Meetup in Palo Alto on Nov 17 2015stevemcpherson
Ledion Bitincka from Splunk spoke at the AWS Big Data Meetup in Palo Alto and give an overview of Splunk’s processing pipeline topology and explained their approach to indexing data at scale.
Open Source Backup Cpnference 2014: Bareos in scientific environments, by Dr....NETWAYS
To backup 110 (partly virtualized) Linux servers the Max Planck Institute for Radio Astronomy has been using Bareos for 5 years now. The full backup volume is constantly growing and has just passed the 35 TiB mark with up to 6 million files per TiB. Naturally there were problems with scalability and flexibility which needed to be addressed.
We are using 2 Spectra Logic T950 (LTO5/LTO6) tape libraries, 40 TiB of disk backup space, and a dedicated 1GbE/10GbE backup LAN.
As it may be an inspiration to other users, we would like to share our experience utilizing virtual full backups, concurrent jobs, backup of Heartbeat/DRBD Failover Clusters and integrating Bareos with REAR for disaster recovery.
Coming from TSM, passing Bacula on the way, we finally found our destination with Bareos!
The Max Planck Institute for Neurological Research operates several brain scanners for human and animal studies. Imaging techniques used here comprise magnetic resonance imaging (MRI), positron emission tomography (PET), optical imaging and microscopy.
Research is often interdisciplinary, including contributions from the fields of biology, physics, medicine, psychology, genetics, biochemistry, radiochemistry – with very heterogeneous characteristics of data and analysis methods. Backup requirements range between file systems with literally millions of very small files (DICOM raw data or FSL intermediate results) to files of 200 GB+ size (PET listmode).
“Good Scientific Practice” mandates backup/archiving primary data and “everything else needed to reproduce published results” (tools, documentation of tool chains, intermediate results) – which is a veritable challenge in a high-end, dynamic lab environment.
Until recently, we have used a HSM system from Sun/Oracle Inc (SAM-FS) to meet our requirements of backup and archiving, in particular, using HSM-type filesystems for scientific computing in order to have a fine-grained backup.
However, a significantly larger and more powerful system was needed and we are now migrating to a Quantum i6000 (LTO-6) tape library with Grau OpenArchiver as HSM frontend. With help from our colleagues in Bonn (MPI for Radio Astronomy), we were able to use Bareos for archiving some vital filesystems (backup-to-disk using a HSM file system with WORM tapes; one job per file; file archives < 5 GB; mostly unixoid backup clients).
We are very pleased with the performance, ease of handling and flexibility this approach offers, e.g. when using incremental backups of virtual machines, listing the 5 largest files can tell a lot about a system’s “health”; pre- and posthooks allow some interesting security features in an ESX-cluster environment (taking network interfaces automatically up before saving sensitive data and shutting the interfaces down afterwards); analysing backup reports reveal longterm trends for hot spots, etc.