Download to read offline
Uploaded byAlluxio, Inc.
Introducing What's New in Alluxio 2.4
Alluxio Community Office Hour November 5, 2020 For more Alluxio events: https://www.alluxio.io/events/ Speaker(s): Calvin Jia, Alluxio We are extremely excited to announce the release of Alluxio 2.4.0! Alluxio 2.4.0 focuses on features critical to large scale, production deployments in Cloud and Hybrid Cloud environments. Features such as highly scalable metadata journaling, aggregate cluster metrics monitoring, and automated detection of JVM pauses further improve Alluxio’s suitability for demanding workloads. Devops tools are also key for triaging issues when they occur. In Alluxio 2.4 we further improve the cluster wide log collection framework. Finally, Alluxio is continually expanding its state of the art integrations with frameworks and storage systems. Alluxio 2.4 introduces and improves integrations with Kubernetes, Azure Data Lake Storage, and Apache Ozone. Alluxio 2.4 is also the first Alluxio release that has support for Java 11. In this Office Hour, we will go over: - Expanded metadata service - Cloud native deployment - Simplified DevOps and system monitoring - Support for Java 11
Introducing What's New in Alluxio 2.4
- 1. What’s New inAlluxio 2.4 11/5 Office Hour - Calvin Jia Please Join https://alluxio.io/slack for Q&A
- 2. 2.4 Focus Areas ●Cloud and Hybrid Cloud compatibility ○ Alluxio Data Orchestration Hub ○ Improve number of supported deployments and storage ○ Large number of requests from community ● Production Readiness ○ Journal Scalability ○ Improved DevOps monitoring and debugging tools
- 3. Alluxio Data OrchestrationHub ● Management console for Alluxio clusters ● Wizards for connecting to external storage and validating integrations ● System monitoring ● Configuration management
- 4. New Integrations ● Java11 Support ○ Observed to be beneficial for compute framework performance (ie. Presto, Spark) ● Terraform scripts for GCP ○ Easily spin up Alluxio clusters and customize resources ● Improved support for Docker, K8s, ADLS, Ozone ○ A result of top requests from the community and collaboration ● Integration with HashiCorp Vault ○ Allows centralized secrets management outside of Alluxio
- 5. Embedded Journal ● Improvescalability to 1B files in the namespace ○ Previous design saw limitations for larger namespaces ● Alluxio deployments shift from 3rd party storage dependence to local storage with embedded journal Standby Master Leading Master Standby Master Raft State Change State Change State Change
- 6. DevOps Monitoring andTools ● Alluxio Data Orchestration Hub provides management capabilities ● Alluxio UI has cluster overview metrics ○ Total read/write of the cluster and total number of metadata operations per mount ● Improved utility for log collection ○ bin/alluxio collectInfo all /tmp ● System monitor passively checks for long JVM pauses
- 7. Questions? Alluxio Website -https://www.alluxio.io Alluxio Community Slack Channel - https://www.alluxio.io/slack Alluxio Office Hours & Webinars - https://www.alluxio.io/events Keep an eye out for the Alluxio Data Orchestration Hub Webinar!