Introducing the Terascala Intelligent Storage Bridge


Published on

In this presentation, Bryan Cote from Terascala describes the company's new Intelligent Storage Bridge technology.

"HPC workflows are most efficient when researchers have the data they need, when the need it. The Intelligent Storage Bridge acts like an air traffic controller, automatically moving large data sets on and of scratch using the maximum available network, compute, and storage resources. Once a researcher's work is complete, the data can then be transferred to more cost-appropriate enterprise storage for publishing and backup. Automating the workflow and placing the control in the hands of the researchers saves time and removes the need for additional training and data management expertise."

Learn more:
Watch the video presentation:

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Key Points:Terascala appliances are sold in every major market. Customers: this is a representative set of customers. We’re deployed at Fortune 500 companies, large and medium-size federal agencies, and a number of major universities.Applications: we’re focused on data-intensive applications that benefit from a high throughput storage appliance.Business need: we’ve heard from customers who have applications that may take as long as 30 hours or even several days to complete, which affects productivity and time to market. Companies may bring in proprietary solutions to handle scratch data that can run at a much higher performance. This meets the needs of the application in terms of throughput, but it comes at a price. That price is that the scratch data volume is large and unprotected—it’s not the typical NAS storage. These scratch storage systems are:high touch and have high administrative overhead that requires specialized expertisethey are often prone to having many points of failureworkflows are not optimizedscratch data then becomes an island that is not connected to the rest of the IT infrastructure. It satisfies the applications, but it doesn’t present an optimized workflow.
  • Key Points:TeraOS brings everything together so all of the components operate as an appliance with high availability. Without TeraOS, there will be many points of failure and a high cost of administration requiring in-house expertise to manage it. TeraOS removes these complexities. The performance comes from the Terascala Performance Engine. The core TPE has the hardware and software to deliver multiple GB/s to the application. The TPE provides all of the management, analytics, and tuning necessary for optimal performance across a variety of workloads. To get more performance, add more performance extension (TPX) modules. These can be scaled modularly with no limit—you can run 50 GB/s if need be. As modules are added, they operate under the umbrella of TeraOS.TeraView is a single pane of glass that let’s users look at the health and performance of system.All of the high-performance appliances on the marketplace sold by Dell, EMC, and NetApp use TeraOS in this unified interface—consistent across all appliances.
  • Key Points:Use the links provided to visit partner websites for more product information.Additional resources are available on
  • Introducing the Terascala Intelligent Storage Bridge

    1. 1. Company Snapshot - Founded in 2005 and based in Boston, MA - Financial investment from Intel and several venture firms TARGET MARKETS - Manufacturing - Life Sciences CUSTOMERS APPLICATION BUSINESS NEED - Westinghouse - Tradeworx - product design - genomics analysis - long time to results - many points of failure - Oil and Gas - Finance - TGen - Sandia - Sold through Dell, NetApp - Media - Education - Government - Florida State University - University of Minnesota - high frequency trading - seismic processing - scratch data unprotected - workflows not optimized * PARTIAL LIST - animation rendering * PARTIAL LIST - system too high touch 2
    2. 2. Our Product • TeraOS » • The enabling software that is distributed across all components to create a single, high-performance storage appliance Terascala Performance Engine (TPE) » » • The foundation of the storage appliance that includes management and orchestration software, performance and analytics, monitoring, and tuning Base performance is 3-7 GB/s Terascala Performance Extension (TPX) » • TeraView User Interface » • Module that can be added to the Terascala Performance Engine to linearly scale performance by 3-4 GB/s Provides a single unified view into the appliance to see the performance and health of the system Intelligent Storage Bridge » Fast and reliable data movement between traditional enterprise and high performance storage 3
    3. 3. Sold Through Dell, EMC, NetApp • Terascala solutions are available through • MORE INFORMATION DT-HSS • • MORE INFORMATION HPS Rack Case Studies: 4
    4. 4. The Problem • Large file sets are stranded in high-performance storage • Researchers can’t access the data they need, when it’s needed • Project data is unprotected and cannot be leveraged by the rest of the organization X HIGH-PERFORMANCE STORAGE X ENTERPRISE STORAGE
    5. 5. ISB Overview The Intelligent Storage Bridge quickly moves large data sets between temporary and permanent storage, ensuring data is preserved and available for other projects Value • First workflow manager purpose-built for Lustre environments • Improves storage efficiency • Decreases time to complete jobs • Increases the number of jobs that can be run • Access data at any point in the workflow • Valuable project data is protected and leveraged Users Specifications • R&D and engineering application users • HPC / storage admins • 1U rack • Minimum base configuration is 3 nodes • Users define policies by • 3 RU 2+1 redundancy • Time • Event • File age • Scales in 1U increments • 48GBs memory • 2 hot swappable 500GB storage drives • Single port FDR InfiniBand • 500 MB/s per 1U active node • Supports NFS v3, CIFS, Lustre 1.88 • Compatible with NIS
    6. 6. How data is moved today • Data movement is done manually or automatically Data transfer takes 5:41 hours with 1 server • Manual data movement 6 » Usually requires expertise of an admin 5 » Can take days to copy, depending on number of servers being used (see chart) Hours 4 3 • Automatic data movement 2 » Requires scripts to automate 1 » Requires admin expertise and constant monitoring 0 1 3 5 7 9 11 Nodes Either method is prone to errors, requiring the process to be re-started from the beginning
    7. 7. Why the ISB is Unique • • • • • • Appliance Solution High Availability Designed to offload Administration Forms based, no scripting necessary Monitoring integrated with TeraOS Scales through the addition of additional ISB Modules
    8. 8. Forms Based Interface
    9. 9. Target Use Cases and Industries • Use Cases » Data ingest and re-ingest » Job benefits from multiple runs » Integrated with other clusters • Industries » Life Sciences – complex workflow » Manufacturing – comparing multiple jobs » Oil & Gas – iterative analysis of TB data sets
    10. 10. Use Case #1: Optimize Workflow Genomics Sequence Workflow 1. Data from sequencers (intermittent, MB/s) 2. Data availability • Golden image to scale-out NAS • Data to high performance storage 3. Data processing • Data to apps ~3 GB/s or higher 4. Store results 5. Move scratch data to scale-out NAS with Intelligent Storage Bridge 11
    11. 11. Use Case #2: Improve Scratch Efficiency Automate Backup of TBs of Data and Improve Scratch Efficiency NFS ─ ─ ─ ─ Intelligent Storage Bridge CIFS ─ ─ NAS Multiple users/applications sharing scratch at multi-GB/s R/W Working data backed up and protected on NAS Scalable performance and redundant Move large data sets quickly via parallel copy between PFS and different NAS configurations Role-based access controls (RBAC) ISB job management Fully managed by TeraView Before ─ 200 users including 50+ power users running ANSYS, Fluent, STAR ─ NFS solution can’t keep up ─ System crashes unpredictably After ─ ANSYS 5x faster (61 to 11 hours) ─ No drop-outs ─ Tools to manage, analyze, predict and optimize 12
    12. 12. Questions • Additional Resources: » Visit for data sheets, videos, white papers » Blog: » Twitter: @Terascala_Inc • How to buy: » Contact Terascala Bryan Cote Product Management Director M: 603-557-3568 Twitter: @BC_HPC Twitter: @Terascala_Inc Blog:
    13. 13. THANK YOU