High Performance Cloud Computing

This document discusses high performance cloud computing. It covers 1) cloud infrastructure like EC2 instance types, 2) tools for provisioning and managing cloud resources such as AWS CloudFormation and Chef, 3) example HPC applications that can run on the cloud including genomics tools, and 4) best practices around optimizing costs and resources for high performance computing workloads in the cloud.

Chainer v4 and v5

Chainer v4 includes performance improvements like Intel integration and cuDNN enhancements. It also introduces usability features like Sequential chains and reorganized documentation. Chainer v4 allows exporting models to Caffe and ONNX formats. Chainer v5 is planned to improve usability with NumPy compatibility, distributions support, and code generation. It also aims to enhance performance through static subgraph caching.

Technology Updates of PG-Strom at Aug-2014 (PGUnconf@Tokyo)

Kohei KaiGai

Finding New Sub-Atomic Particles on the AWS Cloud (BDT402) | AWS re:Invent 2013

This session will describe how members of the US Large Hadron Collider (LHC) community have benchmarked the usage of Amazon Elastic Compute Cloud (Amazon EC2) resource to simulate events observed by experiments at the European Organization for Nuclear Research (CERN). Miron Livny from the University of Wisconsin-Madison who has been collaborating with the US-LHC community for more than a decade will detail the process for benchmarking high-throughput computing (HTC) applications running across multiple AWS regions using the open source HTCondor distributed computing software. The presentation will also outline the different ways that AWS and HTCondor can help meet the needs of compute intensive applications from other scientific disciplines.

themidgame-tube-slides

Pedro Moy

This document describes the architecture of themidgame-tube, a data platform for analyzing YouTube videos and influencers. It discusses scraping over 19 million videos and 500GB of data from YouTube, using a batch-only data pipeline with AWS EMR, YARN, and MapReduce to process the data. It also presents results from system scalability tests using Spot Instances on EMR, showing the platform can scale to process large amounts of data quickly and cost effectively.

CuPy v4 and v5 roadmap

Microsoft Azure in HPC scenarios

mictc

HPC (High Performance Computing) utilizes parallel processing for applications like actuarial sciences, financial modeling, and genetics research. A hybrid HPC-Azure configuration uses on-premise compute nodes and Azure compute nodes for additional capacity. The head node distributes work units to the compute node clusters. Powershell and the Windows Azure Management Library can automate starting and stopping Azure nodes to control costs in this hybrid configuration.

HPC in the Cloud

Guy Tel-Zur

Advances in GPU Computing

This document summarizes Nvidia's GPU technology conference (GTC16) including announcements about their Tesla P100 GPU and DGX-1 deep learning supercomputer. Key points include: - The new Tesla P100 GPU delivers up to 21 teraflops of performance for deep learning and uses new technologies like NVLink, HBM2 memory, and a page migration engine. - The Nvidia DGX-1 is a deep learning supercomputer powered by 8 Tesla P100 GPUs with over 170 teraflops of performance for training neural networks. - CUDA 8 and unified memory improvements on the P100 enable simpler programming and larger datasets by allowing allocations beyond GPU memory size and

Unlimited Virtual Computing Capacity using the Cloud for Automated Parameter ...

Joseph Luchette

This document discusses using cloud computing infrastructure for automated parameter estimation. It outlines some limitations of traditional computing like limited processors, computing power, and availability. Cloud computing allows leveraging third-party computing capabilities over the internet as a service. Examples are provided of using cloud infrastructure for serial and parallel parameter estimation modeling runs. The conclusions state that parameter estimation using PEST can be performed on cloud infrastructure, eliminating limitations of small to mid-size businesses and providing scalable and low-cost computational resources.

Scale-out AI Training on Massive Core System from HPC to Fabric-based SOC

inside-BigData.com

In this video from Arm HPC Asia 2019, Fu Li from Quantum Cloud presents: Scale out AI Training on Massive Core System from HPC to Fabric based SOC. "The purpose of these workshops has been to bring together the leading Arm vendors, end users and open source development community to discuss the latest products, developments and open source software support in HPC on Arm." Learn more: https://www.linaro.org/events/workshop/arm-hpc-asia-2019/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

The Revolution of Deep Learning

This document discusses the rise of deep learning and its key drivers like big data, GPU acceleration, and better algorithms. It outlines NVIDIA's deep learning SDK and frameworks that provide high performance GPU acceleration for computer vision, speech, language, and recommendation applications. The document notes that industries like cloud computing, healthcare, media, security, and autonomous machines are adopting deep learning. It also highlights NVIDIA's DGX-1 deep learning supercomputer and discusses advances in areas like speech recognition, medical imaging, autonomous driving, robotics, and art through deep learning techniques.

Distributed Multi-GPU Computing with Dask, CuPy and RAPIDS

PeterAndreasEntschev

The search for faster computing remains of great importance to the software community. Relatively inexpensive modern hardware, such as GPUs, allows users to run highly parallel code on thousands, or even millions of cores on distributed systems. Building efficient GPU software is not a trivial task, often requiring a significant amount of engineering hours to attain the best performance. Similarly, distributed computing systems are inherently complex. In recent years, several libraries were developed to solve such problems. However, they often target a single aspect of computing, such as GPU computing with libraries like CuPy, or distributed computing with Dask. Libraries like Dask and CuPy tend to provide great performance while abstracting away the complexity from non-experts, being great candidates for developers writing software for various different applications. Unfortunately, they are often difficult to be combined, at least efficiently. With the recent introduction of NumPy community standards and protocols, it has become much easier to integrate any libraries that share the already well-known NumPy API. Such changes allow libraries like Dask, known for its easy-to-use parallelization and distributed computing capabilities, to defer some of that work to other libraries such as CuPy, providing users the benefits from both distributed and GPU computing with little to no change in their existing software built using the NumPy API.

Taking High Performance Computing to the Cloud: Windows HPC and

Saptak Sen

High Performance Computing (HPC) is expected to be the single largest workload on Windows Azure. This session discusses how Windows HPC Server 2008 R2 SP2 enables our customers to easily run their HPC applications on Windows Azure. It covers different usage scenarios (“bursting” to Windows Azure vs. running everything in Windows Azure),differences between running HPC applications on-premise vs. in Azure,best practices,limitations,etc. Real-world customers and their scenarios are highlighted. The key points are illustrated with live demos of HPC applications running in Windows Azure. This session is a must for everyone who wants to know about HPC and Windows Azure.

MesosCon 2018

Pablo Delgado

This document discusses Netflix's use of Spark workflows on Mesos clusters and how they implement autoscaling. It describes how Netflix runs Spark jobs on Mesos clusters, uses Fenzo to schedule tasks, and implements fixed-size Spark executors. It also discusses how the clusters are autoscaled by monitoring executor usage and resource availability, and scaling the cluster up or down by adding/removing Mesos agents based on utilization thresholds.

産総研におけるプライベートクラウドへの取り組み

Ryousei Takano

The document discusses cloud computing and the AIST Super Cloud. It provides details on 3 common cloud platforms: Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). It then describes AIST's transition from its Super Cluster to the Green Cloud and newer Super Cloud, including hardware specifications. Key cloud management platforms are outlined, including Rocks, Eucalyptus, OpenStack, and OpenNebula. The document focuses on AIST's adoption and use of cloud computing technologies for research.

Ai Forum at Computex 2017 - Keynote Slides by Jensen Huang

NVIDIA Taiwan

人工智慧運算正進入一個嶄新的紀元：GPU 深度學習。以模擬人類的思考方式做為出發點，從無人機、自駕車、機器人到智慧城市，NVIDIA GPU 讓機器得以學習、感知、推論，並協助解決問題。如今人工智慧已融入你我的生活當中，且逐步顛覆產業的發展。 NVIDIA 今年首度於 COMPUTEX 台北國際電腦展舉辦「人工智慧論壇」，內容包括精采的主題演說以及熱門應用議題，從雲端到終端裝置，共同探討人工智慧未來趨勢，並引領台灣進入智慧產業革命。

GPUs in Big Data - StampedeCon 2014

StampedeCon

At StampedeCon 2014, John Tran of NVIDIA presented "GPUs in Big Data." Modern graphics processing units (GPUs) are massively parallel general-purpose processors that are taking Big Data by storm. In terms of power efficiency, compute density, and scalability, it is clear now that commodity GPUs are the future of parallel computing. In this talk, we will cover diverse examples of how GPUs are revolutionizing Big Data in fields such as machine learning, databases, genomics, and other computational sciences.

Cloud Friendly Hadoop and Hive

DataWorks Summit

This document discusses Hadoop and Hive as a service in the cloud using Qubole Data Service. It describes how Qubole allows users to run Hadoop and Hive queries on-demand in the cloud without having to manage infrastructure. Key points include: - Qubole automatically provisions Hadoop clusters in AWS as needed to run queries and scales the clusters up and down based on demand. - It archives job logs and histories to S3 for access to past jobs. The service also auto-configures node types and manages bad nodes. - For Hive, Qubole provides query history, allows testing queries on samples first, and implements optimizations like columnar caches to improve query performance.

Intro to the Distributed Version of TensorFlow

Altoros

Yahoo uses Hadoop clusters separately from deep learning clusters to avoid unnecessary data movement. YARN works well for deep learning, allowing multiple experiments to run concurrently on a single cluster in a cost-effective manner. Previously, GPU resources were manually scheduled using a notepad, which was painful and only worked for a small number of users. The document discusses retraining an Inception-v3 model on 3 specific categories in ~30 minutes on a desktop CPU with over 90% accuracy. It also mentions using one server per task and tools for distributed TensorFlow that summarize logs and initialize from checkpoints.

Adventures in RDS Load Testing

Mike Harnish

This document summarizes load testing experiments conducted on Amazon RDS using an Oracle database. The tests aimed to evaluate RDS performance under different configurations and provide a basis for future load testing. Tests were run using m2.4xlarge and m1.xlarge instance types with varying provisioned IOPS. Key results showed that provisioned IOPS had a significant impact on throughput and latency. Higher IOPS configurations achieved thousands of transactions per second but also had periods of high latency. Lower IOPS configurations had more stable performance but lower throughput. The experiments provided insights into how different factors like instance type, IOPS provisioning, and read/write ratios influence RDS and database performance under load.

kubectl internal / Kubernetes Internal #8

Getting The Most Out of VR | Sinjin Bain

Jessica Tams

Delivered at Casual Connect Europe 2016 Virtual Reality and Augmented (or Mixed) Reality have unique and distinct capabilities from other interactive platforms that require new approaches to development and content creation and management. We will explore some innovative approaches to runtime performance, collaborative development, data and live operations to get the most out of your games in development.

Distributed deep learning optimizations - AI WithTheBest

geetachauhan

HPC on AWS

This document discusses how Amazon Web Services (AWS) provides high-performance computing (HPC) capabilities in the cloud. It notes that HPC workloads require large amounts of data, compute, and people. AWS offers elastic, on-demand infrastructure services like Amazon EC2 and S3 that allow users to pay for only what they use. The document outlines how users can run HPC workloads on AWS using EC2 instance types optimized for compute and GPU-accelerated instances. It shares examples of organizations using AWS for applications like computational fluid dynamics and molecular modeling.

HPC on AWS

ReadMaloney

This document discusses how Amazon Web Services (AWS) provides high-performance computing (HPC) capabilities through elastic cloud computing resources. It describes AWS computing instances like EC2 that provide configurable compute capacity and GPU-accelerated instances. The document also outlines how users can get started with HPC on AWS through a simple 4-step process and notes the ecosystem of independent software vendors supporting HPC workloads on AWS.

What's hot

MEW22 22nd Machine Evaluation Workshop Microsoft

Lee Stott

Automating auto-scaled load balancer based on linux and vm orchestrator

Andrew Yongjoon Kong

HybridAzureCloud

Chris Condo

HPC in the Cloud

Guy Tel-Zur

Advances in GPU Computing

Unlimited Virtual Computing Capacity using the Cloud for Automated Parameter ...

Joseph Luchette

Scale-out AI Training on Massive Core System from HPC to Fabric-based SOC

inside-BigData.com

The Revolution of Deep Learning

Distributed Multi-GPU Computing with Dask, CuPy and RAPIDS

PeterAndreasEntschev

Taking High Performance Computing to the Cloud: Windows HPC and

Saptak Sen

MesosCon 2018

Pablo Delgado

産総研におけるプライベートクラウドへの取り組み

Ryousei Takano

Ai Forum at Computex 2017 - Keynote Slides by Jensen Huang

NVIDIA Taiwan

GPUs in Big Data - StampedeCon 2014

StampedeCon

Cloud Friendly Hadoop and Hive

DataWorks Summit

Intro to the Distributed Version of TensorFlow

Altoros

Adventures in RDS Load Testing

Mike Harnish

kubectl internal / Kubernetes Internal #8

Getting The Most Out of VR | Sinjin Bain

Jessica Tams

Distributed deep learning optimizations - AI WithTheBest

geetachauhan

What's hot (20)

MEW22 22nd Machine Evaluation Workshop Microsoft

Automating auto-scaled load balancer based on linux and vm orchestrator

HybridAzureCloud

HPC in the Cloud

Advances in GPU Computing

Unlimited Virtual Computing Capacity using the Cloud for Automated Parameter ...

Scale-out AI Training on Massive Core System from HPC to Fabric-based SOC

The Revolution of Deep Learning

Distributed Multi-GPU Computing with Dask, CuPy and RAPIDS

Taking High Performance Computing to the Cloud: Windows HPC and

MesosCon 2018

産総研におけるプライベートクラウドへの取り組み

Ai Forum at Computex 2017 - Keynote Slides by Jensen Huang

GPUs in Big Data - StampedeCon 2014

Cloud Friendly Hadoop and Hive

Intro to the Distributed Version of TensorFlow

Adventures in RDS Load Testing

kubectl internal / Kubernetes Internal #8

Getting The Most Out of VR | Sinjin Bain

Distributed deep learning optimizations - AI WithTheBest

Similar to High Performance Cloud Computing

HPC on AWS

HPC on AWS

ReadMaloney

Systems Bioinformatics Workshop Keynote

This document discusses how data science platforms can be built on cloud computing infrastructure like Amazon Web Services (AWS). It highlights how AWS provides scalable, on-demand computing and storage resources that allow data and compute needs to scale rapidly. Example applications and customer case studies are presented to show how various organizations are using AWS for large-scale data analysis, including genomics, computational fluid dynamics, and more. The document argues that distributed, programmable cloud infrastructure can support new types of data-driven science by providing massive, rapidly scaling resources.

High Performance Cloud Computing

High Performance Cloud Computing

Talk at West Coast Association of Shared Resource Directors

Platforms for data science require rethinking data and compute architectures to handle large-scale data in a distributed, programmable way. Cloud platforms like Amazon Web Services can remove infrastructure constraints by providing on-demand, global, secure, programmable and elastic compute and storage resources. This allows building data science platforms that accept all data formats, have evolvable APIs, and treat data as a programmable resource to enable large-scale data analysis across multiple locations.

Deep Dive on Delivering Amazon EC2 Instance Performance

Amazon EC2 provides a broad selection of instance types to deliver high performance for a diverse mix of applications. In this session, we overview the drivers of system performance and discuss in depth how Amazon EC2 instances deliver system performance while also providing elasticity and complete control over your infrastructure. We also detail best practices and share performance tips for getting the most out of your Amazon EC2 instances.

AWS Webcast - An Introduction to High Performance Computing on AWS

The document discusses high performance computing (HPC) on AWS. It begins with an agenda that includes HPC concepts, patterns and practices as well as a demo of CloudFormation Cluster (cfncluster). It then discusses using elastic AWS instance clusters that are neither too large nor too small for jobs. The document covers various pricing models on AWS including on-demand, reserved, spot and dedicated instances. It also discusses AWS services that can be used for HPC workloads like EC2, auto scaling, monitoring with CloudWatch, GPU computing, middleware services and integrated solutions. Finally, popular HPC workloads on AWS are listed in various industries.

Deep Dive on Delivering Amazon EC2 Instance Performance

Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...

Amazon Elastic Compute Cloud (Amazon EC2) provides a broad selection of instance types to accommodate a diverse mix of workloads. In this technical session, we provide an overview of the Amazon EC2 instance platform, key platform features, and the concept of instance generations. We dive into the design choices of the different instance families, including the General Purpose, Compute Optimized, Storage Optimized, and Memory Optimized families. We also detail best practices and share performance tips for getting the most out of your Amazon EC2 instances. Learning Objectives: • Understand the differences between instances • Learn best practices and tips for getting the most out of EC2 instances

SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...

Amazon EC2 provides a broad selection of instance types to accommodate a diverse mix of workloads. In this session, we provide an overview of the Amazon EC2 instance platform, key platform features, and the concept of instance generations. We dive into the current generation design choices of the different instance families, including the General Purpose, Compute Optimized, Storage Optimized, Memory Optimized, and Accelerated Computing (GPU and FPGA) instance families. We also detail best practices and share performance tips for getting the most out of your Amazon EC2 instances.

Intro to High Performance Computing in the AWS Cloud

Visit http:aws.amazon.com/hpc for more information about HPC on AWS. High Performance Computing (HPC) allows scientists and engineers to solve complex science, engineering, and business problems using applications that require high bandwidth, low latency networking, and very high compute capabilities. AWS allows you to increase the speed of research by running high performance computing in the cloud and to reduce costs by providing Cluster Compute or Cluster GPU servers on-demand without large capital investments. You have access to a full-bisection, high bandwidth network for tightly-coupled, IO-intensive workloads, which enables you to scale out across thousands of cores for throughput-oriented applications.

AWS reinvent 2019 recap - Riyadh - Containers and Serverless - Paul Maddox

AWS Riyadh User Group

This document provides an overview and agenda for an AWS storage, compute, containers, serverless, and management tools presentation. It includes summaries of several upcoming AWS services and features related to EBS, S3, EC2, EKS, Fargate, Lambda, and AWS Cost Optimizer. The speaker is introduced as Paul Maddox, Principal Architect at AWS, with a background in development, SRE, and systems architecture.

SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...

Amazon EC2 provides a broad selection of instance types to accommodate a diverse mix of workloads. In this session, we provide an overview of the Amazon EC2 instance platform, key platform features, and the concept of instance generations. We dive into the current generation design choices of the different instance families, including the General Purpose, Compute Optimized, Storage Optimized, Memory Optimized, and Accelerated Computing (GPU and FPGA) instance families. We also detail best practices and share performance tips for getting the most out of your Amazon EC2 instances.

Backend.AI Technical Introduction (19.09 / 2019 Autumn)

Lablup Inc.

Deep Dive on Amazon EC2 instances

Amazon Web Services LATAM

Arquitetura Hibrida - Integrando seu Data Center com a Nuvem da AWS

The document discusses using AWS services like EC2, VPC, Auto Scaling and others to build a hybrid architecture that integrates an organization's on-premises data center with the AWS cloud. It provides overviews of EC2 instance types, Auto Scaling capabilities, and how to use VPC to connect networks and define routing and security rules. The hybrid model allows leveraging AWS' elastic infrastructure while integrating it with existing IT systems, enabling innovation without being constrained by data center capacity limits or costs.

OSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp

NETWAYS

Kristian Köhntopp, Principal in Core Infrastructure at Booking.com, currently building Kubernetes Clusters and Datacenters. In previous lives, Kris has been working on getting Openstack, SDN and distributed file systems to work at SysEleven, managing MySQL infrastructure at Booking.com, doing performance and architecture consulting for MySQL AB, and working as a senior security engineer at web.de.

Deep Dive on Delivering Amazon EC2 Instance Performance

Real-world High Performance & High Throughput Computing on AWS - AWS PS Summi...

This session will give you practical approaches to efficiently and securely manage your data using AWS, including ready-made HPC tools and services to quickly get started analyzing that data. We will demonstrate using Alces Flight to interact with and process aerial imagery data to build geographic data sets. We’ll also point to the future of new serverless architectures like AWS Lambda to compute at massive scale. Speakers: Adrian White – Technical Business Development Lead, Research & Technical Computing, Asia Pacific, AWS Dr Matthew Berryman – Managing Director, Across the Cloud Pty Ltd and Chair, University of Wollongong High Performance Steering Committee Level: 300

Similar to High Performance Cloud Computing (20)

HPC on AWS

Systems Bioinformatics Workshop Keynote

High Performance Cloud Computing

Talk at West Coast Association of Shared Resource Directors

Deep Dive on Delivering Amazon EC2 Instance Performance

AWS Webcast - An Introduction to High Performance Computing on AWS

Deep Dive on Delivering Amazon EC2 Instance Performance

Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...

SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...

Intro to High Performance Computing in the AWS Cloud

AWS reinvent 2019 recap - Riyadh - Containers and Serverless - Paul Maddox

SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...

Backend.AI Technical Introduction (19.09 / 2019 Autumn)

Deep Dive on Amazon EC2 instances

Arquitetura Hibrida - Integrando seu Data Center com a Nuvem da AWS

OSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp

Deep Dive on Delivering Amazon EC2 Instance Performance

Real-world High Performance & High Throughput Computing on AWS - AWS PS Summi...

More from Deepak Singh

Intel Theater Presentation - SC11

HPC with Amazon EC2 provides high performance computing resources using Amazon EC2 instances. It offers: 1) Scalable infrastructure with on-demand or reserved instances across multiple regions. 2) High performance instance types optimized for compute, memory, storage and networking. 3) Tools for cluster computing using MPI, placement groups for low latency, and benchmark results showing performance comparable to local clusters. 4) Orchestration using AWS services like CloudFormation for automation and provisioning, as well as open source tools for configuration management and job scheduling.

Platforms for Data Science - Computing on the Brink

#arseniclife

Scientists discovered a bacteria that can use arsenic instead of phosphorus in its DNA, a finding that was controversial. The discovery was first reported on blogs and Twitter six months before being published in a peer-reviewed journal. The scientist leading the research said any discussion of the findings would need to be properly moderated and peer-reviewed. However, some criticized the findings as "hogwash".

Talk at NCRR P41 Director's Meeting

Amazon Web Services (AWS) provides a platform for life science research in the cloud. It offers scalable and secure infrastructure services that can be used to store and analyze large datasets. Researchers can use AWS to run genomic analysis programs in parallel using MapReduce. AWS also provides open datasets that researchers can access and analyze in the cloud. Working with AWS, researchers can build data science platforms tailored to their specific needs and take advantage of AWS's expertise in scalable infrastructure.

Platforms for data science

Platforms for data science in 3 sentences: Data science now deals with vast amounts of data from many sources, and cloud platforms provide scalable and programmable infrastructure that is well-suited to handle large-scale data and computation. The cloud allows data scientists to move analysis to where the data is stored and take advantage of utilities like Amazon Web Services to optimize costly resources. AWS and cloud platforms can partner with data scientists to build customized solutions for their specific computational and data handling needs.

Discovery 2015 Workshop

This document provides an overview of scientific computing using Amazon Web Services. It discusses key AWS services like Amazon EC2, S3, SQS, SNS and how they can be used to build scalable, reliable, secure and cost-effective applications in the cloud. EC2 provides virtual computing resources that can be easily increased or decreased as needed. Other services like S3 provide storage, SQS provides messaging and SNS provides notifications. Together these services allow applications to take advantage of the cloud's on-demand, flexible nature.

Bio-IT World 2010 - Keynote talk

The document discusses scientific computing using Amazon Web Services (AWS). It describes how AWS provides scalable, highly available, and cost-effective infrastructure services that allow users to process massive amounts of data. Key AWS services highlighted include Amazon S3 for storage, EC2 for compute, and tools like auto scaling and load balancing that allow applications to automatically scale up or down based on demand.

Talk at Microsoft Cloud Futures 2010

This document discusses how scientific computing can utilize Amazon Web Services (AWS). It outlines how AWS provides scalable compute and storage resources through services like EC2, S3, and MapReduce. These resources allow researchers to process vast amounts of data and run computationally intensive simulations. The document provides examples of scientific projects in genomics, physics, and other domains that have leveraged AWS to process large datasets and perform parallel computing tasks more quickly and cost effectively than traditional computing methods and hardware.

NHGRI Cloud Computing talk

This document discusses using Amazon Web Services for scientific computing. It describes how AWS provides scalable, highly available, and flexible cloud computing resources including compute, storage, databases, and other services. Some key benefits highlighted include the ability to rapidly scale resources up or down as needed, pay only for what is used, and build applications without worrying about managing infrastructure. Examples are given of how AWS is used for data analysis, genomic research, and distributed computing applications.

Plenary Talk at ACAT 2010

This document discusses using Amazon Web Services for scientific computing. It describes how AWS provides scalable, cost-effective, and reliable cloud infrastructure that researchers can use on-demand. Specific AWS services that can benefit scientific applications are mentioned, including Amazon EC2 for flexible computing power, S3 for data storage, and Elastic MapReduce for hosted Hadoop services. Examples are given of scientific projects that have successfully used AWS, such as genome analysis and astrophysics simulations. The document argues that AWS enables new approaches to doing science by providing access to vast, on-demand computing resources and platforms for distributed applications and data sharing.

Masterworks talk on Big Data and the implications of petascale science

Big Data and Biology: The implications of petascale science This document discusses how big data is impacting biology and biomedical research. It notes that the amount of biological and biomedical data is growing exponentially, from gigabytes per week to petabytes per year. This massive increase in data is due to factors like genome sequencing producing terabytes per week. Storing, managing, processing and analyzing such large amounts of data presents many challenges. The document discusses how distributed file systems and MapReduce frameworks like Hadoop are helping to address these challenges by allowing data to be distributed across clusters and processed in parallel. It also discusses how cloud computing platforms can provide scalable solutions to help researchers manage and analyze big biological data.

Talk given at "Cloud Computing for Systems Biology" workshop

This document discusses the role of cloud computing in biology and big data. It describes how cloud platforms like Amazon Web Services provide scalable, cost-effective and reliable infrastructure for storing and analyzing large biological datasets. The document outlines how researchers are using cloud computing to collaborate on projects, run computationally intensive analyses, and develop new tools and applications for processing life sciences data in the cloud.

Hadoop for Bioinformatics

This document discusses how Hadoop can be used for bioinformatics applications. It provides examples of how Hadoop has been used to efficiently process large genomic datasets, such as read mapping and genome assembly, in a distributed, parallel manner. Hadoop allows bioinformatics workflows and algorithms to be rethought and scaled to handle the growing size of genomic data. Key applications discussed include read mapping, variant discovery, and de novo assembly.

Big Data & the networked future of Science (at Ignite Seattle 7)

The document discusses how big data and networked technologies are transforming science. It notes that the Human Genome Project took 15 years to sequence one genome while now a single genome can be sequenced in weeks. It also notes that the 1000 Genomes Project now generates around 100 terabytes of data per week, over 2 petabytes per year. The document highlights several quotes about how data and networked technologies are allowing more collaboration and new insights in science.

Science Big, Science Connected

The document discusses many topics related to informatics, data science, and collective intelligence on the web. It touches on data storage and management, open data initiatives like 1000 Genomes, challenges of data volume, distributed and networked computing models, harnessing collective intelligence through social networks and communities, and architectural principles for data-driven platforms that enable data collection, analysis, and reuse through visualization and mashups.

Bioscreencast: Capturing the life sciences frame by frame

Searching Science

The document discusses various challenges and topics related to scientific information retrieval. Some of the key topics discussed include searching across literature and repositories for linked information, searching outside traditional literature and repositories, evaluating relevance and trust of search results, integrating patent information into the scientific worldview, limitations of search tools like Google Scholar, and using machine learning in scientific search. The document raises questions about building tools to help non-specialists search and make sense of large amounts of returned information.

Nanotechnology and medicine

An Open Scientific Future