Presentation2013
•Download as PPTX, PDF•
0 likes•320 views
This document discusses the development of a new lightweight version of the Eumicrobedb database called Transcriptomicsdb. The new version reduces the database size and dependencies by decreasing the number of tables and views from 329 to 37 and 127 to 18 respectively in Eumicrobedb-Oracle. It also improves query time from 10 seconds to 1.2 seconds and reduces genome upload time from 12-14 hours to 2 hours. The document describes how the new database will help laboratories with limited hosting facilities to store and analyze sequencing data. It notes that the source code will soon be released to allow others to replicate the database without needing an Oracle license and various bioinformatics packages.
Report
Share
Report
Share
Recommended
Infrastructure coders logstash
Logstash is a tool for managing logs that allows users to collect logs from various sources, parse them, and output the data to multiple destinations. It provides inputs to collect logs from different systems, filters to parse and transform the log data, and outputs to send the data to places like Elasticsearch, Graphite, or other applications. The document provides examples of using Logstash to collect Apache logs, parse them using Grok filters, and output the results to Elasticsearch for searching and Graphite via StatsD for metrics.
Spying on the Linux kernel for fun and profit
Do you ever wonder what the kernel is doing while your code is running? This talk will explore some methodologies and techniques (eBPF, ftrace, etc.) to look under the hood of the Linux kernel and understand what it’s actually doing behind the scenes.
Nvidia in bioinformatics
Presentation by Jonathan Cohen & Mark Berger at Bioinformatics conference July 2013. It covers
- GPU Programming in 10 slides
- GPUs in Bioinformatics
- Porting SeqAn to CUDA
- Resources for developers and bioinformatics professionals
Security Monitoring with eBPF
Using the new extended Berkley Packet Filter capabilities in Linux to the improve performance of auditing security relevant kernel events around network, file and process actions.
Gnocchi v3 brownbag
This document summarizes the author's experience optimizing Gnocchi, an open source time-series database, to store metrics for hundreds of thousands of resources over many months. The author describes improving performance by adding Ceph storage nodes, tuning Ceph configurations, minimizing I/O operations, and improving the storage format. Benchmark results show the new version achieves 50% higher write throughput, 40-60% faster computation times, 30-60% better overall performance, and 30-40% fewer operations. Usage hints are also provided to help optimize for different use cases.
Linux kernel tracing superpowers in the cloud
The Linux 4.x series introduced a new powerful engine of programmable tracing (BPF) that allows to actually look inside the kernel at runtime. This talk will show you how to exploit this engine in order to debug problems or identify performance bottlenecks in a complex environment like a cloud. This talk will cover the latest Linux superpowers that allow to see what is happening “under the hood” of the Linux kernel at runtime. I will explain how to exploit these “superpowers” to measure and trace complex events at runtime in a cloud environment. For example, we will see how we can measure latency distribution of filesystem I/O, details of storage device operations, like individual block I/O request timeouts, or TCP buffer allocations, investigating stack traces of certain events, identify memory leaks, performance bottlenecks and a whole lot more.
Presto Bangalore Meetup1 Presto Raptor@ola
Presto Raptor is a columnar storage system designed to work natively with Presto that provides real-time analytics capabilities. Raptor is optimized for high performance on flash storage and scales to handle large volumes of data and high query throughput. Key features of Raptor include bucketed tables to co-locate related data and enable fast joins, temporal columns to optimize queries on time-series data, and physical data awareness to skip unnecessary data during queries. Raptor can be used for real-time dashboards, funnels, and event analytics on large datasets stored in its distributed database.
zmq.rs - A brief history of concurrency in Rust
This document summarizes the history of concurrency approaches used in the Rust zmq.rs library. It started with a unified 1:1 threading model but later switched to using M:N threading to allow more flexibility. It discusses green threads, native threads, the libgreen and green-rs crates, and how the project will move to using the mio crate for asynchronous I/O instead of libgreen. The goal is to rewrite zmq.rs to follow the C++ ZeroMQ design of using a few worker threads each with an event loop and manually mapping sockets to workers.
Recommended
Infrastructure coders logstash
Logstash is a tool for managing logs that allows users to collect logs from various sources, parse them, and output the data to multiple destinations. It provides inputs to collect logs from different systems, filters to parse and transform the log data, and outputs to send the data to places like Elasticsearch, Graphite, or other applications. The document provides examples of using Logstash to collect Apache logs, parse them using Grok filters, and output the results to Elasticsearch for searching and Graphite via StatsD for metrics.
Spying on the Linux kernel for fun and profit
Do you ever wonder what the kernel is doing while your code is running? This talk will explore some methodologies and techniques (eBPF, ftrace, etc.) to look under the hood of the Linux kernel and understand what it’s actually doing behind the scenes.
Nvidia in bioinformatics
Presentation by Jonathan Cohen & Mark Berger at Bioinformatics conference July 2013. It covers
- GPU Programming in 10 slides
- GPUs in Bioinformatics
- Porting SeqAn to CUDA
- Resources for developers and bioinformatics professionals
Security Monitoring with eBPF
Using the new extended Berkley Packet Filter capabilities in Linux to the improve performance of auditing security relevant kernel events around network, file and process actions.
Gnocchi v3 brownbag
This document summarizes the author's experience optimizing Gnocchi, an open source time-series database, to store metrics for hundreds of thousands of resources over many months. The author describes improving performance by adding Ceph storage nodes, tuning Ceph configurations, minimizing I/O operations, and improving the storage format. Benchmark results show the new version achieves 50% higher write throughput, 40-60% faster computation times, 30-60% better overall performance, and 30-40% fewer operations. Usage hints are also provided to help optimize for different use cases.
Linux kernel tracing superpowers in the cloud
The Linux 4.x series introduced a new powerful engine of programmable tracing (BPF) that allows to actually look inside the kernel at runtime. This talk will show you how to exploit this engine in order to debug problems or identify performance bottlenecks in a complex environment like a cloud. This talk will cover the latest Linux superpowers that allow to see what is happening “under the hood” of the Linux kernel at runtime. I will explain how to exploit these “superpowers” to measure and trace complex events at runtime in a cloud environment. For example, we will see how we can measure latency distribution of filesystem I/O, details of storage device operations, like individual block I/O request timeouts, or TCP buffer allocations, investigating stack traces of certain events, identify memory leaks, performance bottlenecks and a whole lot more.
Presto Bangalore Meetup1 Presto Raptor@ola
Presto Raptor is a columnar storage system designed to work natively with Presto that provides real-time analytics capabilities. Raptor is optimized for high performance on flash storage and scales to handle large volumes of data and high query throughput. Key features of Raptor include bucketed tables to co-locate related data and enable fast joins, temporal columns to optimize queries on time-series data, and physical data awareness to skip unnecessary data during queries. Raptor can be used for real-time dashboards, funnels, and event analytics on large datasets stored in its distributed database.
zmq.rs - A brief history of concurrency in Rust
This document summarizes the history of concurrency approaches used in the Rust zmq.rs library. It started with a unified 1:1 threading model but later switched to using M:N threading to allow more flexibility. It discusses green threads, native threads, the libgreen and green-rs crates, and how the project will move to using the mio crate for asynchronous I/O instead of libgreen. The goal is to rewrite zmq.rs to follow the C++ ZeroMQ design of using a few worker threads each with an event loop and manually mapping sockets to workers.
Stat2013
The document provides information about performing chi-square tests and choosing appropriate statistical tests. It discusses key concepts like the null hypothesis, degrees of freedom, and expected versus observed values. Examples are provided to illustrate chi-square tests for goodness of fit and comparison of proportions. The document also compares parametric and non-parametric tests, providing examples of when each would be used.
Lecture 3,4
Genome projects, Secondgen and Thirdgen genome sequencing, application of genome sequencing in predicting disease genes
26 nov2013seminar
This document summarizes information from several genomics and bioinformatics research groups and projects. It discusses:
- The ENCODE project and its focus areas including databases, data mining, visualization, transcriptomics, alternative splicing, sequencing pipelines, comparative genomics, epigenomics, and population genomics.
- Tools and databases for variant analysis from the 1000 Genomes Project and FORGE Consortium.
- The Genome Modeling System from The Genome Institute at Washington University for analyzing TCGA, ICGC, 1000 Genomes, and PCGP data.
- Using RNA-seq technology to reveal the transcriptome and methods for isolating translated mRNA.
- Resources for analyzing Human Microbiome Project data
Databases Part II
The document discusses several topics related to biodiversity databases and identification tools:
- The Encyclopedia of Life is a collaborative effort to bring together information about 1.9 million named species on the internet freely.
- 17 countries contain 70% of global biodiversity and are considered "megadiverse."
- The Barcode of Life project uses DNA barcoding to identify species using markers like COI for animals, ITS for fungi, and rbcL and matK for plants.
- GenBank and related NCBI databases like PubMed, Nucleotide, and Protein are important tools for depositing and retrieving sequence data using services like ESearch and ESummary.
Lecture5,6
Many diseases are caused by genetic mutations. Over 4000 diseases are linked to altered genes, including heart disease, cancer, autoimmune disorders, and diabetes. Specific mutations are associated with certain cancers, such as mutations in the RB1 and BRCA1 genes which can lead to retinoblastoma and breast cancer respectively. Large-scale projects like the Cancer Genome Atlas and ENCODE project aim to catalogue all mutations in cancers and across the human genome. Immunogenetics research examines the genetic links to immune-related disorders. The HapMap and 1000 Genomes projects studied genetic variants and helped map human genetic diversity.
Genome sequencingprojects
This document provides an overview of genome sequencing. It discusses the history of genome sequencing, from early sequencing of small viruses in the 1970s to larger genomes like yeast and the human genome. The document outlines different sequencing technologies over time, from Sanger sequencing to newer single-molecule approaches. It also summarizes key genome projects like ENCODE and 1000 Genomes that have provided insights into non-coding regulatory elements and human genetic variation.
Lecture 1,2
This document discusses genomics and genome sequencing. It provides an overview of the history of genome sequencing including early organisms sequenced like bacteriophage. It describes how genomes are sequenced through library construction, cloning, and strategies like Sanger sequencing. Applications of genome sequencing are also mentioned such as predicting genes, studying genome organization and evolution, and understanding the genetic basis of disease.
Multithreading Fundamentals
Everybody knows the lock keyword, but how does it implemented? What are its performance characteristics. Gael Fraiteur scratches the surface of multithreaded programming in .NET and goes deep through the Windows Kernel down to CPU microarchitecture.
Hadoop - Disk Fail In Place (DFIP)
Bharath Mundlapudi presented on Disk Fail Inplace in Hadoop. He discussed how a single disk failure currently causes an entire node to be blacklisted. With newer hardware trends of more disks per node, this wastes significant resources. His team developed a Disk Fail Inplace approach where Hadoop can tolerate disk failures until a threshold. This included separating critical and user files, handling failures at startup and runtime in DataNode and TaskTracker, and rigorous testing of the new approach.
Under The Hood Of A Shard-Per-Core Database Architecture
This document summarizes the key design decisions behind ScyllaDB's shard-per-core database architecture. It discusses how ScyllaDB addresses the challenges of scaling databases across hundreds of CPU cores by utilizing an asynchronous task model with one thread and one data shard per CPU core. This allows for linear scalability. It also overhauls the I/O scheduling to prioritize workloads and maximize throughput from SSDs under mixed read/write workloads. Benchmark results show ScyllaDB's architecture can handle petabyte-scale databases with high performance and low latency even on commodity hardware.
Hadoop Operations - Best practices from the field
Talk about Hadoop Operations and Best Practices for building and maintaining Hadoop cluster.
Talk was held at the data2day conference in Karlsruhe, Germany on 27.11.2014
Sphinx at Craigslist in 2012
These are the slides from my talk at the 2012 Sphinx Search Day in Santa Clara, California. It provides a high-level picture of where Sphinx is used at craigslist, a bit of history, issues, and future work.
Lrz kurs: big data analysis
The document provides an overview of big data analysis and parallel programming tools for R. It discusses what constitutes big data, popular big data applications, and relevant hardware and software. It then covers parallel programming challenges and approaches in R, including using multicore processors with the multicore package, SMP and cluster programming with foreach and doMC/doSNOW, NoSQL databases like Redis with doRedis, and job scheduling. The goal is to help users effectively analyze big data in R by leveraging parallelism.
Performance
Lightning talk showing various aspectos of software system performance. It goes through: latency, data structures, garbage collection, troubleshooting method like workload saturation method, quick diagnostic tools, famegraph and perfview
Firefox Crash Reporting (@ Open Source Bridge)
Socorro is Mozilla's crash analysis system that collects, processes, and analyzes crash reports from Firefox. It receives over 2,300 crashes per minute and stores over 110 terabytes of crash data across 120 physical servers. Socorro provides various reports and visualizations to help Mozilla engineers analyze crashes and regressions. Upcoming work includes improving search, analytics, querying capabilities, and configuration management to further scale the system.
Extreme Scripting July 2009
A talk presented at an NSF Workshop on Data-Intensive Computing, July 30, 2009.
Extreme scripting and other adventures in data-intensive computing
Data analysis in many scientific laboratories is performed via a mix of standalone analysis programs, often written in languages such as Matlab or R, and shell scripts, used to coordinate multiple invocations of these programs. These programs and scripts all run against a shared file system that is used to store both experimental data and computational results.
While superficially messy, the flexibility and simplicity of this approach makes it highly popular and surprisingly effective. However, continued exponential growth in data volumes is leading to a crisis of sorts in many laboratories. Workstations and file servers, even local clusters and storage arrays, are no longer adequate. Users also struggle with the logistical challenges of managing growing numbers of files and computational tasks. In other words, they face the need to engage in data-intensive computing.
We describe the Swift project, an approach to this problem that seeks not to replace the scripting approach but to scale it, from the desktop to larger clusters and ultimately to supercomputers. Motivated by applications in the physical, biological, and social sciences, we have developed methods that allow for the specification of parallel scripts that operate on large amounts of data, and the efficient and reliable execution of those scripts on different computing systems. A particular focus of this work is on methods for implementing, in an efficient and scalable manner, the Posix file system semantics that underpin scripting applications. These methods have allowed us to run applications unchanged on workstations, clusters, infrastructure as a service ("cloud") systems, and supercomputers, and to scale applications from a single workstation to a 160,000-core supercomputer.
Swift is one of a variety of projects in the Computation Institute that seek individually and collectively to develop and apply software architectures and methods for data-intensive computing. Our investigations seek to treat data management and analysis as an end-to-end problem. Because interesting data often has its origins in multiple organizations, a full treatment must encompass not only data analysis but also issues of data discovery, access, and integration. Depending on context, data-intensive applications may have to compute on data at its source, move data to computing, operate on streaming data, or adopt some hybrid of these and other approaches.
Thus, our projects span a wide range, from software technologies (e.g., Swift, the Nimbus infrastructure as a service system, the GridFTP and DataKoa data movement and management systems, the Globus tools for service oriented science, the PVFS parallel file system) to application-oriented projects (e.g., text analysis in the biological sciences, metagenomic analysis, image analysis in neuroscience, information integration for health care applications, management of experimental data from X-ray sources, diffusion tensor imaging for computer aided diagnosis), and the creation and operation of national-scale infrastructures, including the Earth System Grid (ESG), cancer Biomedical Informatics Grid (caBIG), Biomedical Informatics Research Network (BIRN), TeraGrid, and Open Science Grid (OSG).
For more information, please see www.ci.uchicago/swift.
You suck at Memory Analysis
From the current offensive and defensive technique arsenal, memory analysis applied to volatile memory is far from being the most explored channel. It is more likely to hear about input validation attacks or attacks against the protocol & cryptography while keys, passphrases, credit card numbers and other precious artifacts are kept unsafely in memory. This analysis arises as a mine waiting to be explored since it is sustained by one of the most vulnerable and unavoidable resource to systems, memory. From Java to Stuxnex, as well as Windows but without forgetting the Cloud, I will try to show some scenarios where these techniques can be applied, its impact as a threat and bring an important and fun subject not just to those who work in forensics but also to penetration testers as myself. Finally, I will also try to show how can this be used for defensive technologies as tools for monitoring and protection in networks with systems in production.
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Twitter's operations team manages software performance, availability, capacity planning, and configuration management for Twitter. They use metrics, logs, and analysis to find weak points and take corrective action. Some techniques include caching everything possible, moving operations to asynchronous daemons, and optimizing databases to reduce replication delay and locks. The team also created several open source projects like CacheMoney for caching and Kestrel for asynchronous messaging.
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter and Finding your own Fail Whale document discusses Twitter operations. The operations team manages software performance, availability, capacity planning, and configuration management using metrics, logs, and data-driven analysis to find weak points and take corrective action. They use managed services for infrastructure to focus on computer science problems. The document outlines Twitter's rapid growth and challenges in maintaining performance as traffic increases. It provides recommendations around caching, databases, asynchronous processing, and other techniques Twitter uses to optimize performance under heavy load.
Fixing twitter
Fixing Twitter and Finding your own Fail Whale document discusses Twitter operations. The Twitter operations team focuses on software performance, availability, capacity planning, and configuration management using metrics, logs, and science. They use a dedicated managed services team and run their own servers instead of cloud services. The document outlines Twitter's rapid growth and challenges in maintaining performance. It discusses strategies for monitoring, analyzing metrics to find weak points, deploying changes, and improving processes through configuration management and peer reviews.
Fixing_Twitter
Twitter's operations team manages software performance, availability, capacity planning, and configuration management. They use metrics, logs, and analysis to find weak points and take corrective action. Some techniques include caching everything possible, moving operations to asynchronous daemons, optimizing databases, and instrumenting all systems. Their goal is to process requests asynchronously when possible and avoid overloading relational databases.
Loadays managing my sql with percona toolkit
Percona Toolkit includes tools for monitoring and optimizing MySQL performance. Pt-diskstats summarizes disk I/O statistics from /proc/diskstats in an interactive table, showing read and write rates, sizes, and other metrics for each disk or partition. Pt-ioprofile measures I/O usage to identify which files MySQL is accessing and how it spends time reading, writing, and syncing data. These tools help administrators understand where disk I/O is going and identify opportunities to optimize storage usage.
More Related Content
Viewers also liked
Stat2013
The document provides information about performing chi-square tests and choosing appropriate statistical tests. It discusses key concepts like the null hypothesis, degrees of freedom, and expected versus observed values. Examples are provided to illustrate chi-square tests for goodness of fit and comparison of proportions. The document also compares parametric and non-parametric tests, providing examples of when each would be used.
Lecture 3,4
Genome projects, Secondgen and Thirdgen genome sequencing, application of genome sequencing in predicting disease genes
26 nov2013seminar
This document summarizes information from several genomics and bioinformatics research groups and projects. It discusses:
- The ENCODE project and its focus areas including databases, data mining, visualization, transcriptomics, alternative splicing, sequencing pipelines, comparative genomics, epigenomics, and population genomics.
- Tools and databases for variant analysis from the 1000 Genomes Project and FORGE Consortium.
- The Genome Modeling System from The Genome Institute at Washington University for analyzing TCGA, ICGC, 1000 Genomes, and PCGP data.
- Using RNA-seq technology to reveal the transcriptome and methods for isolating translated mRNA.
- Resources for analyzing Human Microbiome Project data
Databases Part II
The document discusses several topics related to biodiversity databases and identification tools:
- The Encyclopedia of Life is a collaborative effort to bring together information about 1.9 million named species on the internet freely.
- 17 countries contain 70% of global biodiversity and are considered "megadiverse."
- The Barcode of Life project uses DNA barcoding to identify species using markers like COI for animals, ITS for fungi, and rbcL and matK for plants.
- GenBank and related NCBI databases like PubMed, Nucleotide, and Protein are important tools for depositing and retrieving sequence data using services like ESearch and ESummary.
Lecture5,6
Many diseases are caused by genetic mutations. Over 4000 diseases are linked to altered genes, including heart disease, cancer, autoimmune disorders, and diabetes. Specific mutations are associated with certain cancers, such as mutations in the RB1 and BRCA1 genes which can lead to retinoblastoma and breast cancer respectively. Large-scale projects like the Cancer Genome Atlas and ENCODE project aim to catalogue all mutations in cancers and across the human genome. Immunogenetics research examines the genetic links to immune-related disorders. The HapMap and 1000 Genomes projects studied genetic variants and helped map human genetic diversity.
Genome sequencingprojects
This document provides an overview of genome sequencing. It discusses the history of genome sequencing, from early sequencing of small viruses in the 1970s to larger genomes like yeast and the human genome. The document outlines different sequencing technologies over time, from Sanger sequencing to newer single-molecule approaches. It also summarizes key genome projects like ENCODE and 1000 Genomes that have provided insights into non-coding regulatory elements and human genetic variation.
Lecture 1,2
This document discusses genomics and genome sequencing. It provides an overview of the history of genome sequencing including early organisms sequenced like bacteriophage. It describes how genomes are sequenced through library construction, cloning, and strategies like Sanger sequencing. Applications of genome sequencing are also mentioned such as predicting genes, studying genome organization and evolution, and understanding the genetic basis of disease.
Viewers also liked (7)
Similar to Presentation2013
Multithreading Fundamentals
Everybody knows the lock keyword, but how does it implemented? What are its performance characteristics. Gael Fraiteur scratches the surface of multithreaded programming in .NET and goes deep through the Windows Kernel down to CPU microarchitecture.
Hadoop - Disk Fail In Place (DFIP)
Bharath Mundlapudi presented on Disk Fail Inplace in Hadoop. He discussed how a single disk failure currently causes an entire node to be blacklisted. With newer hardware trends of more disks per node, this wastes significant resources. His team developed a Disk Fail Inplace approach where Hadoop can tolerate disk failures until a threshold. This included separating critical and user files, handling failures at startup and runtime in DataNode and TaskTracker, and rigorous testing of the new approach.
Under The Hood Of A Shard-Per-Core Database Architecture
This document summarizes the key design decisions behind ScyllaDB's shard-per-core database architecture. It discusses how ScyllaDB addresses the challenges of scaling databases across hundreds of CPU cores by utilizing an asynchronous task model with one thread and one data shard per CPU core. This allows for linear scalability. It also overhauls the I/O scheduling to prioritize workloads and maximize throughput from SSDs under mixed read/write workloads. Benchmark results show ScyllaDB's architecture can handle petabyte-scale databases with high performance and low latency even on commodity hardware.
Hadoop Operations - Best practices from the field
Talk about Hadoop Operations and Best Practices for building and maintaining Hadoop cluster.
Talk was held at the data2day conference in Karlsruhe, Germany on 27.11.2014
Sphinx at Craigslist in 2012
These are the slides from my talk at the 2012 Sphinx Search Day in Santa Clara, California. It provides a high-level picture of where Sphinx is used at craigslist, a bit of history, issues, and future work.
Lrz kurs: big data analysis
The document provides an overview of big data analysis and parallel programming tools for R. It discusses what constitutes big data, popular big data applications, and relevant hardware and software. It then covers parallel programming challenges and approaches in R, including using multicore processors with the multicore package, SMP and cluster programming with foreach and doMC/doSNOW, NoSQL databases like Redis with doRedis, and job scheduling. The goal is to help users effectively analyze big data in R by leveraging parallelism.
Performance
Lightning talk showing various aspectos of software system performance. It goes through: latency, data structures, garbage collection, troubleshooting method like workload saturation method, quick diagnostic tools, famegraph and perfview
Firefox Crash Reporting (@ Open Source Bridge)
Socorro is Mozilla's crash analysis system that collects, processes, and analyzes crash reports from Firefox. It receives over 2,300 crashes per minute and stores over 110 terabytes of crash data across 120 physical servers. Socorro provides various reports and visualizations to help Mozilla engineers analyze crashes and regressions. Upcoming work includes improving search, analytics, querying capabilities, and configuration management to further scale the system.
Extreme Scripting July 2009
A talk presented at an NSF Workshop on Data-Intensive Computing, July 30, 2009.
Extreme scripting and other adventures in data-intensive computing
Data analysis in many scientific laboratories is performed via a mix of standalone analysis programs, often written in languages such as Matlab or R, and shell scripts, used to coordinate multiple invocations of these programs. These programs and scripts all run against a shared file system that is used to store both experimental data and computational results.
While superficially messy, the flexibility and simplicity of this approach makes it highly popular and surprisingly effective. However, continued exponential growth in data volumes is leading to a crisis of sorts in many laboratories. Workstations and file servers, even local clusters and storage arrays, are no longer adequate. Users also struggle with the logistical challenges of managing growing numbers of files and computational tasks. In other words, they face the need to engage in data-intensive computing.
We describe the Swift project, an approach to this problem that seeks not to replace the scripting approach but to scale it, from the desktop to larger clusters and ultimately to supercomputers. Motivated by applications in the physical, biological, and social sciences, we have developed methods that allow for the specification of parallel scripts that operate on large amounts of data, and the efficient and reliable execution of those scripts on different computing systems. A particular focus of this work is on methods for implementing, in an efficient and scalable manner, the Posix file system semantics that underpin scripting applications. These methods have allowed us to run applications unchanged on workstations, clusters, infrastructure as a service ("cloud") systems, and supercomputers, and to scale applications from a single workstation to a 160,000-core supercomputer.
Swift is one of a variety of projects in the Computation Institute that seek individually and collectively to develop and apply software architectures and methods for data-intensive computing. Our investigations seek to treat data management and analysis as an end-to-end problem. Because interesting data often has its origins in multiple organizations, a full treatment must encompass not only data analysis but also issues of data discovery, access, and integration. Depending on context, data-intensive applications may have to compute on data at its source, move data to computing, operate on streaming data, or adopt some hybrid of these and other approaches.
Thus, our projects span a wide range, from software technologies (e.g., Swift, the Nimbus infrastructure as a service system, the GridFTP and DataKoa data movement and management systems, the Globus tools for service oriented science, the PVFS parallel file system) to application-oriented projects (e.g., text analysis in the biological sciences, metagenomic analysis, image analysis in neuroscience, information integration for health care applications, management of experimental data from X-ray sources, diffusion tensor imaging for computer aided diagnosis), and the creation and operation of national-scale infrastructures, including the Earth System Grid (ESG), cancer Biomedical Informatics Grid (caBIG), Biomedical Informatics Research Network (BIRN), TeraGrid, and Open Science Grid (OSG).
For more information, please see www.ci.uchicago/swift.
You suck at Memory Analysis
From the current offensive and defensive technique arsenal, memory analysis applied to volatile memory is far from being the most explored channel. It is more likely to hear about input validation attacks or attacks against the protocol & cryptography while keys, passphrases, credit card numbers and other precious artifacts are kept unsafely in memory. This analysis arises as a mine waiting to be explored since it is sustained by one of the most vulnerable and unavoidable resource to systems, memory. From Java to Stuxnex, as well as Windows but without forgetting the Cloud, I will try to show some scenarios where these techniques can be applied, its impact as a threat and bring an important and fun subject not just to those who work in forensics but also to penetration testers as myself. Finally, I will also try to show how can this be used for defensive technologies as tools for monitoring and protection in networks with systems in production.
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Twitter's operations team manages software performance, availability, capacity planning, and configuration management for Twitter. They use metrics, logs, and analysis to find weak points and take corrective action. Some techniques include caching everything possible, moving operations to asynchronous daemons, and optimizing databases to reduce replication delay and locks. The team also created several open source projects like CacheMoney for caching and Kestrel for asynchronous messaging.
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter and Finding your own Fail Whale document discusses Twitter operations. The operations team manages software performance, availability, capacity planning, and configuration management using metrics, logs, and data-driven analysis to find weak points and take corrective action. They use managed services for infrastructure to focus on computer science problems. The document outlines Twitter's rapid growth and challenges in maintaining performance as traffic increases. It provides recommendations around caching, databases, asynchronous processing, and other techniques Twitter uses to optimize performance under heavy load.
Fixing twitter
Fixing Twitter and Finding your own Fail Whale document discusses Twitter operations. The Twitter operations team focuses on software performance, availability, capacity planning, and configuration management using metrics, logs, and science. They use a dedicated managed services team and run their own servers instead of cloud services. The document outlines Twitter's rapid growth and challenges in maintaining performance. It discusses strategies for monitoring, analyzing metrics to find weak points, deploying changes, and improving processes through configuration management and peer reviews.
Fixing_Twitter
Twitter's operations team manages software performance, availability, capacity planning, and configuration management. They use metrics, logs, and analysis to find weak points and take corrective action. Some techniques include caching everything possible, moving operations to asynchronous daemons, optimizing databases, and instrumenting all systems. Their goal is to process requests asynchronously when possible and avoid overloading relational databases.
Loadays managing my sql with percona toolkit
Percona Toolkit includes tools for monitoring and optimizing MySQL performance. Pt-diskstats summarizes disk I/O statistics from /proc/diskstats in an interactive table, showing read and write rates, sizes, and other metrics for each disk or partition. Pt-ioprofile measures I/O usage to identify which files MySQL is accessing and how it spends time reading, writing, and syncing data. These tools help administrators understand where disk I/O is going and identify opportunities to optimize storage usage.
Pilot Hadoop Towards 2500 Nodes and Cluster Redundancy
Hadoop has become a critical part of Criteo's operations. What started out as a proof of concept has turned into two in-house bare-metal clusters of over 2200 nodes. Hadoop contains the data required for billing and, perhaps even more importantly, the data used to create the machine learning models, computed every 6 hours by Hadoop, that participate in real time bidding for online advertising.
Two clusters do not necessarily mean a redundant system, so Criteo must plan for any of the disasters that can destroy a cluster.
This talk describes how Criteo built its second cluster in a new datacenter and how to do it better next time. How a small team is able to run and expand these clusters is explained. More importantly the talk describes how a redundant data and compute solution at this scale must function, what Criteo has already done to create this solution and what remains undone.
Ruby and Distributed Storage Systems
This document discusses using Ruby for distributed storage systems. It describes components like Bigdam, which is Treasure Data's new data ingestion pipeline. Bigdam uses microservices and a distributed key-value store called Bigdam-pool to buffer data. The document discusses designing and testing Bigdam using mocking, interfaces, and integration tests in Ruby. It also explores porting Bigdam-pool from Java to Ruby and investigating Ruby's suitability for tasks like asynchronous I/O, threading, and serialization/deserialization.
There's no magic... until you talk about databases
Norbert Hartl discussed various technologies used in software projects. He noted that complexity, overly complex code, and excessive JavaScript usage can kill projects. He reviewed technologies like Elasticsearch, microservices, Docker Swarm, Kubernetes, and load balancing. He discussed using MongoDB but noted its limitations and what would be needed for a database to support object-oriented features. He proposed a distributed database approach using file locking and load balancing across multiple machines.
PyConDE / PyData Karlsruhe 2017 – Connecting PyData to other Big Data Landsca...
How Apache Arrow and Apache Parquet are helpful technologies to connect the Python Data ecosystem to other landscapes such as the Java/Scala based Big Data ecosystem.
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
Machine Learning at the Limit
John Canny, UC Berkeley
How fast can machine learning and graph algorithms be? In "roofline" design, every kernel is driven toward the limits imposed by CPU, memory, network etc. This can lead to dramatic improvements: BIDMach is a toolkit for machine learning that uses rooflined design and GPUs to achieve two- to three-orders of magnitude improvements over other toolkits on single machines. These speedups are larger than have been reported for *cluster* systems (e.g. Spark/MLLib, Powergraph) running on hundreds of nodes, and BIDMach with a GPU outperforms these systems for most common machine learning tasks. For algorithms (e.g. graph algorithms) which do require cluster computing, we have developed a rooflined network primitive called "Kylix". We can show that Kylix approaches the rooline limits for sparse Allreduce, and empirically holds the record for distributed Pagerank. Beyond rooflining, we believe there are great opportunities from deep algorithm/hardware codesign. Gibbs Sampling (GS) is a very general tool for inference, but is typically much slower than alternatives. SAME (State Augmentation for Marginal Estimation) is a variation of GS which was developed for marginal parameter estimation. We show that it has high parallelism, and a fast GPU implementation. Using SAME, we developed a GS implementation of Latent Dirichlet Allocation whose running time is 100x faster than other samplers, and within 3x of the fastest symbolic methods. We are extending this approach to general graphical models, an area where there is currently a void of (practically) fast tools. It seems at least plausible that a general-purpose solution based on these techniques can closely approach the performance of custom algorithms.
Bio
John Canny is a professor in computer science at UC Berkeley. He is an ACM dissertation award winner and a Packard Fellow. He is currently a Data Science Senior Fellow in Berkeley's new Institute for Data Science and holds a INRIA (France) International Chair. Since 2002, he has been developing and deploying large-scale behavioral modeling systems. He designed and protyped production systems for Overstock.com, Yahoo, Ebay, Quantcast and Microsoft. He currently works on several applications of data mining for human learning (MOOCs and early language learning), health and well-being, and applications in the sciences.
Similar to Presentation2013 (20)
Under The Hood Of A Shard-Per-Core Database Architecture
Under The Hood Of A Shard-Per-Core Database Architecture
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Pilot Hadoop Towards 2500 Nodes and Cluster Redundancy
Pilot Hadoop Towards 2500 Nodes and Cluster Redundancy
There's no magic... until you talk about databases
There's no magic... until you talk about databases
PyConDE / PyData Karlsruhe 2017 – Connecting PyData to other Big Data Landsca...
PyConDE / PyData Karlsruhe 2017 – Connecting PyData to other Big Data Landsca...
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
More from Sucheta Tripathy
Gal
This document describes Genome Annotator light (GAL), a tool for genome analysis and visualization. GAL integrates genome annotation, comparative genomics, and visualization features into a single virtual machine. It uses a MySQL database with a schema based on the Genome Unified Schema. The front end is built with Perl, CGI, GD, PHP, JavaScript, and Ajax. GAL can annotate genomes with varying levels of data, from simple fasta files to fully annotated genomes. It visualizes genomes through features like a genome browser, gene details pages, and synteny viewers. GAL has been implemented on oomycete and cyanobacterial genomes.
Ramorum2016 final
This document summarizes the assembly of the Phytophthora ramorum genome using PacBio long reads. It describes the error correction and assembly process for two P. ramorum strains, Pr102 and ND886. For Pr102, multiple assembly versions (V1-V5) were generated using different error correction and assembly protocols. The V5 assembly resulted in fewer scaffolds, larger size, and fewer gaps compared to previous versions. For ND886, PacBio reads were error corrected and assembled. Both assemblies captured more repetitive elements compared to previous Sanger-based assemblies. Gene predictions were also improved in number and quality.
Primer designgeneprediction
The document discusses various topics related to gene prediction including primer designing, restriction mapping, and gene prediction. It provides guidelines for primer designing such as avoiding non-specific binding and dimer formation between primers. It also discusses key concepts in gene prediction such as reading frame consistency between exons, codon and dicodon frequencies that can help distinguish coding from non-coding regions, and position specific scoring matrices to predict translation start sites.
Motif andpatterndatabase
This document discusses motifs, which are nucleotide or amino acid sequence patterns associated with biological functions. It defines motifs, patterns, and profiles. Motifs are conserved regions, patterns are qualitative expressions, and profiles are quantitative representations. It discusses tools for de novo prediction of motifs like MEME and resources for motif discovery. Finally, it provides examples of motifs, patterns, and building position specific scoring matrices from sample sequences.
Databases ii
The document discusses various types of biological databases including sequence databases, structure databases, genome databases, and model organism databases. It provides examples of nucleotide databases like Genbank, DDBJ, EMBL-EBI, and TIGR. Genome browsers like UCSC Genome Browser, Ensembl browser, and Integrated Genome Browser are also mentioned. Other topics covered include the Encyclopedia of Life, India Biodiversity, Barcode of Life, data retrieval schemes, bibliographic databases, and database journals.
Snps and microarray
This document outlines the schedule and topics for a class on SNPs and gene expression. The class will have 6 sessions, with groups of 2 students presenting on selected papers in each session. Topics to be covered include an introduction to terminology like forward and reverse genetics; how often SNPs occur in the general population; databases of SNPs like dbSNP and haplotype projects like HapMap; gene expression analysis using microarrays; and experimental design, data analysis, and visualization techniques for microarray data. Papers for each group to present on are also listed. The next class will involve discussing one particular paper on SNPs.
Stat2013
The chi-square test is used to determine if an observed distribution of data differs from the distribution expected if the null hypothesis is true. It requires a contingency table of observed and expected frequencies, a probability value, and degrees of freedom. The chi-square test calculates a test statistic to determine if any difference is statistically significant or likely due to chance. Examples show applying the chi-square test to genetics data on tall and dwarf pea plants and to the distribution of sixes rolled in dice.
Lecture7,8
Pharmacogenetics refers to how genetic differences affect individuals' responses to drugs. It influences different metabolic pathways and is responsible for over 106,000 deaths annually in the US. Certain genetic mutations can determine how effectively drugs are processed in the body. Microarrays are DNA chips that allow researchers to analyze large numbers of genes simultaneously, helping to identify genetic factors influencing drug responses and diseases.
Primer designgeneprediction
The document discusses topics to be covered in an IICB course on 8th December 2012, including primer designing, restriction mapping, and gene prediction. It provides information and guidelines on these topics, such as the appropriate length and properties of primers, the four types of restriction enzymes, and methods for gene prediction including patterns, frame consistency, dicodon frequencies, position-specific scoring matrices, and coding potential. Relevant references and websites discussing these techniques are also listed.
Sequence Alignment,Blast, Fasta, MSA
Similarity searches (Blast, Fasta, sequence alignments), pairwise sequence alignments for the aCSIR Ph.D course work
Biological databases
The document discusses the history and process of genome sequencing, as well as several important genome projects such as the Human Genome Project. It also examines the role of databases in genomic research, describing different types of biological databases and how they can be used to store, organize, and retrieve genomic data. Finally, it provides examples of popular databases and genome browsers that are widely used by researchers.
Human encodeproject
A consortium of 440 scientists from 32 laboratories characterized functional elements in the human genome as part of the ENCyclopedia Of DNA Elements (ENCODE) project. They found that 80% of the genome is biochemically active, with millions of regulatory elements such as promoters, enhancers, and insulators. Many of these elements interact with genes over long distances to control gene expression. This study significantly changes understanding of how the genome works.
Tyler presentation
This document discusses oomycete genomics research that has been funded by the National Science Foundation, USDA CSREES, and USDA NIFA from 2007-2016. Over 120 destructive oomycete pathogen species have been studied, including the genera Phytophthora and Hyaloperonospora. The genomes of several Phytophthora species have been sequenced, with efforts underway to sequence all species through a new initiative with BGI. Comparisons between sequenced oomycete genomes show both conserved and unique genes. Effector genes associated with disease are located in repeat-rich regions of genomes and have expanded through evolution.
Tyler presentation
The document discusses past, present, and future work on oomycete genomics. In the past, genome comparisons found conserved gene order and effector genes associated with repeats. The P. sojae genome is being finished using new sequencing methods. In the future, sequencing capacity is rapidly improving which will enable more oomycete genomes to be sequenced cheaply.
Vbi oomycetes2011 final
The document discusses tools from the US DOE Joint Genome Institute (JGI) for eukaryotic genome annotation and analysis of oomycete genomes. It introduces MycoCosm, a database with over 70 annotated fungal and oomycete genomes that allows manual curation of gene models. The automated annotation pipeline and various gene prediction programs used by JGI are described briefly. The document also outlines the manual curation workflow involving validating gene structures, choosing the best model, and annotating genes.
Chibucos annot go_final
This document discusses Gene Ontology (GO) annotation. It describes how GO provides a common language to describe gene products using controlled vocabularies and ontologies. Key points:
- GO annotation involves assigning GO terms to describe the molecular function, biological process and cellular component of a gene product.
- Literature curation and computational methods are used to make GO annotations based on different types of evidence.
- GO annotations are captured in gene annotation files along with the evidence and reference sources.
Rnaseq forgenefinding
The document discusses next generation sequencing methods and RNA sequencing. It covers topics like sequencing formats, data analysis workflows including mapping, clustering, assembly programs, finding new genes and correcting existing ones. It discusses input file types, calculating sequencing depth, available tools for alignment, output file formats, assembly programs, splice junction prediction, and applications of RNA sequencing like gene expression analysis and annotation.
6th July 2012
This document discusses various methods for structural annotation and gene prediction, including:
- Using patterns like dicodon frequencies, position-specific weight matrices, and coding potential to identify coding regions and translation starts.
- Scoring donor and acceptor sites of potential exons based on models of conserved motifs near splice sites.
- Calculating multiple scores like coding potential, donor score, and acceptor score for potential exons and using classifiers to distinguish true exons from non-exons.
Rnaseq forgenefinding
1. Next generation sequencing techniques like 454 sequencing and Illumina sequencing have enabled high-throughput sequencing of genomes and transcriptomes. These techniques can generate billions of bases of sequence data per day.
2. RNA-Seq is a technique that uses next generation sequencing to determine the RNA transcripts present in a cell and their expression levels. It has advantages over previous methods as it is highly sensitive, quantitative, and can discover isoforms and lowly expressed genes.
3. Analyzing RNA-Seq data involves aligning the short reads to a reference genome or transcriptome using tools like Bowtie or BWA. This allows discovery of novel transcripts, splicing patterns, and quantification of expression levels. Numerous studies have used RNA-
Tyler functional annotation thurs 1120
This document discusses functional annotation and the Gene Ontology. It describes how functional annotation attaches biological information to sequences through searches of databases for homology, domains, and pathways as well as manual curation. Searches include BLAST for homology, Pfam and InterPro for domains, and KEGG and Reactome for pathways. Assignments include EC numbers for metabolic pathways and Gene Ontology terms from automated and manual annotation. Manual annotation combines all evidence and allows incorporation of experimental data but requires more time.
More from Sucheta Tripathy (20)
Recently uploaded
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...Nguyen Thanh Tu Collection
https://app.box.com/s/qhtvq32h4ybf9t49ku85x0n3xl4jhr15Chapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering
Syllabus
Chapter-1
Introduction to objective, scope and outcome the subject
Chapter 2
Introduction: Scope and Specialization of Civil Engineering, Role of civil Engineer in Society, Impact of infrastructural development on economy of country.
Chapter 3
Surveying: Object Principles & Types of Surveying; Site Plans, Plans & Maps; Scales & Unit of different Measurements.
Linear Measurements: Instruments used. Linear Measurement by Tape, Ranging out Survey Lines and overcoming Obstructions; Measurements on sloping ground; Tape corrections, conventional symbols. Angular Measurements: Instruments used; Introduction to Compass Surveying, Bearings and Longitude & Latitude of a Line, Introduction to total station.
Levelling: Instrument used Object of levelling, Methods of levelling in brief, and Contour maps.
Chapter 4
Buildings: Selection of site for Buildings, Layout of Building Plan, Types of buildings, Plinth area, carpet area, floor space index, Introduction to building byelaws, concept of sun light & ventilation. Components of Buildings & their functions, Basic concept of R.C.C., Introduction to types of foundation
Chapter 5
Transportation: Introduction to Transportation Engineering; Traffic and Road Safety: Types and Characteristics of Various Modes of Transportation; Various Road Traffic Signs, Causes of Accidents and Road Safety Measures.
Chapter 6
Environmental Engineering: Environmental Pollution, Environmental Acts and Regulations, Functional Concepts of Ecology, Basics of Species, Biodiversity, Ecosystem, Hydrological Cycle; Chemical Cycles: Carbon, Nitrogen & Phosphorus; Energy Flow in Ecosystems.
Water Pollution: Water Quality standards, Introduction to Treatment & Disposal of Waste Water. Reuse and Saving of Water, Rain Water Harvesting. Solid Waste Management: Classification of Solid Waste, Collection, Transportation and Disposal of Solid. Recycling of Solid Waste: Energy Recovery, Sanitary Landfill, On-Site Sanitation. Air & Noise Pollution: Primary and Secondary air pollutants, Harmful effects of Air Pollution, Control of Air Pollution. . Noise Pollution Harmful Effects of noise pollution, control of noise pollution, Global warming & Climate Change, Ozone depletion, Greenhouse effect
Text Books:
1. Palancharmy, Basic Civil Engineering, McGraw Hill publishers.
2. Satheesh Gopi, Basic Civil Engineering, Pearson Publishers.
3. Ketki Rangwala Dalal, Essentials of Civil Engineering, Charotar Publishing House.
4. BCP, Surveying volume 1
Benner "Expanding Pathways to Publishing Careers"
This presentation was provided by Rebecca Benner, Ph.D., of the American Society of Anesthesiologists, for the second session of NISO's 2024 Training Series "DEIA in the Scholarly Landscape." Session Two: 'Expanding Pathways to Publishing Careers,' was held June 13, 2024.
Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...
Whether you're new to SEO or looking to refine your existing strategies, this webinar will provide you with actionable insights and practical tips to elevate your nonprofit's online presence.
Wound healing PPT
This document provides an overview of wound healing, its functions, stages, mechanisms, factors affecting it, and complications.
A wound is a break in the integrity of the skin or tissues, which may be associated with disruption of the structure and function.
Healing is the body’s response to injury in an attempt to restore normal structure and functions.
Healing can occur in two ways: Regeneration and Repair
There are 4 phases of wound healing: hemostasis, inflammation, proliferation, and remodeling. This document also describes the mechanism of wound healing. Factors that affect healing include infection, uncontrolled diabetes, poor nutrition, age, anemia, the presence of foreign bodies, etc.
Complications of wound healing like infection, hyperpigmentation of scar, contractures, and keloid formation.
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh
Gender and Mental Health - Counselling and Family Therapy Applications and In...
A proprietary approach developed by bringing together the best of learning theories from Psychology, design principles from the world of visualization, and pedagogical methods from over a decade of training experience, that enables you to: Learn better, faster!
Stack Memory Organization of 8086 Microprocessor
The stack memory organization of 8086 microprocessor.
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
Physical pharmaceutics notes for B.pharm students
Mule event processing models | MuleSoft Mysore Meetup #47
Mule event processing models | MuleSoft Mysore Meetup #47
Event Link:- https://meetups.mulesoft.com/events/details/mulesoft-mysore-presents-mule-event-processing-models/
Agenda
● What is event processing in MuleSoft?
● Types of event processing models in Mule 4
● Distinction between the reactive, parallel, blocking & non-blocking processing
For Upcoming Meetups Join Mysore Meetup Group - https://meetups.mulesoft.com/mysore/YouTube:- youtube.com/@mulesoftmysore
Mysore WhatsApp group:- https://chat.whatsapp.com/EhqtHtCC75vCAX7gaO842N
Speaker:-
Shivani Yasaswi - https://www.linkedin.com/in/shivaniyasaswi/
Organizers:-
Shubham Chaurasia - https://www.linkedin.com/in/shubhamchaurasia1/
Giridhar Meka - https://www.linkedin.com/in/giridharmeka
Priya Shaw - https://www.linkedin.com/in/priya-shaw
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...Nguyen Thanh Tu Collection
https://app.box.com/s/tacvl9ekroe9hqupdnjruiypvm9rdaneTemple of Asclepius in Thrace. Excavation results
The temple and the sanctuary around were dedicated to Asklepios Zmidrenus. This name has been known since 1875 when an inscription dedicated to him was discovered in Rome. The inscription is dated in 227 AD and was left by soldiers originating from the city of Philippopolis (modern Plovdiv).
Recently uploaded (20)
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...
Elevate Your Nonprofit's Online Presence_ A Guide to Effective SEO Strategies...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
Juneteenth Freedom Day 2024 David Douglas School District
Juneteenth Freedom Day 2024 David Douglas School District
SWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptx
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
RHEOLOGY Physical pharmaceutics-II notes for B.pharm 4th sem students
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
Mule event processing models | MuleSoft Mysore Meetup #47
Mule event processing models | MuleSoft Mysore Meetup #47
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
BÀI TẬP BỔ TRỢ TIẾNG ANH LỚP 9 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2024-2025 - ...
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
Presentation2013
- 1. Sucheta Tripathy, Akash Gupta, Brett M Tyler This is work in progress…
- 2. Yet Again Another Database????????? Its raining sequences!!!!!!!!!! Sequencing is outrunning the ability to store, transmit and analyze the data - NYtimes
- 3. FungiDB VMD ->EuMicrobedb.org Transcriptomicsdb
- 4. Eumicrobedb ◦ Based on Oracle and GUS ◦ Administered at the Virginia Tech
- 5. Based on Mysql Front end remains the same (Based on perl CGI, GD and PHP) Name spaces are downsized. Number of tables/views downsized. Removed dependencies.
- 6. 3 Name spaces 5 name spaces (20+7+10) tables (179+39+40+15+56) (18) Views tables Independent of oracle (84+4+15+24) views Independent of Bioperl Needs Oracle license Needs Bioperl Transcriptomics database
- 7. EuMicrobedb- Eumicrobedb- Oracle Light Total Number of 329 37 Tables Total Number of 127 18 views Query time 10secs 1.2 secs Time for genome 12-14 hours 2 hours upload
- 8. An P. sojae V1.0 not ati P. sojae V5.0 on P. ramorum V1.0 H. arabidopsidis V8.3 Sequence fasta C++ Database API GFF Toolkit
- 9. GFF Genome tools Sequence fasta C++API Database An not ati on Toolkit
- 10. 3 Dell Power Edge R420 servers: 16 GB RAM, 1.5 TB each with NFS. ◦ Data Analysis server ◦ Web server ◦ Data storage R820 server: 128 GB RAM, 16 TB storage. IICB has 2 compute clusters with 64 nodes and each node having 192 GB memory. CSIR-cMMACS has India’s fastest supercomputer. One sequencing and one bioinformatics support.
- 11. Labs sequencing genomes at rapid pace ◦ Draft Assembly ◦ Data not yet in genbank ◦ Gff annotation available : View on browser Labs with limited hosting facilities ◦ Data hosting ◦ Data analysis
- 12. Source will be released soon for people to replicate the database. ◦ No Oracle license. $$$ ◦ Independent of many packages. ◦ Installation time reduced. ◦ Simple user experience.
- 13. Prof. Siddhrtha Roy Prof. Brett Tyler