e-Infrastructure @ Science
•Download as ODP, PDF•
0 likes•229 views
T
TomSARA is a Dutch organization that provides e-infrastructure services to support research, including high-performance computing, data services, and networking. The presenter works on the BiG Grid project, which provides over 6000 compute cores and 10PB of storage to support data-intensive e-science. Challenges include managing diverse stakeholder needs, ensuring continued funding, and adapting infrastructure to keep pace with rapidly changing technologies and scientific requirements. Collaboration across institutions is key to maintaining a strong e-science environment in the Netherlands and Europe.
Report
Share
Report
Share
![E-Infrastructures @ science Tom Visser e-Science consultant SARA [email_address]](https://image.slidesharecdn.com/biggridpresentation-120130093308-phpapp01/85/e-Infrastructure-Science-1-320.jpg)
![Today ,[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Big data management
Big data management requires large data centers to store the vast amounts of data being generated. Hadoop is a technology used to manage big data across clusters of commodity servers in data centers. It uses a distributed file system and is designed to parallelize data processing. Data centers are facilities that house computer systems and associated components, such as telecommunications and storage systems, that are needed to manage big data. They contain rows of server racks and must have sufficient power and cooling to handle large amounts of structured and unstructured data from sources like the New York Stock Exchange, Google, and Facebook. As data volumes continue increasing rapidly, more advanced technologies will be needed to manage big data in the future.
DataViz_What_How_Why
Data visualization uses visual representations of data to help people understand patterns and insights more easily than textual representations. The amount of data generated daily is growing exponentially, making data visualization techniques essential. Effective data visualization adheres to principles of visual perception and Gestalt theory, represents quantitative information accurately without distortion, and chooses visual representations well-suited to the data and the intended audience.
Big Data Story - From An Engineer's Perspective
The document provides an overview of big data from an engineer's perspective. It discusses how (1) the amount of data created daily is exponentially growing, with 90% created in the last two years, (2) data is transforming how we live and work through opportunities in areas like social networking, ecommerce, and smart technologies, and (3) big data is fueling innovation through capabilities like prediction, recommendation, and detection using algorithms.
Introduction to Big Data by Manouj Bongirr
Big Logic is a US-based technology consulting and training firm focused on big data analytics using Apache Hadoop. They help enterprises successfully implement big data projects by sharing experiences and empowering clients to decide whether to build or buy solutions. Big data refers to extremely large data sets that are difficult to process using traditional software tools.
Big data, Big decision
This document discusses how enterprises can leverage big data. It notes that no single solution will meet all needs and not all solutions will be a good fit. It recommends enterprises use big data if improvements and returns on investment are measurable, and outlines steps for getting started such as starting small and organically, reusing existing resources, and initially focusing on internal information. The overall message is that successfully using big data depends on enterprise goals and capabilities.
High Performance Computing and Big Data: The coming wave
This document discusses the rise of high performance computing and big data, and how these technologies are enabling new insights and opportunities. It outlines how computation that once took days can now take seconds, and how big data is providing new levels of insight. It also discusses how Intel technologies like Xeon Phi coprocessors, solid-state drives, and 10 Gigabit Ethernet are providing the infrastructure needed to efficiently capture, store, process, and analyze massive amounts of data.
Puja(801),sanghamitra(819),surabhi(844)
This document discusses big data, defining it as data that is too large and complex for traditional data processing systems due to its volume, variety and velocity. It outlines the 3Vs of big data - volume, referring to the large amount of data being generated daily; variety, referring to different data formats; and velocity, referring to the speed at which data is generated and needs to be processed. The document also discusses characteristics of big data like structured, semi-structured and unstructured data, benefits of big data, challenges of capturing, storing, analyzing and presenting big data, and technologies like Hadoop and MapReduce used for big data solutions.
BIG DATA
This document provides an overview of big data. It defines big data as large volumes of data that come in many varieties and are created quickly. It notes that while 5 billion gigabytes of data were created from the beginning of recorded time until 2003, that same amount was being created every two days by 2011 and every 10 minutes by 2013 due to the growth of data from sources like the internet and smartphones. The document also outlines some of the common tools used to analyze big data, potential benefits and challenges, as well as which industries are most impacted.
Recommended
Big data management
Big data management requires large data centers to store the vast amounts of data being generated. Hadoop is a technology used to manage big data across clusters of commodity servers in data centers. It uses a distributed file system and is designed to parallelize data processing. Data centers are facilities that house computer systems and associated components, such as telecommunications and storage systems, that are needed to manage big data. They contain rows of server racks and must have sufficient power and cooling to handle large amounts of structured and unstructured data from sources like the New York Stock Exchange, Google, and Facebook. As data volumes continue increasing rapidly, more advanced technologies will be needed to manage big data in the future.
DataViz_What_How_Why
Data visualization uses visual representations of data to help people understand patterns and insights more easily than textual representations. The amount of data generated daily is growing exponentially, making data visualization techniques essential. Effective data visualization adheres to principles of visual perception and Gestalt theory, represents quantitative information accurately without distortion, and chooses visual representations well-suited to the data and the intended audience.
Big Data Story - From An Engineer's Perspective
The document provides an overview of big data from an engineer's perspective. It discusses how (1) the amount of data created daily is exponentially growing, with 90% created in the last two years, (2) data is transforming how we live and work through opportunities in areas like social networking, ecommerce, and smart technologies, and (3) big data is fueling innovation through capabilities like prediction, recommendation, and detection using algorithms.
Introduction to Big Data by Manouj Bongirr
Big Logic is a US-based technology consulting and training firm focused on big data analytics using Apache Hadoop. They help enterprises successfully implement big data projects by sharing experiences and empowering clients to decide whether to build or buy solutions. Big data refers to extremely large data sets that are difficult to process using traditional software tools.
Big data, Big decision
This document discusses how enterprises can leverage big data. It notes that no single solution will meet all needs and not all solutions will be a good fit. It recommends enterprises use big data if improvements and returns on investment are measurable, and outlines steps for getting started such as starting small and organically, reusing existing resources, and initially focusing on internal information. The overall message is that successfully using big data depends on enterprise goals and capabilities.
High Performance Computing and Big Data: The coming wave
This document discusses the rise of high performance computing and big data, and how these technologies are enabling new insights and opportunities. It outlines how computation that once took days can now take seconds, and how big data is providing new levels of insight. It also discusses how Intel technologies like Xeon Phi coprocessors, solid-state drives, and 10 Gigabit Ethernet are providing the infrastructure needed to efficiently capture, store, process, and analyze massive amounts of data.
Puja(801),sanghamitra(819),surabhi(844)
This document discusses big data, defining it as data that is too large and complex for traditional data processing systems due to its volume, variety and velocity. It outlines the 3Vs of big data - volume, referring to the large amount of data being generated daily; variety, referring to different data formats; and velocity, referring to the speed at which data is generated and needs to be processed. The document also discusses characteristics of big data like structured, semi-structured and unstructured data, benefits of big data, challenges of capturing, storing, analyzing and presenting big data, and technologies like Hadoop and MapReduce used for big data solutions.
BIG DATA
This document provides an overview of big data. It defines big data as large volumes of data that come in many varieties and are created quickly. It notes that while 5 billion gigabytes of data were created from the beginning of recorded time until 2003, that same amount was being created every two days by 2011 and every 10 minutes by 2013 due to the growth of data from sources like the internet and smartphones. The document also outlines some of the common tools used to analyze big data, potential benefits and challenges, as well as which industries are most impacted.
5 Factors Impacting Your Big Data Project's Performance
Big data analytics projects frequently fail to return ROI due to these five factors. Learn how to make your big data cloud project succeed.
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Big data workloads are growing rapidly and require unprecedented scale in data storage, processing power and bandwidth. Cloud infrastructure provides the flexibility and economics needed to support big data's mixed workloads through software-defined compute, storage, networking and security. This allows organizations to gain real-time insights from massive data sets and harness big data's potential for industries like retail, manufacturing and more.
Overview of big data in cloud computing
Viet-Trung Tran presents information on big data and cloud computing. The document discusses key concepts like what constitutes big data, popular big data management systems like Hadoop and NoSQL databases, and how cloud computing can enable big data processing by providing scalable infrastructure. Some benefits of running big data analytics on the cloud include cost reduction, rapid provisioning, and flexibility/scalability. However, big data may not always be suitable for the cloud due to issues like data security, latency requirements, and multi-tenancy overhead.
Big Data in the Cloud
Adopting Hadoop to manage your Big Data is an important step, but not the end-solution to your Big Data challenges. Here are some of the additional considerations you must face:
Choosing the right cloud for the job: The massive computing and storage resources that are needed to support Big Data applications make cloud environments an ideal fit, and more than ever, there is a growing number of choices of cloud infrastructure types and providers. Given the diverse options, and the dynamic environments involved, it becomes ever more important to maintain the flexibility for all your IT needs.
Big Data is a complex beast: It involves many and different moving parts, in large clusters, and is continually growing and evolving. Managing such an environment manually is not a viable option. The question is, how can you achieve automation of all this complexity?
The world beyond Hadoop: Big Data is not just Hadoop – there is a whole rapidly growing ecosystem to contend with, including NoSQL, data processing, analytics tools… As well as your own application services. How can you manage deployment, configuration, scaling and failover of all the different pieces, in a consistent way?
In this session, you’ll learn how to deploy and manage your Hadoop cluster on any Cloud, as well as manage the rest of your big data application stack using a new open source framework called Cloudify.
Enabling a Data Driven Agile Business
This document discusses enabling a data-driven agile business through data analytics and summarizes WSO2's progress in building products to support big data strategies. It notes that WSO2 released new versions of its BAM and CEP products in 2012 to focus on performance, scalability, and customizability. The document outlines WSO2's vision for 2013 of fusing its products to enable "Data to knowledge" and ambitions to process 10 terabytes of data in 2 seconds. It also notes that the amount of data created and shared globally is expected to grow dramatically between 2012 and 2015.
KNOWLEDGE ARCHITECTURE: IT’S IMPORTANCE TO AN ORGANIZATION
To convert data to knowledge, a convergence of knowledge management, information architecture, and data science is necessary. This is called knowledge architecture. Knowledge architecture involves designing and implementing an organization's intellectual infrastructure to capture, organize, analyze, and utilize information. It transforms information into knowledge through applying context. NASA faces challenges with large amounts of growing and siloed data. Opportunities for knowledge architecture at NASA include improving search capabilities, developing metadata standards, using analytics and visualization, and creating a lessons learned knowledge graph. This could help NASA make better decisions by leveraging past knowledge and reducing waste.
Big data
Big data refers to large, complex datasets that are difficult to process using traditional database management tools. There are four key characteristics of big data: volume, velocity, variety, and veracity. Various sources generate big data, including social media, scientific instruments, mobile devices, sensors, and more. Analyzing big data can provide benefits like cost reductions, time reductions, new product development, and smarter business decisions. Hadoop Distributed File System (HDFS) and Hadoop software platform provide scalable and cost-effective infrastructure for storing and processing big data across commodity servers in a cluster.
Guest Lecture: Introduction to Big Data at Indian Institute of Technology
This document provides an introduction to big data, including definitions of big data and why it is important. It discusses characteristics of big data like volume, velocity, variety and veracity. It provides examples of big data applications in various industries like GE, Boeing, social media, finance, CERN, journalism, politics and more. It also introduces NoSQL and the CAP theorem, and concludes that big data is changing business and technology by enabling new insights from data to reduce costs and optimize operations.
Big data
This document provides an overview of big data and Hadoop. It defines big data as large volumes of diverse data that cannot be processed by traditional systems. Key characteristics are volume, velocity, variety, and veracity. Popular sources of big data include social media, emails, videos, and sensor data. Hadoop is presented as an open-source framework for distributed storage and processing of large datasets across clusters of computers. It uses HDFS for storage and MapReduce as a programming model. Major tech companies like Google, Facebook, and Amazon are discussed as big players in big data.
The rise of “Big Data” on cloud computing
The rise of “Big Data” on cloud computing: Review and open research issues
Paper Link: https://www.researchgate.net/publication/264624667_The_rise_of_Big_Data_on_cloud_computing_Review_and_open_research_issues
Big Data on Public Cloud
Presentation by Dr.Thanachart Numnonda, IMC Institute @Thailand Big Data User Group #2, 13 March 2015
The importance of data
The document discusses the importance of data for evidence-based policymaking, organizational development, detecting security issues, and improving business outcomes. It provides examples of how New Zealand Registry Services (NZRS) uses data for these purposes, including operating a national broadband map and open data portals. The document advocates for making more data openly available to enable reproducible research, more informed policy debates, and increased public trust.
BIG DATA-Seminar Report
Big Data may well be the Next Big Thing in the IT world
Big data burst upon the scene in the first decade of the 21st century
Gail Zhou on "Big Data Technology, Strategy, and Applications"
Dr. Gail Zhou presented this topic at DevNexus on Feb 25, 2014. Big Data history, opportunities, and applications. Big Data key concepts, reference architecture with open source technology stacks. Hadoop architecture explained (HDFS, Map Reduce, and YARN). Big Data start-up challenges and strategies to overcome them. Technology update: Hadoop and Cassandra based technology offerings.
Knowledge Architecture: Graphing Your Knowledge
Ask any project manager and they will tell you the importance of reviewing lessons learned prior to starting a new project. The lesson learned databases are filled with nuggets of valuable information to help project teams increase the likelihood of project success. Why then do most lesson learned databases go unused by project teams? In my experience, they are difficult to search through and require hours of time to review the result set.
Recently I had a project engineer ask me if we could search our lessons learned using a list of 22 key terms the team was interested in. Our current keyword search engine would require him to enter each term individually, select the link, and save the document for review. Also, there was no way to search only the database, the query would search our entire corpus, close to 20 million URLs. This would not do. I asked our search team if they would run a special query against the lesson database only, using the terms provided. They returned a spreadsheet with a link to each document containing the terms. The engineer had his work cut out for him: over 1100 documents were on the list;.
I started thinking there had to be a better way. I had been experimenting with topic modeling, in particular to assist our users in connecting seemingly disparate documents through an easier visualization mechanism. Something better than a list of links on multiple pages. I gathered my toolbox: R/RStudio, for the topic modeling and exploring the data; Neo4j, for modeling and visualizing the topics; and Linkurious, a web front end for our users to search and visualize the graph database.
Big data and hadoop introduction
Before moving to hadoop, one must understand why we need hadoop, irrespective we have all sorts of RDBMS available in the market. This presentation has good understanding of big data and this enables you to properly analyze the use case for big data problems.
VFB 2013 - HP Labs - Horizon Scanning - Technology Trends
HP Labs conducts research across three time horizons - evolutionary research of up to 2 years, applied research of 2-5 years, and exploratory research of 5-20+ years. Its charter is to deliver breakthrough technologies, advance fundamental science, create opportunities for HP and customers, and engage with customers and partners. The data explosion is pushing the limits of today's data centers, with 40 zettabytes of data projected by 2020, requiring next-generation capabilities like business insight at internet speeds and personalized content. HP Labs is working to manage and secure systems as more enterprise operations move to the cloud, through techniques like embedded control points, management across boundaries, and continual assurance.
The internet of things, do we need all that data?
With the arrival of the Internet of Things, plenty of data is being generated. Do we need to maintain all that data in the cloud or are there other ways to do this? This presentation, originally given at the BI symposium in the Netherlands complements a blog entry you can find http://community.hpe.com/t5/Cloud-Source/IoT-Big-Data-let-s-think-differently/ba-p/6813967
Big data
This document proposes a theme on big data analytics research. It notes that the world's data storage capacity doubles every 40 months and discusses how big data can provide value across many areas like health, policymaking, education and more. The proposal recommends that Hong Kong develop a state-of-the-art big data platform to make a difference in areas like smart cities and support aging populations. It outlines objectives like large-scale machine learning from big data and discusses how Hong Kong is well-positioned for this research with experts across universities and potential collaborators in industry. The expected outcomes include new methodologies, applications impacting society and industry, and educational programs to cultivate big data leaders.
Big data
Nandan Shah's presentation covered key topics related to big data including definitions, important technologies like Hadoop and MapReduce, benefits such as enabling real-time analysis and faster decision making, risks around data security and privacy, and applications across various domains such as banking, retail, and healthcare. The presentation also provided interesting facts about the size of data holdings and growth of the big data industry.
Pioneering and Democratizing Scalable HPC+AI at PSC
In this deck from the 2019 Stanford HPC Conference, Nik Nystrom from the Pittsburgh Supercomputing Center presents: Pioneering and Democratizing Scalable HPC+AI.
"PSC's Bridges was the first system to successfully converge HPC, AI, and Big Data. Designed for the U.S. national research community and supported by NSF, Bridges now serves approximately 1800 projects and 7500 users at 380 institutions, and it is the foundation around which new HPC+AI projects have launched. Bridges emphasizes "nontraditional" uses that span the life, physical, and social sciences, computer science, engineering, business, and humanities. Scalable HPC+AI is driving many of those applications, which span diverse topics such as learning root causes of cancer, strategic reasoning, designing new materials, predicting severe storms, recognizing speech including contextual information, and detecting objects in 4k streaming video. To address the demand for scalable AI, PSC recently introduced Bridges-AI, which adds transformative new AI capability. In this presentation, we share our vision in designing HPC+AI systems at PSC and highlight some of the exciting research breakthroughs they are enabling."
Nick Nystrom is Interim Director and Sr. Director of Research at the Pittsburgh Supercomputing Center (PSC). Nick is architect and PI for Bridges, PSC's flagship system that successfully pioneered the convergence HPC, AI, and Big Data. He is also PI for the NIH Human Biomolecular Atlas Program’s HIVE Infrastructure Component and co-PI for projects that bring emerging AI technologies to research (Open Compass), apply machine learning to biomedical data for breast and lung cancer (Big Data for Better Health), and identify causal relationships in biomedical big data (the Center for Causal Discovery, an NIH Big Data to Knowledge Center of Excellence). His current research interests include hardware and software architecture, applications of machine learning to multimodal data (particularly for the life sciences) and to enhance simulation, and graph analytics.
Watch the video: https://youtu.be/ucRs4A_afus
Learn more: https://www.psc.edu/bridges
and
http://hpcadvisorycouncil.com/events/2019/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Easygenomics ISCB Cloud section 2012
The document provides information about EasyGenomics, a cloud-based bioinformatics platform developed by BGI. It discusses:
- BGI's background as the world's largest genome sequencing center with significant sequencing, computing, and storage capabilities.
- The team building EasyGenomics, which is run like a software company and supported by BGI algorithm teams.
- The key features of EasyGenomics, including bioinformatics workflows, data management, high-speed data transfer, and a simple user interface.
- Future directions for the product, including questions around market positioning and sustainable business models.
More Related Content
What's hot
5 Factors Impacting Your Big Data Project's Performance
Big data analytics projects frequently fail to return ROI due to these five factors. Learn how to make your big data cloud project succeed.
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Big data workloads are growing rapidly and require unprecedented scale in data storage, processing power and bandwidth. Cloud infrastructure provides the flexibility and economics needed to support big data's mixed workloads through software-defined compute, storage, networking and security. This allows organizations to gain real-time insights from massive data sets and harness big data's potential for industries like retail, manufacturing and more.
Overview of big data in cloud computing
Viet-Trung Tran presents information on big data and cloud computing. The document discusses key concepts like what constitutes big data, popular big data management systems like Hadoop and NoSQL databases, and how cloud computing can enable big data processing by providing scalable infrastructure. Some benefits of running big data analytics on the cloud include cost reduction, rapid provisioning, and flexibility/scalability. However, big data may not always be suitable for the cloud due to issues like data security, latency requirements, and multi-tenancy overhead.
Big Data in the Cloud
Adopting Hadoop to manage your Big Data is an important step, but not the end-solution to your Big Data challenges. Here are some of the additional considerations you must face:
Choosing the right cloud for the job: The massive computing and storage resources that are needed to support Big Data applications make cloud environments an ideal fit, and more than ever, there is a growing number of choices of cloud infrastructure types and providers. Given the diverse options, and the dynamic environments involved, it becomes ever more important to maintain the flexibility for all your IT needs.
Big Data is a complex beast: It involves many and different moving parts, in large clusters, and is continually growing and evolving. Managing such an environment manually is not a viable option. The question is, how can you achieve automation of all this complexity?
The world beyond Hadoop: Big Data is not just Hadoop – there is a whole rapidly growing ecosystem to contend with, including NoSQL, data processing, analytics tools… As well as your own application services. How can you manage deployment, configuration, scaling and failover of all the different pieces, in a consistent way?
In this session, you’ll learn how to deploy and manage your Hadoop cluster on any Cloud, as well as manage the rest of your big data application stack using a new open source framework called Cloudify.
Enabling a Data Driven Agile Business
This document discusses enabling a data-driven agile business through data analytics and summarizes WSO2's progress in building products to support big data strategies. It notes that WSO2 released new versions of its BAM and CEP products in 2012 to focus on performance, scalability, and customizability. The document outlines WSO2's vision for 2013 of fusing its products to enable "Data to knowledge" and ambitions to process 10 terabytes of data in 2 seconds. It also notes that the amount of data created and shared globally is expected to grow dramatically between 2012 and 2015.
KNOWLEDGE ARCHITECTURE: IT’S IMPORTANCE TO AN ORGANIZATION
To convert data to knowledge, a convergence of knowledge management, information architecture, and data science is necessary. This is called knowledge architecture. Knowledge architecture involves designing and implementing an organization's intellectual infrastructure to capture, organize, analyze, and utilize information. It transforms information into knowledge through applying context. NASA faces challenges with large amounts of growing and siloed data. Opportunities for knowledge architecture at NASA include improving search capabilities, developing metadata standards, using analytics and visualization, and creating a lessons learned knowledge graph. This could help NASA make better decisions by leveraging past knowledge and reducing waste.
Big data
Big data refers to large, complex datasets that are difficult to process using traditional database management tools. There are four key characteristics of big data: volume, velocity, variety, and veracity. Various sources generate big data, including social media, scientific instruments, mobile devices, sensors, and more. Analyzing big data can provide benefits like cost reductions, time reductions, new product development, and smarter business decisions. Hadoop Distributed File System (HDFS) and Hadoop software platform provide scalable and cost-effective infrastructure for storing and processing big data across commodity servers in a cluster.
Guest Lecture: Introduction to Big Data at Indian Institute of Technology
This document provides an introduction to big data, including definitions of big data and why it is important. It discusses characteristics of big data like volume, velocity, variety and veracity. It provides examples of big data applications in various industries like GE, Boeing, social media, finance, CERN, journalism, politics and more. It also introduces NoSQL and the CAP theorem, and concludes that big data is changing business and technology by enabling new insights from data to reduce costs and optimize operations.
Big data
This document provides an overview of big data and Hadoop. It defines big data as large volumes of diverse data that cannot be processed by traditional systems. Key characteristics are volume, velocity, variety, and veracity. Popular sources of big data include social media, emails, videos, and sensor data. Hadoop is presented as an open-source framework for distributed storage and processing of large datasets across clusters of computers. It uses HDFS for storage and MapReduce as a programming model. Major tech companies like Google, Facebook, and Amazon are discussed as big players in big data.
The rise of “Big Data” on cloud computing
The rise of “Big Data” on cloud computing: Review and open research issues
Paper Link: https://www.researchgate.net/publication/264624667_The_rise_of_Big_Data_on_cloud_computing_Review_and_open_research_issues
Big Data on Public Cloud
Presentation by Dr.Thanachart Numnonda, IMC Institute @Thailand Big Data User Group #2, 13 March 2015
The importance of data
The document discusses the importance of data for evidence-based policymaking, organizational development, detecting security issues, and improving business outcomes. It provides examples of how New Zealand Registry Services (NZRS) uses data for these purposes, including operating a national broadband map and open data portals. The document advocates for making more data openly available to enable reproducible research, more informed policy debates, and increased public trust.
BIG DATA-Seminar Report
Big Data may well be the Next Big Thing in the IT world
Big data burst upon the scene in the first decade of the 21st century
Gail Zhou on "Big Data Technology, Strategy, and Applications"
Dr. Gail Zhou presented this topic at DevNexus on Feb 25, 2014. Big Data history, opportunities, and applications. Big Data key concepts, reference architecture with open source technology stacks. Hadoop architecture explained (HDFS, Map Reduce, and YARN). Big Data start-up challenges and strategies to overcome them. Technology update: Hadoop and Cassandra based technology offerings.
Knowledge Architecture: Graphing Your Knowledge
Ask any project manager and they will tell you the importance of reviewing lessons learned prior to starting a new project. The lesson learned databases are filled with nuggets of valuable information to help project teams increase the likelihood of project success. Why then do most lesson learned databases go unused by project teams? In my experience, they are difficult to search through and require hours of time to review the result set.
Recently I had a project engineer ask me if we could search our lessons learned using a list of 22 key terms the team was interested in. Our current keyword search engine would require him to enter each term individually, select the link, and save the document for review. Also, there was no way to search only the database, the query would search our entire corpus, close to 20 million URLs. This would not do. I asked our search team if they would run a special query against the lesson database only, using the terms provided. They returned a spreadsheet with a link to each document containing the terms. The engineer had his work cut out for him: over 1100 documents were on the list;.
I started thinking there had to be a better way. I had been experimenting with topic modeling, in particular to assist our users in connecting seemingly disparate documents through an easier visualization mechanism. Something better than a list of links on multiple pages. I gathered my toolbox: R/RStudio, for the topic modeling and exploring the data; Neo4j, for modeling and visualizing the topics; and Linkurious, a web front end for our users to search and visualize the graph database.
Big data and hadoop introduction
Before moving to hadoop, one must understand why we need hadoop, irrespective we have all sorts of RDBMS available in the market. This presentation has good understanding of big data and this enables you to properly analyze the use case for big data problems.
VFB 2013 - HP Labs - Horizon Scanning - Technology Trends
HP Labs conducts research across three time horizons - evolutionary research of up to 2 years, applied research of 2-5 years, and exploratory research of 5-20+ years. Its charter is to deliver breakthrough technologies, advance fundamental science, create opportunities for HP and customers, and engage with customers and partners. The data explosion is pushing the limits of today's data centers, with 40 zettabytes of data projected by 2020, requiring next-generation capabilities like business insight at internet speeds and personalized content. HP Labs is working to manage and secure systems as more enterprise operations move to the cloud, through techniques like embedded control points, management across boundaries, and continual assurance.
The internet of things, do we need all that data?
With the arrival of the Internet of Things, plenty of data is being generated. Do we need to maintain all that data in the cloud or are there other ways to do this? This presentation, originally given at the BI symposium in the Netherlands complements a blog entry you can find http://community.hpe.com/t5/Cloud-Source/IoT-Big-Data-let-s-think-differently/ba-p/6813967
Big data
This document proposes a theme on big data analytics research. It notes that the world's data storage capacity doubles every 40 months and discusses how big data can provide value across many areas like health, policymaking, education and more. The proposal recommends that Hong Kong develop a state-of-the-art big data platform to make a difference in areas like smart cities and support aging populations. It outlines objectives like large-scale machine learning from big data and discusses how Hong Kong is well-positioned for this research with experts across universities and potential collaborators in industry. The expected outcomes include new methodologies, applications impacting society and industry, and educational programs to cultivate big data leaders.
Big data
Nandan Shah's presentation covered key topics related to big data including definitions, important technologies like Hadoop and MapReduce, benefits such as enabling real-time analysis and faster decision making, risks around data security and privacy, and applications across various domains such as banking, retail, and healthcare. The presentation also provided interesting facts about the size of data holdings and growth of the big data industry.
What's hot (20)
5 Factors Impacting Your Big Data Project's Performance
5 Factors Impacting Your Big Data Project's Performance
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
KNOWLEDGE ARCHITECTURE: IT’S IMPORTANCE TO AN ORGANIZATION
KNOWLEDGE ARCHITECTURE: IT’S IMPORTANCE TO AN ORGANIZATION
Guest Lecture: Introduction to Big Data at Indian Institute of Technology
Guest Lecture: Introduction to Big Data at Indian Institute of Technology
Gail Zhou on "Big Data Technology, Strategy, and Applications"
Gail Zhou on "Big Data Technology, Strategy, and Applications"
VFB 2013 - HP Labs - Horizon Scanning - Technology Trends
VFB 2013 - HP Labs - Horizon Scanning - Technology Trends
Similar to e-Infrastructure @ Science
Pioneering and Democratizing Scalable HPC+AI at PSC
In this deck from the 2019 Stanford HPC Conference, Nik Nystrom from the Pittsburgh Supercomputing Center presents: Pioneering and Democratizing Scalable HPC+AI.
"PSC's Bridges was the first system to successfully converge HPC, AI, and Big Data. Designed for the U.S. national research community and supported by NSF, Bridges now serves approximately 1800 projects and 7500 users at 380 institutions, and it is the foundation around which new HPC+AI projects have launched. Bridges emphasizes "nontraditional" uses that span the life, physical, and social sciences, computer science, engineering, business, and humanities. Scalable HPC+AI is driving many of those applications, which span diverse topics such as learning root causes of cancer, strategic reasoning, designing new materials, predicting severe storms, recognizing speech including contextual information, and detecting objects in 4k streaming video. To address the demand for scalable AI, PSC recently introduced Bridges-AI, which adds transformative new AI capability. In this presentation, we share our vision in designing HPC+AI systems at PSC and highlight some of the exciting research breakthroughs they are enabling."
Nick Nystrom is Interim Director and Sr. Director of Research at the Pittsburgh Supercomputing Center (PSC). Nick is architect and PI for Bridges, PSC's flagship system that successfully pioneered the convergence HPC, AI, and Big Data. He is also PI for the NIH Human Biomolecular Atlas Program’s HIVE Infrastructure Component and co-PI for projects that bring emerging AI technologies to research (Open Compass), apply machine learning to biomedical data for breast and lung cancer (Big Data for Better Health), and identify causal relationships in biomedical big data (the Center for Causal Discovery, an NIH Big Data to Knowledge Center of Excellence). His current research interests include hardware and software architecture, applications of machine learning to multimodal data (particularly for the life sciences) and to enhance simulation, and graph analytics.
Watch the video: https://youtu.be/ucRs4A_afus
Learn more: https://www.psc.edu/bridges
and
http://hpcadvisorycouncil.com/events/2019/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Easygenomics ISCB Cloud section 2012
The document provides information about EasyGenomics, a cloud-based bioinformatics platform developed by BGI. It discusses:
- BGI's background as the world's largest genome sequencing center with significant sequencing, computing, and storage capabilities.
- The team building EasyGenomics, which is run like a software company and supported by BGI algorithm teams.
- The key features of EasyGenomics, including bioinformatics workflows, data management, high-speed data transfer, and a simple user interface.
- Future directions for the product, including questions around market positioning and sustainable business models.
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
In this deck from the Stanford HPC Conference, Nick Nystrom and Paola Buitrago provide an update from the Pittsburgh Supercomputing Center.
Nick Nystrom is Chief Scientist at the Pittsburgh Supercomputing Center (PSC). Nick is architect and PI for Bridges, PSC's flagship system that successfully pioneered the convergence of HPC, AI, and Big Data. He is also PI for the NIH Human Biomolecular Atlas Program’s HIVE Infrastructure Component and co-PI for projects that bring emerging AI technologies to research (Open Compass), apply machine learning to biomedical data for breast and lung cancer (Big Data for Better Health), and identify causal relationships in biomedical big data (the Center for Causal Discovery, an NIH Big Data to Knowledge Center of Excellence). His current research interests include hardware and software architecture, applications of machine learning to multimodal data (particularly for the life sciences) and to enhance simulation, and graph analytics.
Watch the video: https://youtu.be/LWEU1L1o7yY
Learn more: https://www.psc.edu/
and
http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Internet2 Bio IT 2016 v2
The document discusses Internet2, an advanced networking consortium that operates a 15,000 mile fiber optic network for research and education. It provides very high speed connectivity and collaboration technologies to facilitate large data sharing and frictionless research. Examples are given of life sciences projects utilizing Internet2's high-speed network for genomic research and agricultural applications involving terabytes of satellite and sensor data. The network is expanding to include cloud computing resources and supercomputing centers to enable global-scale distributed scientific computing and collaboration.
Internet of things (IoT) and big data- r.nabati
This document provides an introduction to Internet of Things (IoT) and Big Data Analytics. It discusses why IoT is an important and growing trend, defining IoT as connecting physical objects to the internet. It notes that billions of devices are expected to be connected by 2020. The document then discusses some key IoT technologies like communication standards, sensors, and embedded systems. It also outlines some common IoT applications and opportunities in areas like smart homes, healthcare, transportation and more. Finally, the document introduces some characteristics of big data, such as the scale, variety and velocity of data that is now being generated and needs to be analyzed.
BeSTGRID OpenGridForum 29 GIN session
BeSTGRID is a New Zealand research infrastructure organization that aims to enhance e-Research capabilities. It provides skills, services, and tools to help researchers access computational resources and collaborate on shared data. Since 2006, BeSTGRID has delivered services for bioscience, geoscience, and other applications. It established foundations for shared infrastructure and piloted identity management services. Going forward, BeSTGRID will focus on coordinating access to compute and data resources and providing discipline-specific applications and services through collaboration across institutions.
SKA NZ R&D BeSTGRID Infrastructure
BeSTGRID aims to enhance research capability in New Zealand by providing skills training and infrastructure support. Since 2006, BeSTGRID has delivered services and tools to support research collaboration on shared data and computational resources. BeSTGRID coordinates access to compute and storage resources across New Zealand and provides discipline-specific applications and services to support researchers.
Data-intensive bioinformatics on HPC and Cloud
The document discusses data-intensive bioinformatics and challenges with analyzing large genomic datasets on high performance computing (HPC) resources. It summarizes that storage is the biggest challenge as sequencing projects generate very large amounts of data and users do not clean up data. The strategies discussed to address this include assessing costs of storage and analysis upfront, limiting project lifetimes, moving to tiered storage, and improving efficiency. It also discusses using cloud computing resources through virtual clusters and containers to enable flexible, on-demand access and pay-per-use pricing models. Scientific workflows and microservices approaches are presented as ways to automate and orchestrate large-scale genomic analyses on distributed computing resources.
SomeSlides
The document outlines the vision, mission, and strategy of the STFC (Science and Technology Facilities Council) in implementing e-Science technologies. The goals are to exploit data from STFC facilities through innovative infrastructure, integrate activities nationally and internationally, and improve computation and data management capabilities to enable new scientific discoveries.
Session 33 - Production Grids
The document provides an overview of several production grids including the Open Science Grid, DEISA, NAREGI, the Nordic DataGrid Facility, EGEE, and TeraGrid. It describes the organizations, resources, users, operations, software, and partnerships of each grid. The grids provide petascale resources and support a wide range of scientific applications in fields like high energy physics, life sciences, earth sciences, and engineering. They are internationally collaborative efforts that aim to enable open scientific research through distributed computing infrastructures.
Shared services - the future of HPC and big data facilities for UK research
Slides from Jisc panel session at HPC & Big Data 2016 with contributions from the Francis Crick Institute, QMUL and King's College London covering their use of the Jisc shared data centre and the eMedLab project
Cloud Computing,雲端運算-中研院網格計畫主持人林誠謙
The document discusses the limits of information and communication technologies (ICT) such as computing power, data storage, and network bandwidth. It proposes that future networks will need to scale in both size and functionality through approaches like federation of multiple networks. Cloud computing is presented as a potential approach to tackle these limits by providing on-demand access to shared computing resources over a network in a scalable and elastic manner. However, cloud computing is still associated with many marketing hype and open questions remain regarding its impact and how it can integrate with existing technologies.
Toward a National Research Platform
- The Pacific Research Platform (PRP) interconnects campus DMZs across multiple institutions to provide high-speed connectivity for data-intensive research.
- The PRP utilizes specialized data transfer nodes called FIONAs that provide disk-to-disk transfer speeds of 10-100Gbps.
- Early applications of the PRP include distributing telescope data between UC campuses, connecting particle physics experiments to computing resources, and enabling real-time wildfire sensor data analysis.
Toward a National Research Platform
The document summarizes Dr. Larry Smarr's presentation on the Pacific Research Platform (PRP) and its role in working toward a national research platform. It describes how PRP has connected research teams and devices across multiple UC campuses for over 15 years. It also details PRP's innovations like Flash I/O Network Appliances (FIONAs) and use of Kubernetes to manage distributed resources. Finally, it outlines opportunities to further integrate PRP with the Open Science Grid and expand the platform internationally through partnerships.
100503 bioinfo instsymp
Presentation on the work we've done within BeSTGRID as it relates to bioinformatics in NZ, for the 2010 Bioinformatics Symposium https://www.bestgrid.org/NZ-Bioinformatics-Symposium-2010
100503 bioinfo instsymp
100503 bioinfo instsympBeSTGRID - NZ Research Computing (eResearch, Grid, eScience, Cyberinfrastructure)
BeSTGRID aims to enhance research capability in New Zealand by providing skills and infrastructure to help researchers engage with new eResearch services and kick start centralized infrastructure. Since 2006, BeSTGRID has delivered services and tools to support research collaboration on shared datasets and computational resources. BeSTGRID coordinates access to compute and data resources, provides discipline-specific services and applications, and builds a sustainable community to develop middleware, applications, and services.Grid computing assiment
Grid computing combines the resources of multiple computers from different organizations to solve large problems. It works by sharing computing power, memory, storage and other resources across an authorized network. Examples of grid computing include projects that analyze large datasets like genome sequencing or simulate complex systems like climate modeling. Major grid computing projects include those run by scientific organizations like CERN and SETI@home, which analyzes radio telescope data using volunteers' computers. Grid computing infrastructure allows resources to be accessed easily like a utility over the network.
Thoughts on Cybersecurity
This talk was given at a workshop entitled "Cybersecurity Engagement in a Research Environment" at Rady School of Management at UCSD. The workshop was organized by Michael Corn, the UCSD CISO. It tries to provoke discussion around the cybersecurity features and requirements of international science collaborations, as well as more generally, federated cyberinfrastructure systems.
IoT 2014 Value Creation Workshop: SDIL
Structuring Big Data results to create new information: Smart Data. These Smart Data can be used to advance knowledge and support decision-making processes.
A close cooperation between industry and science creates better conditions for cutting-edge research in Data Engineering/Smart Data.
CINECA webinar slides: Data Gravity in the Life Sciences: Lessons learned fro...
We live in an era of cloud computing. Many of the services in the life sciences are keenly planning cloud transformations, seeking to create globally distributed ecosystems of harmonised data based on standards from organisations like GA4GH. CINECA faces similar challenges, gathering cohort datasets from all over the globe, many of which are pinned in place, due to their size, legal restrictions, or other considerations. But is “bringing compute to the data” always the right choice? In this webinar, based on experiences from the Human Cell Atlas Data Coordination Platform and other projects from EMBL-EBI, we will explore the concept of “data gravity”: The idea that whilst there are forces that may hold data in one place, there are others that require it to be mobile. We’ll consider how effectively planning a cloud strategy requires consideration of the gravity of datasets, and the impact it may have on team skills required, incentives for good practice, and storage and compute costs.
The CINECA webinar series aims to discuss ways to address common challenges and share best practices in the field of cohort data analysis, as well as distribute CINECA project results. All CINECA webinars include an audience Q&A session during which attendees can ask questions and make suggestions. Please note that all webinars are recorded and available for posterior viewing. CINECA webinars include an audience Q&A session during which attendees can ask questions and make suggestions.
This webinar took place on 12th November 2020 and is part of the CINECA webinar series.
For previous and upcoming CINECA webinars see:
https://www.cineca-project.eu/webinars
Similar to e-Infrastructure @ Science (20)
Pioneering and Democratizing Scalable HPC+AI at PSC
Pioneering and Democratizing Scalable HPC+AI at PSC
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
Evolving Cyberinfrastructure, Democratizing Data, and Scaling AI to Catalyze ...
Shared services - the future of HPC and big data facilities for UK research
Shared services - the future of HPC and big data facilities for UK research
CINECA webinar slides: Data Gravity in the Life Sciences: Lessons learned fro...
CINECA webinar slides: Data Gravity in the Life Sciences: Lessons learned fro...
Recently uploaded
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays, June 2024 with Maria Copot 20
Choosing The Best AWS Service For Your Website + API.pptx
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Main news related to the CCS TSI 2023 (2023/1695)
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
Programming Foundation Models with DSPy - Meetup Slides
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Trusted Execution Environment for Decentralized Process Mining
Presentation of the paper "Trusted Execution Environment for Decentralized Process Mining" given during the CAiSE 2024 Conference in Cyprus on June 7, 2024.
Generating privacy-protected synthetic data using Secludy and Milvus
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Azure API Management to expose backend services securely
How to use Azure API Management to expose backend service securely
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Inconsistent user experience and siloed data, high costs, and changing customer expectations – Citizens Bank was experiencing these challenges while it was attempting to deliver a superior digital banking experience for its clients. Its core banking applications run on the mainframe and Citizens was using legacy utilities to get the critical mainframe data to feed customer-facing channels, like call centers, web, and mobile. Ultimately, this led to higher operating costs (MIPS), delayed response times, and longer time to market.
Ever-changing customer expectations demand more modern digital experiences, and the bank needed to find a solution that could provide real-time data to its customer channels with low latency and operating costs. Join this session to learn how Citizens is leveraging Precisely to replicate mainframe data to its customer channels and deliver on their “modern digital bank” experiences.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Introduction of Cybersecurity with OSS at Code Europe 2024
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
Predictive maintenance is a proactive approach that anticipates equipment failures before they happen. At the forefront of this innovative strategy is Artificial Intelligence (AI), which brings unprecedented precision and efficiency. AI in predictive maintenance is transforming industries by reducing downtime, minimizing costs, and enhancing productivity.
TrustArc Webinar - 2024 Global Privacy Survey
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Fueling AI with Great Data with Airbyte Webinar
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
JavaLand 2024: Application Development Green Masterplan
My presentation slides I used at JavaLand 2024
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Recently uploaded (20)
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Azure API Management to expose backend services securely
Azure API Management to expose backend services securely
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
e-Infrastructure @ Science
- 1. E-Infrastructures @ science Tom Visser e-Science consultant SARA [email_address]
- 3. Who is SARA
- 4. Who am I
- 5. What is BiG Grid
- 6. Why is BiG Grid
- 8. National
- 14. HTC computing
- 15. HPC computing
- 16. Data-services & mass storage
- 18. Huygens National Super IBM Power 6, 3328 cores, 15.25 TB of memory, 700 TB of disk space, 60 TFlop/s LISA National Compute Cluster Dell cluster 4480 cores, 12 TB of memory, 20 TFlop/s Grid Rerouces 2376 Cores, 3408 TB of disk, 2000 TB tape 12 BioInfo Sites Life Science Grid High Energy Physics, Astronomy, Bio Info Visualization Tiled Panel Display Remote Visualization Network SURFnet 6 AMSix Netherlight Innovative Infrastructures Cloud GPU Hadoop Beehub
- 20. Online scientific Collaboration in european project
- 21. 2 years IBED
- 23. Guardian angel
- 28. Providing a world class e-science infrastructure
- 29. Part of the larger european grid
- 30. > 6000 compute cores
- 31. > 10 PB disk
- 32. Tape storage
- 34. E-Infrastructure NL WUR Life Science Grid 16 Grid cores 18 TB disk RUN Life Science Grid 32 Grid cores 18 TB disk UMCG Life Science Grid 32 Grid cores 18 TB disk Keygene Life Science Grid 32 Grid cores 18 TB disk Erasmus MC Life Science Grid 32 Grid cores 18 TB disk LUMC Life Science Grid 32 Grid cores 18 TB disk UU Life Science Grid 32 Grid cores 18 TB disk SARA Central Facillity 2400 cores 3450 TB disk 4000 TB tape 128 Cloud cores Nikhef Central Facillity 2500 Grid cores 1350 TB disk Philips Research Central Facillity 1648 Grid cores 20 TB disk RUG Central Facillity 294 Grid cores 34 TB disk AMC Life Science Grid 32 Grid cores 18 TB disk NKI Life Science Grid 32 Grid cores 18 TB disk SARA Huygens Super Lisa Cluster Visualization Hadoop TUD Life Science Grid 32 Grid cores 18 TB disk
- 37. computation
- 38. collaboration
- 39. lots of data
- 41. Google paper:
- 42. “The unreasonable effectiveness of data”
- 45. OCW (NWO)
- 47. Companies
- 48. Us @ work
- 51. Are you serious?
- 55. Set up experimental ingest
- 56. March 2012 in production
- 60. Support
- 61. Development
- 64. Keeping it all operational
- 67. Easy scalable – complex integrated pipelines
- 68. Legacy; you never start from scratch
- 70. Local policies
- 71. Who decides
- 73. Exploit BIG Grid infrastructure in the life science R&D
- 74. Create functional Problem Solving Environments (PSEs) for the selected technology areas that deal with high demand in computing resources
- 75. Connect with the NBIC-BioAssist and BiG Grid programs.
- 82. Collaboration can bite individual excellence
- 83. Loose coupling or tight integration
- 84. Keeping up with fast changes
- 88. Trans-disciplinarity and integration are key
- 94. BBMRI image NY times: http://goo.gl/I130Q
- 97. http://www.necen.nl concept drawing by J.J.Bot
Editor's Notes
- BiG Grid project proposal Both a problem and opportunity; combine data-set, colloborate on analysis, share the maintenance and curation best practices We need; reliable archiving, secure and easy access, retrieval facilities (discovery / search), communication about the data (now you access it)
- BiG Grid project proposal Both a problem and opportunity; combine data-set, colloborate on analysis, share the maintenance and curation best practices We need; reliable archiving, secure and easy access, retrieval facilities (discovery / search), communication about the data (now you access it)