The document discusses open source visualization of scientific data using tools like the Visualization Toolkit (VTK) and ParaView. It describes how these tools are being used to improve and open up computational chemistry workflows by enabling interactive 3D visualization of molecular structures, properties, and simulation results. Future directions include improved standard representations for different data types, linked multi-view visualization, mobile/tablet apps, and integrating with database and informatics tools.
Avogadro is being rewritten and architected to put semantic chemical meaning at the center of its internal data structures in order to fully support data-centric workflows. Computational and experimental chemistry both suffer when semantic meaning is lost; through the use of expressive formats such as CML, along with lightweight data-exchange formats such as JSON, workflows that previously demanded manual intervention to retain semantic meaning can be used. Integration with projects like JUMBO and Open Babel when conversion is required, coupled with codes such as NWChem where direct support for CML is being added, allow for much richer storage, analysis, and indexing of data. As web-based data sources add more semantic structure to their data, Avogadro will take advantage of those resources.
Chemical Databases and Open Chemistry on the DesktopMarcus Hanwell
The modern chemist has access to large databases containing both experimental and calculated data. The power of HPC resources continues to increase, with more practitioners having routine access to powerful computational chemistry tools. This places an increasingly high burden on users to assimilate these resources into their workflow in order to effectively utilize resources. The creation of an open, extensible application framework that puts computational tools, data, and domain specific knowledge at the fingertips of chemists is increasingly important. A data-centric approach to chemistry, storing all data in a searchable database, will empower users to efficiently collaborate, innovate, and push the frontiers of research. Providing an open, user-friendly and extensible application will open up new tools to experimental chemists, while providing computational chemists the ability to address greater challenges. Additionally, by distributing experimental and computational data across the research community, incorporating cheminformatics analytics techniques, and providing visual search for chemical structures, the workflow of both groups can be significantly improved. This requires suitable data formats for data exchange, and databases with appropriate APIs for querying, and uploading data in order to effectively share. This talk will discuss recent progress made in developing a suite of open chemistry applications on the desktop. The applications can query online databases, such as the NIH structure resolver service, download and manipulate structures, and prepare input files for standalone computational chemistry codes. Another application developed to submit jobs, monitor and retrieve results from HPC resources will also be shown, and a desktop chemistry database browser. The Quixote project aims to establish standards for data exchange in computational chemistry, along with data repositories for organizations. Establishing these standards is important to promote open, reproducible chemistry, and their integration into user-friendly desktop applications will promote their integration in the standard workflow of researchers.
Open Chemistry: Realizing Open Data, Open Standards, and Open SourceMarcus Hanwell
The Blue Obelisk has brought together the computational chemistry community and those who are passionate about Open Chemistry and realizing the promise of Open Data, Open Standards, and Open Software (ODOSOS); the three pillars the group promotes. We will present current work that has taken place over the past five years, which is inspired by these pillars, and present plans for future work.
The group is actively engaged in multiple open source projects that rely on and promote open standards and open data including: Avogadro (a powerful 3D molecular editor), OpenQube (a library for quantum mechanics), ChemData (a tool for large-scale chemical data analysis and visualization), Chemkit (a library for cheminformatics), MoleQueue (a HPC queue manager), and VTK (a library for scientific data visualization). The Open Chemistry project benefits greatly from the activities of the Blue Obelisk and makes use of several prominent open-source projects including Qt and MongoDB.
Avogadro: Open Source Libraries and Application for Computational ChemistryMarcus Hanwell
In order to tackle upcoming molecular simulation and visualization challenges in key areas of materials science, chemistry and biology it is necessary to move beyond fixed software applications. The Avogadro project is in the final stages of an ambitious rewrite of its core data structures, algorithms and visualization capabilities. The project began as a grass roots effort to address deficiencies observed by many of the early contributors in existing commercial and open source solutions. Avogadro is now a robust, flexible solution that can tie in to and harness the power of VTK for additional analysis and visualization capabilities.
Presented at the first Avogadro User Meeting, and presents an overview of the history of Avogadro development. It discusses changes in the rewrite, and the broader Open Chemistry project.
Avogadro is being rewritten and architected to put semantic chemical meaning at the center of its internal data structures in order to fully support data-centric workflows. Computational and experimental chemistry both suffer when semantic meaning is lost; through the use of expressive formats such as CML, along with lightweight data-exchange formats such as JSON, workflows that previously demanded manual intervention to retain semantic meaning can be used. Integration with projects like JUMBO and Open Babel when conversion is required, coupled with codes such as NWChem where direct support for CML is being added, allow for much richer storage, analysis, and indexing of data. As web-based data sources add more semantic structure to their data, Avogadro will take advantage of those resources.
Chemical Databases and Open Chemistry on the DesktopMarcus Hanwell
The modern chemist has access to large databases containing both experimental and calculated data. The power of HPC resources continues to increase, with more practitioners having routine access to powerful computational chemistry tools. This places an increasingly high burden on users to assimilate these resources into their workflow in order to effectively utilize resources. The creation of an open, extensible application framework that puts computational tools, data, and domain specific knowledge at the fingertips of chemists is increasingly important. A data-centric approach to chemistry, storing all data in a searchable database, will empower users to efficiently collaborate, innovate, and push the frontiers of research. Providing an open, user-friendly and extensible application will open up new tools to experimental chemists, while providing computational chemists the ability to address greater challenges. Additionally, by distributing experimental and computational data across the research community, incorporating cheminformatics analytics techniques, and providing visual search for chemical structures, the workflow of both groups can be significantly improved. This requires suitable data formats for data exchange, and databases with appropriate APIs for querying, and uploading data in order to effectively share. This talk will discuss recent progress made in developing a suite of open chemistry applications on the desktop. The applications can query online databases, such as the NIH structure resolver service, download and manipulate structures, and prepare input files for standalone computational chemistry codes. Another application developed to submit jobs, monitor and retrieve results from HPC resources will also be shown, and a desktop chemistry database browser. The Quixote project aims to establish standards for data exchange in computational chemistry, along with data repositories for organizations. Establishing these standards is important to promote open, reproducible chemistry, and their integration into user-friendly desktop applications will promote their integration in the standard workflow of researchers.
Open Chemistry: Realizing Open Data, Open Standards, and Open SourceMarcus Hanwell
The Blue Obelisk has brought together the computational chemistry community and those who are passionate about Open Chemistry and realizing the promise of Open Data, Open Standards, and Open Software (ODOSOS); the three pillars the group promotes. We will present current work that has taken place over the past five years, which is inspired by these pillars, and present plans for future work.
The group is actively engaged in multiple open source projects that rely on and promote open standards and open data including: Avogadro (a powerful 3D molecular editor), OpenQube (a library for quantum mechanics), ChemData (a tool for large-scale chemical data analysis and visualization), Chemkit (a library for cheminformatics), MoleQueue (a HPC queue manager), and VTK (a library for scientific data visualization). The Open Chemistry project benefits greatly from the activities of the Blue Obelisk and makes use of several prominent open-source projects including Qt and MongoDB.
Avogadro: Open Source Libraries and Application for Computational ChemistryMarcus Hanwell
In order to tackle upcoming molecular simulation and visualization challenges in key areas of materials science, chemistry and biology it is necessary to move beyond fixed software applications. The Avogadro project is in the final stages of an ambitious rewrite of its core data structures, algorithms and visualization capabilities. The project began as a grass roots effort to address deficiencies observed by many of the early contributors in existing commercial and open source solutions. Avogadro is now a robust, flexible solution that can tie in to and harness the power of VTK for additional analysis and visualization capabilities.
Presented at the first Avogadro User Meeting, and presents an overview of the history of Avogadro development. It discusses changes in the rewrite, and the broader Open Chemistry project.
Evolution of database access technologies in Java-based software projectsTom Mens
This talk was presented by Tom Mens and Loup Meurice at the EvoLille'15 European Open Symposium on Empirical Software Engineering (December 2015). We report on the evolution of open source Java projects relying on relational database technologies. We empirically analysed how the use of Java database access technologies evolves over time. We carried out a coarse-grained source-code analysis over several thousands of Java projects on GitHub, and complemented this by a fine-grained longitudinal analysis of the co-evolution between database schema changes and source code changes within three large Java projects (OSCAR, OpenEMR and BroadLeaf). The presented results are a first step towards a recommendation system supporting developers in writing database-centered code. The research was part of an interuniversity research project between University of Mons and University of Namur, financed by the F.R.S.-FNRS, Belgium.
Survival analysis of database technologies in open source Java projectsTom Mens
Presentation at ICSME 2015 (Bremen Germany) of an ERA track research article, entitled "Towards a survival analysis of database framework usage in Java projects". Joint work by Tom Mens and Mathieu Goeminne in the context of an F.R.S.-FNRS funded research project carried out at the Software Engineering Lab of the University of Mons, Belgiu
f people think about big data, they always think about Twitter or Facebook. But there are other areas where much more data amounts incurred and the analyzes are more complex. In this talk, we talk about a real example from bioinformatics. We will explain the actual scenario and how the various Microsoft platforms from SQL Server to Azure Analytics and HDInsight could help us – or not.
This is a presentation by Prof. Anne Elster at the International Workshop on Open Source Supercomputing held in conjunction with the 2017 ISC High Performance Computing Conference.
WBDB 2015 Performance Evaluation of Spark SQL using BigBencht_ivanov
In this paper we present the initial results of our work to run BigBench on Spark. First, we evaluated the data scalability behavior of the existing MapReduce implementation of BigBench. Next, we executed the group of 14 pure HiveQL queries on Spark SQL and compared the results with the respective Hive results. Our experiments show that: (1) for both MapReduce and Spark SQL, BigBench queries perform with the increase of the data size on average better than the linear scaling behavior and (2) pure HiveQL queries perform faster on Spark SQL than on Hive.
http://clds.sdsc.edu/wbdb2015.ca/program
Presentation slides for SDCSB Cytoscape Workshop on 5/19/2016. The presentation contains current status of Cytoscape project and overview of the Cytoscape ecosystem. It briefly mentions the Cytoscape Cyberinfrastructure.
In this talk, we discuss QuTrack, a Blockchain-based approach to track experiment and model changes primarily for AI and ML models. In addition, we discuss how change analytics can be used for process improvement and to enhance the model development and deployment processes.
DISCLAIMER: These are not my slides, but these slides are for the IPAW paper by Quan Pham, Tanu Malik, and Ian Foster: Auditing and Maintaining Provenance in Software Packages
How to expand the Galaxy from genes to Earth in six simple steps (and live sm...Raffaele Montella
FACE-IT is an effort to develop a new IT infrastructure to accelerate existing disciplinary research and enable information transfer among traditionally separate fields. At present, finding data and processing it into usable form can dominate research efforts. By providing ready access to not only data but also the software tools used to process it for specific uses (e.g., climate impact and economic model inputs), FACE-IT allows researchers to concentrate their efforts on analysis. Lowering barriers to data access allows researchers to stretch in new directions and allows researchers to learn and respond to the needs of other fields. FACE-IT builds on the Globus Galaxies platform, which has been developed over the past several years at the University of Chicago. FACE-IT also benefit from substantial software development undertaken by the communities who have developed most of the domain-specific tools required to populate FACE-IT with useful capabilities. The FACE-IT Galaxy manages earth system datatypes (as NetCDF), new tool parameters (dates, map, opendap), aggregated datatypes (RAFT), service providers and cool map visualizers.
Reproducibility - The myths and truths of pipeline bioinformaticsSimon Cockell
In a talk for the Newcastle Bioinformatics Special Interest Group (http://bsu.ncl.ac.uk/fms-bioinformatics) I explored the topic of reproducibility. Looking at the pros and cons of pipelining analyses, as well as some tools for achieving this. I also considered some additional tools for enabling reproducible bioinformatics, and look at the 'executable paper', and whether it represents the future for bioinformatics publishing.
Evolution of database access technologies in Java-based software projectsTom Mens
This talk was presented by Tom Mens and Loup Meurice at the EvoLille'15 European Open Symposium on Empirical Software Engineering (December 2015). We report on the evolution of open source Java projects relying on relational database technologies. We empirically analysed how the use of Java database access technologies evolves over time. We carried out a coarse-grained source-code analysis over several thousands of Java projects on GitHub, and complemented this by a fine-grained longitudinal analysis of the co-evolution between database schema changes and source code changes within three large Java projects (OSCAR, OpenEMR and BroadLeaf). The presented results are a first step towards a recommendation system supporting developers in writing database-centered code. The research was part of an interuniversity research project between University of Mons and University of Namur, financed by the F.R.S.-FNRS, Belgium.
Survival analysis of database technologies in open source Java projectsTom Mens
Presentation at ICSME 2015 (Bremen Germany) of an ERA track research article, entitled "Towards a survival analysis of database framework usage in Java projects". Joint work by Tom Mens and Mathieu Goeminne in the context of an F.R.S.-FNRS funded research project carried out at the Software Engineering Lab of the University of Mons, Belgiu
f people think about big data, they always think about Twitter or Facebook. But there are other areas where much more data amounts incurred and the analyzes are more complex. In this talk, we talk about a real example from bioinformatics. We will explain the actual scenario and how the various Microsoft platforms from SQL Server to Azure Analytics and HDInsight could help us – or not.
This is a presentation by Prof. Anne Elster at the International Workshop on Open Source Supercomputing held in conjunction with the 2017 ISC High Performance Computing Conference.
WBDB 2015 Performance Evaluation of Spark SQL using BigBencht_ivanov
In this paper we present the initial results of our work to run BigBench on Spark. First, we evaluated the data scalability behavior of the existing MapReduce implementation of BigBench. Next, we executed the group of 14 pure HiveQL queries on Spark SQL and compared the results with the respective Hive results. Our experiments show that: (1) for both MapReduce and Spark SQL, BigBench queries perform with the increase of the data size on average better than the linear scaling behavior and (2) pure HiveQL queries perform faster on Spark SQL than on Hive.
http://clds.sdsc.edu/wbdb2015.ca/program
Presentation slides for SDCSB Cytoscape Workshop on 5/19/2016. The presentation contains current status of Cytoscape project and overview of the Cytoscape ecosystem. It briefly mentions the Cytoscape Cyberinfrastructure.
In this talk, we discuss QuTrack, a Blockchain-based approach to track experiment and model changes primarily for AI and ML models. In addition, we discuss how change analytics can be used for process improvement and to enhance the model development and deployment processes.
DISCLAIMER: These are not my slides, but these slides are for the IPAW paper by Quan Pham, Tanu Malik, and Ian Foster: Auditing and Maintaining Provenance in Software Packages
How to expand the Galaxy from genes to Earth in six simple steps (and live sm...Raffaele Montella
FACE-IT is an effort to develop a new IT infrastructure to accelerate existing disciplinary research and enable information transfer among traditionally separate fields. At present, finding data and processing it into usable form can dominate research efforts. By providing ready access to not only data but also the software tools used to process it for specific uses (e.g., climate impact and economic model inputs), FACE-IT allows researchers to concentrate their efforts on analysis. Lowering barriers to data access allows researchers to stretch in new directions and allows researchers to learn and respond to the needs of other fields. FACE-IT builds on the Globus Galaxies platform, which has been developed over the past several years at the University of Chicago. FACE-IT also benefit from substantial software development undertaken by the communities who have developed most of the domain-specific tools required to populate FACE-IT with useful capabilities. The FACE-IT Galaxy manages earth system datatypes (as NetCDF), new tool parameters (dates, map, opendap), aggregated datatypes (RAFT), service providers and cool map visualizers.
Reproducibility - The myths and truths of pipeline bioinformaticsSimon Cockell
In a talk for the Newcastle Bioinformatics Special Interest Group (http://bsu.ncl.ac.uk/fms-bioinformatics) I explored the topic of reproducibility. Looking at the pros and cons of pipelining analyses, as well as some tools for achieving this. I also considered some additional tools for enabling reproducible bioinformatics, and look at the 'executable paper', and whether it represents the future for bioinformatics publishing.
In this presentation, Kris Thorleifsson and Agust Egilsson from QuantCell Research describe how their easy-to-use big data spreadsheet and end-user programming environment.
"QuantCell is a big data spreadsheet and an end-user programming tool. It improves turnaround time and enables SMEs to benefit from big data. It enables non-developers to build complex analysis, models and applications, and it brings the capabilities of major programming languages to the spreadsheet user. QuantCell supports real time Big Data frameworks and Apache Hadoop installations. It allows you to build MapReduce analysis or submit real time queries right from the spreadsheet, submit it to a Hadoop server and watch it hack away. The results can be delivered back to the spreadsheet for further analysis and visualization."
Learn more: http://www.quantcell.com
Watch the video presentation: http://wp.me/p3RLEV-1aI
Clouds, Clusters, and Containers: Tools for responsible, collaborative computingMatthew Vaughn
Intro slides from AKES workshop at ISMB2016. This workshop addresses the challenges and requirements for working effectively on cloud computing and high performance computing resources, discusses the key principles that should guide responsible scientific computation and collaboration, and using hands-on sessions presents practical solutions using emergent software tools that are becoming widely adopted in the global scientific community. Specifically, we will look at using “containers” to bundle software applications and their full execution environment in a portable way. We will look at managing and sharing data across distributed resources. And finally, we will tackle how to orchestrate job execution across systems and capture metadata on the results (and the process) so that parameters and methodologies are not lost.
Keynote at Gateways 2017 Conference, Ann Arbor MI
Speaker: Ian Stokes-Rees
"Connecting Cyberinfrastructure Back To The Laptop"
Science Gateways today are generally built to provide a web-accessible interface for a particular scientific community to access a combination of software, hardware, and data deployed in an expertly managed computing center. But what happens when the scientist wants to repatriate their data? Or perform some analysis that is not supported by the gateway? Both for the purposes of encouraging innovative workflows and serving an audience with a wide range of computational experience it is important to consider how a gateway can fit into the broader computational ecosystem of a particular researcher or research group. One simple starting point for this is to ask the question "how can the gateway connect back to the laptop?". This talk will consider how this is being done today in science gateways and present some ideas for how this could be expanded in the future.
A “meta‑cloud” for building clouds
Build your own cloud on our hardware resources
Agnostic to specific cloud software
Run existing cloud software stacks (like OpenStack, Hadoop, etc.)
... or new ones built from the ground up
Control and visibility all the way to the bare metal
“Sliceable” for multiple, isolated experiments at once
The Tale of Two Deployments: Greenfield and Monolith Apps with Docker Enterpr...Docker, Inc.
Docker use at Cornell University has been increasing steadily over the last 3 years in our central departments and various colleges - particularly as we move more workloads to the cloud. In this talk, we’ll give an overview of our Docker use cases across campus, featuring in detail two specific projects that highlight the versatility of this technology: Containerizing our central financial system (a traditional monolithic system); and building new researcher-focused financial tools natively in Docker (a microservice architecture built with the cloud in mind). We’ll discuss the design and implementation of both projects in detail. We’ll also describe how Docker has enabled us to develop consistent DevOps and CI practices spanning these two very different architectures.
How static analysis supports quality over 50 million lines of C++ codecppfrug
Retour d’expérience sur l'utilisation d'outils d'analyse statique pour maintenir des activités Q&A dans le développement de logiciel scientifique de grande échelle au sein du CERN.
Big Data Europe SC6 WS 3: Ron Dekker, Director CESSDA European Open Science A...BigData_Europe
Slides for keynote talk at the Big Data Europe workshop nr 3 on 11.9.2017 in Amsterdam co-located with SEMANTiCS2017 conference by Ron Dekker, Director CESSDA: European Open Science Agenda: where we are and where we are going?
In this presentation from the Dell booth at SC13, Joseph Antony from NCI describes how they are using HPC Virtualization to meet user needs.
Watch the video presentation: http://insidehpc.com/2013/12/05/panel-discussion-thought-hpc-virtualization-never-going-happen/
The mission of the IHME is to apply rigorous measurement and analysis to help policy makers make better decisions on a range of health policy issues. Like other organizations, the IHME have embraced containers and micro-services aggressively to better support hundreds of collaborating researchers.
In addition to containerized workloads, the IHME run a wide-variety of traditional analytic, simulation and high-performance computing workloads on an HPC cluster with 15,000 cores and 13PB of storage. Researchers increasingly need to combine both containerized and non-containerized elements into workflow pipelines, and a key challenge has been ensuring SLAs for various departments and avoiding duplicate infrastructure and unnecessary data movement and duplication. In collaboration with industry partners, IHME have deployed a unique solution based on Univa’s Navops technology that allows them to combine containerized and traditional analytic and high-performance application workloads on a single shared Kubernetes cluster, ensuring departmental SLAs and helping contain infrastructure costs.
In this talk Dr. Grandison will discuss IHME, their experience deploying containerized applications and how they went about using Kubernetes to support a variety of new containerized applications as well as a variety of traditional analytic applications.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Climate Impact of Software Testing at Nordic Testing Days
Open Source Visualization of Scientific Data
1. Open Source Visualization of
Scientific Data
8 August 2011
Dr. Marcus D. Hanwell
marcus.hanwell@kitware.com
1
2. Outline
• Background
• Why is open science important?
• Opening up chemistry over the last four years
• The Visualization Toolkit (VTK)
• ParaView – a client-server Qt based VTK GUI
• New frontiers – web, mobile and tablets
• Future directions
2
3. My Background
• Ph.D. (Physics) – University of Sheffield
• Google Summer of Code – Avogadro
• Postdoc (Chemistry) – University of Pittsburgh
• R&D engineer – Kitware, Inc
• Passionate about physics, chemistry, and the
growing need to improve computational tools
• See the need for powerful open source, cross
platform frameworks and applications
• Develop(ed): Gentoo, KDE, Kalzium, Avogadro,
Open Babel, VTK, ParaView, Titan, CMake
3
4. Kitware
• Founded in 1998: 5 former GE Research employees
• 95 employees: 42% PhD
• Privately held, profitable from creation, no debt
• Rapidly Growing: >30% in 2010, 7M web-visitors/quarter
• Offices • 2011 Small Business
– Albany, NY Administration’s
Tibbetts Award
– Carrboro, NC
• HPCWire Readers
– Lyon, France and Editor’s Choice
– Bangalore, India • Inc’s 5000 List: 2008
to 2010
6. What Is “Open Science”?
“Open science is the idea that scientific
knowledge of all kinds should be openly
shared as early as is practical in the
discovery process.”
openscience.org
6
7. What Is The Problem?
“…when the journal system was developed
in the 17th and 18th centuries it was an
excellent example of open science. The
journals are perhaps the most open system
for the dissemination of knowledge that can
be constructed — if you’re working with 17th
century technology. But, of course, today
we can do a lot better.”
openscience.org
7
8. Opening Up Chemistry
• Computational chemistry is currently one
of the more closed sciences
• Lots of black box proprietary codes
– Only a few have access to the code
– Publishing results from black box codes
– Many file formats in use, little agreement
• More papers should be including data
• Growing need for open standards
8
9. Movements for Open Chemistry
• Formed an “unorganization” – Blue Obelisk
– Published first article in 2005
– Open data, open standards and open source
– Meet at ACS and other conferences when possible
– Follow-up article currently in press
• Quixote collaboration more recently
– Provide meaningful data storage and exchange
– Principally targeting computational chemistry
9
10. Typical Chemistry Workflow
Log File Input File
Edit/Analyze
Results
Data
Job
Submission
Local
Calcula@on
Remote
10
11. Problem: Pretty Complex/Manual
• Most steps require user intervention
• Obtain starting structure (previous work, databases)
• Edit structure
• Write input file
• Move input file to cluster
• Submit to queue
• Wait for completion
• Retrieve input file
• Analyze output file
• Extract the relevant data, change formats
• Store results
• Repeat
11
12. Improved Chemistry Workflow
Log File Input File
Edit/Analyze
Results
Data
Job
Submission
Local
Calcula@on
Remote
12
13. Avogadro
• Project began 2006
• Split into library and
application (plugin based)
• One of very few open source editors
• Designed to be extensible from the start
• Generate input & read output from many codes
• An active and growing community
• Chemistry needs a free, open framework
13
14. Avogadro’s Roots
• Avogadro projected started in 2006
• First funded work in 2007 by Marcus Hanwell
– Google Summer of Code student
– Final year of Ph.D. spent the summer coding
– Funded as part of KDE project – Kalzium editor
• Built on several other open source projects
– Qt, Eigen, Open Babel, Blue Obelisk Data Repository
• Also uses open standards, e.g. OpenGL
• Cross platform, open source stack
14
15. Avogadro Vital Statistics
• Supports Linux, Windows and Mac OS X
• Contributions from over 20 developers
• Over 180,000 downloads over 4 years
• Translated into 19 languages
• Used by Kalzium for molecular editor
• Featured by Trolltech/Nokia,
– Qt in use
– Qt ambassador program
15
17. Desktop Database
• Use of “document store” NoSQL
• Doesn’t force too much structure
• Some entries have experimental data available
• Some have computational jobs
• Employ a “pile of stuff” approach
• Can store both source and derived data
• Calculate identifiers, QSAR properties, etc
• MongoDB is a scalable, open solution
• Proven scaling with large web applications
17
18. Chemistry Data Explorer
• Qt application
• Connects to local or remote database
• Uses VTK for visual data exploration
• Can ingest new data
– Uses Open Babel to generate descriptors
– Standard InChi, SMILES, molecular weight
– More could be added
• All derived from files stored in the database
18
20. Database Interaction on the Web
• Avogadro directly accesses some (read-
only) public databases:
• PDB, NIH “fetch by name”
• More could be added
• ChemData follows this approach
• Quixote aims to support both public and
private sharing models – open framework
20
25. OpenQube – Quantum Data
• Reads in key quantum data
– Basis set used in calculation
– Eigenvectors for molecular orbitals
– Density matrix for electron density
– Standard geometry
• Multithreaded calculation
– Produce regular grids of scalar data
– Molecular orbitals, electron density…
25
26. Molecular Orbitals and Electron Density
• Quantum files store basis sets and
matrices
−αr 2
GTO = ce
φ i = ∑ c µiφ µ
µ
ρ(r) = ∑ ∑ Pµν φ µ φν
µ ν
• Using these equations, and the supplied
matrices – calculate cubes
26
27. Advanced Visualization: VTK
• New Avogadro plugin:
• Takes volumetric data from Avogadro
• Uses GPU accelerated rendering in VTK
• Widespread excitement from many in the
chemistry community
• Several groups interested in collaborating
• Google Summer of Code project
• Leverage significant capabilities in VTK
27
31. VTK: The Toolkit
• Collection of C++ libraries
– Leveraged by many applications
– Divided into logical areas, e.g.
• Filtering – data processing in visualization pipeline
• InfoVis – informatics visualization
• Widgets – 3D interaction widgets
• VolumeRendering – 3D volume rendering
• Cross platform, using OpenGL
• Wrapped in Python, Tcl and Java
31
32. VTK Development Team
• From Ohloh: Very large, active development team: Over the past twelve
months, 100 developers contributed new code to VTK. This is one of the largest open-source
teams in the world, and is in the top 2% of all project teams on Ohloh.
and many others...
32
33. ParaView
• Parallel visualization application
• Open source, BSD licensed
• Turn-key application wrapper around VTK
• Parallel data processing and rendering
33
34. Large Data Visualization
• BlueGene/L at LLNL
– 65,536 compute nodes (32 bit PPC)
– 1,024 I/O nodes (32 bit PPC)
– 512 MB of RAM per node
• Sandia Red Storm
– 12,960 compute nodes (AMD Opteron dual)
– 640 service and I/O nodes
– 40 TB of DDR RAM per node
34
38. 3D Chemistry Visualization
• Some existing features specific to chemistry
– Gaussian cube, PDB, and a few others
• Excellent handling of volumetric data:
– Marching cubes
– Volume rendering
– Contouring
• Advanced rendering:
– Point sprites
– Manta – real time ray tracing
38
39. Titan: VTK and Informatics
• Led by Sandia National Laboratories
• Substantial expansion of VTK:
– Informatics & analysis
• Actively developed, growing feature set
• Improved 2D rendering and API
• Database connectivity, client-server, pipeline
based approach
• Uses web technologies such as ProtoViz
• Scalable, interactive infoviz
39
41. New Frontiers
• New work porting VTK
– Use C++ as the common core
• iOS port in the early stages
• Android port
– Use OpenGL ES 2.0 – new rendering code
• Also ParaViewWeb – delivering over web
– Use image delivery and rendering on server
– Also using WebGL for rendering (optionally)
41
42. Future Directions
• VTK modularization (in progress)
– Developing more agile build systems
– Automating more with CMake
• Using Git more fully to improve stability
– Use of master and next
– Topic branches - merge when ready
• Code review using Gerrit
– Integration with continuous integration
– Test before merge
42