Docker for Integration Testing
•
0 likes•1,773 views
Docker for Integration Testing, presentation at Docker Amsterdam/Utrecht August 2014.
Report
Share
Report
Share
Download to read offline
Recommended
Using Docker to Develop, Test and Run Maven Projects - Wouter Danes
Docker recently hit version 1.0 and is being picked up around the world by Ops teams to ease running their applications. Docker can also play a big role in easing the development of applications. In this talk I will address how to use docker to: - create a more scalable build environment using jenkins and docker; - integration test your software using maven and docker; - package your software and run the images in different environments.
Using Docker for Testing
Demo of how to dockerise and deploy your microservices application to the test environment, how to run selenium tests inside docker and how to put this all together to integrate your tests in your CI/CD pipeline using Jenkins.
Presented at ATA GTR 2016 in Pune.
Zalenium - SeleniumConf Austin 2017
We know how complicated it is to have a stable grid, and how hard it is to maintain over time with enough capabilities to cover most browsers and platforms. Internally, we found that ~75% of our tests were executed in Firefox/Chrome, and the remaining were executed in Safari/IE. We decided to develop a tool where docker-selenium nodes are created, used and disposed on demand. For Safari/IE, we just forward the tests to Sauce Labs/BrowserStack.
Zalenium is an OSS extension to scale up and down your local grid dynamically with Docker containers. It uses Docker-Selenium to run tests in Firefox/Chrome, and when a different browser is needed, tests get redirected to a cloud testing service. Result: our tests suites run faster since most of the tests run on local Firefox/Chrome nodes, and we use in a smarter way the cloud testing service we pay for.
Diego Molina – Software Engineer in Test, Zalando SE
Leo Gallucci – Software Engineer, Tools and Infrastructure, Zalando SE
Docker Best Practices Workshop
A hands-on workshop that covers 18 best practices in 4 categories or in other words ✅️ Dos & Don'ts.
After a general introduction, we will have a look at the essential practices (aka must do), then move to the image practices, then we will go through the security practices, and finally, some general practices.
Please note, this workshop assumes that you have a basic knowledge of Docker.
Hands-on repo:
https://github.com/aabouzaid/docker-best-practices-workshop
Stop Being Lazy and Test Your Software
DockerCon EU 2015 in Barcelona
Practical tips for using Docker to run tests during development, CI/CD, and different strategies for speeding up the run of your test suite by using parallel pipelines in containers.
The jet tool used to demonstrate parallel testing is available here: https://codeship.com/documentation/docker/installation/
Introduction to Docker
Introduction to Docker slide in Docker for DevOps and Continuous Delivery at OSS Festival 2014 Thailand
Recommended
Using Docker to Develop, Test and Run Maven Projects - Wouter Danes
Docker recently hit version 1.0 and is being picked up around the world by Ops teams to ease running their applications. Docker can also play a big role in easing the development of applications. In this talk I will address how to use docker to: - create a more scalable build environment using jenkins and docker; - integration test your software using maven and docker; - package your software and run the images in different environments.
Using Docker for Testing
Demo of how to dockerise and deploy your microservices application to the test environment, how to run selenium tests inside docker and how to put this all together to integrate your tests in your CI/CD pipeline using Jenkins.
Presented at ATA GTR 2016 in Pune.
Zalenium - SeleniumConf Austin 2017
We know how complicated it is to have a stable grid, and how hard it is to maintain over time with enough capabilities to cover most browsers and platforms. Internally, we found that ~75% of our tests were executed in Firefox/Chrome, and the remaining were executed in Safari/IE. We decided to develop a tool where docker-selenium nodes are created, used and disposed on demand. For Safari/IE, we just forward the tests to Sauce Labs/BrowserStack.
Zalenium is an OSS extension to scale up and down your local grid dynamically with Docker containers. It uses Docker-Selenium to run tests in Firefox/Chrome, and when a different browser is needed, tests get redirected to a cloud testing service. Result: our tests suites run faster since most of the tests run on local Firefox/Chrome nodes, and we use in a smarter way the cloud testing service we pay for.
Diego Molina – Software Engineer in Test, Zalando SE
Leo Gallucci – Software Engineer, Tools and Infrastructure, Zalando SE
Docker Best Practices Workshop
A hands-on workshop that covers 18 best practices in 4 categories or in other words ✅️ Dos & Don'ts.
After a general introduction, we will have a look at the essential practices (aka must do), then move to the image practices, then we will go through the security practices, and finally, some general practices.
Please note, this workshop assumes that you have a basic knowledge of Docker.
Hands-on repo:
https://github.com/aabouzaid/docker-best-practices-workshop
Stop Being Lazy and Test Your Software
DockerCon EU 2015 in Barcelona
Practical tips for using Docker to run tests during development, CI/CD, and different strategies for speeding up the run of your test suite by using parallel pipelines in containers.
The jet tool used to demonstrate parallel testing is available here: https://codeship.com/documentation/docker/installation/
Introduction to Docker
Introduction to Docker slide in Docker for DevOps and Continuous Delivery at OSS Festival 2014 Thailand
Drone CI
A talk about Drone CI I gave at the local Golang Meetup in Nuremberg, Germany. Just a simple intro how Drone CI works and how to get started.
Ice breaker with dev ops
Want to get started with DevOps, but dont know where to start? Refer to this ppt which will help you get started with DevOps.
Contribute 101: Compose/Kitematic/Machine by Ben Bonnefoy
Technical staff at Docker, Ben Bonnefoy, David Gageot, Nathan LeClaire and Aanand Prasad, introduce the Docker Project and how to contribute.
CI/CD with Jenkins and Docker - DevOps Meetup Day Thailand
Slide for talking in "CI/CD with Jenkins and Docker"
Introduction to jenkins
@abe238 - Introduction to Jenkins - presented on September 12, 2015 at the Seattle CodeCamp 2015 in Seattle University by Abe Diaz
"Workstation Up" - Docker Development at Flow by Mike Roth
Docker is an integral part of Flow's technology stack, supporting everything from a developer's local environment to Production containers in AWS.
"Workstation" has become central to a developer's toolset at Flow, giving them the ability to bring up/down a service, along with any upstream/downstream dependencies, in a single, simple command implemented with GOlang CLI. For example, developers can run “workstation up --app www” - and reliably have the www app running along with its dozens of transitive dependencies. It truly is reliable - requiring no additional configuration - and just continues to work.
The team has recently transitioned to Docker for Mac Beta and just love referencing containers via localhost!
DockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose
Presented by Sandro Cirulli, Platform Tech Lead, Oxford University Press
Oxford University Press (OUP) recently started the Oxford Global Languages (OGL) initiative (http://www.oxforddictionaries.com/words/oxfordlanguages) which aims at providing language resources for digitally under represented languages. In August 2015 OUP launched two African languages websites for Zulu (http://zu.oxforddictionaries.com) and Northern Sotho (http://nso.oxforddictionaries.com). The backend of these websites is based on an API retrieving data in RDF from a triple store and delivering data to the frontend in JSON-LD.
The entire micro-service infrastructure for development, staging, and production runs on Docker containers in Amazon EC2 instances. In particular, we use Jenkins to rebuild the Docker image for the API based on a Python Flask application and Docker Compose to orchestrate the containers. A typical CI workflow is as follows:
- a developer commits code to the codebase
- Jenkins triggers a job to run unit tests
- if the unit tests are successful, the Docker image of the Python Flask application is rebuilt and the container is restarted via Docker Compose
- if the unit tests or the Docker build failed, the monitor view shows the Jenkins jobs in red and displays the name of the possible culprit who broke the build.
A demo of this CI workflow is available at http://www.sandrocirulli.net/continuous-integration-with-jenkins-docker-and-compose
CI/CD Pipeline with Docker
Basic Idea
Develop a build system that leverages Docker for implementing continuous integration/deployment(CI/CD) pipeline. A git commit must kick off packaging a Docker Image and provisioning it in a VM.
A git based commit should be used for starting of a build for a docker image which would then be run and provisioned in a Virtual Machine. After every commit a series of test cases is then run on the code to ensure the correctness of the code. After all the test-cases pass, the image gets updated on docker-hub registry, and a VM gets provisioned which can then run the software directly (after pulling the image from the docker-hub).
This entire process ensures that the most recent and updated version of the code is available to the person who is using the software and this speeds up the overall process by at least 2-3 folds.
vodQA(Pune) 2018 - Visual testing of web apps in headless environment manis...
vodQA(Pune) 2018 - Visual testing of web apps in headless environment manish katoch
Shipping NodeJS with Docker and CoreOS
These are my slides from the November BayNode Talk Night. I spoke about our experience moving our NodeJS architecture to Docker and CoreOS as well as some tips/tricks we've learned along the way.
calmio-cicd-containers
In this talk, we will discuss the construction of a CI/CD pipeline consisting of Docker Engine, GitHub, Jenkins, Docker Registry and calm.io. The pipeline will be kicked off by a commit to a GitHub repository. The commit will cause Jenkins to run a build job and, upon successful completion of that job, push a Docker image up to Docker Registry. Once the new docker image is made available, Jenkins will trigger calm.io to deploy the new images on staging and production systems.
Git and GitHub for Documentation
I have evidence that using git and GitHub for documentation and community doc techniques can give us 300 doc changes in a month. I’ve bet my career on these methods and I want to share with you.
Javaone 2014 - Git & Docker with Jenkins
JavaOne 2014 demo deck --- topics: Advanced Git Integration w/Jenkins & Docker Jenkins integration
Building Efficient Parallel Testing Platforms with Docker
We often use containers to maintain parity across development, testing, and production environments, but we can also use containerization to significantly reduce time needed for testing by spinning up multiple instances of fully isolated testing environments and executing tests in parallel. This strategy also helps you maximize the utilization of infrastructure resources. The enhanced toolset provided by Docker makes this process simple and unobtrusive, and you’ll see how Docker Engine, Registry, and Compose can work together to make your tests fast.
Continuous Delivery Pipeline with Docker and Jenkins
Example of a Continuous Delivery Pipeline using Docker and Jenkins. github: https://github.com/camiloribeiro/cdeasy
.Net OSS Ci & CD with Jenkins - JUC ISRAEL 2013
A talk I [Haggai Philip Zagury] gave @ JUC 2013 - https://speakerdeck.com/hagzag/juc-israel-2013
Jenkins, pipeline and docker
The Jenkins open source continuous integration server now provides a “pipeline” scripting language which can define jobs that persist across server restarts, can be stored in a source code repository and can be versioned with the source code they are building. By defining the build and deployment pipeline in source code, teams can take full control of their build and deployment steps. The Docker project provides lightweight containers and a system for defining and managing those containers. The Jenkins pipeline and Docker containers are a great combination to improve the portability, reliability, and consistency of your build process.
This session will demonstrate Jenkins and Docker in the journey from continuous integration to DevOps.
Testing strategies for Docker containers
There are multiple testing strategies you can apply to test #Docker containers. In these slides, I’ll highlight some of them, discussing benefits and drawbacks of each approach.
This is the companion presentation for my blog post about #Docker #testing: https://goo.gl/IEQLKx
Docker.pptx
Docker Basics
docker windows authentication
docker visual studio
docker dot net
docker files
docker-compose
docker networking
container
Microsoft containers
More Related Content
What's hot
Drone CI
A talk about Drone CI I gave at the local Golang Meetup in Nuremberg, Germany. Just a simple intro how Drone CI works and how to get started.
Ice breaker with dev ops
Want to get started with DevOps, but dont know where to start? Refer to this ppt which will help you get started with DevOps.
Contribute 101: Compose/Kitematic/Machine by Ben Bonnefoy
Technical staff at Docker, Ben Bonnefoy, David Gageot, Nathan LeClaire and Aanand Prasad, introduce the Docker Project and how to contribute.
CI/CD with Jenkins and Docker - DevOps Meetup Day Thailand
Slide for talking in "CI/CD with Jenkins and Docker"
Introduction to jenkins
@abe238 - Introduction to Jenkins - presented on September 12, 2015 at the Seattle CodeCamp 2015 in Seattle University by Abe Diaz
"Workstation Up" - Docker Development at Flow by Mike Roth
Docker is an integral part of Flow's technology stack, supporting everything from a developer's local environment to Production containers in AWS.
"Workstation" has become central to a developer's toolset at Flow, giving them the ability to bring up/down a service, along with any upstream/downstream dependencies, in a single, simple command implemented with GOlang CLI. For example, developers can run “workstation up --app www” - and reliably have the www app running along with its dozens of transitive dependencies. It truly is reliable - requiring no additional configuration - and just continues to work.
The team has recently transitioned to Docker for Mac Beta and just love referencing containers via localhost!
DockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose
Presented by Sandro Cirulli, Platform Tech Lead, Oxford University Press
Oxford University Press (OUP) recently started the Oxford Global Languages (OGL) initiative (http://www.oxforddictionaries.com/words/oxfordlanguages) which aims at providing language resources for digitally under represented languages. In August 2015 OUP launched two African languages websites for Zulu (http://zu.oxforddictionaries.com) and Northern Sotho (http://nso.oxforddictionaries.com). The backend of these websites is based on an API retrieving data in RDF from a triple store and delivering data to the frontend in JSON-LD.
The entire micro-service infrastructure for development, staging, and production runs on Docker containers in Amazon EC2 instances. In particular, we use Jenkins to rebuild the Docker image for the API based on a Python Flask application and Docker Compose to orchestrate the containers. A typical CI workflow is as follows:
- a developer commits code to the codebase
- Jenkins triggers a job to run unit tests
- if the unit tests are successful, the Docker image of the Python Flask application is rebuilt and the container is restarted via Docker Compose
- if the unit tests or the Docker build failed, the monitor view shows the Jenkins jobs in red and displays the name of the possible culprit who broke the build.
A demo of this CI workflow is available at http://www.sandrocirulli.net/continuous-integration-with-jenkins-docker-and-compose
CI/CD Pipeline with Docker
Basic Idea
Develop a build system that leverages Docker for implementing continuous integration/deployment(CI/CD) pipeline. A git commit must kick off packaging a Docker Image and provisioning it in a VM.
A git based commit should be used for starting of a build for a docker image which would then be run and provisioned in a Virtual Machine. After every commit a series of test cases is then run on the code to ensure the correctness of the code. After all the test-cases pass, the image gets updated on docker-hub registry, and a VM gets provisioned which can then run the software directly (after pulling the image from the docker-hub).
This entire process ensures that the most recent and updated version of the code is available to the person who is using the software and this speeds up the overall process by at least 2-3 folds.
vodQA(Pune) 2018 - Visual testing of web apps in headless environment manis...
vodQA(Pune) 2018 - Visual testing of web apps in headless environment manish katoch
Shipping NodeJS with Docker and CoreOS
These are my slides from the November BayNode Talk Night. I spoke about our experience moving our NodeJS architecture to Docker and CoreOS as well as some tips/tricks we've learned along the way.
calmio-cicd-containers
In this talk, we will discuss the construction of a CI/CD pipeline consisting of Docker Engine, GitHub, Jenkins, Docker Registry and calm.io. The pipeline will be kicked off by a commit to a GitHub repository. The commit will cause Jenkins to run a build job and, upon successful completion of that job, push a Docker image up to Docker Registry. Once the new docker image is made available, Jenkins will trigger calm.io to deploy the new images on staging and production systems.
Git and GitHub for Documentation
I have evidence that using git and GitHub for documentation and community doc techniques can give us 300 doc changes in a month. I’ve bet my career on these methods and I want to share with you.
Javaone 2014 - Git & Docker with Jenkins
JavaOne 2014 demo deck --- topics: Advanced Git Integration w/Jenkins & Docker Jenkins integration
Building Efficient Parallel Testing Platforms with Docker
We often use containers to maintain parity across development, testing, and production environments, but we can also use containerization to significantly reduce time needed for testing by spinning up multiple instances of fully isolated testing environments and executing tests in parallel. This strategy also helps you maximize the utilization of infrastructure resources. The enhanced toolset provided by Docker makes this process simple and unobtrusive, and you’ll see how Docker Engine, Registry, and Compose can work together to make your tests fast.
Continuous Delivery Pipeline with Docker and Jenkins
Example of a Continuous Delivery Pipeline using Docker and Jenkins. github: https://github.com/camiloribeiro/cdeasy
.Net OSS Ci & CD with Jenkins - JUC ISRAEL 2013
A talk I [Haggai Philip Zagury] gave @ JUC 2013 - https://speakerdeck.com/hagzag/juc-israel-2013
Jenkins, pipeline and docker
The Jenkins open source continuous integration server now provides a “pipeline” scripting language which can define jobs that persist across server restarts, can be stored in a source code repository and can be versioned with the source code they are building. By defining the build and deployment pipeline in source code, teams can take full control of their build and deployment steps. The Docker project provides lightweight containers and a system for defining and managing those containers. The Jenkins pipeline and Docker containers are a great combination to improve the portability, reliability, and consistency of your build process.
This session will demonstrate Jenkins and Docker in the journey from continuous integration to DevOps.
What's hot (20)
Contribute 101: Compose/Kitematic/Machine by Ben Bonnefoy
Contribute 101: Compose/Kitematic/Machine by Ben Bonnefoy
CI/CD with Jenkins and Docker - DevOps Meetup Day Thailand
CI/CD with Jenkins and Docker - DevOps Meetup Day Thailand
"Workstation Up" - Docker Development at Flow by Mike Roth
"Workstation Up" - Docker Development at Flow by Mike Roth
DockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose
DockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose
vodQA(Pune) 2018 - Visual testing of web apps in headless environment manis...
vodQA(Pune) 2018 - Visual testing of web apps in headless environment manis...
Building Efficient Parallel Testing Platforms with Docker
Building Efficient Parallel Testing Platforms with Docker
Continuous Delivery Pipeline with Docker and Jenkins
Continuous Delivery Pipeline with Docker and Jenkins
Similar to Docker for Integration Testing
Testing strategies for Docker containers
There are multiple testing strategies you can apply to test #Docker containers. In these slides, I’ll highlight some of them, discussing benefits and drawbacks of each approach.
This is the companion presentation for my blog post about #Docker #testing: https://goo.gl/IEQLKx
Docker.pptx
Docker Basics
docker windows authentication
docker visual studio
docker dot net
docker files
docker-compose
docker networking
container
Microsoft containers
Codefresh + Cloud 66 webinar: Testing Strategies for Docker Driven Development
Slides from Alexei Ledenev's talk on testing strategies for Docker containers. Abstract: Being able to build a Docker container is only part of its development story. You need to be able to test your newly created containers. Ideally, we’d all like our container testing to be repeatable and portable. But there’s more than one way to achieve this so how do you know which method to choose? During this talk, we will demonstrate several approaches for testing Docker containers, discussing benefits and drawbacks with each approach.
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...
This DevOps Docker Tutorial on what is docker ( Docker Tutorial Blog Series: https://goo.gl/32kupf ) will help you understand how to use Docker Hub, Docker Images, Docker Container & Docker Compose. This tutorial explains Docker's working Architecture and Docker Engine in detail. This Docker tutorial also includes a Hands-On session around Docker by the end of which you will learn to pull a centos Docker Image and spin your own Docker Container. You will also see how to launch multiple docker containers using Docker Compose. Finally, it will also tell you the role Docker plays in the DevOps life-cycle.
The Hands-On session is performed on an Ubuntu-64bit machine in which Docker is installed.
Containers and Microservices for Realists
Talk for Oracle Code Conference- New York 2017. Container, Microservices, Serverless by Karthik Gaekwad
Containers and microservices for realists
Talk for Oracle Code Conference- New York 2017. Container, Microservices, Serverless
Docker Birthday #3 - Intro to Docker Slides
High level overview of Docker + Birthday #3 overview (app and challenge portion)!
Learn more about Docker Birthday #3 celebrations here: https://www.docker.com/community/docker-birthday-3
Docker for dev
Nelis and I used this presentation to guide us through the Docker for Dev session at XP Days Benelux 2016.
Dockerize the World
Presentation about docker from Java User Group in Ostrava CZ (23th of November 2015). Presented by Martin Damovsky (@damovsky).
Demos are available at https://github.com/damovsky/jug-ostrava-docker
Docker and OpenStack Boston Meetup
OpenStack, Containers, and Docker: The Future of Application Deployment
Twenty years ago, developers built static applications on well-defined stacks that ran on proprietary, monolithic hardware. Developers today want freedom to build applications using their choice of services and stacks and, ideally, want to be able to run those applications on any available hardware. Of course, this raises questions about service interaction, the practicality of migrating applications across environments, and the challenges of managing unlimited combinations of services and hardware environment.
By promoting an opensource approach to flexible and inter-operable infrastructure, OpenStack goes a long way towards achieving this vision of the future. This talk discusses the application and platform side of the equation, and the interplay between OpenStack, Container technology (e.g. LXC), and the opensource Docker.io project. Docker.io enables any application and its dependencies to be deployed as lightweight containers that run consistently virtually anywhere. The same containerized application that runs on a developer's laptop can run consistently on a bare metal server, an OpenStack cluster, a Rackspace cloud, a VM,etc. While providing isolation and compatibility, containers have significant size, performance, and deployment advantages over traditional VMs.
Recently, the community created an integration between Docker and OpenStack Nova, opening up exciting possibilities for web scale application deployment, continuous integration and deployment, private PaaS, and hybrid cloud. This session will give an introduction to Docker and containers in the context of OpenStack, and will then demonstrate cross-environment deployment of applications.
Docker Dhahran Nov 2016 meetup
Docker "Global Mentor Week" is your opportunity to #learndocker. to learn how to build, ship, and run modern distributed applications with ease. thanks to the Docker platform.
Right now, Docker has developed out a series of self-paced online labs that will be available during the meetup. Docker’s meetup groups worldwide are hosting a series of complimentary events to help newcomers and intermediate users learn Docker.
We'll have hands-on labs for both beginners and intermediate users, labs targeting both developers and operations. There is something for everyone. Docker mentor will be on hand at this event to help you prepare. and work through the self-paced materials. Bring your laptop, have fun and learn Docker!
Docker for Developers
Heard about Docker and are wondering how to make it part of your stack? Excited by the potential of developing faster and smarter? We’re going to show you how to build development environments with Docker. Then teach you how to integrate that environment into your development and testing workflow.
We’ll show you how to easily install Docker on Linux, OS X and Windows and how to manage it.
You’ll learn how to build replicas of your production stack to develop in.
We’ll demonstrate some simple Docker-enabled workflows that include local development, Git Hub integration, and container-based testing.
You’ll get an introduction to taking the next step and leveling up your local workflow to a group or team-centric continuous integration environment with Jenkins.
You’ll leave the session with the basic tools to add Docker to your development lifecycle and information about how to learn more to build on your knowledge.
Dockerize the World - presentation from Hradec Kralove
Introduction to Docker + many useful tips and hacks + use cases for docker.
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
This was a talk given at the second CT Software Developers Meetup (http://www.meetup.com/CT-Software-Developers-Meetup/). It covers how NorthPage is using Docker and Vagrant with a home grown Preview tool to increase the efficiency of the GitHub Pull Request Workflow.
Docker From Scratch
Docker provides a new, powerful way of prototyping, testing and deploying applications on cloud-based infrastructures. In this seminar we delve into the concept of Docker containers without requiring any previous knowledge from the audience.
Similar to Docker for Integration Testing (20)
Codefresh + Cloud 66 webinar: Testing Strategies for Docker Driven Development
Codefresh + Cloud 66 webinar: Testing Strategies for Docker Driven Development
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...
What is Docker | Docker Tutorial for Beginners | Docker Container | DevOps To...
Dockerize the World - presentation from Hradec Kralove
Dockerize the World - presentation from Hradec Kralove
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
CT Software Developers Meetup: Using Docker and Vagrant Within A GitHub Pull ...
Recently uploaded
Quarkus Hidden and Forbidden Extensions
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
GlobusWorld 2024 Opening Keynote session
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Developing Distributed High-performance Computing Capabilities of an Open Sci...
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Understanding variable roles in code has been found to be helpful by students
in learning programming -- could variable roles help deep neural models in
performing coding tasks? We do an exploratory study.
- These are slides of the talk given at InteNSE'23: The 1st International Workshop on Interpretability and Robustness in Neural Software Engineering, co-located with the 45th International Conference on Software Engineering, ICSE 2023, Melbourne Australia
Cracking the code review at SpringIO 2024
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Enhancing Research Orchestration Capabilities at ORNL.pdf
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
How to Position Your Globus Data Portal for Success Ten Good Practices
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
Vitthal Shirke Java Microservices Resume.pdf
Software Engineering, Software Consulting, Tech Lead, Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Transaction, Spring MVC, OpenShift Cloud Platform, Kafka, REST, SOAP, LLD & HLD.
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
This session covers GraalVM & Spring Boot, Spring AOT, performance optimizations and using libraries
GraphSummit Paris - The art of the possible with Graph Technology
Sudhir Hasbe, Chief Product Officer, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Globus Compute Introduction - GlobusWorld 2024
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Graspan: A Big Data System for Big Code Analysis
We built a disk-based parallel graph system, Graspan, that uses a novel edge-pair centric computation model to compute dynamic transitive closures on very large program graphs.
We implement context-sensitive pointer/alias and dataflow analyses on Graspan. An evaluation of these analyses on large codebases such as Linux shows that their Graspan implementations scale to millions of lines of code and are much simpler than their original implementations.
These analyses were used to augment the existing checkers; these augmented checkers found 132 new NULL pointer bugs and 1308 unnecessary NULL tests in Linux 4.4.0-rc5, PostgreSQL 8.3.9, and Apache httpd 2.2.18.
- Accepted in ASPLOS ‘17, Xi’an, China.
- Featured in the tutorial, Systemized Program Analyses: A Big Data Perspective on Static Analysis Scalability, ASPLOS ‘17.
- Invited for presentation at SoCal PLS ‘16.
- Invited for poster presentation at PLDI SRC ‘16.
Large Language Models and the End of Programming
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
BoxLang: Review our Visionary Licenses of 2024
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
Globus Compute wth IRI Workflows - GlobusWorld 2024
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Understanding Globus Data Transfers with NetSage
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
Prosigns: Transforming Business with Tailored Technology Solutions
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
Text summarization of braking news using fine tuning BART model
Navigating the Metaverse: A Journey into Virtual Evolution"
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
Recently uploaded (20)
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
How to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good Practices
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)
GraphSummit Paris - The art of the possible with Graph Technology
GraphSummit Paris - The art of the possible with Graph Technology
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology Solutions
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
Text-Summarization-of-Breaking-News-Using-Fine-tuning-BART-Model.pptx
Navigating the Metaverse: A Journey into Virtual Evolution"
Navigating the Metaverse: A Journey into Virtual Evolution"
Docker for Integration Testing
- 1. Docker for Integration Testing Wouter Danes Docker Amsterdam / Utrecht, 28 August 2014
- 3. Agenda • Ways I use docker during development • Fitting docker into your build process • Docker maven integration
- 4. Ways I use docker during development
- 5. Shadow IT
- 6. Run ! “production” ! locally
- 8. CI environment • Spawn build slaves • No port collisions • Replace D/T with feature branch deploys • No version / stack collisions in “environment”
- 9. Fitting docker into your build process
- 10. High level build process build test package test package publish
- 11. Docker commands per build step build test package test package publish build run start stop link expose tag push
- 13. build test package test package publish build run start stop link expose tag push resources compile test package integration-test verify install deploy Maven vs Docker
- 14. docker-maven-plugin • Build docker images • Start containers • Link containers • Wait for containers to finish initialisation • Expose URLs / port info to integration tests • Clean up • Push images to an index / registry
- 15. Demo project
- 16. References • Two docker blog posts: www.wouterdanes.net • Docker maven plugin: github.com/wouterd/docker-maven-plugin • 12 principles for developing containers well: 12factor.net