This document discusses key considerations for architecting distributed cloud applications. It notes that cloud applications must be designed to embrace failure as individual instances will inevitably fail at some point. It recommends running multiple redundant instances of services behind load balancers to provide high availability. It also discusses using messaging and queues for communication between services to make the applications more resilient and scalable. The document provides guidance on various architectural patterns and best practices for building cloud-native applications.
Microservice With Spring Boot and Spring CloudEberhard Wolff
Spring Boot and Spring Cloud are an ideal foundation for creating Microservices based on Java. This presentation explains basic concepts of these libraries.
Today’s cutting edge companies have software release cycles measured in days instead of months. This agility is enabled by the DevOps practice of continuous integration and delivery, which automates building, testing, and deploying all code changes. This automation helps you catch bugs sooner and accelerates developer productivity. In this session, we’ll share the processes followed by Amazon engineers and discuss how you can bring them to your company by using a set of application lifecycle management tools from AWS: the newly announced AWS CodeBuild service, AWS CodePipeline, and AWS CodeDeploy.
Immutable Infrastructure: the new App DeploymentAxel Fontaine
Immutable Infrastructure: the new App Deployment
App deployment and server setup are complex, error-prone and time-consuming. They require OS installers, package managers, configuration recipes, install and deployment scripts, server tuning, hardening and more. But... Is this really necessary? Are we trapped in a mindset of doing things this way just because that's how they've always done?
What if we could start over and radically simplify all this? What if, within seconds, and with a single command, we could wrap our application into the bare minimal machine required to run it? What if this machine could then be transported and run unchanged on our laptop and in the cloud? How do the various platforms and tools like AWS, Docker, Heroku and Boxfuse fit into this picture? What are their strengths and weaknesses? When should you use them?
This talk is for developers and architects wishing to radically improve and simplify how they deploy their applications. It takes Continuous Delivery to a level far beyond what you've seen today. Welcome to Immutable Infrastructure generation. This is the new black.
Continuous Integration and Deployment Best Practices on AWSAmazon Web Services
With AWS, organizations now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100% API-driven enables organizations to use lean methodologies and realize these benefits. In this session, we will explore some key concepts and design patterns for continuous deployment and continuous integration, two elements of lean application and infrastructure development. We will look at several use cases where IT organizations leveraged AWS to rapidly develop and iterate on applications for scale, high availability and cost optimization.
Speaker: Adrian White, Solutions Architect, Amazon Web Services
Adopting Java for the Serverless world at AWS User Group PretoriaVadym Kazulkin
Java is for many years one of the most popular programming languages, but it used to have hard times in the Serverless Community. Java is known for its high cold start times and high memory footprint. For both you have to pay to the cloud providers of your choice. That's why most developers tried to avoid using Java for such use cases. But the times change: Community and cloud providers improve things steadily for Java developers. In this talk we look at the features and possibilities AWS cloud provider offers for the Java developers and look the most popular Java frameworks, like Micronaut, Quarkus and Spring (Boot) and look how (AOT compiler and GraalVM native images play a huge role) they address Serverless challenges and enable Java for broad usage in the Serverless world.
Microservice With Spring Boot and Spring CloudEberhard Wolff
Spring Boot and Spring Cloud are an ideal foundation for creating Microservices based on Java. This presentation explains basic concepts of these libraries.
Today’s cutting edge companies have software release cycles measured in days instead of months. This agility is enabled by the DevOps practice of continuous integration and delivery, which automates building, testing, and deploying all code changes. This automation helps you catch bugs sooner and accelerates developer productivity. In this session, we’ll share the processes followed by Amazon engineers and discuss how you can bring them to your company by using a set of application lifecycle management tools from AWS: the newly announced AWS CodeBuild service, AWS CodePipeline, and AWS CodeDeploy.
Immutable Infrastructure: the new App DeploymentAxel Fontaine
Immutable Infrastructure: the new App Deployment
App deployment and server setup are complex, error-prone and time-consuming. They require OS installers, package managers, configuration recipes, install and deployment scripts, server tuning, hardening and more. But... Is this really necessary? Are we trapped in a mindset of doing things this way just because that's how they've always done?
What if we could start over and radically simplify all this? What if, within seconds, and with a single command, we could wrap our application into the bare minimal machine required to run it? What if this machine could then be transported and run unchanged on our laptop and in the cloud? How do the various platforms and tools like AWS, Docker, Heroku and Boxfuse fit into this picture? What are their strengths and weaknesses? When should you use them?
This talk is for developers and architects wishing to radically improve and simplify how they deploy their applications. It takes Continuous Delivery to a level far beyond what you've seen today. Welcome to Immutable Infrastructure generation. This is the new black.
Continuous Integration and Deployment Best Practices on AWSAmazon Web Services
With AWS, organizations now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100% API-driven enables organizations to use lean methodologies and realize these benefits. In this session, we will explore some key concepts and design patterns for continuous deployment and continuous integration, two elements of lean application and infrastructure development. We will look at several use cases where IT organizations leveraged AWS to rapidly develop and iterate on applications for scale, high availability and cost optimization.
Speaker: Adrian White, Solutions Architect, Amazon Web Services
Adopting Java for the Serverless world at AWS User Group PretoriaVadym Kazulkin
Java is for many years one of the most popular programming languages, but it used to have hard times in the Serverless Community. Java is known for its high cold start times and high memory footprint. For both you have to pay to the cloud providers of your choice. That's why most developers tried to avoid using Java for such use cases. But the times change: Community and cloud providers improve things steadily for Java developers. In this talk we look at the features and possibilities AWS cloud provider offers for the Java developers and look the most popular Java frameworks, like Micronaut, Quarkus and Spring (Boot) and look how (AOT compiler and GraalVM native images play a huge role) they address Serverless challenges and enable Java for broad usage in the Serverless world.
Why does DevOps matter? How can you use continuous integration to build your product faster, make it more highly available, and be able to recover from bugs quickly? Let one of our solutions architects walk you through continuous integration and continuous delivery on AWS. This session includes live demos of our tools AWS CodeCommit, AWS CodePipeline, and AWS CodeDeploy.
Speaker: Leo Zhandovsky, Solutions Architect, Amazon Web services
recordings to the Canberra Summit can be found here
https://aws.amazon.com/events/anz/on-demand/canberra-summit/
Performance Testing using Real Browsers with JMeter & WebdriverBlazeMeter
Learn how to easily run performance tests with real browsers using Selenium WebDriver.
Ophir Prusak, BlazeMeter’s Chief Evangelist, gives step-by-step instructions on doing this using BlazeMeter and/or JMeter.
Learn how to:
- Correlate actual browser-based user experience with the load tests
- Run multiple Selenium Webdriver tests in parallel at scale by using the power of the cloud
- Do it all without any prior JMeter knowledge or experience!
We are now witnessing a new wave of IT revolution and its effect is very similar to the Cloud and Virtualization revolutions that started in the last decade. This new wave, called Containerization, is related to technologies such as Docker and Kubernetes, which now fuel large scale solutions including Big Data and IoT.
Learn about:
- Typical DevOps challenges and modern solutions
- Using Docker as Amazon EC2 Container Service Evolution of Enterprise Architecture (Containers, IoT, Machine Learning and technologies of tomorrow)
- Business value of using advances DevOps technologies with real-life case study
Developing Resilient Cloud Native Apps with Spring CloudDustin Ruehle
Distributed and massively scalable systems are difficult to design, implement, and operate. Further, microservice architectures are supposed to enable your business to be disruptive and innovative. Thankfully, two communities have emerged to facilitate easier solutions for these concerns and do a lot of the work for you: Spring Cloud OSS and Cloud Foundry. In this talk, we will take a deeper look at preventing cascading failures using Hystrix, as well as illustrate a mechanism for A/B testing using Eureka and blue-green deployments on Cloud Foundry.
Weaveworks at AWS re:Invent 2016: Operations Management with Amazon ECSWeaveworks
Alfonso described how Weave open source projects (Weave Net and Weave Scope) can help with networking, visualization, and control for ECS. Specifically, Weave acts as a key communicator for networking containers with its multi-host overlay and additional features (including automatic DNS service discovery and multicast).
FaaS or not to FaaS. Visible and invisible benefits of the Serverless paradig...Vadym Kazulkin
When we talk about prices, we often only talk about Lambda costs. In our applications, however, we rarely use only Lambda. Usually we have other building blocks like API Gateway, data sources like SNS, SQS or Kinesis. We also store our data either in S3 or in serverless databases like DynamoDB or recently in Aurora Serverless. All of these AWS services have their own pricing models to look out for. In this talk, we will draw a complete picture of the total cost of ownership in serverless applications and present a decision-making list for determining if and whether to rely on serverless paradigm in your project. In doing so, we look at the cost aspects as well as other aspects such as understanding application lifecycle, software architecture, platform limitations, organizational knowledge and plattform and tooling maturity. We will also discuss current challenges adopting serverless such as lack of high latency ephemeral storage, unsufficient network performance and missing security features.
Adopting Java for the Serverless world at Serverless Meetup SingaporeVadym Kazulkin
Java is for many years one of the most popular programming languages, but it used to have hard times in the Serverless Community. Java is known for its high cold start times and high memory footprint. For both you have to pay to the cloud providers of your choice. That's why most developers tried to avoid using Java for such use cases. But the times change: Community and cloud providers improve things steadily for Java developers. In this talk we look at the features and possibilities AWS cloud provider offers for the Java developers and look the most popular Java frameworks, like Micronaut, Quarkus and Spring (Boot) and look how (AOT compiler and GraalVM native images play a huge role) they address Serverless challenges and enable Java for broad usage in the Serverless world.
SPEAKER: Alisa Petivotova, Quality Architect @EPAM.
TOPIC DESCRIPTION:
We'll talk about the approach not only in terms of how to build contract tests, but when and why we should use them. This will also be a discussion on how to build testing process with close collaboration between development and testing teams.
WinOps Conf 2016 - Michael Greene - Release PipelinesWinOps Conf
There are benefits to be gained when patterns and practices from developer techniques are applied to operations. Notably, a fully automated solution where infrastructure is managed as code and all changes are automatically validated before reaching production. This is a process shift that is recognized among industry innovators. For organizations already leveraging these processes, it should be clear how to leverage Microsoft platforms. For organizations that are new to the topic, it should be clear how to bring this process to your environment and what it means to your organizational culture. This presentation explains the components of a Release Pipeline for configuration as code, the value to operations, and solutions that are used when designing a new Release Pipeline architecture.
Ed Seymour
Containerisation Lead – Red Hat
Ed has over 20 years experience working in software development and IT automation. With a career that started with a small software start-up, working efficiently and with agility was a necessity, and through his experience working at a global IT services company, gained valuable experience in promoting and effecting organisational change, adoption of agile methods, and automation of the software development life-cycle. At Red Hat, Ed’s role has focused on enabling customers as they embrace new organisational behaviours and structures, for example DevOps, and developing new IT services through adoption of emerging technologies, such as Cloud Management, OpenStack; Ed specialises in solutions based on containers through Docker, Kubernetes and OpenShift.
Serhiy Kalinets "Embracing architectural challenges in the modern .NET world"Fwdays
For more than decade .NET has been used primarily in enterprise software development. We all remember intranet deployment, IIS, SQL Server, N-tier applications and so on. The toolset (Visual Studio, SQL Management Studio, IIS Management snap-in etc) seemed to be set in stone as well as architecture (controllers, services, repositories). .NET people were isolated from other folks, who were using clusters, containers, clouds, and Linux.
However, adoption of clouds during few past years, the release of .NET Core made much more choices available to developers. It turned out that traditional way of building application is not that efficient from many viewpoints, including costs, time, performance or robustness. It happens because the environment has been changed and many assumptions are not still relevant.
In this talk, we will discuss what and why has been changed and how to deal with that. What are new requirements for our applications? What are new services available, and how to use them wisely? And finally, how should we design our applications to be cost-effective, competitive and have a lot of fun working with .NET Core.
Why does DevOps matter? How can you use continuous integration to build your product faster, make it more highly available, and be able to recover from bugs quickly? Let one of our solutions architects walk you through continuous integration and continuous delivery on AWS. This session includes live demos of our tools AWS CodeCommit, AWS CodePipeline, and AWS CodeDeploy.
Speaker: Leo Zhandovsky, Solutions Architect, Amazon Web services
recordings to the Canberra Summit can be found here
https://aws.amazon.com/events/anz/on-demand/canberra-summit/
Performance Testing using Real Browsers with JMeter & WebdriverBlazeMeter
Learn how to easily run performance tests with real browsers using Selenium WebDriver.
Ophir Prusak, BlazeMeter’s Chief Evangelist, gives step-by-step instructions on doing this using BlazeMeter and/or JMeter.
Learn how to:
- Correlate actual browser-based user experience with the load tests
- Run multiple Selenium Webdriver tests in parallel at scale by using the power of the cloud
- Do it all without any prior JMeter knowledge or experience!
We are now witnessing a new wave of IT revolution and its effect is very similar to the Cloud and Virtualization revolutions that started in the last decade. This new wave, called Containerization, is related to technologies such as Docker and Kubernetes, which now fuel large scale solutions including Big Data and IoT.
Learn about:
- Typical DevOps challenges and modern solutions
- Using Docker as Amazon EC2 Container Service Evolution of Enterprise Architecture (Containers, IoT, Machine Learning and technologies of tomorrow)
- Business value of using advances DevOps technologies with real-life case study
Developing Resilient Cloud Native Apps with Spring CloudDustin Ruehle
Distributed and massively scalable systems are difficult to design, implement, and operate. Further, microservice architectures are supposed to enable your business to be disruptive and innovative. Thankfully, two communities have emerged to facilitate easier solutions for these concerns and do a lot of the work for you: Spring Cloud OSS and Cloud Foundry. In this talk, we will take a deeper look at preventing cascading failures using Hystrix, as well as illustrate a mechanism for A/B testing using Eureka and blue-green deployments on Cloud Foundry.
Weaveworks at AWS re:Invent 2016: Operations Management with Amazon ECSWeaveworks
Alfonso described how Weave open source projects (Weave Net and Weave Scope) can help with networking, visualization, and control for ECS. Specifically, Weave acts as a key communicator for networking containers with its multi-host overlay and additional features (including automatic DNS service discovery and multicast).
FaaS or not to FaaS. Visible and invisible benefits of the Serverless paradig...Vadym Kazulkin
When we talk about prices, we often only talk about Lambda costs. In our applications, however, we rarely use only Lambda. Usually we have other building blocks like API Gateway, data sources like SNS, SQS or Kinesis. We also store our data either in S3 or in serverless databases like DynamoDB or recently in Aurora Serverless. All of these AWS services have their own pricing models to look out for. In this talk, we will draw a complete picture of the total cost of ownership in serverless applications and present a decision-making list for determining if and whether to rely on serverless paradigm in your project. In doing so, we look at the cost aspects as well as other aspects such as understanding application lifecycle, software architecture, platform limitations, organizational knowledge and plattform and tooling maturity. We will also discuss current challenges adopting serverless such as lack of high latency ephemeral storage, unsufficient network performance and missing security features.
Adopting Java for the Serverless world at Serverless Meetup SingaporeVadym Kazulkin
Java is for many years one of the most popular programming languages, but it used to have hard times in the Serverless Community. Java is known for its high cold start times and high memory footprint. For both you have to pay to the cloud providers of your choice. That's why most developers tried to avoid using Java for such use cases. But the times change: Community and cloud providers improve things steadily for Java developers. In this talk we look at the features and possibilities AWS cloud provider offers for the Java developers and look the most popular Java frameworks, like Micronaut, Quarkus and Spring (Boot) and look how (AOT compiler and GraalVM native images play a huge role) they address Serverless challenges and enable Java for broad usage in the Serverless world.
SPEAKER: Alisa Petivotova, Quality Architect @EPAM.
TOPIC DESCRIPTION:
We'll talk about the approach not only in terms of how to build contract tests, but when and why we should use them. This will also be a discussion on how to build testing process with close collaboration between development and testing teams.
WinOps Conf 2016 - Michael Greene - Release PipelinesWinOps Conf
There are benefits to be gained when patterns and practices from developer techniques are applied to operations. Notably, a fully automated solution where infrastructure is managed as code and all changes are automatically validated before reaching production. This is a process shift that is recognized among industry innovators. For organizations already leveraging these processes, it should be clear how to leverage Microsoft platforms. For organizations that are new to the topic, it should be clear how to bring this process to your environment and what it means to your organizational culture. This presentation explains the components of a Release Pipeline for configuration as code, the value to operations, and solutions that are used when designing a new Release Pipeline architecture.
Ed Seymour
Containerisation Lead – Red Hat
Ed has over 20 years experience working in software development and IT automation. With a career that started with a small software start-up, working efficiently and with agility was a necessity, and through his experience working at a global IT services company, gained valuable experience in promoting and effecting organisational change, adoption of agile methods, and automation of the software development life-cycle. At Red Hat, Ed’s role has focused on enabling customers as they embrace new organisational behaviours and structures, for example DevOps, and developing new IT services through adoption of emerging technologies, such as Cloud Management, OpenStack; Ed specialises in solutions based on containers through Docker, Kubernetes and OpenShift.
Serhiy Kalinets "Embracing architectural challenges in the modern .NET world"Fwdays
For more than decade .NET has been used primarily in enterprise software development. We all remember intranet deployment, IIS, SQL Server, N-tier applications and so on. The toolset (Visual Studio, SQL Management Studio, IIS Management snap-in etc) seemed to be set in stone as well as architecture (controllers, services, repositories). .NET people were isolated from other folks, who were using clusters, containers, clouds, and Linux.
However, adoption of clouds during few past years, the release of .NET Core made much more choices available to developers. It turned out that traditional way of building application is not that efficient from many viewpoints, including costs, time, performance or robustness. It happens because the environment has been changed and many assumptions are not still relevant.
In this talk, we will discuss what and why has been changed and how to deal with that. What are new requirements for our applications? What are new services available, and how to use them wisely? And finally, how should we design our applications to be cost-effective, competitive and have a lot of fun working with .NET Core.
What aspects must a developer be aware of when a Web Services will be run in clustered environment such as a server farm?
Do Web Services implementations need to be \"cluster aware\", or can this be handled transparently by the runtime platform?
We revisit the subject of why keeping Web Services implementations as stateless as possible really helps in these circumstances, and the effect of using session-based facilities on scalability.
Have ZENworks with current maintenance?
If the answer is yes, you may be entitled to two free licenses for Novell Service Desk.
See the slide presentation to learn more about Novell Service Desk capabilities.
Overview of web services and web service architectures.
Web services have come of age and are the foundation of today's enterprise application architectures.
Service Oriented Architecture (SOA) break up traditional application silos into reusable services shared and used by different applications.
Web services group and encapsulate related functionality into reusable functional units.
Web service intermediaries complement the business functionality exposed by web services with
functions such as authentication, load balancing, logging and caching.
To control web service consumer and producer compatibility, web services should carry the version of the service in the interface.
Ever sit in a meeting and hear “I can't get my job done because ‘Chris’ is not done with the code I need to develop my code”? This is the content from an education session enabling attendees to learn about Service Virtualization and find out which problems it solves and how it differs from developers stubbing and mocking. It describes the differences between virtualizing hardware and hypervisors vs. simulating business behavior and services. Presentation by Chris Kraus
Service Fabric is a distributed systems platform that makes it easy to package, deploy, and manage scalable and reliable microservices. Service Fabric also addresses the significant challenges in developing and managing cloud applications. Developers and administrators can avoid solving complex infrastructure problems and focus instead on implementing mission-critical, demanding workloads knowing that they are scalable, reliable, and manageable. Service Fabric represents the next-generation middleware platform for building and managing these enterprise-class, Tier-1 cloud-scale applications
Fundamental and Practice.
Explain about microservices characters and pattern. And also how to be good build microservices. And also additional the scale cube and CAP theory.
Arsitektur Aplikasi Modern - Faisal Henry SusantoDicodingEvent
Baparekraf Developer Day adalah kegiatan yang diadakan oleh Kementerian Pariwisata dan Ekonomi Kreatif/Badan Pariwisata dan Ekonomi Kreatif (Kemenparekraf/Baparekraf) dengan tujuan mengasah kemampuan teknis pengembang aplikasi di Indonesia. Kegiatan ini memungkinkan transfer pengetahuan dan standar industri secara langsung dari para praktisi yang telah sukses, khususnya pada bidang pengembangan aplikasi.
Back-End Session
Tema: Arsitektur Aplikasi Modern
Speaker: Faisal Henry Susanto (Praktisi IT)
Migration to cloud is no easy task. Start small and learn the core technologies before leveraging the advanced features of the cloud. The cultural change will affect the whole organization from development to business management and sales.
Cloud native applications are the future of software. Modern software is stateless, provided from cloud to heterogeneous clients on demand and designed to be scalable and resilient.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?XfilesPro
Worried about document security while sharing them in Salesforce? Fret no more! Here are the top-notch security standards XfilesPro upholds to ensure strong security for your Salesforce documents while sharing with internal or external people.
To learn more, read the blog: https://www.xfilespro.com/how-does-xfilespro-make-document-sharing-secure-and-seamless-in-salesforce/
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
Accelerate Enterprise Software Engineering with PlatformlessWSO2
Key takeaways:
Challenges of building platforms and the benefits of platformless.
Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience.
How Choreo enables the platformless experience.
How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo.
Demo of an end-to-end app built and deployed on Choreo.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Strategies for Successful Data Migration Tools.pptxvarshanayak241
Data migration is a complex but essential task for organizations aiming to modernize their IT infrastructure and leverage new technologies. By understanding common challenges and implementing these strategies, businesses can achieve a successful migration with minimal disruption. Data Migration Tool like Ask On Data play a pivotal role in this journey, offering features that streamline the process, ensure data integrity, and maintain security. With the right approach and tools, organizations can turn the challenge of data migration into an opportunity for growth and innovation.
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...informapgpstrackings
Keep tabs on your field staff effortlessly with Informap Technology Centre LLC. Real-time tracking, task assignment, and smart features for efficient management. Request a live demo today!
For more details, visit us : https://informapuae.com/field-staff-tracking/
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Hivelance Technology
Cryptocurrency trading bots are computer programs designed to automate buying, selling, and managing cryptocurrency transactions. These bots utilize advanced algorithms and machine learning techniques to analyze market data, identify trading opportunities, and execute trades on behalf of their users. By automating the decision-making process, crypto trading bots can react to market changes faster than human traders
Hivelance, a leading provider of cryptocurrency trading bot development services, stands out as the premier choice for crypto traders and developers. Hivelance boasts a team of seasoned cryptocurrency experts and software engineers who deeply understand the crypto market and the latest trends in automated trading, Hivelance leverages the latest technologies and tools in the industry, including advanced AI and machine learning algorithms, to create highly efficient and adaptable crypto trading bots
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
3. We must do things differently when building
cost-effective, failure-resilient solutions
Why cloud apps?
Past Present
Clients Enterprise/Intranet Public/Internet
Demand Stable (small) Dynamic (small massive)
Datacenter Single tenant Multi-tenant
Operations People (expensive) Automation (cheap)
Scale Up via few reliable (expensive) PCs Out via lots of (cheap) commodity PCs
Failure Unlikely but possible Very likely
Machine loss Catastrophic Normal (no big deal)
Examples Past Present
Exceptions Catch, swallow & keep running Crash & restart
Communication In order
Exactly once
Out of order
Clients must retry & servers must be idempotent
4. Some reasons why a service instance may fail (stop)
Developer: Unhandled exception
DevOps: Scaling the number of service instances down
DevOps: Updating service code to a new version
Orchestrator: Moving service code from one machine to another
Force majeure: Hardware failure (power supply, fans [overheating], hard disk,
network controller, router, bad network cable, etc.)
Force majeure: Data center outages (natural disasters, attacks)
Since failure is inevitable & unavoidable, embrace it
Architect assuming failures will happen
Operate services using infrastructure that avoids single points of failure
Run multiple instances of services, replicate data, etc.
Cloud computing is all about embracing failure
6. E-Commerce Application
Load
Balancer
Applications consist of many (micro)services
Inventory #1
Inventory #2
Orders #1
Orders #2
Orders #3
Orders #4
Web Site #1
Web Site #2
Web Site #3
Each service solves a domain-
specific problem & has exclusive
access to its own data store
7. Thumbnail
Service
Thumbnail
ServicePhoto Share
Service
Photo Share
Service
Photo Share
Service
4 reasons to split a monolith into microservices
Photo Share
Service
Thumbnail
Service
Photo Share Service
Thumbnail
SharedLib-v7
Photo Share
Service
SharedLib-v1
Photo Share
Service
node.js
Thumbnail
Service
.NET
Photo Share
Service (V1) Thumbnail
Service
V1
Thumbnail
Service
SharedLib-v7
Thumbnail
Service
V2
SharedLib-v1
Video Share
Service (V1)
Backward compatibility
must be maintained
8. Myth: Microservices offer small,
easy-to-understand/manage code bases
A monolith can use OOP & libraries (requires developer discipline)
Library changes cause build failures (not runtime failures)
Myth: A failing service doesn’t impact other services
Many services require dependencies be fully functioning
Hard to write/test code that gracefully recovers when dependency fails
We run multiple service instances so there is no such thing as “failure”
A monolith is up/down completely; no recovery code
Infrastructure restarts failed instances keeping them up
Microservice architecture benefits myths
9. Composing SLAs for dependent services
Service-A Service-B Service-C Service-D
99.990% (264s/month)
99.998% ( 52s/month)
99.985% (396s/month)
99.997% ( 78s/month)
99.980% (528s/month)
99.996% (104s/month)
99.995% (132s/month)
99.999% ( 26s/month)
What about the network’s SLA?
11. 1. Single root repo; don’t share code with another service
2. Deploy dependent libs with service
3. No config in code; read from environment vars
4. Handle unresponsive service dependencies robustly
5. Strictly separate build, release, & run steps
Build: Builds a version of the code repo & gathers dependencies
Release: Combines build with config ReleaseId (immutable)
Run: Runs service in execution environment
12-factor services (1-5)
12. 6. Service is 1+ stateless processes & shares nothing
7. Service listens on ports; avoid using (web) hosts
8. Use processes for isolation; multiple for concurrency
9. Processes can crash/be killed quickly & start fast
10. Keep dev, staging, & prod environments similar
11. Log to stdout (dev=console; prod=file & archive it)
12. Deploy & run admin tasks (scripts) as processes
12-factor services (6-12)
14. 8 fallacies of distributed computing
http://www.rgoarchitects.com/Files/fallacies.pdf
Fallacy Effect
The network is reliable App needs error handling/retry
Latency is zero App must restrict its traffic
Bandwidth is infinite App must restrict its traffic
The network is secure App must secure its data/authenticate servers
Topology doesn't change Changes affect latency & bandwidth
There is one administrator Changes affect ability to reach destination
Transport cost is zero Costs must be budgeted
The network is homogeneous Affects reliability, latency, & bandwidth
15. We run multiple instances of a service
For service failure/recovery & scale up/down
So, instances’ endpoints dynamically change over the service’s lifetime
Ideally, we’d like to abstract this from client code
Each client wants a single stable endpoint as the face of the
dynamically-changing service instance endpoints
Typically, this is accomplished via a reverse proxy
NOTE: Every request goes through the RP; causes an extra network hop
We’re losing some performance to gain a lot of benefits
Client uses DNS (at well-known static endpoint) to get RP’s stable endpoint
DNS endpoints are usually cached & re-resolved infrequently
Service high-availability & scalability
17. Cluster DNS & service reverse proxy
Load
Balancer
Web Site #1
Web Site #2
Web Site #3
Inventory #1
Inventory #3
Inventory #2
Orders #1
Orders #2
⚠ WS #1 could fail
before I #3 replies
⚠
18. Comparing an in-process call to a network request
Performance: Worse, increases network congestion, unpredictable
Unreliable: Requires retry loops with exponential backup/circuit breakers
Server code must be idempotent
Security: Requires authentication, authorization, & encryption
Diagnostics: network issues, perf counters/events/logs, causality/call stacks
Turning a monolith into a microservice
IntelliSense, refactoring & compile-time type-safety)
19. Thumbnail
Service
Thumbnail
ServicePhoto Share
Service
Photo Share
Service
Photo Share
Service
4 reasons to split a monolith into microservices
Photo Share
Service
Thumbnail
Service
Photo Share Service
Thumbnail
SharedLib-v7
Photo Share
Service
SharedLib-v1
Photo Share
Service
node.js
Thumbnail
Service
.NET
Photo Share
Service (V1) Thumbnail
Service
V1
Thumbnail
Service
SharedLib-v7
Thumbnail
Service
V2
SharedLib-v1
Video Share
Service (V1)
Backward compatibility
must be maintained
20. Define explicit, formal cross-language API/data contracts
“Contracts” defined via code do not work; do not do this
Ex: DateTime can be null in Java but not in .NET
Use cross-language data transfer formats
Ex: JSON/XML, Avro, Protocol Buffers, FlatBuffers, Thrift, Bond, etc.
Consider embedding a version number in the data structure
Optional: (De)serialize data into language-specific types
Beware of RAM/CPU costs with this; keep types “disposable” (not contracts)
Defining network API contracts
21. Technologies try to map method call network request
Examples: RPC, RMI, CORBA, DCOM, WCF, etc.
These frequently don’t work well due to
Network fallacies (lack of retry/circuit breaker)
Language-specific data type conversions (ex: dates, times, durations)
Versioning: Which version to call on the server?
Authentication: expiring tokens
Logging: Log request parameters/headers/payload, reply headers/payload?
Beware leaky RPC-like abstractions
23. The request/reply pattern is frequently not the best
Client sends to server but selected server may be busy; other server may be idle
Client may crash/scale down/reconfigure while waiting for server’s reply
So, consider messaging communication instead
Resource efficient
Client doesn’t wait for server reply (no blocked threads/long-lived locks)
Idle consumers pull work vs busy consumer pushed more work
Consumers don’t need listening endpoints; producers talk to queue service
Resilient: Producer/consumer instances can come, go, and move at will
If consumer fails, another consumer processes the message (1+ delivery, not ordered)
Consumers/producers can be offline without message loss
Elastic: Use queue length to determine need to scale up/down
Messaging communication
24. Messaging with queues
Load
Balancer
WebSite #1
WebSite #2
WebSite #3
Service-A
#1
Service-A
#3
Service-A
#2
Service-B
#1
Service-B
#2
🛈 Request/reply isn’t required; Service-B #1
could post to Q-WS1; not to Q-A
🛈 All Service-A instances could
go down; but not WebSite #1
26. Building reliable & scalable services that manage state is
substantially harder than building stateless services
Due to data size/speed, partitioning, replication, consistency, disaster recovery,
backup/restore, costs, administration, security, etc.
Because of this, most devs do not build their own stateful
services; they use a robust/hardened service instead
When selecting a stateful service, you must fully understand your service’s
requirements and understand the trade-offs when comparing available services
It is common to use multiple stateful services within a single solution
Stateful service considerations
27. The most frequently-used stateful service
Used for documents, images, audio, video, etc.
Fast & inexpensive: GB/month storage, I/O requests, and egress bytes
All cloud providers offer a file storage service
No lock-in: It’s relatively easy to move files across providers if you avoid
provider-specific features
File storage services offer public (read-only) access
Send clients file URLs for them to access; reduces load on your other services!
Use a Content Delivery Network (CDN) to improve performance even more
Files (blobs & objects) storage services
28. Store many small related entities
Common: query, joins, indexing, sorting, stored proc, viewers/editors, etc.
As data increases, relational DBs (SQL) require expensive
hardware to address size & performance
ACID goal: give impression that 1 thing at a time is happening no matter how
complex the work (looks like a single PC)
NonRel-DBs (noSQL) spread data across many cheap PCs
For customer preferences, shopping carts, product catalogs, session state, etc.
Con: Can’t easily access all data (no sort/join); many are eventually consistency
Pro: Cheaper & have flexible data models (entity ≈ in-memory object)
Rel-DBs & NonRel-DBs will co-exist for years to come
DB storage services
29. Non-Relational
Database
Relational DB vs non-relational DB:
speed, size, simplicity, & price
Service #1
Relational
Database
(1 partition)
Service #2
Service #3
Service #4
Service #5
Service #1
Service #2
Service #3
Service #4
Service #5
Partition #1
Partition #2
Partition #3
Simple CRUD
Joins, sorts,
etc.
Complex CRUD,
joins, sorts,
stored procs,
X-table txns
30. Data is partitioned for size, speed, or both
Architecting a service’s partitions is often the hardest part of designing a service
X-partition ops require network hops & different/distributed transactions
How many partitions depends on how much data you’ll have in the future
And how you intend to access that data
Each partition’s data is replicated for reliability
Replicating state increases chance of data surviving 1+ simultaneous failures
But, more replicas increase cost & network latency to sync replicas
For some scenarios, data loss is OK
Replicas go across fault/update domains; avoids single point of failure
Data partitioning & replicas
31. CAP theorem states
When facing a network Partition (replicas can’t talk to each other):
You can maintain Consistency by not allowing writes (loss of availability)
You can maintain Availability by not replicating data (loss of consistency)
Strong: all replicas see same data at same time
Done via distributed transactions/locks across replicas communication
Weak: replicas see different data at a moment in time but
eventually see the same data
There are many factors pushing us towards weak consistency
Txs rarely work across DBs & each microservice selects its own DB
Caches improve perf by copying data which is out of sync with the truth
CQRS pattern: writes data asynchronously but reads data synchronously
Data consistency
32. Load
Balancer
A cache can improve performance
but introduces stale (inconsistent) data
Stateful
Data
Other
Internal
Tiers
?
Stateless
Compute
Cache
Stateless
Web
33. Concurrency control
Pessimistic: accessor locks 1+ entries (blocking other accessors), modifies entries,
& then unlock them (unblocking another accessor)
Bad scalability (1 accessor at a time) & what if locker fails to release the lock?
Optimistic: accessor gets 1+ entries/version IDs, modifies entries if IDs haven’t
changed (contains the read value)
Data schema versioning (without downtime)
Backup & Restore Needed due to app bug/hacker
Recovery Point Objective(RPO): Max data (minutes) business can afford to lose
Recovery Time Objective(RTO): Max downtime business can afford to restore data
NOTE: smaller RPO/RTO increases costs
Other DB concerns
Motivation
Embracing failure
When to split a monolith into Microservices and when not to
Containers
Networking
Messaging
Versioning & upgrades
Managing state
Jeffrey Richter: Software Architect, Microsoft Azure
Jeffrey Richter is a Software Engineer on Microsoft’s Azure team. He is also a co-founder of Wintellect, a software consulting and training company. He has authored many videos available on WintellectNOW, has spoken at many industry conferences, and is the author of several best-selling Windows and .NET Framework programming books including Windows Runtime via C#, CLR via C#, 4th Edition, and Windows via C/C++, 5th Edition. Jeffrey has also been a contributing editor to MSDN Magazine where he authored many feature articles and columns.
http://www.morganclaypool.com/doi/abs/10.2200/S00516ED2V01Y201306CAC024
http://channel9.msdn.com/Shows/ARCast.TV/ARCastTV-Pat-Helland-on-Memories-Guesses-and-Apologies
Server/Service v/s Cloud
Enterprise intranet services, stable demand, security is within intranet
Cloud :: scale – plan for phenomenal growth, scale-up & scale-out – availability is business.
Scale-up :: expensive, utilization can be low if the usage pattern varies, geo-aspect (replication). # of instances is lower.
Scale-out :: economical, cost-effective, larger replication presence.
http://xiard.wordpress.com/2008/11/06/pdc-2008-designing-for-scale-out/
http://channel9.msdn.com/pdc2008/BB54/
http://mschnlnine.vo.llnwd.net/d1/pdc08/PPTX/BB54.pptx
Consistency Levels
Strong: Changes visible now via synchronization
Eventual: Changes occur in the future (address change for mail)
Optimistic: Changes occur MAYBE in the future (stock ticker)
Message Assurance
Exactly once: no loss, no dups
At least once: no loss, duplicates
At most once: loss, no duplicates
Best effort: loss & duplicates
Level 4:
Hash-based traffic distribution (5-tuple: Src IP/Port to Dst IP/Port, protocol)
TCP/UDP support
Port forwarding
Idle timeout adjustment
Client IP affinity (3-tuple: Src IP to Dst IP, protocol); all requests from a client go to same server
TCP & HTTP health monitoring
NAT & SNAT
Level 7:
Cookie persistence
SSL/TLS Offload
HTTPS monitoring
URL or HTTP path LB
WAF rules
PaaS Scale out
Beware: a new service instance could be assigned a previous instance’s endpoint
This requires certificates or some ID/uniqueness so client knows whichservice it’s communicating with
MS OneAPI document? https://github.com/Microsoft/api-guidelines/blob/master/Guidelines.md
JMR MOVE: Method that could return an unbounded collection, must implement paging & may offer filtering/sorting
Use simple data types and shallow object graphs
JSON: null, true/false, number, string, array (ordered set of values),object (unordered set of name/value pairs)
For richer values (guid, date, time, duration), use string & clearly document format
Encourages scalable, resilient, versioning patterns (ex: CQRS & Event Sourcing)
JMR: Election of a single role instance to perform a task