This document discusses API gateways as a solution for challenges that arise in microservices architectures. It describes how a monolithic architecture can become complex as services grow quickly. In a microservices architecture, clients could communicate directly with each service but this introduces problems around endpoint management, multiple requests, and refactoring difficulties. An API gateway provides a single entry point, routes requests to appropriate services, and aggregates results to address these issues. It then demonstrates Netflix Zuul, an open source API gateway, and provides a demo of its use with Eureka service discovery and routing between hello and goodbye microservices.
Irfan Baqui, Senior Engineer at LunchBadger, breaks down the important role of the API Gateway in Microservices. Additionally, Irfan covers how to get started with Express Gateway, an open source API Gateway built entirely on Express.js. Originally presented at the San Francisco Node Meetup.
API Gateways are the well suited service for microservices architecture. It provides many security and performance related features along with reliability of the system. These slides explains what is API Gateway. What is microservices architecture, its benefits and how API Gateway empower this architecture. Further more API Gateway aggregation is explained with an example.
A brief overview of the significance of API Gateways in microservices architecture by providing Kong as an example.
Slide 2: Monolith Vs Microservices
Monolith:
Pros-
Simple to implement
Less integration test - easy to test
Easy to ship
Fast development
Cons-
Violates Open-Close principle
Nightmare when it comes to managing the code
Difficult to enhance
Bigger artifacts
Hard to replace individual components like DB, Logger etc.
Microservices-
Pros-
Easy to manage
One reason to change
Dynamic scaling
Single responsibility
Cons-
Multiple points of failure
Hard to test - rich integration tests required
Heterogeneity in infrastructure
Slide 3: API Gateway Pattern
It is microservices design pattern.
An API gateway is a service which is the entry point into the application from the outside world. It’s responsible for request routing, API composition, and other functions, such as authentication.
There are a lot of issues when client is talking to multiple components to get the job done. These include multiple proxies at client side, different logic to handle different calls, client needs to know the implementation details of server.
A much better approach is for a client to make a single request to what’s known as an API gateway. An API gateway is a service which is the single entry-point for API requests into an application. It’s similar to the Facade pattern from object-oriented design. Like a facade, an API gateway encapsulates the application’s internal architecture and provides an API to its clients. It might also have other responsibilities, such as authentication, monitoring, and rate limiting.
These are also termed as BFF - Backend For Frontend
Slide 4: API Gateway in Action
It acts as a “backend for the frontend”. The clients do not know which services they are talking to. They communicate with a single interface - API Gateway. The gateway resolves the client requests and distributes them to respective services.
Slide 7: Kong Architecture
Kong is a cloud-native, fast, scalable, and distributed Microservice Abstraction Layer (also known as an API Gateway, API Middleware or in some cases Service Mesh). Made available as an open-source project in 2015, its core values are high performance and extensibility.
Actively maintained, Kong is widely used in production at companies ranging from startups to Global 5000 as well as government organizations.
Looking for a way to build beautiful, cloud-native web and mobile apps? Seeking to eliminate the undifferentiated heavy lifting of a traditional, three-tier web application? AWS Amplify makes it a snap with an easy-to-use library, a powerful toolchain, a user-interface component library, and built-in capabilities such as authentication and analytics. Uncover how to bring your idea to life, scale it to the max, and delight your users in less time than before.
Irfan Baqui, Senior Engineer at LunchBadger, breaks down the important role of the API Gateway in Microservices. Additionally, Irfan covers how to get started with Express Gateway, an open source API Gateway built entirely on Express.js. Originally presented at the San Francisco Node Meetup.
API Gateways are the well suited service for microservices architecture. It provides many security and performance related features along with reliability of the system. These slides explains what is API Gateway. What is microservices architecture, its benefits and how API Gateway empower this architecture. Further more API Gateway aggregation is explained with an example.
A brief overview of the significance of API Gateways in microservices architecture by providing Kong as an example.
Slide 2: Monolith Vs Microservices
Monolith:
Pros-
Simple to implement
Less integration test - easy to test
Easy to ship
Fast development
Cons-
Violates Open-Close principle
Nightmare when it comes to managing the code
Difficult to enhance
Bigger artifacts
Hard to replace individual components like DB, Logger etc.
Microservices-
Pros-
Easy to manage
One reason to change
Dynamic scaling
Single responsibility
Cons-
Multiple points of failure
Hard to test - rich integration tests required
Heterogeneity in infrastructure
Slide 3: API Gateway Pattern
It is microservices design pattern.
An API gateway is a service which is the entry point into the application from the outside world. It’s responsible for request routing, API composition, and other functions, such as authentication.
There are a lot of issues when client is talking to multiple components to get the job done. These include multiple proxies at client side, different logic to handle different calls, client needs to know the implementation details of server.
A much better approach is for a client to make a single request to what’s known as an API gateway. An API gateway is a service which is the single entry-point for API requests into an application. It’s similar to the Facade pattern from object-oriented design. Like a facade, an API gateway encapsulates the application’s internal architecture and provides an API to its clients. It might also have other responsibilities, such as authentication, monitoring, and rate limiting.
These are also termed as BFF - Backend For Frontend
Slide 4: API Gateway in Action
It acts as a “backend for the frontend”. The clients do not know which services they are talking to. They communicate with a single interface - API Gateway. The gateway resolves the client requests and distributes them to respective services.
Slide 7: Kong Architecture
Kong is a cloud-native, fast, scalable, and distributed Microservice Abstraction Layer (also known as an API Gateway, API Middleware or in some cases Service Mesh). Made available as an open-source project in 2015, its core values are high performance and extensibility.
Actively maintained, Kong is widely used in production at companies ranging from startups to Global 5000 as well as government organizations.
Looking for a way to build beautiful, cloud-native web and mobile apps? Seeking to eliminate the undifferentiated heavy lifting of a traditional, three-tier web application? AWS Amplify makes it a snap with an easy-to-use library, a powerful toolchain, a user-interface component library, and built-in capabilities such as authentication and analytics. Uncover how to bring your idea to life, scale it to the max, and delight your users in less time than before.
Docker containers have become a key component of modern application design. Increasingly, developers are breaking their applications apart into smaller components and distributing them across a pool of compute resources.
Do you want to run your code without the cost and effort of provisioning and managing servers? Find out how in this deep dive session on AWS Lambda, which allows you to run code for virtually any type of application or back end service – all with zero administration. During the session, we’ll look at a number of key AWS Lambda features and benefits, including automated application scaling with high availability; pay-as-you-consume billing; and the ability to automatically trigger your code from other AWS services or from any web or mobile app.
Continuous Deployment Practices, with Production, Test and Development Enviro...Amazon Web Services
With AWS companies now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100% API driven enables businesses to use lean methodologies and realize these benefits. This in turn leads to greater success for those who make use of these practices. In this session we'll talk about some key concepts and design patterns for Continuous Deployment and Continuous Integration, two elements of lean development of applications and infrastructures.
Threat protection and application access controls are key security mechanisms that protect APIs when exposed to internal or external users and developers.
In this technical deep-dive webcast, Apigee's security team, led by Subra Kumaraswamy, will discuss API threats and the protection mechanisms that every API and app developer must implement for safe and secure API management.
This webcast will cover:
- the API threat model
- how to design and implement appropriate guardrails for API security using build-in policies and configuration
- a demo of Apigee Edge threat protection features, including TLS encryption, XML/JSON/SQL injection attacks, and rate limiting
Whether you're an IT security architect or an API or app developer, this webcast will help you understand secure API management.
Download Podcast: http://bit.ly/1biiJQS
Watch Video: http://youtu.be/ffs35w1RYRI
How to migrate an application in IBM APIc, and preserve its client credentialShiu-Fun Poon
This provides the rest and toolkit command on how to migrate an application from one environment to another without know the client_secret in the plaintext format.
by Omar Lari, Partner Solutions Architect, AWS
Amazon Elastic Container Service for Kubernetes (Amazon EKS) is a new managed service for running Kubernetes on AWS. This session will provide an overview of Amazon EKS, why we built it, and how it works.
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. With a few clicks in the AWS Management Console, you can create an API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services, such as workloads running on Amazon Elastic Compute Cloud (Amazon EC2), code running on AWS Lambda, or any Web application. Amazon API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, authorization and access control, monitoring, and API version management.
Presented by: Danilo Poccia, Technical Evangelist, Amazon Web Services
Auto Scaling helps you ensure that you have the correct number of Amazon EC2 instances available to handle the load for your application. You create collections of EC2 instances, called Auto Scaling groups.
You can specify the minimum number of instances in each Auto Scaling group, and Auto Scaling ensures that your group never goes below this size.
You can specify the maximum number of instances in each Auto Scaling group, and Auto Scaling ensures that your group never goes above this size.
If you specify the desired capacity, either when you create the group or at any time thereafter, Auto Scaling ensures that your group has this many instances.
If you specify scaling policies, then Auto Scaling can launch or terminate instances as demand on your application increases or decreases
Designing security & governance via AWS Control Tower & Organizations - SEC30...Amazon Web Services
Whether it is per business unit or per application, many AWS customers use multiple accounts to meet their infrastructure isolation, separation of duties, and billing requirements. In this session, we cover considerations, limitations, and security patterns when building a multi-account strategy. We explore topics such as thought pattern, identity federation, cross-account roles, consolidated logging, and account governance. We conclude by presenting an enterprise-ready landing-zone framework and providing the background needed to implement an AWS Landing Zone using AWS Control Tower and AWS Organizations.
While many organizations have started to automate their software development processes, many still engineer their infrastructure largely by hand. Treating your infrastructure just like any other piece of code creates a “programmable infrastructure” that allows you to take full advantage of the scalability and reliability of the AWS cloud. This session will walk through practical examples of how AWS customers have merged infrastructure configuration with application code to create application-specific infrastructure and a truly unified development lifecycle. You will learn how AWS customers have leveraged tools like CloudFormation, orchestration engines, and source control systems to enable their applications to take full advantage of the scalability and reliability of the AWS cloud, create self-reliant applications, and easily recover when things go seriously wrong with their infrastructure.
Building Cloud-Native App Series - Part 2 of 11
Microservices Architecture Series
Event Sourcing & CQRS,
Kafka, Rabbit MQ
Case Studies (E-Commerce App, Movie Streaming, Ticket Booking, Restaurant, Hospital Management)
The presentation from our online webinar "Design patterns for microservice architecture".
Full video from webinar available here: https://www.youtube.com/watch?v=826aAmG06KM
If you’re a CTO or a Lead Developer and you’re planning to design service-oriented architecture, it’s definitely a webinar tailored to your needs. Adrian Zmenda, our Lead Dev, will explain:
- when microservice architecture is a safe bet and what are some good alternatives
- what are the pros and cons of the most popular design patterns (API Gateway, Backend for Frontend and more)
- how to ensure that the communication between services is done right and what to do in case of connection issues
- why we’ve decided to use a monorepo (monolithic repository)
- what we’ve learned from using the remote procedure call framework gRPC
- how to monitor the efficiency of individual services and whole SOA-based systems.
Wipro & Experitest Co webinar: Enhance your Mobile App with Load Testing and Increased Productivity.
Wipro and Experitest on a live webinar on improving the performance of your mobile application while expanding productivity. The solutions we present will allow you to test faster, and more diligently than ever, combining load, performance, and functionality all at once.
The session covers:
- A case study involving Wipro and SeeTest working with a major American Bank
- Live demo to show SeeTest Automation integrate into HP LoadRunner
- How Wipro developed a mainframe component to perform backend validation
- Live demo of Wipro's SeeTest Framework, and how it improves productivity by 50%
Speakers:
Guy Arieli, CTO, Experitest
Sudheer Mohan, Director - Mobility Certification & Automation, Wipro
Docker containers have become a key component of modern application design. Increasingly, developers are breaking their applications apart into smaller components and distributing them across a pool of compute resources.
Do you want to run your code without the cost and effort of provisioning and managing servers? Find out how in this deep dive session on AWS Lambda, which allows you to run code for virtually any type of application or back end service – all with zero administration. During the session, we’ll look at a number of key AWS Lambda features and benefits, including automated application scaling with high availability; pay-as-you-consume billing; and the ability to automatically trigger your code from other AWS services or from any web or mobile app.
Continuous Deployment Practices, with Production, Test and Development Enviro...Amazon Web Services
With AWS companies now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100% API driven enables businesses to use lean methodologies and realize these benefits. This in turn leads to greater success for those who make use of these practices. In this session we'll talk about some key concepts and design patterns for Continuous Deployment and Continuous Integration, two elements of lean development of applications and infrastructures.
Threat protection and application access controls are key security mechanisms that protect APIs when exposed to internal or external users and developers.
In this technical deep-dive webcast, Apigee's security team, led by Subra Kumaraswamy, will discuss API threats and the protection mechanisms that every API and app developer must implement for safe and secure API management.
This webcast will cover:
- the API threat model
- how to design and implement appropriate guardrails for API security using build-in policies and configuration
- a demo of Apigee Edge threat protection features, including TLS encryption, XML/JSON/SQL injection attacks, and rate limiting
Whether you're an IT security architect or an API or app developer, this webcast will help you understand secure API management.
Download Podcast: http://bit.ly/1biiJQS
Watch Video: http://youtu.be/ffs35w1RYRI
How to migrate an application in IBM APIc, and preserve its client credentialShiu-Fun Poon
This provides the rest and toolkit command on how to migrate an application from one environment to another without know the client_secret in the plaintext format.
by Omar Lari, Partner Solutions Architect, AWS
Amazon Elastic Container Service for Kubernetes (Amazon EKS) is a new managed service for running Kubernetes on AWS. This session will provide an overview of Amazon EKS, why we built it, and how it works.
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. With a few clicks in the AWS Management Console, you can create an API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services, such as workloads running on Amazon Elastic Compute Cloud (Amazon EC2), code running on AWS Lambda, or any Web application. Amazon API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls, including traffic management, authorization and access control, monitoring, and API version management.
Presented by: Danilo Poccia, Technical Evangelist, Amazon Web Services
Auto Scaling helps you ensure that you have the correct number of Amazon EC2 instances available to handle the load for your application. You create collections of EC2 instances, called Auto Scaling groups.
You can specify the minimum number of instances in each Auto Scaling group, and Auto Scaling ensures that your group never goes below this size.
You can specify the maximum number of instances in each Auto Scaling group, and Auto Scaling ensures that your group never goes above this size.
If you specify the desired capacity, either when you create the group or at any time thereafter, Auto Scaling ensures that your group has this many instances.
If you specify scaling policies, then Auto Scaling can launch or terminate instances as demand on your application increases or decreases
Designing security & governance via AWS Control Tower & Organizations - SEC30...Amazon Web Services
Whether it is per business unit or per application, many AWS customers use multiple accounts to meet their infrastructure isolation, separation of duties, and billing requirements. In this session, we cover considerations, limitations, and security patterns when building a multi-account strategy. We explore topics such as thought pattern, identity federation, cross-account roles, consolidated logging, and account governance. We conclude by presenting an enterprise-ready landing-zone framework and providing the background needed to implement an AWS Landing Zone using AWS Control Tower and AWS Organizations.
While many organizations have started to automate their software development processes, many still engineer their infrastructure largely by hand. Treating your infrastructure just like any other piece of code creates a “programmable infrastructure” that allows you to take full advantage of the scalability and reliability of the AWS cloud. This session will walk through practical examples of how AWS customers have merged infrastructure configuration with application code to create application-specific infrastructure and a truly unified development lifecycle. You will learn how AWS customers have leveraged tools like CloudFormation, orchestration engines, and source control systems to enable their applications to take full advantage of the scalability and reliability of the AWS cloud, create self-reliant applications, and easily recover when things go seriously wrong with their infrastructure.
Building Cloud-Native App Series - Part 2 of 11
Microservices Architecture Series
Event Sourcing & CQRS,
Kafka, Rabbit MQ
Case Studies (E-Commerce App, Movie Streaming, Ticket Booking, Restaurant, Hospital Management)
The presentation from our online webinar "Design patterns for microservice architecture".
Full video from webinar available here: https://www.youtube.com/watch?v=826aAmG06KM
If you’re a CTO or a Lead Developer and you’re planning to design service-oriented architecture, it’s definitely a webinar tailored to your needs. Adrian Zmenda, our Lead Dev, will explain:
- when microservice architecture is a safe bet and what are some good alternatives
- what are the pros and cons of the most popular design patterns (API Gateway, Backend for Frontend and more)
- how to ensure that the communication between services is done right and what to do in case of connection issues
- why we’ve decided to use a monorepo (monolithic repository)
- what we’ve learned from using the remote procedure call framework gRPC
- how to monitor the efficiency of individual services and whole SOA-based systems.
Wipro & Experitest Co webinar: Enhance your Mobile App with Load Testing and Increased Productivity.
Wipro and Experitest on a live webinar on improving the performance of your mobile application while expanding productivity. The solutions we present will allow you to test faster, and more diligently than ever, combining load, performance, and functionality all at once.
The session covers:
- A case study involving Wipro and SeeTest working with a major American Bank
- Live demo to show SeeTest Automation integrate into HP LoadRunner
- How Wipro developed a mainframe component to perform backend validation
- Live demo of Wipro's SeeTest Framework, and how it improves productivity by 50%
Speakers:
Guy Arieli, CTO, Experitest
Sudheer Mohan, Director - Mobility Certification & Automation, Wipro
As application development becomes more agile, and the ability to rapidly create and iterate new innovations escalates, so too does the need to be able to rapidly scale up the solutions that become successful. Equally it is common to create solutions with relatively short life-cycles and so we need to be able to scale down to recover resources too. On a more fine grained level, to make efficient use of shared platforms such as Kubernetes, we need to be able to dynamically scale applications up and down based on fine grained demand. Inevitably all these challenges are just as important for the integration between applications. This session explores what scalability means for the key areas of integration technology - application integration, API management and messaging.
Pros and Cons of a MicroServices Architecture talk at AWS ReInventSudhir Tonse
Netflix morphed from a private datacenter based monolithic application into a cloud based Microservices architecture. This talk highlights the pros and cons of building software applications as suites of independently deployable services, as well as practical approaches for overcoming challenges - especially in the context of an elastic but ephemeral cloud ecosystem. What were the lessons learned while building and managing these services? What are the best practices and anti-patterns?
What aspects must a developer be aware of when a Web Services will be run in clustered environment such as a server farm?
Do Web Services implementations need to be \"cluster aware\", or can this be handled transparently by the runtime platform?
We revisit the subject of why keeping Web Services implementations as stateless as possible really helps in these circumstances, and the effect of using session-based facilities on scalability.
Presentation from DDD Sydney, May 28th, 2016
Buzz word! More buzz words! And another buzz word!! Now that that's out of the way, if you're thinking of heading down the microservices path, then how do you do it? How do you build the services? What do you need to think about if you're starting from scratch? What if you're converting a legacy app? How do we deal with versioning? Do we have to use a NoSQL solution, just because Netflix does? Do we need to use docker/containers? What about the code? Show me the code! Well, that's what this session is all about. Designing and building microservices in .NET and then handling a bunch of other concerns that a microservices approach will force you to think about. Sounds interesting, doesn't it? You betcha.
Building Event Driven (Micro)services with Apache KafkaGuido Schmutz
What is a Microservices architecture and how does it differ from a Service-Oriented Architecture? Should you use traditional REST APIs to bind services together? Or is it better to use a richer, more loosely-coupled protocol? This talk will start with quick recap of how we created systems over the past 20 years and how different architectures evolved from it. The talk will show how we piece services together in event driven systems, how we use a distributed log (event hub) to create a central, persistent history of events and what benefits we achieve from doing so.
Apache Kafka is a perfect match for building such an asynchronous, loosely-coupled event-driven backbone. Events trigger processing logic, which can be implemented in a more traditional as well as in a stream processing fashion. The talk will show the difference between a request-driven and event-driven communication and show when to use which. It highlights how the modern stream processing systems can be used to hold state both internally as well as in a database and how this state can be used to further increase independence of other services, the primary goal of a Microservices architecture.
Subscription based control system to automate management of events for robotsdbpublications
In Industrial Robots, a human machine interfaces (HMI) provide means to command and control robots for various purposes. Generally, this is implemented in a reactive manner by using polling (or pull) methodology to manage the events. This methodology restricts the possibilities of automation for reacting to events thus requires a human operator to poll and react to the events based on the skill of the human operator. This sometimes causes non-optimal or wrong responses. This paper proposes a design methodology to dynamically tap the events using a subscription based control system for event based management of robots. This design enables the client to be light-weight, cost effective and makes the responses more reliable. The paper also scrutinizes the alternative design options using semantic web for multirobot coordinative activities.
Migrate a on-prem platform to the public cloud with Java - SpringBoot and PCFRoy Braam
This describes a story about a couple of teams that started their migration to the public cloud so the platform becomes available for ~300 teams. War stories, their journey, bloopers and their choices all shared.
#JaxLondon keynote: Developing applications with a microservice architectureChris Richardson
The micro-service architecture, which structures an application as a set of small, narrowly focused, independently deployable services, is becoming an increasingly popular way to build applications. This approach avoids many of the problems of a monolithic architecture. It simplifies deployment and let’s you create highly scalable and available applications. In this keynote we describe the micro-service architecture and how to use it to build complex applications. You will learn how techniques such as Command Query Responsibility Segregation (CQRS) and Event Sourcing address the key challenges of developing applications with this architecture. We will also cover some of the various frameworks such as Spring Boot that you can use to implement micro-services.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Understanding Globus Data Transfers with NetSageGlobus
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
3. Scenario
Core services grow very quickly as well as whole system complexity
Hundreds of services involved in rendering of one complex webpage Ex:
Amazon
4. Monolithic Architecture
A mobile client retrieves this data by making a single REST call to the
application, such as:
GET api.company.com/productdetails/productId
A load balancer routes the request to one of several identical application
instances.
The application then queries various database tables and return the
response to the client
5. Microservices Architecture
Data displayed on the product details page is owned by multiple
microservices
Shopping Cart Service – Number of items in the shopping cart
Order Service – Order history
Catalog Service – Basic product information
such as product name, image, and price
Review Service – Customer reviews
Inventory Service – Low inventory warning
Shipping Service – Shipping options, deadlines, and costs, drawn separately
from the shipping provider’s API
Recommendation Service(s) – Suggested items
6. Direct Client-to-Microservice
Communication
Client could make requests to each of the microservices directly
Each microservice would have a public endpoint:
https://serviceName.api.company.name
This URL would map to the microservice’s load balancer, which distributes
requests across the available instances.
To retrieve the product-specific page information, the mobile client would
make requests to each of the services listed in the previous slide
7. Challenges & Limitations
Necessity to know all endpoints addresses
The client in this example has to make seven separate requests
Makes the client code much more complex
Perform http request for each piece of information separately
Merge the result on a client side
Difficult to refactor the microservices
Non web-friendly protocols
9. API Gateway
A single entry point into the system
Handle requests
by routing them to the appropriate backend service or
by invoking multiple backend services
Aggregating the results
12. Netflix Zuul Project
Zuul is a gateway service that provides dynamic routing, monitoring,
resiliency, security and more.
Zuul = Spring Cloud + Netflix