Ever since the “CloudNative revolution” took over our development environment (devenv), we have never been more challenged (or more excited). With Kubernetes, Docker (Containerd) & many other microservice-related technologies, we have a handful of technologies to master before we write the first line of code.
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Amazon Web Services
Using Terraform to automate your infrastructure on AWS. What is Terraform and how is it different from Ansible. How to control cloud deployments using Terraform.
The document discusses software defined data centers and private clouds. It introduces Stratoscale Symphony, which is described as a turnkey, optimized OpenStack distribution for private clouds. Symphony is claimed to be a true hyper-converged infrastructure delivered in a single software stack that is infinitely scalable. It provides benefits such as self-service, programmability, portability, flexibility, ease of management, and lower total cost of ownership compared to traditional data center and private cloud approaches.
Terraform for azure: the good, the bad and the ugly - Giulio Vian
Terraform is an open-source tool for building, changing, and versioning infrastructure safely and efficiently. It allows users to define infrastructure as code and provision resources on cloud platforms like Azure. The document discusses both the benefits of Terraform for Azure, such as treating infrastructure as code and standardized configuration, as well as some challenges, such as limitations in Terraform's language and potential issues with state management. It provides tips for best practices like organizing code and folders, importing existing resources, and handling credentials and IPs. Overall, Terraform is useful for provisioning Azure infrastructure in a repeatable way but requires care around its language and state management.
Using Terraform to manage the configuration of a Cisco ACI fabric.Joel W. King
Terraform is an open-source infrastructure as code software tool created by HashiCorp. It is written in GoLang. Cisco has developed an ACI terraform provider used to interact with the Cisco APIC. Network engineers define and provision the ACI infrastructure using a declarative configuration language known as HCL, HashiCorp Configuration Language.
This session will begin with a short presentation on Terraform and how it can be used to manage resources in an ACI fabric. There is a companion GitLab repository (https://gitlab.com/joelwking/terraform_aci) which will be used as a demo environment. Attendees can download Vagrant and VirtualBox to their laptop and execute the demonstration using the Cisco DevNet Always-on ACI sandbox.
This document discusses Terraform, an open-source tool that allows users to define and provision infrastructure resources in a declarative configuration file. It summarizes that Terraform allows users to build, change, and destroy infrastructure components like compute instances, storage buckets, and networking through declarative configuration files, enabling an infrastructure-as-code approach that is easy to version, track changes for, and integrate with continuous delivery practices.
Agenda
1. The changing landscape of IT Infrastructure
2. Containers - An introduction
3. Container management systems
4. Kubernetes
5. Containers and DevOps
6. Future of Infrastructure Mgmt
About the talk
In this talk, you will get a review of the components & the benefits of Container technologies - Docker & Kubernetes. The talk focuses on making the solution platform-independent. It gives an insight into Docker and Kubernetes for consistent and reliable Deployment. We talk about how the containers fit and improve your DevOps ecosystem and how to get started with containerization. Learn new deployment approach to effectively use your infrastructure resources to minimize the overall cost.
Ever since the “CloudNative revolution” took over our development environment (devenv), we have never been more challenged (or more excited). With Kubernetes, Docker (Containerd) & many other microservice-related technologies, we have a handful of technologies to master before we write the first line of code.
Using HashiCorp’s Terraform to build your infrastructure on AWS - Pop-up Loft...Amazon Web Services
Using Terraform to automate your infrastructure on AWS. What is Terraform and how is it different from Ansible. How to control cloud deployments using Terraform.
The document discusses software defined data centers and private clouds. It introduces Stratoscale Symphony, which is described as a turnkey, optimized OpenStack distribution for private clouds. Symphony is claimed to be a true hyper-converged infrastructure delivered in a single software stack that is infinitely scalable. It provides benefits such as self-service, programmability, portability, flexibility, ease of management, and lower total cost of ownership compared to traditional data center and private cloud approaches.
Terraform for azure: the good, the bad and the ugly - Giulio Vian
Terraform is an open-source tool for building, changing, and versioning infrastructure safely and efficiently. It allows users to define infrastructure as code and provision resources on cloud platforms like Azure. The document discusses both the benefits of Terraform for Azure, such as treating infrastructure as code and standardized configuration, as well as some challenges, such as limitations in Terraform's language and potential issues with state management. It provides tips for best practices like organizing code and folders, importing existing resources, and handling credentials and IPs. Overall, Terraform is useful for provisioning Azure infrastructure in a repeatable way but requires care around its language and state management.
Using Terraform to manage the configuration of a Cisco ACI fabric.Joel W. King
Terraform is an open-source infrastructure as code software tool created by HashiCorp. It is written in GoLang. Cisco has developed an ACI terraform provider used to interact with the Cisco APIC. Network engineers define and provision the ACI infrastructure using a declarative configuration language known as HCL, HashiCorp Configuration Language.
This session will begin with a short presentation on Terraform and how it can be used to manage resources in an ACI fabric. There is a companion GitLab repository (https://gitlab.com/joelwking/terraform_aci) which will be used as a demo environment. Attendees can download Vagrant and VirtualBox to their laptop and execute the demonstration using the Cisco DevNet Always-on ACI sandbox.
This document discusses Terraform, an open-source tool that allows users to define and provision infrastructure resources in a declarative configuration file. It summarizes that Terraform allows users to build, change, and destroy infrastructure components like compute instances, storage buckets, and networking through declarative configuration files, enabling an infrastructure-as-code approach that is easy to version, track changes for, and integrate with continuous delivery practices.
Agenda
1. The changing landscape of IT Infrastructure
2. Containers - An introduction
3. Container management systems
4. Kubernetes
5. Containers and DevOps
6. Future of Infrastructure Mgmt
About the talk
In this talk, you will get a review of the components & the benefits of Container technologies - Docker & Kubernetes. The talk focuses on making the solution platform-independent. It gives an insight into Docker and Kubernetes for consistent and reliable Deployment. We talk about how the containers fit and improve your DevOps ecosystem and how to get started with containerization. Learn new deployment approach to effectively use your infrastructure resources to minimize the overall cost.
Must Know Azure Kubernetes Best Practices And Features For Better Resiliency ...CodeOps Technologies LLP
Running day-1 Ops on your Kubernetes is somewhat easy, but it is quite daunting to manage day two challenges. Learn about AKS best practices for your cloud-native applications so that you can avoid blow up your workloads.
The Mushroom Cloud Effect or What Happens When Containers Fail? by Alois Mayr...Docker, Inc.
This document discusses the "mushroom cloud effect" that can occur when containers fail in highly dynamic container environments. It describes how a failure in one container due to a lack of disk space on the host led to cascading failures that affected many dependent services. The failure spread as container health checks failed and orchestration rescheduled containers, eventually exhausting disk space and preventing any new containers from running. Automated monitoring is needed to pinpoint the root cause of such cascading failures in complex systems with many interdependent containers and services.
Docker for Ops: Docker Networking Deep Dive, Considerations and Troubleshooti...Docker, Inc.
Overview;
What is libnetwork
New features in 1.12
Deep Dive;
Multihost networking
Secure Control Plane
Secure Data plane
Service Discovery
Native Loadbalacing
Routing Mesh
This document provides instructions for installing Hybris Telco Accelerator on an Amazon EC2 instance. It outlines the prerequisites, including having an EC2 instance and SSH access. The steps include downloading the Hybris software, unzipping it, installing Java if needed, building the configuration using Ant, verifying the build, backing up default configurations, copying sample Telco configurations, and starting the Hybris server. Useful commands are also provided for finding files, installing editors, tailing logs, rebooting the VM, and checking system resources.
Janos Matyas discusses SequenceIQ's technology for provisioning Hadoop clusters. They use Docker containers and Apache Ambari for easy cluster setup across cloud providers. Key components are building Docker images, using Ansible to provision cloud templates, and running Serf and dnsmasq for service discovery and dynamic cluster membership changes. Their Cloudbreak product provides an API for on-demand Hadoop provisioning on various clouds.
A New Centralized Volume Storage Solution for Docker and Container Cloud by W...Docker, Inc.
Elara is a Docker volume plugin that provides a distributed volume management tool and connection between containers and shared storage providers. It supports advanced volume operations like snapshots, backups, migration, IOPS, disk quotas and extending volumes. Elara uses a key-value storage as a database to store node status remotely. It focuses on shared storage and supports multiple backend drivers like NFS, Cinder and filesystem/block devices.
AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)Amazon Web Services
Keeping consistent environments across your development, test, and production systems can be a complex task. Docker containers offer a way to develop and test your application in the same environment in which it runs in production. You can use tools such as the ECS CLI and Docker Compose for local testing of applications; Jenkins and AWS CodePipeline for building and workflow orchestration; Amazon EC2 Container Registry to store your container images; and Amazon EC2 Container Service to manage and scale containers. In this session, you will learn how to build containers into your development workflow and orchestrate container deployments using Amazon ECS. You will hear how Okta runs 30,000 tests per developer commit and releases 10,000 new lines of code each week to production with a CI system based on 100% AWS services. We'll also discuss how Okta uses ECS for parallelized testing in CI and for production microservices in a multi-region, always on cloud service.
In deploying apps that have been containerized, you have a lot to think about regarding what to use in production. There are a lot of things to manage, so orchestrators become a huge help. providing many services together such as scheduling, container communication, scaling, health, and more. There are major platforms to consider from Kubernetes, Swarm to ECS. In this talk we'll go through the overview of orchestrators and some of the differences between the big players. You should come out of the talk knowing where to go next in determining your orchestrator needs.
Case Study: Using Terraform and Packer to deploy go applications to AWSPatrick Bolduan
Terraform and Packer were used to automate the deployment of a Go application to AWS. Terraform was used to define and provision the infrastructure in AWS including networks, security groups, load balancers, auto scaling groups, and databases. Packer was used to build machine images with the application and dependencies installed to populate the auto scaling groups. Together, Terraform and Packer provided an automated and repeatable process for deploying the application to different environments like development, testing, and production.
Dev ops for big data cluster management toolsRan Silberman
What are the tools that we can find to day to manage Hadoop cluster and its ecosystem?
There are two tools ready today:
Cloudera Manager and Ambari from Hortonworks.
In this presentation I explain what they do and why to use them, as well as Pros. and Cons.
Serverless frameworks are changing the way we do computing. In open source container world, Kubernetes is playing a pivotal role in manifesting this. This presentation will go deep into various features of Kubernetes to create serverless functions.
Also includes a comparative study of various serverless frameworks such as Kubeless, Fission and Funktion are available in open source world. Will conclude with an implementation demo and some real world use cases.
Presented in serverless summit 2017: www.inserverless.com
Achieve big data analytic platform with lambda architecture on cloudScott Miao
This document discusses achieving a big data analytic platform using the Lambda architecture on cloud infrastructure. It begins by explaining why moving to the cloud provides benefits like elastic scaling, reduced operational overhead, and increased focus on innovation. Common cloud services at Trend Micro like an analytic engine and cloud storage are then described. The document introduces the Lambda architecture and proposes a serving layer as a service. Key lessons learned from building big data solutions on AWS include the pros of unlimited scalability and easy disaster recovery compared to on-premises infrastructure.
Overview of Openshift 3.5 on Azure. Openshift is a enterprise class implementation of kubernetes and docker. Designed to allow the creation of high scalable applications in multiple languages.
This document discusses migrating an Oracle database to Amazon RDS. It outlines that DBaaS like RDS reduces management costs but also has less flexibility than self-managed EC2 databases. The document then describes a process for migrating an Oracle database to RDS using Oracle Data Pump utilities to export the database, transfer the export file to RDS using DBMS_FILE_TRANSFER, and import the file into a new database on RDS. This process leverages an EC2 instance as a pivot to help mitigate issues with directly uploading large export files to RDS.
1. The document discusses running NetflixOSS microservices on Docker locally and in the cloud. It demonstrates Docker local setup with Eureka for service discovery and Microscaler for auto scaling and recovery.
2. Key lessons from running SkyDNS and Eureka together for service discovery in Docker include that both work well but Eureka provides more application awareness while SkyDNS has looser coupling.
3. Microscaler is an open source auto scaling system developed to handle auto scaling, recovery, and version rolling for Docker local deployments, providing functionality similar to Amazon Auto Scaling and RightScale for Docker environments.
WinOps Conference London 2017 session
Public Cloud IaaS vs traditional on prem and how Hashicorp Terraform is a great tool to configure Azure. Recorded here: https://www.youtube.com/watch?v=LDZXRBBuXCU
This document provides guidance on deploying Helix Core, a version control system, on Amazon Web Services (AWS). It discusses example architectures like using multiple availability zones for high performance and reliability. It also covers setting up virtual private clouds (VPCs) for security, using placement groups for continuous integration/delivery, storage options, and advanced topologies. The document concludes by discussing hybrid deployments, compliance, and options for managing Helix Core deployments through Assembla which specializes in hosting version control systems on AWS.
This document provides an agenda and overview for an AWS Fundamentals Basecamp training session. The agenda includes sessions on AWS Basecamp introduction, introduction to cloud computing, AWS services and regions overview, AWS networking, AWS security overview, AWS compute services, AWS storage services, and AWS databases. It also includes a break for lunch and a hands-on lab building AWS infrastructure in the afternoon. The document provides details on the topics that will be covered in each session.
Must Know Azure Kubernetes Best Practices And Features For Better Resiliency ...CodeOps Technologies LLP
Running day-1 Ops on your Kubernetes is somewhat easy, but it is quite daunting to manage day two challenges. Learn about AKS best practices for your cloud-native applications so that you can avoid blow up your workloads.
The Mushroom Cloud Effect or What Happens When Containers Fail? by Alois Mayr...Docker, Inc.
This document discusses the "mushroom cloud effect" that can occur when containers fail in highly dynamic container environments. It describes how a failure in one container due to a lack of disk space on the host led to cascading failures that affected many dependent services. The failure spread as container health checks failed and orchestration rescheduled containers, eventually exhausting disk space and preventing any new containers from running. Automated monitoring is needed to pinpoint the root cause of such cascading failures in complex systems with many interdependent containers and services.
Docker for Ops: Docker Networking Deep Dive, Considerations and Troubleshooti...Docker, Inc.
Overview;
What is libnetwork
New features in 1.12
Deep Dive;
Multihost networking
Secure Control Plane
Secure Data plane
Service Discovery
Native Loadbalacing
Routing Mesh
This document provides instructions for installing Hybris Telco Accelerator on an Amazon EC2 instance. It outlines the prerequisites, including having an EC2 instance and SSH access. The steps include downloading the Hybris software, unzipping it, installing Java if needed, building the configuration using Ant, verifying the build, backing up default configurations, copying sample Telco configurations, and starting the Hybris server. Useful commands are also provided for finding files, installing editors, tailing logs, rebooting the VM, and checking system resources.
Janos Matyas discusses SequenceIQ's technology for provisioning Hadoop clusters. They use Docker containers and Apache Ambari for easy cluster setup across cloud providers. Key components are building Docker images, using Ansible to provision cloud templates, and running Serf and dnsmasq for service discovery and dynamic cluster membership changes. Their Cloudbreak product provides an API for on-demand Hadoop provisioning on various clouds.
A New Centralized Volume Storage Solution for Docker and Container Cloud by W...Docker, Inc.
Elara is a Docker volume plugin that provides a distributed volume management tool and connection between containers and shared storage providers. It supports advanced volume operations like snapshots, backups, migration, IOPS, disk quotas and extending volumes. Elara uses a key-value storage as a database to store node status remotely. It focuses on shared storage and supports multiple backend drivers like NFS, Cinder and filesystem/block devices.
AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)Amazon Web Services
Keeping consistent environments across your development, test, and production systems can be a complex task. Docker containers offer a way to develop and test your application in the same environment in which it runs in production. You can use tools such as the ECS CLI and Docker Compose for local testing of applications; Jenkins and AWS CodePipeline for building and workflow orchestration; Amazon EC2 Container Registry to store your container images; and Amazon EC2 Container Service to manage and scale containers. In this session, you will learn how to build containers into your development workflow and orchestrate container deployments using Amazon ECS. You will hear how Okta runs 30,000 tests per developer commit and releases 10,000 new lines of code each week to production with a CI system based on 100% AWS services. We'll also discuss how Okta uses ECS for parallelized testing in CI and for production microservices in a multi-region, always on cloud service.
In deploying apps that have been containerized, you have a lot to think about regarding what to use in production. There are a lot of things to manage, so orchestrators become a huge help. providing many services together such as scheduling, container communication, scaling, health, and more. There are major platforms to consider from Kubernetes, Swarm to ECS. In this talk we'll go through the overview of orchestrators and some of the differences between the big players. You should come out of the talk knowing where to go next in determining your orchestrator needs.
Case Study: Using Terraform and Packer to deploy go applications to AWSPatrick Bolduan
Terraform and Packer were used to automate the deployment of a Go application to AWS. Terraform was used to define and provision the infrastructure in AWS including networks, security groups, load balancers, auto scaling groups, and databases. Packer was used to build machine images with the application and dependencies installed to populate the auto scaling groups. Together, Terraform and Packer provided an automated and repeatable process for deploying the application to different environments like development, testing, and production.
Dev ops for big data cluster management toolsRan Silberman
What are the tools that we can find to day to manage Hadoop cluster and its ecosystem?
There are two tools ready today:
Cloudera Manager and Ambari from Hortonworks.
In this presentation I explain what they do and why to use them, as well as Pros. and Cons.
Serverless frameworks are changing the way we do computing. In open source container world, Kubernetes is playing a pivotal role in manifesting this. This presentation will go deep into various features of Kubernetes to create serverless functions.
Also includes a comparative study of various serverless frameworks such as Kubeless, Fission and Funktion are available in open source world. Will conclude with an implementation demo and some real world use cases.
Presented in serverless summit 2017: www.inserverless.com
Achieve big data analytic platform with lambda architecture on cloudScott Miao
This document discusses achieving a big data analytic platform using the Lambda architecture on cloud infrastructure. It begins by explaining why moving to the cloud provides benefits like elastic scaling, reduced operational overhead, and increased focus on innovation. Common cloud services at Trend Micro like an analytic engine and cloud storage are then described. The document introduces the Lambda architecture and proposes a serving layer as a service. Key lessons learned from building big data solutions on AWS include the pros of unlimited scalability and easy disaster recovery compared to on-premises infrastructure.
Overview of Openshift 3.5 on Azure. Openshift is a enterprise class implementation of kubernetes and docker. Designed to allow the creation of high scalable applications in multiple languages.
This document discusses migrating an Oracle database to Amazon RDS. It outlines that DBaaS like RDS reduces management costs but also has less flexibility than self-managed EC2 databases. The document then describes a process for migrating an Oracle database to RDS using Oracle Data Pump utilities to export the database, transfer the export file to RDS using DBMS_FILE_TRANSFER, and import the file into a new database on RDS. This process leverages an EC2 instance as a pivot to help mitigate issues with directly uploading large export files to RDS.
1. The document discusses running NetflixOSS microservices on Docker locally and in the cloud. It demonstrates Docker local setup with Eureka for service discovery and Microscaler for auto scaling and recovery.
2. Key lessons from running SkyDNS and Eureka together for service discovery in Docker include that both work well but Eureka provides more application awareness while SkyDNS has looser coupling.
3. Microscaler is an open source auto scaling system developed to handle auto scaling, recovery, and version rolling for Docker local deployments, providing functionality similar to Amazon Auto Scaling and RightScale for Docker environments.
WinOps Conference London 2017 session
Public Cloud IaaS vs traditional on prem and how Hashicorp Terraform is a great tool to configure Azure. Recorded here: https://www.youtube.com/watch?v=LDZXRBBuXCU
This document provides guidance on deploying Helix Core, a version control system, on Amazon Web Services (AWS). It discusses example architectures like using multiple availability zones for high performance and reliability. It also covers setting up virtual private clouds (VPCs) for security, using placement groups for continuous integration/delivery, storage options, and advanced topologies. The document concludes by discussing hybrid deployments, compliance, and options for managing Helix Core deployments through Assembla which specializes in hosting version control systems on AWS.
This document provides an agenda and overview for an AWS Fundamentals Basecamp training session. The agenda includes sessions on AWS Basecamp introduction, introduction to cloud computing, AWS services and regions overview, AWS networking, AWS security overview, AWS compute services, AWS storage services, and AWS databases. It also includes a break for lunch and a hands-on lab building AWS infrastructure in the afternoon. The document provides details on the topics that will be covered in each session.
Azure Event Hubs - Behind the Scenes With Kasun Indrasiri | Current 2022HostedbyConfluent
Azure Event Hubs - Behind the Scenes With Kasun Indrasiri | Current 2022
Azure Event Hubs is a hyperscale PaaS event stream broker with protocol support for HTTP, AMQP, and Apache Kafka RPC that accepts and forwards several trillion (!) events per day and is available in all global Azure regions. This session is a look behind the curtain where we dive deep into the architecture of Event Hubs and look at the Event Hubs cluster model, resource isolation, and storage strategies and also review some performance figures.
This document provides best practices for naming and tagging AWS resources. Key recommendations include:
1. Use standardized naming conventions to provide context and allow resources to be identified, filtered, and queried.
2. Automate infrastructure provisioning and tagging to ensure consistency at scale.
3. For EC2 instances, include environment, cluster/platform, application, and other context in names.
4. For VPCs, subnets, and security groups, include relevant details like environment, platform, availability zone, and accessibility in names.
5. Tag resources according to usage, ownership, cost structure, and other attributes to enable cost tracking and automation.
VMworld 2013: How SRP Delivers More Than Power to Their Customers VMworld
VMworld 2013
Sheldon Brown, SRP
Girish Manmadkar, VMware
Learn more about VMworld and register at http://www.vmworld.com/index.jspa?src=socmed-vmworld-slideshare
The document provides an overview of AWS Free Tier and key AWS services. It discusses how AWS provides global infrastructure across multiple regions and availability zones to provide high availability and meet regulatory requirements. Key services summarized include IAM for access control, S3 for object storage, EC2 for virtual servers, EBS for block storage, load balancers, CloudWatch for monitoring, auto scaling, RDS for databases, VPC for virtual networks, and the AWS CLI.
The document discusses using Apache Kafka for event detection pipelines. It describes how Kafka can be used to decouple data pipelines and ingest events from various source systems in real-time. It then provides an example use case of using Kafka, Hadoop, and machine learning for fraud detection in consumer banking, describing the online and offline workflows. Finally, it covers some of the challenges of building such a system and considerations for deploying Kafka.
The document provides an overview of the major OpenStack components from both a tenant and operator perspective. It describes the key services that OpenStack provides (Compute, Networking, Block Storage, Object Storage, Image Storage, Identity) and how each would be used and managed differently by tenants consuming infrastructure resources versus operators configuring and maintaining the cloud platform. It aims to explain the similarities and differences in how these services are experienced by tenants versus operators.
IT Infrastructure for Beginners Understandingraj4u1oct
This document summarizes key components of an IT infrastructure including lease lines, PRI lines, networking equipment, servers, storage, virtualization, cloud services, and AWS services. It discusses EC2 virtual servers, EBS storage volumes, RDS relational database services, DynamoDB non-relational database, Redshift data warehousing, EMR big data processing, and compares AWS, Azure, and Google Cloud Platform.
Those who out-compute can many times out-compete. The cloud gives you access to a massive amount of compute power when you need it. This talk will present an introduction to HPC in the cloud, including, the benefits of HPC in the cloud, how to get started, some tools to use, and how you can manage data. We will showcase several examples of HPC in the cloud by a number of public sector and commercial customers.
Created by: Dr. Jeff Layton, Principal, Solutions Architect
This presentation provides an overview of Apache CloudStack, an open source cloud computing platform. It discusses CloudStack's history and licensing, its ability to provide infrastructure as a service across multiple hypervisors, and how it enables multi-tenancy, high availability, scalability, and resource allocation. Key CloudStack components and concepts are also summarized, such as networking models, security groups, primary and secondary storage, usage tracking, and its management architecture.
Join Marc Trouard-Riolle from Citrix Cloud Product Marketing for the latest presentation in the Citrix Cloud Master Class series.
In this session you will hear about building private enterprise clouds with Citrix CloudPlatform:
Learn about hypervisor, storage and networking considerations within private cloud use cases
Build a tailored availability zone for traditional workloads
See a step-by-step demonstration of building an enterprise private cloud
If you need to build highly performant, mission critical ,microservice-based system following DevOps best practices, you should definitely check Service Fabric!
Service Fabric is one of the most interesting services Azure offers today. It provide unique capabilities outperforming competitor products.
We are seeing global companies start to use Service Fabric for their mission critical solutions.
In this talk we explore the current state of Service Fabric and dive deeper to highlight best practices and design patterns.
We will cover the following topics:
• Service Fabric Core Concepts
• Cluster Planning and Management
• Stateless Services
• Stateful Services
• Actor Model
• Availability and reliability
• Scalability and perfromance
• Diganostics and Monitoring
• Containers
• Testing
• IoT
Live broadcast on https://www.youtube.com/watch?v=Zuxfhpab6xo
GE IOT Predix Time Series & Data Ingestion Service using Apache Apex (Hadoop)Apache Apex
This presentation will introduce usage of Apache Apex for Time Series & Data Ingestion Service by General Electric Internet of things Predix platform. Apache Apex is a native Hadoop data in motion platform that is being used by customers for both streaming as well as batch processing. Common use cases include ingestion into Hadoop, streaming analytics, ETL, database off-loads, alerts and monitoring, machine model scoring, etc.
Abstract: Predix is an General Electric platform for Internet of Things. It helps users develop applications that connect industrial machines with people through data and analytics for better business outcomes. Predix offers a catalog of services that provide core capabilities required by industrial internet applications. We will deep dive into Predix Time Series and Data Ingestion services leveraging fast, scalable, highly performant, and fault tolerant capabilities of Apache Apex.
Speakers:
- Venkatesh Sivasubramanian, Sr Staff Software Engineer, GE Predix & Committer of Apache Apex
- Pramod Immaneni, PPMC member of Apache Apex, and DataTorrent Architect
The Enterprise IT Checklist for Docker Operations Nicola Kabar
Enterprises often have hundreds of legacy applications developed by development teams across multiple business units. This presents a series of challenges to IT teams as they architect and support a complex and diverse IT environment. Add to that Docker, containers, and cloud - going beyond the pilot environment to production requires both the technology and best practices. In this session, we will go through a checklist of considerations and best practices providing a framework for smooth Docker production operations.
Walk through this hands-on workshop to expand your AWS technical skills. Gain credibility for your experience working with AWS by building proficiency with services and solutions in the areas of AWS Architecture Fundamentals.
Cassandra Tools and Distributed Administration (Jeffrey Berger, Knewton) | C*...DataStax
At Knewton we operate across five different VPCs a total of 29 clusters, each ranging from 3 nodes to 24 nodes. For a team of three to maintain this is not herculean, however good tools to diagnose issues and gather information in a distributed manner are vital to moving quickly and minimizing engineering time spent.
The database team at Knewton has been successfully using a combination of Ansible and custom open sourced tools to maintain and improve the Cassandra deployment at Knewton. I will be talking about several of these tools and giving examples of how we are using them. Specifically I will discuss the cassandra-tracing tool, which analyzes the contents of the system_traces keyspace, and the cassandra-stat tool, which gives real-time output of the operations of a cassandra cluster. Distributed administration with ad-hoc Ansible will also be covered and I will walk through examples of using these commands to identify and remediate clusterwide issues.
About the Speaker
Jeffrey Berger Lead Database Engineer, Knewton
Dr. Jeffrey Berger is currently the lead database engineer at Knewton, an education tech startup in NYC. He joined the tech scene in NYC in 2013 and spent two years working with MongoDB, becoming a certified MongoDB administrator and a MongoDB Master. He received his Cassandra Administrator certification at Cassandra Summit 2015. He holds a Ph.D. in Theoretical Physics from Penn State and spent several years working on high energy nuclear interactions.
Terratest - Automation testing of infrastructureKnoldus Inc.
TerraTest is a testing framework specifically designed for testing infrastructure code written with HashiCorp's Terraform. It helps validate that your Terraform configurations create the desired infrastructure, and it can be used for both unit testing and integration testing.
Getting Started with Apache Spark (Scala)Knoldus Inc.
In this session, we are going to cover Apache Spark, the architecture of Apache Spark, Data Lineage, Direct Acyclic Graph(DAG), and many more concepts. Apache Spark is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.
Secure practices with dot net services.pptxKnoldus Inc.
Securing .NET services is paramount for protecting applications and data. Employing encryption, strong authentication, and adherence to best coding practices ensures resilience against potential threats, enhancing overall cybersecurity posture.
Distributed Cache with dot microservicesKnoldus Inc.
A distributed cache is a cache shared by multiple app servers, typically maintained as an external service to the app servers that access it. A distributed cache can improve the performance and scalability of an ASP.NET Core app, especially when the app is hosted by a cloud service or a server farm. Here we will look into implementation of Distributed Caching Strategy with Redis in Microservices Architecture focusing on cache synchronization, eviction policies, and cache consistency.
Introduction to gRPC Presentation (Java)Knoldus Inc.
gRPC, which stands for Remote Procedure Call, is an open-source framework developed by Google. It is designed for building efficient and scalable distributed systems. gRPC enables communication between client and server applications by defining a set of services and message types using Protocol Buffers (protobuf) as the interface definition language. gRPC provides a way for applications to call methods on a remote server as if they were local procedures, making it a powerful tool for building distributed and microservices-based architectures.
Using InfluxDB for real-time monitoring in JmeterKnoldus Inc.
Explore the integration of InfluxDB with JMeter for real-time performance monitoring. This session will cover setting up InfluxDB to capture JMeter metrics, configuring JMeter to send data to InfluxDB, and visualizing the results using Grafana. Learn how to leverage this powerful combination to gain real-time insights into your application's performance, enabling proactive issue detection and faster resolution.
Intoduction to KubeVela Presentation (DevOps)Knoldus Inc.
KubeVela is an open-source platform for modern application delivery and operation on Kubernetes. It is designed to simplify the deployment and management of applications in a Kubernetes environment. KubeVela is a modern software delivery platform that makes deploying and operating applications across today's hybrid, multi-cloud environments easier, faster and more reliable. KubeVela is infrastructure agnostic, programmable, yet most importantly, application-centric. It allows you to build powerful software, and deliver them anywhere!
Stakeholder Management (Project Management) PresentationKnoldus Inc.
A stakeholder is someone who has an interest in or who is affected by your project and its outcome. This may include both internal and external entities such as the members of the project team, project sponsors, executives, customers, suppliers, partners and the government. Stakeholder management is the process of managing the expectations and the requirements of these stakeholders.
Introduction To Kaniko (DevOps) PresentationKnoldus Inc.
Kaniko is an open-source tool developed by Google that enables building container images from a Dockerfile inside a Kubernetes cluster without requiring a Docker daemon. Kaniko executes each command in the Dockerfile in the user space using an executor image, which runs inside a container, such as a Kubernetes pod. This allows building container images in environments where the user doesn’t have root access, like a Kubernetes cluster.
Efficient Test Environments with Infrastructure as Code (IaC)Knoldus Inc.
In the rapidly evolving landscape of software development, the need for efficient and scalable test environments has become more critical than ever. This session, "Streamlining Development: Unlocking Efficiency through Infrastructure as Code (IaC) in Test Environments," is designed to provide an in-depth exploration of how leveraging IaC can revolutionize your testing processes and enhance overall development productivity.
Exploring Terramate DevOps (Presentation)Knoldus Inc.
Terramate is a code generator and orchestrator for Terraform that enhances Terraform's capabilities by adding features such as code generation, stacks, orchestration, change detection, globals, and more . It's primarily designed to help manage Terraform code at scale more efficiently . Terramate is particularly useful for managing multiple Terraform stacks, providing support for change detection and code generation 2. It allows you to create relationships between stacks to improve your understanding and control over your infrastructure . One of the key features of Terramate is its ability to detect changes at both the stack and module level. This capability allows you to identify which stacks and resources have been altered and selectively determine where you should execute commands.
Clean Code in Test Automation Differentiating Between the Good and the BadKnoldus Inc.
This session focuses on the principles of writing clean, maintainable, and efficient code in the context of test automation. The session will highlight the characteristics that distinguish good test automation code from bad, ultimately leading to more reliable and scalable testing frameworks.
Integrating AI Capabilities in Test AutomationKnoldus Inc.
Explore the integration of artificial intelligence in test automation. Understand how AI can enhance test planning, execution, and analysis, leading to more efficient and reliable testing processes. Explore the cutting-edge integration of Artificial Intelligence (AI) capabilities in Test Automation, a transformative approach shaping the future of software testing. This session will delve into practical applications, benefits, and considerations associated with infusing AI into test automation workflows.
State Management with NGXS in Angular.pptxKnoldus Inc.
NGXS is a state management pattern and library for Angular. NGXS acts as a single source of truth for your application's state - providing simple rules for predictable state mutations. In this session we will go through the main for components of NGXS -Store, Actions, State, and Select.
Authentication in Svelte using cookies.pptxKnoldus Inc.
Svelte streamlines authentication with cookies, offering a secure and seamless user experience. Effortlessly manage sessions by storing tokens in cookies, ensuring persistent logins. With Svelte's simplicity, implement robust authentication mechanisms, enhancing user security and interaction.
OAuth2 Implementation Presentation (Java)Knoldus Inc.
The OAuth 2.0 authorization framework is a protocol that allows a user to grant a third-party web site or application access to the user's protected resources, without necessarily revealing their long-term credentials or even their identity. It is commonly used in scenarios such as user authentication in web and mobile applications and enables a more secure and user-friendly authorization process.
Supply chain security with Kubeclarity.pptxKnoldus Inc.
Kube clarity is a comprehensive solution designed to enhance supply chain security within Kubernetes environments. Kube clarity enables organizations to identify and mitigate potential security threats throughout the software development and deployment process.
Mastering Web Scraping with JSoup Unlocking the Secrets of HTML ParsingKnoldus Inc.
In this session, we will delve into the world of web scraping with JSoup, an open-source Java library. Here we are going to learn how to parse HTML effectively, extract meaningful data, and navigate the Document Object Model (DOM) for powerful web scraping capabilities.
Akka gRPC Essentials A Hands-On IntroductionKnoldus Inc.
Dive into the fundamental aspects of Akka gRPC and learn to leverage its power in building compact and efficient distributed systems. This session aims to equip attendees with the essential skills and knowledge to leverage Akka and gRPC effectively in building robust, scalable, and distributed applications.
Entity Core with Core Microservices.pptxKnoldus Inc.
How Developers can use Entity framework(ORM) which provides a structured and consistent way for microservices to interact with their respective database, prompting independence, scaliblity and maintainiblity in a distributed system, and also provide a high-level abstraction for data access.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
2. Lack of etiquette and manners is a huge turn off.
KnolX Etiquettes
Punctuality
Respect Knolx session timings, you
are requested not to join sessions
after a 5 minutes threshold post
the session start time.
Feedback
Make sure to submit a constructive
feedback for all sessions as it is
very helpful for the presenter.
Silent Mode
Keep your mobile devices in silent
mode, feel free to move out of
session in case you need to attend
an urgent call.
Avoid Disturbance
Avoid unwanted chit chat during
the session.
3. Agenda
01 Introduction to Cloud Providers - AWS
02 Benefits
03 AWS Infrastructure
04 Setting up Infrastructure - From Scratch
05 Infrastructure as Code - Terraform
04 Demo
4. Introduction to Cloud Providers - AWS
General Client-Server Architecture
Server Components
● Compute (CPU)
● Memory (RAM)
● Storage
Database (Structured Storage)
Network Devices
● Routers
● Switches
● DNS server
5. Benefits
● On Demand service
● Pay as you go
● Access resources instantly
● Easy to access/use
6. AWS Infrastructure
● AWS is global
○ Regions
■ Availability Zones
● Data Centers
○ Edge Locations (Point of Presence)
AWS Infrastructure
8. Setting up Infrastructure -
From Scratch
● VPC - Logically Isolated sections where AWS Resources are launched.
● Subnet - Logical subdivision of an IP network.
● Internet Gateway - Highly available VPC component that allows communication between your
VPC and the internet.
● Elastic IP - An Elastic IP address is a static IPv4 address.
● NAT Gateway - Enable instances in a private subnet to connect to the internet or other AWS
services, but prevent the internet from initiating a connection with those instances.
● Route Tables - Contains a set of rules, called routes, that are used to determine where network
traffic from your subnet or gateway is directed.
● Instance - Server holding compute, storage and memory
9. Infrastructure as Code
Provisioning Infrastructure via s/w to achieve consistent and predictable
environments.
Benefits of IaC
● Automate deployment
● Consistent environments
● Reusable components
● Documented Architecture
10. IaC - Terraform
Features of Terraform
● Defined in code
● Stored in source control
● Declarative Syntax
● Idempotent
● Follows Push mechanism
14. IaC - Terraform
Terraform Concepts
● Data
data "aws_ami" "web" {
filter {
name = "state"
values = ["available"]
}
filter {
name = "tag:Component"
values = ["web"]
}
most_recent = true
}
resource "aws_instance" "web" {
ami = data.aws_ami.web.id
instance_type = "t1.micro"
}
15. IaC - Terraform
Terraform Concepts
● Output
output "instance_ip_addr" {
value = aws_instance.server.private_ip
description = "The private IP address of the main server instance."
}
16. Terraform
Benefits out of the box
● Inspect state file
● Dependency graph: Tf knows the order in which resources needs to be created first
● Calculates Additions, updations and deletions
● Parallel Execution
● Allow saving the plan