This deck was presented at Lendingkart meetup in Bangalore covering our experiences with creating CI/CD Pipeline with Kubernetes. Here is the video link of the meetup.
https://youtu.be/YraPL_NGmcs
Navigate the universe of CI/CD tools.
As the fastest way to production, the CI/CD pipeline is now mainstream among software companies, forming the backbone of the modern DevOps environment. While DevOps handles the culture aspect, CI/CD focuses on the process and tools.
With this guide, we hope to provide a clear overview of the various CI/CD tools categories and give a broad sampling of the various tools that are available.
Azure DevOps Tutorial | Developing CI/ CD Pipelines On Azure | EdurekaEdureka!
(** DevOps Certification Training: https://www.edureka.co/devops **)
This Edureka "Azure DevOps” PPT will give you a thorough and insightful overview Microsoft Azure and DevOps approach and help you create a CI/CD pipeline using Microsoft Azure.
Following are the offerings of this PPT:
1. What Is DevOps?
2. What Is Azure DevOps?
3. Components Of Azure DevOps
4. Demo – Azure DevOps
Check out our Playlists: https://goo.gl/Xpx77b
Blog Series: https://goo.gl/KvjU4h
Follow us to never miss an update in the future.
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
This deck was presented at Lendingkart meetup in Bangalore covering our experiences with creating CI/CD Pipeline with Kubernetes. Here is the video link of the meetup.
https://youtu.be/YraPL_NGmcs
Navigate the universe of CI/CD tools.
As the fastest way to production, the CI/CD pipeline is now mainstream among software companies, forming the backbone of the modern DevOps environment. While DevOps handles the culture aspect, CI/CD focuses on the process and tools.
With this guide, we hope to provide a clear overview of the various CI/CD tools categories and give a broad sampling of the various tools that are available.
Azure DevOps Tutorial | Developing CI/ CD Pipelines On Azure | EdurekaEdureka!
(** DevOps Certification Training: https://www.edureka.co/devops **)
This Edureka "Azure DevOps” PPT will give you a thorough and insightful overview Microsoft Azure and DevOps approach and help you create a CI/CD pipeline using Microsoft Azure.
Following are the offerings of this PPT:
1. What Is DevOps?
2. What Is Azure DevOps?
3. Components Of Azure DevOps
4. Demo – Azure DevOps
Check out our Playlists: https://goo.gl/Xpx77b
Blog Series: https://goo.gl/KvjU4h
Follow us to never miss an update in the future.
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
While many organizations have started to automate their software development processes, many still engineer their infrastructure largely by hand. Treating your infrastructure just like any other piece of code creates a “programmable infrastructure” that allows you to take full advantage of the scalability and reliability of the AWS cloud. This session will walk through practical examples of how AWS customers have merged infrastructure configuration with application code to create application-specific infrastructure and a truly unified development lifecycle. You will learn how AWS customers have leveraged tools like CloudFormation, orchestration engines, and source control systems to enable their applications to take full advantage of the scalability and reliability of the AWS cloud, create self-reliant applications, and easily recover when things go seriously wrong with their infrastructure.
Everyone heard about Kubernetes. Everyone wants to use this tool. However, sometimes we forget about security, which is essential throughout the container lifecycle.
Therefore, our journey with Kubernetes security should begin in the build stage when writing the code becomes the container image.
Kubernetes provides innate security advantages, and together with solid container protection, it will be invincible.
During the sessions, we will review all those features and highlight which are mandatory to use. We will discuss the main vulnerabilities which may cause compromising your system.
Contacts:
LinkedIn - https://www.linkedin.com/in/vshynkar/
GitHub - https://github.com/sqerison
-------------------------------------------------------------------------------------
Materials from the video:
The policies and docker files examples:
https://gist.github.com/sqerison/43365e30ee62298d9757deeab7643a90
The repo with the helm chart used in a demo:
https://github.com/sqerison/argo-rollouts-demo
Tools that showed in the last section:
https://github.com/armosec/kubescape
https://github.com/aquasecurity/kube-bench
https://github.com/controlplaneio/kubectl-kubesec
https://github.com/Shopify/kubeaudit#installation
https://github.com/eldadru/ksniff
Further learning.
A book released by CISA (Cybersecurity and Infrastructure Security Agency):
https://media.defense.gov/2021/Aug/03/2002820425/-1/-1/1/CTR_KUBERNETES%20HARDENING%20GUIDANCE.PDF
O`REILLY Kubernetes Security:
https://kubernetes-security.info/
O`REILLY Container Security:
https://info.aquasec.com/container-security-book
Thanks for watching!
AManaging Kong API Gateway with TerraformByungjin Park
2018년 10월 23일 진행된 하시코프 한국 사용자 모임 두 번째 밋업 발표 자료 입니다.
# Terraform으로 Kong API Gateway 관리하기
박병진, Site Reliability Engineer @ Kasa
Abstract:
Kong은 Nginx와 Lua를 기반으로 하는 오픈소스 API Gateway입니다. 본 세션에서는 API Gateway가 무엇인지, Kong을 선택한 이유와 사용 방법, Terraform을 이용하여 Kong을 관리한 경험을 공유합니다.
In this session we will take an introduction look to Continuous Integration and Continuous Delivery workflow.
This is an introduction session to CI/CD and is best for people new to the CI/CD concepts, or looking to brush up on benefits of using these approaches.
* What CI & CD actually are
* What good looks like
* A method for tracking confidence
* The business value from CI/CD
DevSecOps: Taking a DevOps Approach to SecurityAlert Logic
More organisations are embracing DevOps and automation to realise compelling business benefits, such as more frequent feature releases, increased application stability, and more productive resource utilization. However, many security and compliance monitoring tools have not kept up. In fact, they often represent the largest single remaining barrier to continuous delivery.
Infrastructure as Code, tools, benefits, paradigms and more.
Presentation from DigitalOnUs DevOps: Infrastructure as Code Meetup (September 20, 2018 - Monterrey Nuevo Leon MX)
YouTube Link: https://youtu.be/8sFTdzz55KU
** Cloud Certification Training: https://www.edureka.co/cloud-computing-certification-courses **
This Edureka ”Azure Pipelines” session will give you a complete walkthrough to Microsoft Azure Pipelines and introduce to Agile Development on Azure Cloud platform.
Following are the offerings of this PPT:
What is Azure DevOps?
Azure DevOps Services
What is Azure Pipelines
Demo: Azure Pipelines Walkthrough
Follow us to never miss an update in the future.
YouTube: https://www.youtube.com/user/edurekaIN
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
Castbox: https://castbox.fm/networks/505?country=in
Microservices, Containers, Docker and a Cloud-Native Architecture in the Midd...Kai Wähner
Microservices are the next step after SOA: Services implement a limited set of functions. Services are developed, deployed and scaled independently. Continuous Integration and Continuous Delivery automate deployments. This way you get shorter time to results and increased flexibility. Containers improve these even more offering a very lightweight and flexible deployment option.
In the middleware world, you use concepts and tools such as an Enterprise Service Bus (ESB), Complex Event Processing (CEP), Business Process Management (BPM) or API Gateways. Many people still think about complex, heavyweight central brokers here. However, Microservices and containers are relevant not just for custom self-developed applications, but they are also a key requirement to make the middleware world more flexible, agile and automated.
This session discusses the requirements, best practices and challenges for creating a good Microservices architecture in the middleware world. A live demo with the open source PaaS framework CloudFoundry shows how technologies and frameworks such as Java, SOAP / REST Web Services, Jenkins and Docker are used to create an agile software development lifecycle to realize “Middleware Microservices”. It also discusses other modern cloud-native alternatives such as Kubernetes, Docker, Mesos, Mesosphere or Amazon ECS / AWS.
If you’re working with just a few containers, managing them isn't too complicated. But what if you have hundreds or thousands? Think about having to handle multiple upgrades for each container, keeping track of container and node state, available resources, and more. That’s where Kubernetes comes in. Kubernetes is an open source container management platform that helps you run containers at scale. This talk will cover Kubernetes components and show how to run applications on it.
This slide is the speech provided by me for InfoSec2020 (https://2020.infosec.org.tw/) conference in Taiwan. It describes the container security, what issues is. how to exploit it and how to defense it.
- Archeology: before and without Kubernetes
- Deployment: kube-up, DCOS, GKE
- Core Architecture: the apiserver, the kubelet and the scheduler
- Compute Model: the pod, the service and the controller
Continuous Delivery is hot. As we all increasingly compete using software, the business always wants more change faster. However, change is seen as risky. How do we deliver quickly while not exposing the business to excessive risk? What does this imply for how we update our mission critical databases?
Successful continuous delivery efforts use quality as an enabler of rapid change. Rapid feedback on the quality of the application, and a disciplined, high quality process support frequent delivery of business value, rather than frequent outage.
IBM UrbanCode’s Eric Minick and DBmaestro’s Yaniv Yehuda present how to build safety in to your delivery process. We will look at database change in some detail while delivering generally applicable lessons.
While many organizations have started to automate their software development processes, many still engineer their infrastructure largely by hand. Treating your infrastructure just like any other piece of code creates a “programmable infrastructure” that allows you to take full advantage of the scalability and reliability of the AWS cloud. This session will walk through practical examples of how AWS customers have merged infrastructure configuration with application code to create application-specific infrastructure and a truly unified development lifecycle. You will learn how AWS customers have leveraged tools like CloudFormation, orchestration engines, and source control systems to enable their applications to take full advantage of the scalability and reliability of the AWS cloud, create self-reliant applications, and easily recover when things go seriously wrong with their infrastructure.
Everyone heard about Kubernetes. Everyone wants to use this tool. However, sometimes we forget about security, which is essential throughout the container lifecycle.
Therefore, our journey with Kubernetes security should begin in the build stage when writing the code becomes the container image.
Kubernetes provides innate security advantages, and together with solid container protection, it will be invincible.
During the sessions, we will review all those features and highlight which are mandatory to use. We will discuss the main vulnerabilities which may cause compromising your system.
Contacts:
LinkedIn - https://www.linkedin.com/in/vshynkar/
GitHub - https://github.com/sqerison
-------------------------------------------------------------------------------------
Materials from the video:
The policies and docker files examples:
https://gist.github.com/sqerison/43365e30ee62298d9757deeab7643a90
The repo with the helm chart used in a demo:
https://github.com/sqerison/argo-rollouts-demo
Tools that showed in the last section:
https://github.com/armosec/kubescape
https://github.com/aquasecurity/kube-bench
https://github.com/controlplaneio/kubectl-kubesec
https://github.com/Shopify/kubeaudit#installation
https://github.com/eldadru/ksniff
Further learning.
A book released by CISA (Cybersecurity and Infrastructure Security Agency):
https://media.defense.gov/2021/Aug/03/2002820425/-1/-1/1/CTR_KUBERNETES%20HARDENING%20GUIDANCE.PDF
O`REILLY Kubernetes Security:
https://kubernetes-security.info/
O`REILLY Container Security:
https://info.aquasec.com/container-security-book
Thanks for watching!
AManaging Kong API Gateway with TerraformByungjin Park
2018년 10월 23일 진행된 하시코프 한국 사용자 모임 두 번째 밋업 발표 자료 입니다.
# Terraform으로 Kong API Gateway 관리하기
박병진, Site Reliability Engineer @ Kasa
Abstract:
Kong은 Nginx와 Lua를 기반으로 하는 오픈소스 API Gateway입니다. 본 세션에서는 API Gateway가 무엇인지, Kong을 선택한 이유와 사용 방법, Terraform을 이용하여 Kong을 관리한 경험을 공유합니다.
In this session we will take an introduction look to Continuous Integration and Continuous Delivery workflow.
This is an introduction session to CI/CD and is best for people new to the CI/CD concepts, or looking to brush up on benefits of using these approaches.
* What CI & CD actually are
* What good looks like
* A method for tracking confidence
* The business value from CI/CD
DevSecOps: Taking a DevOps Approach to SecurityAlert Logic
More organisations are embracing DevOps and automation to realise compelling business benefits, such as more frequent feature releases, increased application stability, and more productive resource utilization. However, many security and compliance monitoring tools have not kept up. In fact, they often represent the largest single remaining barrier to continuous delivery.
Infrastructure as Code, tools, benefits, paradigms and more.
Presentation from DigitalOnUs DevOps: Infrastructure as Code Meetup (September 20, 2018 - Monterrey Nuevo Leon MX)
YouTube Link: https://youtu.be/8sFTdzz55KU
** Cloud Certification Training: https://www.edureka.co/cloud-computing-certification-courses **
This Edureka ”Azure Pipelines” session will give you a complete walkthrough to Microsoft Azure Pipelines and introduce to Agile Development on Azure Cloud platform.
Following are the offerings of this PPT:
What is Azure DevOps?
Azure DevOps Services
What is Azure Pipelines
Demo: Azure Pipelines Walkthrough
Follow us to never miss an update in the future.
YouTube: https://www.youtube.com/user/edurekaIN
Instagram: https://www.instagram.com/edureka_learning/
Facebook: https://www.facebook.com/edurekaIN/
Twitter: https://twitter.com/edurekain
LinkedIn: https://www.linkedin.com/company/edureka
Castbox: https://castbox.fm/networks/505?country=in
Microservices, Containers, Docker and a Cloud-Native Architecture in the Midd...Kai Wähner
Microservices are the next step after SOA: Services implement a limited set of functions. Services are developed, deployed and scaled independently. Continuous Integration and Continuous Delivery automate deployments. This way you get shorter time to results and increased flexibility. Containers improve these even more offering a very lightweight and flexible deployment option.
In the middleware world, you use concepts and tools such as an Enterprise Service Bus (ESB), Complex Event Processing (CEP), Business Process Management (BPM) or API Gateways. Many people still think about complex, heavyweight central brokers here. However, Microservices and containers are relevant not just for custom self-developed applications, but they are also a key requirement to make the middleware world more flexible, agile and automated.
This session discusses the requirements, best practices and challenges for creating a good Microservices architecture in the middleware world. A live demo with the open source PaaS framework CloudFoundry shows how technologies and frameworks such as Java, SOAP / REST Web Services, Jenkins and Docker are used to create an agile software development lifecycle to realize “Middleware Microservices”. It also discusses other modern cloud-native alternatives such as Kubernetes, Docker, Mesos, Mesosphere or Amazon ECS / AWS.
If you’re working with just a few containers, managing them isn't too complicated. But what if you have hundreds or thousands? Think about having to handle multiple upgrades for each container, keeping track of container and node state, available resources, and more. That’s where Kubernetes comes in. Kubernetes is an open source container management platform that helps you run containers at scale. This talk will cover Kubernetes components and show how to run applications on it.
This slide is the speech provided by me for InfoSec2020 (https://2020.infosec.org.tw/) conference in Taiwan. It describes the container security, what issues is. how to exploit it and how to defense it.
- Archeology: before and without Kubernetes
- Deployment: kube-up, DCOS, GKE
- Core Architecture: the apiserver, the kubelet and the scheduler
- Compute Model: the pod, the service and the controller
Continuous Delivery is hot. As we all increasingly compete using software, the business always wants more change faster. However, change is seen as risky. How do we deliver quickly while not exposing the business to excessive risk? What does this imply for how we update our mission critical databases?
Successful continuous delivery efforts use quality as an enabler of rapid change. Rapid feedback on the quality of the application, and a disciplined, high quality process support frequent delivery of business value, rather than frequent outage.
IBM UrbanCode’s Eric Minick and DBmaestro’s Yaniv Yehuda present how to build safety in to your delivery process. We will look at database change in some detail while delivering generally applicable lessons.
Dopo aver annunciato la nuova partnership commerciale con DBMaestro, Emerasoft ha realizzato un webinar volto ad illustrare le principali caratteristiche di questo nuovo strumento innovativo: DBMaestro TeamWork.
Scopri DBmaestro Teamwork: la soluzione DevOps per il Database, che permette l’Agile Database Development, la Continuous Integration e la Continuous Delivery.
Guarda il video del webinar: https://www.youtube.com/watch?v=YzPB9Y6Y8tA
DevOps Interview Questions Part - 1 | Devops Interview Questions And Answers ...Simplilearn
This presentation about "DevOps interview questions part - 1" will take you through some of the most popular questions that you face in a DevOps interview. This video covers interview questions related to source code management, continuous integration, continuous testing, configuration management, containerization, and continuous monitoring. "The DevOps Hiring Boom” claims that as many as 80 percentages of Fortune 1000 organizations are expected to adopt DevOps by 2019. If you’ve started cross-training to prepare for development and operations roles in the IT industry, you know it’s a challenging field that will take some real preparation to break into. Here are some of the most common DevOps interview questions and answers that can help you while you prepare for DevOps roles in the industry. Learn and get a deeper understanding of these questions to set you apart from the crowd in this booming industry.
This "DevOps interview questions" presentation will answer the questions related to the topics mentioned below:
1. General DevOps questions
2. Source code management - Git
3. Continuous integration - Jenkins
4. Continuous testing - Selenium
Why learn DevOps?
Simplilearn’s DevOps training course is designed to help you become a DevOps practitioner and apply the latest in DevOps methodology to automate your software development lifecycle right out of the class. You will master configuration management; continuous integration deployment, delivery, and monitoring using DevOps tools such as Git, Docker, Jenkins, Puppet, and Nagios in a practical, hands-on and interactive approach. The DevOps training course focuses heavily on the use of Docker containers, a technology that is revolutionizing the way apps are deployed in the cloud today and is a critical skillset to master in the cloud age.
After completing the DevOps training course you will achieve hands-on expertise in various aspects of the DevOps delivery model. The practical learning outcomes of this Devops training course are:
An understanding of DevOps and the modern DevOps toolsets
The ability to automate all aspects of a modern code delivery and deployment pipeline using:
1. Source code management tools
2. Build tools
3. Test automation tools
4. Containerization through Docker
5. Configuration management tools
6. Monitoring tools
DevOps jobs are the third-highest tech role ranked by employer demand on Indeed.com but have the second-highest talent deficit.
1. This DevOps training course will be of benefit the following professional roles:
2. Software Developers
3. Technical Project Managers
4. Architects
5. Operations Support
6. Deployment engineers
7. IT managers
8. Development managers
Learn more at https://www.simplilearn.com/cloud-computing/devops-practitioner-certification-training
Building an Automated Database Deployment PipelineGrant Fritchey
The pace of business accelerates fairly continuously and application development moves right with it. But we’re still trying to deploy databases the same way we did 10 years ago. This session addresses the need for changes in organizational structure, process and technology necessary to arrive at a nimble, fast, automatable and continuous database deployment process. We’ll use actual customer case studies to illustrate both the common methods and the unique context that led to a continuous delivery process that is best described as a pipeline. You will learn how to customize common practices and tool sets to build a database deployment pipeline unique to your environment in order to speed your own database delivery while still protecting your organization’s most valuable asset, it’s data.
Deployment automation efforts tend to start with easier scenarios like moving builds of web applications to servers and getting them installed. However, some parts of our applications aren’t simple builds. They may be updated incrementally; changes may be non-repeatable; or they may be dependent on knowledge contained within some other tool or framework. When we fail to automate changes to these “tricky” parts of our application, errors and delays materialize.
Eric Minick from IBM, and Robert Reeves, database guru from Datical, discuss what makes certain things hard to deploy, and practical techniques and tools for deploying them. Topics covered include:
* What causes certain deployments to be trickier to automate than others
* Successful patterns for overcoming those challenges
* Application of those techniques to mainframe changes, WebSphere configuration and database schema updates
XP teams try to keep systems fully integrated at all times, and shorten the feedback cycle to minutes and hours instead of weeks or months. The sooner you know, the sooner you can adapt.
Watch our record for the webinar "Continuous Integration" to explore how Azure DevOps helps us in achieving continuous feedback using continuous integration.
Continuous Integration and Continuous Delivery on AzureCitiusTech
Healthcare organizations are increasingly turning to cloud computing to address business and patient needs of their rapidly evolving environment and modernize legacy applications. With Azure DevOps, healthcare IT teams can drive innovation, build new products and modernize their application environment.
Geek Sync I In Database Automation We TrustIDERA Software
You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/4ab250A5qUq
Continuous delivery is all the hype in software development. However, databases are being neglected as DBAs just don't seem to trust database automation.
This is not surprising when you consider the number of incidents and instances of downtime that were caused by out-of-process updates, code overrides and other database glitches.
In this session, we will explain what continuous delivery for databases is all about, why DBAs are so mistrusting, and ways to overcome that mistrust and conquer database automation.
Enterprise DevOps and the Modern Mainframe Webcast PresentationCompuware
Compuware and CloudBees demonstrate how you can apply modern DevOps practices to your mainframe applications using Compuware ISPW and Topaz for Total Test with CloudBees Jenkins. Compuware Product Manager Steve Kansa and CloudBees DevOps Evangelist Brian Dawson will:
- Position the mainframe as part of your DevOps and CI/CD journey
- Explain how Jenkins automates mainframe source code management and testing
- Demo a CI/CD workflow on a COBOL application
Watch the full presentation on YouTube: https://www.youtube.com/watch?v=x4MWrPy3bKM.
Similar to The Science of database CICD - UKOUG Breakthrough (20)
Myth Busting - NoSQL vs SQL Data Stores - Video EditionJasmin Fluri
Slides to the Youtube Video about Myth Busting SQL vs NoSQL Data Stores. What questions do you need to ask when you are looking for a new data storage solution?
AskTom Office Hours about Database MigrationsJasmin Fluri
Oracle AskTom Office Hours Session about Continuous Database Migrations and the first steps on how to automate your Migrations. It shows the preconditions to establish an automation and shows basic examples.
Prediction of Skierdays with Oracle Data Mining - Analytics and Data Techcast...Jasmin Fluri
Introductionary Presentation about Data Mining / Machine Learning with the Free Oracle Data Mining Features of the OCI Free Tier. Experiences from a Data Mining Research Project.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
3. About me
Jasmin Fluri
Schaltstelle GmbH
Database Development
Development Process
Automation and Tooling
Continuous Integration
Continuous Delivery
Software Engineering
@jasminfluri
@jasminfluri
jasminfluri
Pipelining
8. DevOps Goal:
We want to ship changes
continuously to get fast and
continuous feedback!
8
9. High performing teams deploy changes at least once a day!
Source: DevOps Report DORA 2021
9
10. High performing teams have a low change failure rate!
10
Source: DevOps Report DORA 2021
11. Why do they recover so much
better from incidents?
11
12. Small changes = small risk = small possible damage = small feedback
loops
12
Source: NYTimes – The best path to long term success is slow, simple and boring!
13. What positive effect do small changes have?
13
Small changes have small impact! It’s easy to see
all the elements that are affected!
Small changes only introduce a small risk of
change!
Small changes can more easily be reverted or fixed
when they are faulty.
14. … so deploying small changes,
often is a good practice to do!
14
15. … but it’s not common in
database development!
15
20. Agile
• Developing small
increments
• Always having a
runnable version of the
product
20
DevOps CI/CD
• Interconnecting tasks
and processes
• Whole app lifecycle
managed by a single
team
• Service-Infrastructure
• Automation
• Hands-off integration
and deployment
• Fast feedback
• Automated quality
assurance
22. Developer
Version Control
System
Continuous
Integration
Server
Continuous
Integration
Pipeline
Pushing changes
into Version Control
Trigger pipeline
on commit or
on merge request
execute
1 - Checkout source code
2 - Build and
test of application
3 – Run system tests,
static code analysis
and metrics
4 - Generate reports
and notifications
5 - notify
Continuous Integration
File-based
Development
Packages
can replace old
versions
No state
No order of
changes!
25. Developer
Version Control
System
Continuous
Integration
Server
Continuous
Integration
Pipeline
Pushing changes
into Version Control
Trigger pipeline
on commit or
on merge request
execute
1 - Checkout source code
2 - Build and
test of database
3 – Run system tests,
static code analysis
and metrics
4 - Generate reports
and notifications
5 - notify
Continuous Integration
File-based
Development and
Generated Code
Changes have
an order! Builds are
deployments!
We depend
upon state!
26. What do we know are
preconditions for good
database CI/CD?
26
27. Before you start building a database CI/CD pipeline you need…
Static
Code
Analysis
Automated
Tests
Version
Control of
Everything
Database
Migration
Tool
Decoupled
Codebase
27
30. You need to store EVERYTHING in version
control!
30
The scary 36.6%
31. There’s no best practice or
standard of how to version
SQL code! – But there are
some ways that work…
31
32. Pure File based development isn’t common in database
development
Problem: you often not only want your migration scripts in VCS but also the
DDLs!
DDLs allow you to build a system from scratch fast!
DMLs allow you to migrate to the next version!
32
33. Database Source Code in Version Control
myproject/
├── …
├── docs/
│ ├── documentation.md
├── db/
│ ├── 0_sequences/
│ ├── 1_tables/
│ ├── 2_mviews/
│ ├── 3_views/
│ ├── 4_packages/
│ └── 5_utils/
├── migrations/
│ ├── scripts/
└── db-tests/
├── packages/
└── data/
DDL
code
Test code
and test data
Sample Project Structure
• DDLs contain the definition of our
database objects
• Migration scripts allow us to migrate to
the next version!
• Test code tests our database logic and
behaviour.
Migration Scripts
34. The majority of people uses a mix of generating and writing
migration scripts!
34
35. To be able to do file-based
development for database migration
scripts we need repeatability!
35
36. What happens if your database scripts are not
repeatable?
Non Repeatable Migration Script
If it fails, we need a new one with the remaining changes (and
corrections)
36
Repeatable Migration Script
If it fails, we alter it, and run it again – it will continue where it failed.
43. Merging Application Development with Database Development
Working Trunk-based Submodules – Mixing Workflows
43
APP Repository
Gitflow
DB Repository
Trunk Based
DB
Repository
Trunk Based
Submodule
APP & DB
Repository
Trunk Based
44. How do you review code when
you do trunk-based
development?
44
45. Daily struggle to wait for code reviews of your feature
branches…
45
46. «But we don’t have the time to do pair programming»
46
57. Unit Testing in your
Application
- API Tests
- Integration tests
- Unit Tests of the
application backend
Testing Tools
Unit Testing in your
Database
- Unit Tests
- Integration tests
59. What happens when release cycles are coupled to other
teams (or components) ?
59
Team A
Team B
Team C
Release 1 Release 2 Release 3 Release 4
This is not Continuous Delivery!
63. Coupling of release
activities must be
eliminated!
63
Application A
Service XY
Infrastructure
provide
Services
consume
Services
Version 1 Version 2 Version 3
Team A
Version 4
Team B
Teams want to work independently and continuously!
65. Application A Application B Application C
Service XY (Database)
Infrastructure
provide
Services
consume
Services
Service Delivery Organisation
Version 1 Version 2 Version 3
This is even more important,
when we are shipping small
changes continuously!
66. Now we have all preconditions
to build a CI Pipeline!
67. Continuous
Integration &
Release
Pipeline
67
Development Environment
Version Control
Pipelines
Infrastructure
New Functionality
develops
pushes
CI Server
CI
Database
triggers
starts
Static
Code
Analysis
Check
out
Code
Deploy
SQL
Code
Unit
Tests
System
Tests
Set Release
Number
End of
Release
Backu
p
Developer
Artefact Repository
Push
Artefact
Deployment
Artefact
New Commit
on Main
Branch
Release Pipeline
Static
Code
Analysis
Tool
Integration
Pipeline
End of
Integration
72. Introducing CI/CD into a database development
project…
… increased the amount of deployments by over 5x!
More deployments means more testing!
More deployments also means less risk!
72
73. Introducing CI/CD into a database development
project…
… decreased the amount of failed deployments by
over 75%!
If it hurts, do it more often!
Automation introduces a repeatable process, without human
interventions!
73
74. Introducing CI/CD into a database development
project…
… decreased cognitive load of the
developers!
Focus now lies on the development of functionality, not in
manually deploying changes and troubleshooting!
74
75. Introducing CI/CD into a database development
project…
… did not change the lead time for changes!
All participating projects used fixed release windows for
deploying changes into production!
Fixed release windows = Fixed lead time!
75
77. Ressources / References
Accelerate Book : https://itrevolution.com/book/accelerate/
DORA 2022 : https://cloud.google.com/blog/products/devops-sre/dora-2022-accelerate-state-of-
devops-report-now-out
DORA 2021 : https://services.google.com/fh/files/misc/state-of-devops-2021.pdf
The best path to long term success is slow : https://www.nytimes.com/2017/07/31/your-money/the-
best-path-to-long-term-change-is-slow-simple-and-boring.html
Martin Fowler : Continuous Integration
https://www.martinfowler.com/articles/continuousIntegration.htm
77
Editor's Notes
What are we going to see today?
First we will have a look at what are preconditions of database schema evolutions or database migrations?
Second we will explore how database deployments or schema migrations are conducted
And third we will have time for questions and then proceed with a hands-on lab.
Now If you have a version control system you will need to think about a directory structure of your repository.
The main thing when it comes to structuring is storing production code separate from test code and test data.
There are a couple of things to consider:
Store the ddl of all of your objects per object type for easier navigation
Store migration scripts separately
Store tests and test data separate from production code
Use the same naming for test packages and production packages to simplify navigation
Use one test package per productive package that is tested
Test data and tests should be stored separate from each other to ensure reusability of the test data in different test contexts
Now that we have our directory structure we can have a look at our source code workflow.
Let’s have a look at the first precondition. The repository structure of our version control repository like git, and the workflow that we need to set up in our development process.
So The problem with long-running feature branches is that you can't test the commits in the right order. In both database and infrastructure development, it often depends on which patch or change is applied first before another is applied. If you have long running feature branches, you don't know what changes have been made to your application in the meantime. Therefore, testing is not very efficient or safe.
Well, Continuous Integration works best for databases and for infrastructure code when version control is trunk-based.
Because many changes to database applications cannot be tested properly if feature branches are used! The same is true for infrastructure code.
So if you are doing either infrastructure provisioning or database engineering, you should develop trunk-based.
Let’s have a look why!
If we look at how trunk-based development and continuous integration work, the continuous integration pipeline is triggered with each commit on the mainline.
CI is therefore executed when a new commit is made and starts:
Migration scripts.
Post-migration scripts such as recompiling, restoring synonyms, creating permissions.
Tests are executed
Metrics are calculated
Static code analysis is performed
The delivery process can then be started manually and only performs migrations and post-migration scripts.
Now there are two different approaches to implementing trunk-based development.
Either you start completely trunk-based as shown here on the left, so that the whole application including the database is developed trunk-based. This requires a big amount of pair programming and putting changes of small size into production at a time.
Or you work in Git with submodules, and split the application and the database into two repositories.
The database repository is then included as a submodule in the application repository so that the CI/CD pipeline has access to both. The application part can be then developed for example with a gitflow workflow and the database trunk based. Database changes would then be deployed, when a new release of the application is deployed. This is easier to implement when a versioned API above the database exists that decouples the two parts.
If I tell people that they should to pair programming, I often hear that they don’t have time for pair programming, because they have lot’s of stuff to do.
The thinking is that they are faster if they don’t do pair or mob programming.
But the nature of things is, that you usually have dependencies in development. Those can be knowledge gaps that you have, where you need support from a colleague, or access dependencies.
As an example the lead time is shown of the yellow task that needs both developers. It is a lot longer, than if those two would just sit together and do it in pair programming. With the side effect that if they would do pair programming, they would also share knowledge about the feature at the same time. So they would be faster and also smarter afterwards.
A database migration tool helps us migrate the database schema from one state to another.
This means we start with an initial DDL script that builds database version 1.
From here on we will always have migrations that transform our database into the next version. And our database migration tool tracks those migrations, checks if a migration script was already run against a database and if not, installs the migration.
This way we can apply changes to our database like adding tables, adding foreign keys, removing data or refactoring the database with new migration scripts.
The tool that we are going to see today is flyway that follows a migration based approach.
Database migration files are stored as a series of files with continuous enumeration, telling the tool in which order the scripts need to be run, to succeed.
Usually there are migraiton scripts that run only once against a database instance and we have so called repeatable scripts, that run after every migration. Those scripts can set for example synonyms that might get lost during migrations or grants or create database views and procedures that don’t include persistence or state. Also recompilation script are quite common repeatable scripts, that check if all the database objects are still in a valid state and can be executed.
The third precondition is that we need to test our database.
If you have a java Application you can use jUnit and the jUnit extension DBUnit to write database integration tests or also database unit tests. The problem here is, that you need a running database instance to run the tests against, so they are more integration tests than they are unit tests, because you also test the interaction of the two systems with every test and cannot run them without having the database present.
We have primarily two kinds of teams, teams that provide services to others and teams that consume services from other teams.
For application teams or software delivery teams it is especially important to have a service layer abstracting the infrastructure, to make them independent and allowing them to build and run their application independently of any other team.
That’s the main goal of DevOps – You build it, you run it. So everything a team builds, they can also independently run in production. There’s no hand-off to an operations team that takes over production, the application teams does everything on its own.
So, Especially above the infrastructure layer, a service layer is essential so that application teams can use them. These interfaces should be made available in different versions whenever possible.
Versioning looks something like this.
Our service XY provides several versions of its API.
This allows the consuming applications to change the version individually if they either want the benefit of new functionalities or if they want to invest time in the lifecycle of the application and switch to the latest service version.
A very Important aspect is: Applications should be allowed to decide for themselves when they switch to newer versions of an API. Forcing applications to do this synchronously undermines the prioritization within an application team of what is important to them. It may be enough for a team to use functionalities from version 1, because they don’t require new functionalities of newer verisons.