Microservices: why you're doing them wrong_Dmytro Lahoza
•Download as PPTX, PDF•
1 like•304 views
Most developers love microservice and treat them as a panacea, but they are wrong. How to stop creating problems using microservices and start solving them.
Report
Share
Report
Share
Recommended
Test driven development for infrastructure as-a-code, the future trend_Gianfr...
Test driven development for infrastructure as-a-code, the future trend_Gianfr...Katherine Golovinova
This document discusses testing infrastructure as code (IaaC) using test-driven development (TDD) principles. It recommends applying different types of automated tests for IaaC: unit tests to check for errors, integration tests to validate functionality, and acceptance/security tests on deployed infrastructure. Various tools are mentioned for testing IaaC written in languages like Terraform, Ansible, Chef, and Puppet at the unit, integration, and security levels. Adopting a testing mindset and tools can help catch errors and non-compliance early in development pipelines.DevSecOps overview and what one engineer can do_Dmytro Batiievskyi
We will talk about what is DevSecOps and why security matters in context of DevOps and automation as well as general overview of the scope and the approach. Than we will go through cheap yet effective security improvements that can be implemented with a small team without significant extra effort.
Matt carroll - "Security patching system packages is fun" said no-one ever
This document summarizes Matt Carroll's talk on making security patching of system packages less tedious for engineers. It discusses how automating work distribution and feedback through tools like JIRA, establishing clear deadlines, and streamlining documentation can increase agency and motivation. The overall goal is to reduce pain points and uncertainty to encourage proactive security practices rather than treating it as an "arcane ritual".
Nick Drage & Fraser Scott - Epic battle devops vs security
Fraser and Nick debate the relationship between DevOps and security. Nick argues security is too complex for DevOps approaches, while Fraser argues DevOps and security ultimately have the same goals of reducing risk and increasing value. They propose defining a "risk budget" to measure and manage risk like an "error budget", allowing more frequent deployments if risk is reduced through practices like testing and security engagement. Ultimately they agree DevOps and security need cooperation rather than separation, with security helping scale out practices while DevOps takes security responsibilities.
Dev seccon london 2016 intelliment security
This document discusses writing firewall policies in application manifests from a DevSecOps perspective. It describes how defining network and security requirements as code can help automate infrastructure delivery and reduce bottlenecks. The presenter advocates applying a "shift left" paradigm to define requirements early. A demo is outlined showing how Puppet can be used to define an application's network visibility needs, which are then automatically validated and deployed to firewalls by Intelliment for consistent security compliance across teams.
Azaz_Sharepoint & Security_Admin
This document contains information about Azaz Ahmed's skills, experience, education, and certifications. It details his 3+ years of experience in areas like server administration, network security, firewall administration, and penetration testing. It provides summaries of some of his key projects, including migrating a website from SharePoint 2007 to 2013, setting up an Active Directory network, virtualization using Hyper-V, penetration testing, and security auditing. It also lists his technical skills and experience levels in various technologies.
DevSecOps What Why and How
This document discusses DevSecOps, including what it is, why it is needed, and how to implement it. DevSecOps aims to integrate security into development tools and processes to promote a "secure by default" culture. It is needed because traditional security approaches cannot keep up with the rapid pace of DevOps. Implementing DevSecOps involves automating security checks and tests into the development pipeline and promoting collaboration between development, security, and operations teams. The document provides examples of tools that can be used and case studies of DevSecOps implementations.
Why should developers care about container security?
Slides from my talk at SF Bay Cloud Native Containers Meetup Feb 2022 and SnykLive Stranger Danger on April 27, 2022.
https://www.meetup.com/cloudnativecontainers/events/283721735/
Recommended
Test driven development for infrastructure as-a-code, the future trend_Gianfr...
Test driven development for infrastructure as-a-code, the future trend_Gianfr...Katherine Golovinova
This document discusses testing infrastructure as code (IaaC) using test-driven development (TDD) principles. It recommends applying different types of automated tests for IaaC: unit tests to check for errors, integration tests to validate functionality, and acceptance/security tests on deployed infrastructure. Various tools are mentioned for testing IaaC written in languages like Terraform, Ansible, Chef, and Puppet at the unit, integration, and security levels. Adopting a testing mindset and tools can help catch errors and non-compliance early in development pipelines.DevSecOps overview and what one engineer can do_Dmytro Batiievskyi
We will talk about what is DevSecOps and why security matters in context of DevOps and automation as well as general overview of the scope and the approach. Than we will go through cheap yet effective security improvements that can be implemented with a small team without significant extra effort.
Matt carroll - "Security patching system packages is fun" said no-one ever
This document summarizes Matt Carroll's talk on making security patching of system packages less tedious for engineers. It discusses how automating work distribution and feedback through tools like JIRA, establishing clear deadlines, and streamlining documentation can increase agency and motivation. The overall goal is to reduce pain points and uncertainty to encourage proactive security practices rather than treating it as an "arcane ritual".
Nick Drage & Fraser Scott - Epic battle devops vs security
Fraser and Nick debate the relationship between DevOps and security. Nick argues security is too complex for DevOps approaches, while Fraser argues DevOps and security ultimately have the same goals of reducing risk and increasing value. They propose defining a "risk budget" to measure and manage risk like an "error budget", allowing more frequent deployments if risk is reduced through practices like testing and security engagement. Ultimately they agree DevOps and security need cooperation rather than separation, with security helping scale out practices while DevOps takes security responsibilities.
Dev seccon london 2016 intelliment security
This document discusses writing firewall policies in application manifests from a DevSecOps perspective. It describes how defining network and security requirements as code can help automate infrastructure delivery and reduce bottlenecks. The presenter advocates applying a "shift left" paradigm to define requirements early. A demo is outlined showing how Puppet can be used to define an application's network visibility needs, which are then automatically validated and deployed to firewalls by Intelliment for consistent security compliance across teams.
Azaz_Sharepoint & Security_Admin
This document contains information about Azaz Ahmed's skills, experience, education, and certifications. It details his 3+ years of experience in areas like server administration, network security, firewall administration, and penetration testing. It provides summaries of some of his key projects, including migrating a website from SharePoint 2007 to 2013, setting up an Active Directory network, virtualization using Hyper-V, penetration testing, and security auditing. It also lists his technical skills and experience levels in various technologies.
DevSecOps What Why and How
This document discusses DevSecOps, including what it is, why it is needed, and how to implement it. DevSecOps aims to integrate security into development tools and processes to promote a "secure by default" culture. It is needed because traditional security approaches cannot keep up with the rapid pace of DevOps. Implementing DevSecOps involves automating security checks and tests into the development pipeline and promoting collaboration between development, security, and operations teams. The document provides examples of tools that can be used and case studies of DevSecOps implementations.
Why should developers care about container security?
Slides from my talk at SF Bay Cloud Native Containers Meetup Feb 2022 and SnykLive Stranger Danger on April 27, 2022.
https://www.meetup.com/cloudnativecontainers/events/283721735/
Devops security-An Insight into Secure-SDLC
The integration of Security into DevOps is already happening out of necessity. DevOps is a powerful paradigm shift and companies often don’t understand how security fits. Aim of this session is to give an overview of DevOps security and How security can be integrated and automated into each phases of software development life-cycle.
DevSecOps : an Introduction
DevSecOps is a cultural change that incorporates security practices into software development through people, processes, and technologies. It aims to address security without slowing delivery by establishing secure-by-design approaches, automating security tools and processes, and promoting collaboration between developers, security engineers, and operations teams. As software and connected devices continue proliferating, application security must be a central focus of the development lifecycle through a DevSecOps methodology.
Integrate Security into DevOps - SecDevOps
1.Security Controls Must Be Programmable and Automated Wherever Possible
2.Implement a Simple Risk and Threat Model for All Applications
3.Scan Custom Code, Applications and APIs
4.Scan for OSS Issues in Development
5.Treat Scripts/Recipes/Templates/Layers as Sensitive Code
6.Measure System Integrity and Ensure Correct Configuration at Load
7.Use Whitelisting on Production Systems, Including Container-Based Implementations
8.Assume Compromise; Monitor Everything; Architect for Rapid Detection and Response
9.Lock Down Production Infrastructure and Services
10.Tokenization and Payment Processing
DevSecOps Singapore introduction
The document announces events from DevSecOps Singapore to bring together developers, operations, and security professionals. It describes monthly meetups for talks and networking, workshops over 4 months on integrating security testing into the SDLC, and an annual conference in 2017. It provides announcements for the workshops and conference and calls for speakers, office space, and volunteers to help build the community.
DevSecOps reference architectures 2018
DevSecOps reference architectures: Sonatype Nexus, Sonatype Nexus Lifecycle, HP Fortify, SonarQube, Jenkins, Twistlock, JIRA, Contrast, aqua, OWASP Zap, Find Bugs, Gaunltl, OWASP Depedency check, NESSUS, ThreadFix
DevOps 2018
DevSecOps: essential tooling to enable continuous security 2019-09-16
Richard Mills discusses how DevSecOps enables continuous security in Agile development through integrating security tools and processes into CI/CD pipelines. He outlines essential categories of security tools, including static analysis, software composition analysis, vulnerability scanning, dynamic testing, and monitoring. These tools can run tests at various stages of the pipeline to catch issues early. Mills also stresses the importance of integrating security teams with development teams through structures like technical guilds to build a culture of security.
Clean architecture with Python
I gave this presentation Jan 24th 2019 here:
https://www.meetup.com/BAyPIGgies/events/257819623/
The code can be found here:
https://github.com/jairvercosa/ca_auth
DevSecOps | DevOps Sec
DevSecOps enables organizations to deliver secure software at DevOps speed.
Development - Software releases and updates
Operations - Reliability Performance & Scaling
Security - Confidentiality, Availability, Integrity
DevSecops: Defined, tools, characteristics, tools, frameworks, benefits and c...
In this presentation, it is outlined about DevOps, DevSecOps, Characteristics of DevSecOps, DevSecops Practises, Benefits of Implementing DevSecOps, Implementation Frameworks and the Challenges in Implementing DevSecOps.
DevSecOps - The big picture
This document discusses the concepts of DevSecOps at a high level. It begins with a brief history of development methodologies, from Waterfall to Agile, and how Ops became a bottleneck. This led to trends in Agile Operations and collaboration between Dev and Ops, known as DevOps. DevSecOps expands this to incorporate security. It discusses the importance of culture, processes, and technologies for effective communication, automation, and collaboration across Dev, Ops, and Security. The goal is to enable organizations to deliver inherently secure software at DevOps speed through a high-trust environment and automated security pipelines integrated into the software development lifecycle.
DevSecOps Training Bootcamp - A Practical DevSecOps Course
DevSecOps means integrating security practices into the DevOps workflow from the beginning. The goal is to make everyone responsible for security and implement security decisions at the same speed as development and operations. This helps find vulnerabilities early and improve overall security. Implementing DevSecOps requires planning, building, deploying, monitoring and improving security continuously. It provides benefits like improved compliance and identifying issues earlier.
ABN AMRO DevSecOps Journey
This document summarizes ABN AMRO's DevSecOps journey and initiatives. It discusses their implementation of continuous integration and delivery pipelines to improve software quality, reduce lead times, and increase developer productivity. It also covers their work to incorporate security practices like open source software management, container security, and credentials management into the development lifecycle through techniques like dependency scanning, security profiling, and a centralized secrets store. The presentation provides status updates on these efforts and outlines next steps to further mature ABN AMRO's DevSecOps capabilities.
Benefits of DevSecOps
Brief of benefits of DevSecOps
DevSecOps = Security +DevOps
DevSecOps = New culture + new skills + automation
Code Quality - Security
Quality of software code for a given product shipped effectively translates not only to its functional quality but as well to its non functional aspects say security. Many of the issues in code can be addressed much before they reach SCM.
Dev week cloud world conf2021
Talk to executives in IT divisions of large enterprises about security and invariably the conversation will hover around
DevSecOps pipeline.
Is DevSecOps the only thing you need to do for security in your IT division or is there more?
What impact does bringing in secure culture in an engineering context mean?
What handshake is needed between the IT function and the security / risk function for large enterprises?
How does this impact roles and responsibilities of a developer?
This talk is an attempt to answer questions such as these using a real world examples of transformations seen in Fortune 100 companies.
DevSecOps, The Good, Bad, and Ugly
This document discusses DevSecOps and covers the good, bad, and ugly aspects. DevSecOps aims to integrate security practices into the development lifecycle like threat modeling, security testing, and monitoring. The good aspects include finding vulnerabilities early through testing and reviewable infrastructure policies. The bad parts are potential performance issues and loss of availability from tools. The ugly challenges are misunderstandings causing disasters, unstable new tools causing false alarms, and responsibility over security. Overall, DevSecOps is about people, process, and integrating security throughout the development lifecycle rather than just tools.
Strengthen and Scale Security Using DevSecOps - OWASP Indonesia
Strengthen and Scale Security Using DevSecOps - OWASP Indonesia
More details here - https://www.practical-devsecops.com/
Zero to Ninety in Securing DevOps
Delivered at DevSecOps Days 2018, RSA Conference
j. Wolfgang Goerlich
About J. Wolfgang Goerlich
About J Wolfgang Goerlich
CBI (Creative Breakthroughs, Inc.)
Cyber Security Strategist
J Wolfgang Goerlich provides strategic guidance for securing development and DevOps programs in the healthcare, education, financial services, and energy. He is currently with CBI, a cyber security consultancy, as the VP for strategic security programs. Wolfgang also leads the CBI Academy teams, providing mentoring and coaching to the junior-level talent. Prior roles included VP for a managed security services provider, VP for an IT firm specializing in high speed high secure networks, and IT security officer and manager for a financial services firm. He is an active part of the security community; co-founding the Converge Detroit and organizing the BSides Detroit conferences. Wolfgang regularly advises on and presents on the topics of secure development life cycle, DevOps, risk management, incident response, business continuity, and more.
Introduction to DevSecOps
An introduction to the devsecops webinar will be presented by me at 10.30am EST on 29th July,2018. It's a session focussed on high level overview of devsecops which will be followed by intermediate and advanced level sessions in future.
Agenda:
-DevSecOps Introduction
-Key Challenges, Recommendations
-DevSecOps Analysis
-DevSecOps Core Practices
-DevSecOps pipeline for Application & Infrastructure Security
-DevSecOps Security Tools Selection Tips
-DevSecOps Implementation Strategy
-DevSecOps Final Checklist
PIACERE - DevSecOps Automated
This document summarizes the PIACERE project, which aims to integrate security into DevSecOps processes. It receives funding from the EU Horizon 2020 program. The project develops tools like the DevSecOps Modeling Language (DOML) and Verification Tool to integrate security principles into infrastructure modeling and deployment. It also includes a Canary Sandbox Environment for testing deployments and an Infrastructure Optimization Platform for optimizing cloud resources. The overall goal is to provide a unified platform for secure, automated deployment to multiple clouds.
infrastructure management at digital ages
This document discusses infrastructure management in the digital era and the transition to DevOps practices. It provides the following key points:
1) Infrastructure management is evolving from traditional techniques to new ways of working inspired by software development practices like infrastructure as code.
2) Infrastructure as code uses configuration management tools and version control to define infrastructure in code for provisioning, deployment, and updates.
3) For infrastructure managers to support DevOps, their role is shifting from solely managing infrastructure to providing a digital toolbox and optimized experiences through services like a managed cloud platform.
Realtime search at Yammer
This presentation is focused on the architecture, scalability concerns, performance bottlenecks, operational characteristics and lessons learned while designing and implementing Yammer distributed real-time search system. Yammer is an enterprise social network SaaS offering with over 100,000 networks (including 85% of the Fortune 100) and nearly 2 million users. The search system we developed scales well up to 1B messages and serves a foundation of knowledge base analysis services Yammer is developing.
More Related Content
What's hot
Devops security-An Insight into Secure-SDLC
The integration of Security into DevOps is already happening out of necessity. DevOps is a powerful paradigm shift and companies often don’t understand how security fits. Aim of this session is to give an overview of DevOps security and How security can be integrated and automated into each phases of software development life-cycle.
DevSecOps : an Introduction
DevSecOps is a cultural change that incorporates security practices into software development through people, processes, and technologies. It aims to address security without slowing delivery by establishing secure-by-design approaches, automating security tools and processes, and promoting collaboration between developers, security engineers, and operations teams. As software and connected devices continue proliferating, application security must be a central focus of the development lifecycle through a DevSecOps methodology.
Integrate Security into DevOps - SecDevOps
1.Security Controls Must Be Programmable and Automated Wherever Possible
2.Implement a Simple Risk and Threat Model for All Applications
3.Scan Custom Code, Applications and APIs
4.Scan for OSS Issues in Development
5.Treat Scripts/Recipes/Templates/Layers as Sensitive Code
6.Measure System Integrity and Ensure Correct Configuration at Load
7.Use Whitelisting on Production Systems, Including Container-Based Implementations
8.Assume Compromise; Monitor Everything; Architect for Rapid Detection and Response
9.Lock Down Production Infrastructure and Services
10.Tokenization and Payment Processing
DevSecOps Singapore introduction
The document announces events from DevSecOps Singapore to bring together developers, operations, and security professionals. It describes monthly meetups for talks and networking, workshops over 4 months on integrating security testing into the SDLC, and an annual conference in 2017. It provides announcements for the workshops and conference and calls for speakers, office space, and volunteers to help build the community.
DevSecOps reference architectures 2018
DevSecOps reference architectures: Sonatype Nexus, Sonatype Nexus Lifecycle, HP Fortify, SonarQube, Jenkins, Twistlock, JIRA, Contrast, aqua, OWASP Zap, Find Bugs, Gaunltl, OWASP Depedency check, NESSUS, ThreadFix
DevOps 2018
DevSecOps: essential tooling to enable continuous security 2019-09-16
Richard Mills discusses how DevSecOps enables continuous security in Agile development through integrating security tools and processes into CI/CD pipelines. He outlines essential categories of security tools, including static analysis, software composition analysis, vulnerability scanning, dynamic testing, and monitoring. These tools can run tests at various stages of the pipeline to catch issues early. Mills also stresses the importance of integrating security teams with development teams through structures like technical guilds to build a culture of security.
Clean architecture with Python
I gave this presentation Jan 24th 2019 here:
https://www.meetup.com/BAyPIGgies/events/257819623/
The code can be found here:
https://github.com/jairvercosa/ca_auth
DevSecOps | DevOps Sec
DevSecOps enables organizations to deliver secure software at DevOps speed.
Development - Software releases and updates
Operations - Reliability Performance & Scaling
Security - Confidentiality, Availability, Integrity
DevSecops: Defined, tools, characteristics, tools, frameworks, benefits and c...
In this presentation, it is outlined about DevOps, DevSecOps, Characteristics of DevSecOps, DevSecops Practises, Benefits of Implementing DevSecOps, Implementation Frameworks and the Challenges in Implementing DevSecOps.
DevSecOps - The big picture
This document discusses the concepts of DevSecOps at a high level. It begins with a brief history of development methodologies, from Waterfall to Agile, and how Ops became a bottleneck. This led to trends in Agile Operations and collaboration between Dev and Ops, known as DevOps. DevSecOps expands this to incorporate security. It discusses the importance of culture, processes, and technologies for effective communication, automation, and collaboration across Dev, Ops, and Security. The goal is to enable organizations to deliver inherently secure software at DevOps speed through a high-trust environment and automated security pipelines integrated into the software development lifecycle.
DevSecOps Training Bootcamp - A Practical DevSecOps Course
DevSecOps means integrating security practices into the DevOps workflow from the beginning. The goal is to make everyone responsible for security and implement security decisions at the same speed as development and operations. This helps find vulnerabilities early and improve overall security. Implementing DevSecOps requires planning, building, deploying, monitoring and improving security continuously. It provides benefits like improved compliance and identifying issues earlier.
ABN AMRO DevSecOps Journey
This document summarizes ABN AMRO's DevSecOps journey and initiatives. It discusses their implementation of continuous integration and delivery pipelines to improve software quality, reduce lead times, and increase developer productivity. It also covers their work to incorporate security practices like open source software management, container security, and credentials management into the development lifecycle through techniques like dependency scanning, security profiling, and a centralized secrets store. The presentation provides status updates on these efforts and outlines next steps to further mature ABN AMRO's DevSecOps capabilities.
Benefits of DevSecOps
Brief of benefits of DevSecOps
DevSecOps = Security +DevOps
DevSecOps = New culture + new skills + automation
Code Quality - Security
Quality of software code for a given product shipped effectively translates not only to its functional quality but as well to its non functional aspects say security. Many of the issues in code can be addressed much before they reach SCM.
Dev week cloud world conf2021
Talk to executives in IT divisions of large enterprises about security and invariably the conversation will hover around
DevSecOps pipeline.
Is DevSecOps the only thing you need to do for security in your IT division or is there more?
What impact does bringing in secure culture in an engineering context mean?
What handshake is needed between the IT function and the security / risk function for large enterprises?
How does this impact roles and responsibilities of a developer?
This talk is an attempt to answer questions such as these using a real world examples of transformations seen in Fortune 100 companies.
DevSecOps, The Good, Bad, and Ugly
This document discusses DevSecOps and covers the good, bad, and ugly aspects. DevSecOps aims to integrate security practices into the development lifecycle like threat modeling, security testing, and monitoring. The good aspects include finding vulnerabilities early through testing and reviewable infrastructure policies. The bad parts are potential performance issues and loss of availability from tools. The ugly challenges are misunderstandings causing disasters, unstable new tools causing false alarms, and responsibility over security. Overall, DevSecOps is about people, process, and integrating security throughout the development lifecycle rather than just tools.
Strengthen and Scale Security Using DevSecOps - OWASP Indonesia
Strengthen and Scale Security Using DevSecOps - OWASP Indonesia
More details here - https://www.practical-devsecops.com/
Zero to Ninety in Securing DevOps
Delivered at DevSecOps Days 2018, RSA Conference
j. Wolfgang Goerlich
About J. Wolfgang Goerlich
About J Wolfgang Goerlich
CBI (Creative Breakthroughs, Inc.)
Cyber Security Strategist
J Wolfgang Goerlich provides strategic guidance for securing development and DevOps programs in the healthcare, education, financial services, and energy. He is currently with CBI, a cyber security consultancy, as the VP for strategic security programs. Wolfgang also leads the CBI Academy teams, providing mentoring and coaching to the junior-level talent. Prior roles included VP for a managed security services provider, VP for an IT firm specializing in high speed high secure networks, and IT security officer and manager for a financial services firm. He is an active part of the security community; co-founding the Converge Detroit and organizing the BSides Detroit conferences. Wolfgang regularly advises on and presents on the topics of secure development life cycle, DevOps, risk management, incident response, business continuity, and more.
Introduction to DevSecOps
An introduction to the devsecops webinar will be presented by me at 10.30am EST on 29th July,2018. It's a session focussed on high level overview of devsecops which will be followed by intermediate and advanced level sessions in future.
Agenda:
-DevSecOps Introduction
-Key Challenges, Recommendations
-DevSecOps Analysis
-DevSecOps Core Practices
-DevSecOps pipeline for Application & Infrastructure Security
-DevSecOps Security Tools Selection Tips
-DevSecOps Implementation Strategy
-DevSecOps Final Checklist
PIACERE - DevSecOps Automated
This document summarizes the PIACERE project, which aims to integrate security into DevSecOps processes. It receives funding from the EU Horizon 2020 program. The project develops tools like the DevSecOps Modeling Language (DOML) and Verification Tool to integrate security principles into infrastructure modeling and deployment. It also includes a Canary Sandbox Environment for testing deployments and an Infrastructure Optimization Platform for optimizing cloud resources. The overall goal is to provide a unified platform for secure, automated deployment to multiple clouds.
What's hot (20)
DevSecOps: essential tooling to enable continuous security 2019-09-16
DevSecOps: essential tooling to enable continuous security 2019-09-16
DevSecops: Defined, tools, characteristics, tools, frameworks, benefits and c...
DevSecops: Defined, tools, characteristics, tools, frameworks, benefits and c...
DevSecOps Training Bootcamp - A Practical DevSecOps Course
DevSecOps Training Bootcamp - A Practical DevSecOps Course
Strengthen and Scale Security Using DevSecOps - OWASP Indonesia
Strengthen and Scale Security Using DevSecOps - OWASP Indonesia
Similar to Microservices: why you're doing them wrong_Dmytro Lahoza
infrastructure management at digital ages
This document discusses infrastructure management in the digital era and the transition to DevOps practices. It provides the following key points:
1) Infrastructure management is evolving from traditional techniques to new ways of working inspired by software development practices like infrastructure as code.
2) Infrastructure as code uses configuration management tools and version control to define infrastructure in code for provisioning, deployment, and updates.
3) For infrastructure managers to support DevOps, their role is shifting from solely managing infrastructure to providing a digital toolbox and optimized experiences through services like a managed cloud platform.
Realtime search at Yammer
This presentation is focused on the architecture, scalability concerns, performance bottlenecks, operational characteristics and lessons learned while designing and implementing Yammer distributed real-time search system. Yammer is an enterprise social network SaaS offering with over 100,000 networks (including 85% of the Fortune 100) and nearly 2 million users. The search system we developed scales well up to 1B messages and serves a foundation of knowledge base analysis services Yammer is developing.
Real Time Search at Yammer
Presentation on the architecture, scalability concerns, performance bottlenecks, operational characteristics and lessons learned while designing and implementing Yammer distributed real-time search system.
Real-time Search at Yammer - By Aleksandrovsky Boris
See conference video - http://www.lucidimagination.com/devzone/events/conferences/revolution/2011
This talk will be focused on the architecture, scalability concerns, performance bottlenecks,
operational characteristics and lessons learned while designing and implementing Yammer
distributed real-time search system. Yammer is an enterprise social network SaaS offering with over
100,000 networks (including 85% of the Fortune 100) and nearly 2 million users. The search system
we developed scales well up to 1B messages and serves a foundation of knowledge base analysis
services Yammer is developing.
Cytoscape CI Chapter 2
The Cytoscape Cyberinfrastructure extends Cytoscape and its community into web-connected services.The CI is a Service Oriented Architecture that supports network biology oriented computations that can be orchestrated into repeatable workflows.
Kerberos, Token and Hadoop
It introduces and illustrates use cases, benefits and problems for Kerberos deployment on Hadoop; how Token support and TokenPreauth can help solve the problems. It also briefly introduces Haox project, a Java client library for Kerberos.
Banking malware zeu s zombies are using in online banking theft.
Video: https://www.youtube.com/watch?v=VE-w-AsfcGk
I'm take picture from here and there by goggling not mentioning all source please let me know if anyone has any objection. This presentation was presented in “securITy” Information Security Conference at BASIS SoftExpo 2014,Digital world 2014
Cloud Computing Was Built for Web Developers—What Does v2 Look Like for Deep...
What we call the public cloud was developed primarily to manage and deploy web servers. The target audience for these products is Dev Ops. While this is a massive and exciting market, the world of Data Science and Deep Learning is very different — and possibly even bigger. Unfortunately, the tools available today are not designed for this new audience and the cloud needs to evolve. This talk would cover what the next 10 years of cloud computing will look like.
Von JavaEE auf Microservice in 6 Monaten - The Good, the Bad, and the wtfs...
This document summarizes a talk about transitioning from JavaEE monoliths to microservices architecture in 6 months. It discusses the reasons for moving to microservices (faster development and deployment, lower costs), and the challenges including organizing configuration, communication between services, and deployment. It then outlines the steps taken to implement microservices at a company, including setting up continuous integration, using Spring Boot and Cloud, and establishing vertical feature teams to overcome organizational barriers. The key lessons are that the transition does not require a "big bang", can start with a single service, and works best by automating the development and deployment process from development to production.
JCConf.tw 2022 - DevOps for Java developers
In recent years, how we approach development has dramatically changed with the rise of DevOps, Cloud Computing, and Container technologies. Many stakeholders at our organizations are ecstatic about Shift Left, DevSecOps and VSM, but how are we as Java developers embracing this new cultural shift?
In this session we will explore the most common Critical Challenges to Adopt DevOps Culture in
Software Organizations, best practices and some excellent news with success stories.
Microservices Corporate Style
Micro service architecture (MSA) is an approach to building software systems that decomposes business domain models into smaller, consistent, bounded-contexts implemented by services.
Typically implemented and operated by small teams.
Switching from SOAP to REST doesn’t make a micro services architecture.
Micro services are not a technology-only discussion.
Ultralight Data Movement for IoT with SDC Edge
Edge computing and the Internet of Things bring great promise, but often just getting data from the edge requires moving mountains. Let's learn how to make edge data ingestion and analytics easier using StreamSets Data Collector edge, an ultralight, platform independent and small-footprint Open Source solution written in Go for streaming data from resource-constrained sensors and personal devices (like medical equipment or smartphones) to Apache Kafka, Amazon Kinesis and many others. This talk includes an overview of the SDC Edge main features, supported protocols and available processors for data transformation, insights on how it solves some challenges of traditional approaches to data ingestion, pipeline design basics, a walk-through some practical applications (Android devices and Raspberry Pi) and its integration with other technologies such as Streamsets Data Collector, Apache Kafka, Apache Hadoop, InfluxDB and Grafana. The goal here is to make attendees ready to quickly become IoT data intake and SDC Edge Ninjas.
Speaker
Guglielmo Iozzia, Big Data Delivery Manager, Optum (United Health)
Meetup Devops-Geneva-19.10.2019
Bonjour à tous,
Pour ce meetup, nous avons la chance d'être reçu dans les locaux de Richemont.
Je remercie particulièrement Cédric Georg ainsi que l'équipe de Richemont pour leur accueil.
A ce meetup DevOps, nous aurons 2 Retours d'Expérience, voici l'agenda de la soirée:
18:30 - Ouverture des portes
(il faudra donner votre nom et prénom ainsi que votre numéro de plaque d'immatriculation si vous êtes venu en voiture, c'est pour la sécurité, et oui, on ne rigole pas ici :-))
18:50 - Introduction de Matthieu et de Cédric
19:00 - Richemont et sa transformation DevOps
Richemont, fort de sa transformation digitale, a dû s'adapter afin de faire travailler ensemble, avec des outils d'automatisation et de communication, les équipes de développeurs et les équipes opérationnelles.
Squad, DevOps, Tests, Sécurité, Agile et Scrum, comment tous ces termes ont sû devenir le quotidien de Richemont en seulement quelques années.
Nous verrons comment nous avons mis cela en place, quels ont été les points positifs et négatifs de cette transformation.
19:40 - SixSq et l'automatisation du docker sur des edge points (DEMO)
Edge computing is gaining in popularity to address the explosion of data produced by IoT sensors, and the need to better manage AI both in the cloud and at the edge. To address this paradigm shift, SixSq has launched two open source projects: Nuvla for managing applications, and NuvlaBox, a cloud-in-a-box edge solution.
Using these open source projects, in this session we'll demonstrate how edge computing can now be integrated to agnostically operate containerized applications on CaaS infrastructures anywhere, using a Raspberry Pi-based platform.
Grails and the World of Tomorrow
A look at the changing development landscape and how we may have to rearchitect our Grails applications.
Also looks at existing, new, or potential Grails features that can help navigate this new world order.
Building an IoT Massive Multiplayer Game in 60 Minutes - TechBash 2017
The slide deck for my session at TechBash 2017 on "Building an IoT Massive Multiplayer Game in 60 Minutes".
Sample code can be found at https://github.com/estiller/iot-game
Cloud Security and OSI layer 2 - the layer oft forgotten
The document discusses security concerns regarding the virtual layer in cloud computing. It notes that while cloud service providers provide some security measures like encryption, the virtualization layer introduces additional risks that are often overlooked. It identifies several specific security risks in areas like privileged access, network architecture within the cloud, and vulnerabilities specific to virtualization platforms. The document argues that thorough understanding of the virtual layer is needed to properly assess security risks when using cloud computing.
Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)
This session does a quick recap of microservices: why do we want them, what problems do they solve and what are the principles around designing and implementing them? The Dapr.io runtime framework for distributed applications is introduced. Dapr provides a sidecar (almost like a personal assistant to a manager) to an application or microservice, a companion process that handles common tasks such as storing and retrieving state, consuming and publishing messages and events, invoking external services and other microservices as well as handling incoming requests. Participants will do a handson lab with Dapr.io and learn how to quickly implement interactions with various technologies, including Redis and MySQL.
Node(JS) is introduced – a server side JavaScript-based programming language that can be used well for implementing microservices. Some of the main characteristics of NodeJS are discussed (functional programming, asynchronous flows, NPM package manager) as well as common use cases (handle incoming HTTP requests, invoke REST APIs). In the second lab, Node and Dapr are used together to implement microservices that interact with databases and message brokers and each other – in a decoupled fashion.
Microservices Patterns with GoldenGate
Deep-dive into Microservices Patterns with Replication and Stream Analytics
Target Audience: Microservices and Data Architects
This is an informational presentation about microservices event patterns, GoldenGate event replication, and event stream processing with Oracle Stream Analytics. This session will discuss some of the challenges of working with data in a microservices architecture (MA), and how the emerging concept of a “Data Mesh” can go hand-in-hand to improve microservices-based data management patterns. You may have already heard about common microservices patterns like CQRS, Saga, Event Sourcing and Transaction Outbox; we’ll share how GoldenGate can simplify these patterns while also bringing stronger data consistency to your microservice integrations. We will also discuss how complex event processing (CEP) and stream processing can be used with event-driven MA for operational and analytical use cases.
Business pressures for modernization and digital transformation drive demand for rapid, flexible DevOps, which microservices address, but also for data-driven Analytics, Machine Learning and Data Lakes which is where data management tech really shines. Join us for this presentation where we take a deep look at the intersection of microservice design patterns and modern data integration tech.
What it feels like to live in a Security Enabled DevOps World
Security in DevOps world - Evolving frameworks. Cluster Hardening best practices. Automation pipelines for managing infrastructure and PaaS. Continuous Security and DevOps Maturity Model.
Secure Your DevOps Pipeline Best Practices Meetup 08022024.pptx
Our technology, work processes, and activities all depend on if we trust our software to be developed in a safe and secure manner. Join us virtually for our upcoming "Secure Your DevOps Pipeline: Best Practices" Meetup to learn how to integrate security in the development process, DevSecOps advance methods, manage the implement secure coding analysis and how to manage software security risks.
Similar to Microservices: why you're doing them wrong_Dmytro Lahoza (20)
Real-time Search at Yammer - By Aleksandrovsky Boris
Real-time Search at Yammer - By Aleksandrovsky Boris
Banking malware zeu s zombies are using in online banking theft.
Banking malware zeu s zombies are using in online banking theft.
Cloud Computing Was Built for Web Developers—What Does v2 Look Like for Deep...
Cloud Computing Was Built for Web Developers—What Does v2 Look Like for Deep...
Von JavaEE auf Microservice in 6 Monaten - The Good, the Bad, and the wtfs...
Von JavaEE auf Microservice in 6 Monaten - The Good, the Bad, and the wtfs...
Building an IoT Massive Multiplayer Game in 60 Minutes - TechBash 2017
Building an IoT Massive Multiplayer Game in 60 Minutes - TechBash 2017
Cloud Security and OSI layer 2 - the layer oft forgotten
Cloud Security and OSI layer 2 - the layer oft forgotten
Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)
Microservices, Node, Dapr and more - Part One (Fontys Hogeschool, Spring 2022)
What it feels like to live in a Security Enabled DevOps World
What it feels like to live in a Security Enabled DevOps World
Secure Your DevOps Pipeline Best Practices Meetup 08022024.pptx
Secure Your DevOps Pipeline Best Practices Meetup 08022024.pptx
More from Katherine Golovinova
Contract-based Testing Approach as a Tool for Shift Lef
The document discusses contract based testing and shifting testing left. It describes testing at different levels, including UI, integration, and unit testing. It outlines how to implement contract testing between a consumer and provider by creating pacts, publishing them to a broker, and having the provider verify against the pacts. Benefits include apps working together continuously, avoiding duplicative work, and visibility into dependencies. The presentation encourages attendees to try out contract testing.
Speed up application testing with azure container instances
SPEAKER: Anton Boyko, Founder and main speaker of the Ukrainian Microsoft Azure Community.
TOPIC DESCRIPTION:
We recently faced an issue with our test automation on a huge project. We had become victims of our own success: we were focused on eliminating manual testing efforts by increasing automated test coverage, but at some point, our single 8CPU/32GB test agent was not enough anymore. We used to wait for more than an hour for all our tests to run. We started thinking about how to improve the situation. The concept that will be presented is about utilizing Azure Container Instances as a hosting platform for spinning up multiple parallel environments and running all our test automation against them in parallel. This session will be useful for those who want to reduce total product cost by reducing team members’ idle time. I will share code examples that you will be able to use in your own environment.
Analyzing application activities with KSQL and Elasticsearch
IEVGENII VLASYUK, Delivery manager @EPAM
Capturing application events in distributed system is becoming a more and more common task. The modern Kafka eco-system can help to solve this task in an easy and elegant way. A bunch of already-created sources and sinks, SQL syntax and ease of joining data streams will save a lot of time and reduce complexity. However, capturing data is only half the battle. We will also explore how to make use of Elasticsearch to provide advanced analysis of user activities.
Testing Big Data solutions fast and furiously
DMYTRO SOBKO, Lead automation QA engineer @EPAM.
We are well aware of how to test the REST API with N endpoints, with relational and non-relational (NonSQL) databases. Same thing with UI testing. Frameworks like Selenium, Selenide, Selenoid are not a mystery to anyone. Moreover, creating a reliable, extensible and really cool automated test framework for such applications from scratch is not difficult. But what about BigData projects that have no back-end or front-end in the classical sense? How can we test them? What parts should we cover with tests in the first place? And, besides, how do we introduce automation and make it an effective way for such projects?
Dmytro will show you how to create a test framework for Cloud Big Data projects from scratch and to develop it in the most optimal way using the most interesting technologies.
"Fast & Fail in real life of DevTestSecOps"
Kostiantyn Severenchuk is a Systems Architect with 10+ years’ experience in IT production, including support/QA-QC/System Engineering & DevOps.
TOPIC
TOPIC DESCRIPTION:
Let’s talk about best practices and how they are helping us to adapt/survive and resolve any issues asap; real-life examples and implementations will be provided.
Link to video: https://epa.ms/devops-meetup-nov2019
Geodistributed databases - what, how, and why?
SPEAKER:
Yevhen Nedaskivskyi is a professional with over ten years of experience in databases. MCP, MCSA, MCSE. He was twice awarded the Microsoft Data Platform. Regular speaker at IT conferences and seminars. Co-organizer of Ukrainian Data Community Kyiv. He also teaches databases at Igor Sikorsky Kyiv Polytechnic Institute.
In this session, we will review geographically distributed databases and their architecture. The speaker will tell about products that exist on market, their features and principles of work, and he will share his own experience in the design and implementation of similar solutions.
Attenders will learn why such technologies are needed, what tasks they solve, and what principles they are based on.
The lecturer will also tell where to start if there is a need to implement a geodistributed database on your project.
The style is aimed at a wide range of listeners, and the presentation requires minimal knowledge of DBMS principals and database theories.
COSMOS DB - geodistributed database for anyone
SPEAKER:
Illia Lubenets is a Lead Software Engineer at EPAM. Microsoft Azure MVP. Co-founder of .NET Core Ukrainian User Group. He is interested in distributed systems development, mainly based on Microsoft Azure. Active meetup and conference organizer and speaker.
COSMOS DB is an interesting database developed by Microsoft. In this session, we will look at what opportunities it provides and what problems can be solved with this database.
Since this database is an SAAS service, its performance depends on how much you pay for it. The speaker will tell you how to minimize the coefficient in this dependency and what approaches you should take when working with COSMOS.
Migrating from a monolith to microservices – is it worth it?
IURII IVON, EPAM Solution Architect, Microsoft Competency Center Expert.
The term ‘microservices’ has become so popular that many people see it as a silver bullet for all architectural problems, or at least as a trend that should be followed. If your project is a monolith today, does it make sense to move towards microservices? This presentation overviews painful issues to be considered when migrating from a monolith to microservice architecture, ways to solve them, and ideas on the feasibility of such migration.
Azure Functions - the evolution of microservices platform or marketing gibber...
Azure Functions - the evolution of microservices platform or marketing gibber...Katherine Golovinova
ANTON BOYKO, Founder and main speaker of Ukrainian Microsoft Azure Community.
When many people hear the word microservices, they think of Kubernetes. And it’s hard to blame them: the utilization of Docker containerization together with Kubernetes orchestration really seems like a match made in heaven.
But, there is a new name in town: serverless (or FaaS). It seems it can offer you all the stuff that Kubernetes has and more. Native support for .NET, Java, Python, Node. Out-of-the-box binding for things like HTTP incoming requests (for Web API), incoming queue message (for processing async tasks), time-based trigger, etc. It also comes with micro billing (you pay only for execution time, not for idle time) and 0-to-infinite scalability. But is it all too good to be true? Aren't there any drawbacks? Is it all a huge marketing scheme to make us actually pay more? Let's find out.Gatling and Page Object: a way to performance testing
By Ravil Ianbekov at Automation in Action: summer conference.
Video: https://youtu.be/ambUpPYepL4
TOPIC DESCRIPTION
This talk will be interesting for those who have a lot of experience in UI testing but would like to start testing performance. I will outline the capabilities of the Gatling framework and how to build test logic.
Automation of Security scanning easy or cheese
By Karen Florykian at Automation in Action: summer conference.
Video: https://youtu.be/4fUwEvnFo_Q
TOPIC DESCRIPTION
I will share my experience of SDLC enablement on the enterprise level. In the process I will reveal pitfalls and gotchas about the building of a developer-friendly CI-enabled service using industry standard static and dynamic scanning tools, CI platforms, ReportPortal, Carrier platform and Jira integration service.
Gradle plugins for Test Automation
By Maksym Barvinskyi at Automation in Action: summer conference.
Video: https://youtu.be/YoIJ8AcS9Og
TOPIC DESCRIPTION
In this talk we are going to meet a powerful tool – Gradle plugins – and explore what kinds of problems they can solve in Test Automation. These could be additional quality gates for an automation framework build, or automatic code generation of SUT API endpoints and entities that could be used in tests, or other areas depending on your needs. We will create a couple of plugins right there during the talk and discuss their benefits and drawbacks when compared to other options. By the end, you are going to have one more tool in your Test Automation toolset, and be better prepared for solving different kinds of automation problems in your current and future projects.
Automation world under the DevTestSecOps umbrella
By Kostiantyn Severenchuk at Automation in Action: summer conference.
Video: https://youtu.be/mvbElkLtY0Y
TOPIC DESCRIPTION
The DevTestSecOps approach and its implementation on real projects. How to cook it, how to eat it, and its value. Let’s dive deep into the world of automation and its coverage with real examples. Why it is so important? Bonus funny stories as well.
"Disaster Recovery in Azure" by Viktor Kocherha
Do you need to protect your Azure environment? I will talk about the solutions Azure offers for addressing disaster recovery needs, and share real experience on how these can be applied on a project.
"Certified Kubernetes Administrator Exam – how it was" by Andrii Fedenishin
A talk about my experience of passing the CKA exam. I will provide you with advice on how to properly prepare for the exam and describe the strategy you need to follow to pass it successfully.
"Modern CI/CD" by Dmytro Batiievskyi
Requirements, approaches, and tools for CI/CD have been changing actively in recent years. I will provide an overview of the changes we see happening, plus discuss new challenges and ways to address these.
EPAM DevOps community meetup: Building CI/CD for microservice architecture
Building small microservice based architecture is easy. When architecture consists of large set of microservices, you will face a number of challenges.
We will share our approach how to build CI/CD for a complex build, test and deploy procedures for microservice based architecures.
EPAM DevOps community meetup: Designing bare metal Kubernetes clusters
Practical approach for designing and deploying Kubernetes cluster on commodity hardware. Caveats with high availability, networking and storage on bare metal clusters.
Hosting Microservices in Microsoft Azure
Global Azure Bootcamp | Kharkiv, Ukraine.
By Sergii Kryshtop, Software Engineering Manager @EPAM Ukraine.
Infrastructure as Code for Azure: ARM or Terraform?
Global Azure Bootcamp 2018 | Kharkiv, UA.
By Serhii Abanichev, Senior Systems Engineer @EPAM Ukraine
More from Katherine Golovinova (20)
Contract-based Testing Approach as a Tool for Shift Lef
Contract-based Testing Approach as a Tool for Shift Lef
Speed up application testing with azure container instances
Speed up application testing with azure container instances
Analyzing application activities with KSQL and Elasticsearch
Analyzing application activities with KSQL and Elasticsearch
Migrating from a monolith to microservices – is it worth it?
Migrating from a monolith to microservices – is it worth it?
Azure Functions - the evolution of microservices platform or marketing gibber...
Azure Functions - the evolution of microservices platform or marketing gibber...
Gatling and Page Object: a way to performance testing
Gatling and Page Object: a way to performance testing
"Certified Kubernetes Administrator Exam – how it was" by Andrii Fedenishin
"Certified Kubernetes Administrator Exam – how it was" by Andrii Fedenishin
EPAM DevOps community meetup: Building CI/CD for microservice architecture
EPAM DevOps community meetup: Building CI/CD for microservice architecture
EPAM DevOps community meetup: Designing bare metal Kubernetes clusters
EPAM DevOps community meetup: Designing bare metal Kubernetes clusters
Infrastructure as Code for Azure: ARM or Terraform?
Infrastructure as Code for Azure: ARM or Terraform?
Recently uploaded
Essentials of Automations: Exploring Attributes & Automation Parameters
Building automations in FME Flow can save time, money, and help businesses scale by eliminating data silos and providing data to stakeholders in real-time. One essential component to orchestrating complex automations is the use of attributes & automation parameters (both formerly known as “keys”). In fact, it’s unlikely you’ll ever build an Automation without using these components, but what exactly are they?
Attributes & automation parameters enable the automation author to pass data values from one automation component to the next. During this webinar, our FME Flow Specialists will cover leveraging the three types of these output attributes & parameters in FME Flow: Event, Custom, and Automation. As a bonus, they’ll also be making use of the Split-Merge Block functionality.
You’ll leave this webinar with a better understanding of how to maximize the potential of automations by making use of attributes & automation parameters, with the ultimate goal of setting your enterprise integration workflows up on autopilot.
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Dmitrii Kamaev, PhD
Senior Product Owner - QIAGEN
What is an RPA CoE? Session 2 – CoE Roles
In this session, we will review the players involved in the CoE and how each role impacts opportunities.
Topics covered:
• What roles are essential?
• What place in the automation journey does each role play?
Speaker:
Chris Bolin, Senior Intelligent Automation Architect Anika Systems
Apps Break Data
How information systems are built or acquired puts information, which is what they should be about, in a secondary place. Our language adapted accordingly, and we no longer talk about information systems but applications. Applications evolved in a way to break data into diverse fragments, tightly coupled with applications and expensive to integrate. The result is technical debt, which is re-paid by taking even bigger "loans", resulting in an ever-increasing technical debt. Software engineering and procurement practices work in sync with market forces to maintain this trend. This talk demonstrates how natural this situation is. The question is: can something be done to reverse the trend?
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...Pitangent Analytics & Technology Solutions Pvt. Ltd
Discover top-tier mobile app development services, offering innovative solutions for iOS and Android. Enhance your business with custom, user-friendly mobile applications.Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Gursev Pirge, PhD
Senior Data Scientist - JohnSnowLabs
Session 1 - Intro to Robotic Process Automation.pdf
👉 Check out our full 'Africa Series - Automation Student Developers (EN)' page to register for the full program:
https://bit.ly/Automation_Student_Kickstart
In this session, we shall introduce you to the world of automation, the UiPath Platform, and guide you on how to install and setup UiPath Studio on your Windows PC.
📕 Detailed agenda:
What is RPA? Benefits of RPA?
RPA Applications
The UiPath End-to-End Automation Platform
UiPath Studio CE Installation and Setup
💻 Extra training through UiPath Academy:
Introduction to Automation
UiPath Business Automation Platform
Explore automation development with UiPath Studio
👉 Register here for our upcoming Session 2 on June 20: Introduction to UiPath Studio Fundamentals: https://community.uipath.com/events/details/uipath-lagos-presents-session-2-introduction-to-uipath-studio-fundamentals/
ScyllaDB Tablets: Rethinking Replication
ScyllaDB is making a major architecture shift. We’re moving from vNode replication to tablets – fragments of tables that are distributed independently, enabling dynamic data distribution and extreme elasticity. In this keynote, ScyllaDB co-founder and CTO Avi Kivity explains the reason for this shift, provides a look at the implementation and roadmap, and shares how this shift benefits ScyllaDB users.
JavaLand 2024: Application Development Green Masterplan
My presentation slides I used at JavaLand 2024
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
Direct losses from downtime in 1 minute = $5-$10 thousand dollars. Reputation is priceless.
As part of the talk, we will consider the architectural strategies necessary for the development of highly loaded fintech solutions. We will focus on using queues and streaming to efficiently work and manage large amounts of data in real-time and to minimize latency.
We will focus special attention on the architectural patterns used in the design of the fintech system, microservices and event-driven architecture, which ensure scalability, fault tolerance, and consistency of the entire system.
From Natural Language to Structured Solr Queries using LLMs
This talk draws on experimentation to enable AI applications with Solr. One important use case is to use AI for better accessibility and discoverability of the data: while User eXperience techniques, lexical search improvements, and data harmonization can take organizations to a good level of accessibility, a structural (or “cognitive” gap) remains between the data user needs and the data producer constraints.
That is where AI – and most importantly, Natural Language Processing and Large Language Model techniques – could make a difference. This natural language, conversational engine could facilitate access and usage of the data leveraging the semantics of any data source.
The objective of the presentation is to propose a technical approach and a way forward to achieve this goal.
The key concept is to enable users to express their search queries in natural language, which the LLM then enriches, interprets, and translates into structured queries based on the Solr index’s metadata.
This approach leverages the LLM’s ability to understand the nuances of natural language and the structure of documents within Apache Solr.
The LLM acts as an intermediary agent, offering a transparent experience to users automatically and potentially uncovering relevant documents that conventional search methods might overlook. The presentation will include the results of this experimental work, lessons learned, best practices, and the scope of future work that should improve the approach and make it production-ready.
"Choosing proper type of scaling", Olena Syrota
Imagine an IoT processing system that is already quite mature and production-ready and for which client coverage is growing and scaling and performance aspects are life and death questions. The system has Redis, MongoDB, and stream processing based on ksqldb. In this talk, firstly, we will analyze scaling approaches and then select the proper ones for our system.
Main news related to the CCS TSI 2023 (2023/1695)
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
Must Know Postgres Extension for DBA and Developer during Migration
Mydbops Opensource Database Meetup 16
Topic: Must-Know PostgreSQL Extensions for Developers and DBAs During Migration
Speaker: Deepak Mahto, Founder of DataCloudGaze Consulting
Date & Time: 8th June | 10 AM - 1 PM IST
Venue: Bangalore International Centre, Bangalore
Abstract: Discover how PostgreSQL extensions can be your secret weapon! This talk explores how key extensions enhance database capabilities and streamline the migration process for users moving from other relational databases like Oracle.
Key Takeaways:
* Learn about crucial extensions like oracle_fdw, pgtt, and pg_audit that ease migration complexities.
* Gain valuable strategies for implementing these extensions in PostgreSQL to achieve license freedom.
* Discover how these key extensions can empower both developers and DBAs during the migration process.
* Don't miss this chance to gain practical knowledge from an industry expert and stay updated on the latest open-source database trends.
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: info@mydbops.com
Visit: https://www.mydbops.com/
Follow us on LinkedIn: https://in.linkedin.com/company/mydbops
For more details and updates, please follow up the below links.
Meetup Page : https://www.meetup.com/mydbops-databa...
Twitter: https://twitter.com/mydbopsofficial
Blogs: https://www.mydbops.com/blog/
Facebook(Meta): https://www.facebook.com/mydbops/
AppSec PNW: Android and iOS Application Security with MobSF
Mobile Security Framework - MobSF is a free and open source automated mobile application security testing environment designed to help security engineers, researchers, developers, and penetration testers to identify security vulnerabilities, malicious behaviours and privacy concerns in mobile applications using static and dynamic analysis. It supports all the popular mobile application binaries and source code formats built for Android and iOS devices. In addition to automated security assessment, it also offers an interactive testing environment to build and execute scenario based test/fuzz cases against the application.
This talk covers:
Using MobSF for static analysis of mobile applications.
Interactive dynamic security assessment of Android and iOS applications.
Solving Mobile app CTF challenges.
Reverse engineering and runtime analysis of Mobile malware.
How to shift left and integrate MobSF/mobsfscan SAST and DAST in your build pipeline.
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Tomaz Bratanic
Graph ML and GenAI Expert - Neo4j
Northern Engraving | Nameplate Manufacturing Process - 2024
Manufacturing custom quality metal nameplates and badges involves several standard operations. Processes include sheet prep, lithography, screening, coating, punch press and inspection. All decoration is completed in the flat sheet with adhesive and tooling operations following. The possibilities for creating unique durable nameplates are endless. How will you create your brand identity? We can help!
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
Recently uploaded (20)
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Crafting Excellence: A Comprehensive Guide to iOS Mobile App Development Serv...
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
From Natural Language to Structured Solr Queries using LLMs
From Natural Language to Structured Solr Queries using LLMs
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Microservices: why you're doing them wrong_Dmytro Lahoza
- 2. What is microservices Small as RaspberryPI Written in NodeJS Docker, Kubernetes etc.
- 3. What is microservices Function calls are too reliable nowadays Transactions are boring 50 API contracts is not enough
- 5. Solve Team size Distributed team problems Team intercommunication
- 7. Problems You need stable APIs and communication contracts No more shared DB no you need consensus Testing! Strong DevOps culture Service discovery Networking Monitoring and instrumentation Distributed tracing
- 8. API and contracts Swagger RAML Synchronization Validation gRPC?
- 9. Data No more state, no more local files No more single database (MongoDB, Cassandra ?) No more system-wide transactions No more fast and reliable point of consensus (Zookeeper ?) You need messaging (RabbitMQ, ZeroMQ ?)
- 10. Testing No one cares about testing single service Now you have 50 points of failure No one really knows how to test microservices (hard mocking) No one really knows how to test infrastructure for microservices
- 11. DevOps It is about culture, not people You create it, you support it Culture should cover all areas of software lifecycle You cannot deploy 50 microservices manually Service could be written in any language Tons of new shiny tools (Docker, Kubernetes, Terraform)
- 12. Networking Distributed networking is cool (ha!) Discovery is simple (ha!!!) Configuration distribution is dope (ha!!!) Circuit breaker? What is that?
- 13. Monitoring No more single system, no more centralized monitoring Distributed tracing (Zipkin) Logs and metrics collecting (Beats, Statsd) Aggregation (Logstash, Carbon) Visualization (Prometheus, Graphite, Kibana) Store and maintain all this data and systems (SRE?) Incident management (ITIL, Support)
- 14. Security Internal request security (SSL, JWT?) Sensitive data storing and delivery (Vault?) No more access to single instance and fix production app
- 15. Questions?
- 16. Dmytro Lahoza GDFS Inc. Twitter: @dlagoza