This document discusses tools and techniques for optimizing Ruby performance. It begins by looking at common expensive tasks like database operations, network access, and inefficient algorithms. It then discusses tools for benchmarking and profiling Ruby code like Benchmark, benchmark-ips, and stackprof. The document provides examples of optimizing ActiveRecord queries and using caching and memoization. It also discusses optimizing the environment through server, database, and caching configuration. Finally, it notes that in some CPU-intensive or async tasks, Ruby may not be the best tool.
Erik Wendel - Beyond JavaScript Frameworks: Writing Reliable Web Apps With El...Codemotion
In times where a jungle of JavaScript frameworks wants to solve every conceivable problem in web app development, Elm offers a different approach. Elm is a functional language that compiles to JavaScript. It has a user-friendly compiler, a sound type system, built-in immutability and lots of other features that come in handy when developing large, hopefully bug-free, single-page apps. While having fun in the process! In this talk you'll see how Elm works and learn how to use it to build a web app. More importantly, you'll learn the pros and cons of using it over a JavaScript-based solution.
The talk gives an example on how to introduce the Job DSL Plugin in a real life company. After explaining the importance of Configuration as Code and why we use the Job DSL Plugin for it, I will show how to take it to the next level. That means Configuration as Code does not automatically solve all configuration problems, there are a lot of pitfalls people need to look out for. On the one hand, you have to give people the freedom they need, one the other you always have to assure quality and stability at the same time.
The document discusses the DevOpsSec approach which aims to integrate security testing into the development process through automation. It outlines how DevOpsSec can help address issues that arise from the traditional separation of development and operations teams. The document provides examples of different types of tests that can be automated, such as unit testing, performance testing, and security testing of an application's attack surface. It promotes automating as many tests as possible and sharing test automation code to continuously monitor for vulnerabilities and issues.
This document introduces Phoenix, a web framework for building scalable and fault-tolerant distributed systems with Elixir and Erlang. It discusses how Moore's Law has led to more multi-core machines requiring better support for concurrency. Phoenix provides productivity benefits like Rails while enabling applications to handle massive concurrency through Elixir and Erlang's actor model and lightweight processes. The document demonstrates building basic and real-time web apps with Phoenix as well as using it as the web layer for distributed systems.
Mobile Library Development - stuck between a pod and a jar file - Zan Markan ...Codemotion
Isaac Newton, the father of modern software engineering, called it “Standing on the shoulders of giants”. Modern development is exciting as it gets easier and easier, partly because of the wealth of resources available at our fingertips. One category of these resources are libraries, SDKs, and frameworks. This talk will be a guide into the considerations that go into building a library for both iOS/Swift and Java/Android. We will be taking cues from both my personal experience, as well as from studying how the leaders in the field do it.
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensCitus Data
Postgres is a powerful database, it continues to improve in terms of performance, extensibility, and more broadly in features. However it is not perfect.
Here I'll cover a highly opinionated view of all the areas Postgres falls flat, with some rough thought ideas on how we can make it better. Opinions are all informed by 10 years of interacting with customers running literally millions of databases for users.
This document discusses tools and techniques for optimizing Ruby performance. It begins by looking at common expensive tasks like database operations, network access, and inefficient algorithms. It then discusses tools for benchmarking and profiling Ruby code like Benchmark, benchmark-ips, and stackprof. The document provides examples of optimizing ActiveRecord queries and using caching and memoization. It also discusses optimizing the environment through server, database, and caching configuration. Finally, it notes that in some CPU-intensive or async tasks, Ruby may not be the best tool.
Erik Wendel - Beyond JavaScript Frameworks: Writing Reliable Web Apps With El...Codemotion
In times where a jungle of JavaScript frameworks wants to solve every conceivable problem in web app development, Elm offers a different approach. Elm is a functional language that compiles to JavaScript. It has a user-friendly compiler, a sound type system, built-in immutability and lots of other features that come in handy when developing large, hopefully bug-free, single-page apps. While having fun in the process! In this talk you'll see how Elm works and learn how to use it to build a web app. More importantly, you'll learn the pros and cons of using it over a JavaScript-based solution.
The talk gives an example on how to introduce the Job DSL Plugin in a real life company. After explaining the importance of Configuration as Code and why we use the Job DSL Plugin for it, I will show how to take it to the next level. That means Configuration as Code does not automatically solve all configuration problems, there are a lot of pitfalls people need to look out for. On the one hand, you have to give people the freedom they need, one the other you always have to assure quality and stability at the same time.
The document discusses the DevOpsSec approach which aims to integrate security testing into the development process through automation. It outlines how DevOpsSec can help address issues that arise from the traditional separation of development and operations teams. The document provides examples of different types of tests that can be automated, such as unit testing, performance testing, and security testing of an application's attack surface. It promotes automating as many tests as possible and sharing test automation code to continuously monitor for vulnerabilities and issues.
This document introduces Phoenix, a web framework for building scalable and fault-tolerant distributed systems with Elixir and Erlang. It discusses how Moore's Law has led to more multi-core machines requiring better support for concurrency. Phoenix provides productivity benefits like Rails while enabling applications to handle massive concurrency through Elixir and Erlang's actor model and lightweight processes. The document demonstrates building basic and real-time web apps with Phoenix as well as using it as the web layer for distributed systems.
Mobile Library Development - stuck between a pod and a jar file - Zan Markan ...Codemotion
Isaac Newton, the father of modern software engineering, called it “Standing on the shoulders of giants”. Modern development is exciting as it gets easier and easier, partly because of the wealth of resources available at our fingertips. One category of these resources are libraries, SDKs, and frameworks. This talk will be a guide into the considerations that go into building a library for both iOS/Swift and Java/Android. We will be taking cues from both my personal experience, as well as from studying how the leaders in the field do it.
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensCitus Data
Postgres is a powerful database, it continues to improve in terms of performance, extensibility, and more broadly in features. However it is not perfect.
Here I'll cover a highly opinionated view of all the areas Postgres falls flat, with some rough thought ideas on how we can make it better. Opinions are all informed by 10 years of interacting with customers running literally millions of databases for users.
Immutable Server generation: The new App DeploymentAxel Fontaine
From JavaZone 2014
Video: https://vimeo.com/105851488
Abstract:
App deployment and server setup are complex, error-prone and time-consuming. They require OS installers, package managers, configuration recipes, install and deployment scripts, server tuning, hardening and more. But... Is this really necessary? Are we trapped in a mindset of doing things this way just because that's how they've always done?
What if we could start over and radically simplify all this? What if, within seconds, and with a single command, we could wrap our application into the bare minimal machine required to run it? What if this machine could then be transported and run unchanged on our laptop and in the cloud? How do the various tools like Docker and Boxfuse fit into this picture? What are their strengths and weaknesses? When should you use them?
This talk is for developers and architects wishing to radically improve and simplify how they deploy their applications. It takes Continuous Delivery to a level far beyond what you've seen today. Welcome to Immutable Server generation. This is the new black.
When it all goes wrong | PGConf EU 2019 | Will LeinweberCitus Data
This document summarizes a presentation about troubleshooting Postgres performance problems. It discusses how to determine if the issue is with the database, system resources, or the application. It provides examples of common problems like running out of CPU, memory, disk, or parallelism. It also recommends tools to diagnose issues like perf, gdb, iostat, iotop, htop, bwm-ng, and pg_stat_statements. Finally, it discusses setting boundaries around economics, workload, performance, and errors to avoid instability.
When it all goes wrong (with Postgres) | RailsConf 2019 | Will LeinweberCitus Data
You're woken up in the middle of the night to your phone. Your app is down and you're on call to fix it. Eventually you track it down to "something with the db," but what exactly is wrong? And of course, you're sure that nothing changed recently…
Knowing what to fix, and even where to start looking, is a skill that takes a long time to develop. Especially since Postgres normally works very well for months at a time, not letting you get practice!
In this talk, I'll share not only the more common failure cases and how to fix them, but also a general approach to efficiently figuring out what's wrong in the first place.
Best practices say: Software testing should start early in the Software Development Life Cycle. Since automation testing follows SDLC we should start from API (when UI has not ready yet). On practice, most projects start from UI automation. It’s can be quite a hard and painful way especially for manual test engineers who only begin automate testing. API automation is much simple and easier to learn. During the talk, Roman share his knowledge and experience how quickly start API automation testing on the project.
Karate - Web-Service API Testing Made SimpleVodqaBLR
Karate is a DSL for writing web service API acceptance tests that is built on Cucumber and uses BDD syntax. It allows tests to be written in a concise and readable way while enabling faster test execution and better assertion capabilities. Karate tests can be easily integrated into existing CI/CD pipelines.
The document outlines the new features of Java EE 7 including JSF 2.2, CDI 1.1, EJB 3.2, JPA 2.1, Bean Validation 1.1, JMS 2.0, JSON processing, batch applications, JAX-RS 2.0, and the Java API for WebSockets. It also advertises job openings and provides contact information.
While we’re drawing ever closer to Java 9, and even hearing about features in Java 10, it’s also true that many of us are still working with an older version. Even if your project has technically adopted Java 8, and even if you’re using it when coding new features, it’s likely the majority of your code base is still not making the most of what’s available in Java 8 - features like Lambda Expressions, the Streams API, and new Date/Time.
In this presentation, Trisha:
- Highlights some of the benefits of using Java 8 - after all, you’ll probably have to persuade The Management that tampering with existing code is worthwhile
- Demonstrates how to identify areas of code that can be updated to use Java 8 features
- Shows how to automatically refactor your code to make use of features like lambdas and streams.
- Covers some of the pros and cons of using the new features - including suggestions of when refactoring may NOT be the best idea.
Stackato is a PaaS cloud platform from ActiveState that allows developers to easily deploy applications to the cloud. It supports multiple languages including Perl, Ruby, and JavaScript. The presentation demonstrated deploying simple Perl apps to Stackato using the Mojolicious framework. Key benefits of Stackato include minimal differences between development and production environments, one-click deployments, and allowing developers to manage infrastructure. ActiveState is very open and provides documentation, examples, and a community forum to support Stackato users.
Nowadays, TDD, BDD, continuous testing and other methodologies have come into our attention when developing. Yet, we barely know what needs to be tested and why are we testing it? During the talk we will go through a bunch of testing methodologies.
Abstract:
A apresentação centra-se na temática de ter forma de controlar, versionar e actualizar toda a parte de Base de Dados de um projecto. Estamos a falar, desde a produção de modelos ER, a versionamento de scripts, passando pelo deploy dos mesmos e terminado na documentação. A apresentação conta ainda com uma breve demonstração do uso da ferramenta Flyway para versionar e controlar a execução de scripts nos diversos ambientes de um projecto.
Sobre o Nuno Alves:
Chamo-me Nuno Alves nascido em Coimbra, Portugal e vivi maioritariamente em Leiria. Licenciado em Engenharia Informática na ESTG-IPLeiria (Escola Superior de Tecnologia e Gestão) onde o gosto por dados e bases de dados se começou a desenvolver. Daí, profissionalmente a minha área de actuação ser em torno de bases de dados e infra-estruturas. Tenho cerca de 10 anos de experiência repartidos pelas áreas Financeira, Seguros, Governo, Militar em tecnologias que vão desde Oracle, PostgreSQL, MSSQLServer a DB2.
Alberto Maria Angelo Paro - Isomorphic programming in Scala and WebDevelopmen...Codemotion
Scala is the only language that can be used to produce code that can be "trans/compiled" for the JVM, in Javascript and in native binary. This allows to write libraries that are usable in JVM and JS using the power of functional programming (i.e. cats, scalaz), generic programming (i.e. shapeless) and macro/scalameta available in Scala. In this talk, we will see how to write a Scala application backend and a SPA (scala.js/scala-js-react) that share the same code as a business logic, datamodels and transparent API call (JVM/JS) in Scala (via autowire/akka-http/circe).
Eric Lubow discussed key questions to consider when building an infrastructure architecture. He outlined questions around where systems will live, how data will move and be stored, and how to ensure systems can be maintained and scaled. Lubow emphasized the importance of monitoring systems to understand if anything is wrong and highlighted lessons learned from SimpleReach's architecture evolution.
Karate - powerful and simple framework for REST API automation testingRoman Liubun
Karate is a powerful yet simple framework for REST API automation testing. It uses a behavior-driven development style with tests written in a domain-specific language that resembles simple English. Key features include reusable tests, schema validation, data-driven testing with examples, parallel test execution, and built-in reporting. While requiring learning a new language, Karate provides many capabilities out of the box and removes the need for complex programming.
This document discusses strategies for securely operating Docker containers. It begins by acknowledging valid security concerns with Docker and advocating understanding threat models and limitations. The document then explores issues of trust in Docker images and potential solutions like auditing tools and private registries. It outlines opportunities to automate security configuration checks and generate AppArmor profiles. Finally, it presents a vision for a registry of security profiles that could reduce the burden of container hardening. Overall, the document takes an empathetic approach to security concerns while highlighting practical steps and potential technical improvements.
A Deep Dive into the W3C WebDriver SpecificationPeter Thomas
This document provides an overview of WebDriver including:
- It describes WebDriver as a W3C specification for remotely controlling user agents through a REST API.
- It outlines the WebDriver architecture including drivers, bindings, and frameworks.
- It discusses the history and evolution of Selenium to the current WebDriver standard.
This document summarizes several stories about debugging production issues in web applications. It describes common problems like services hanging, exceptions, and high memory usage. It then walks through the troubleshooting process for each story, including the steps taken to identify the issue and how it was fixed. The document highlights various tools used for debugging, such as Performance Monitor, Process Explorer, Fiddler, DebugDiag, and WinDbg. It emphasizes starting with understanding the problem and being able to reproduce it before choosing the appropriate debugging techniques and tools.
Serverless in production, an experience report (LNUG)Yan Cui
AWS Lambda has changed the way we deploy and run software, but this new serverless paradigm has created new challenges to old problems - how do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures?
In this talk Yan and Scott will discuss solutions to these challenges by drawing from real-world experience running Lambda in production and migrating from an existing monolithic architecture.
Love it or hate it (and a lot of people seem to hate it), Maven is a widely used tool. We can consider that Maven has been the de-facto standard build tool for Java over the last 10 years. Most experienced developers already got their share of Maven headaches. Unfortunately, new developers are going through the same hard learning process, because they don't know how to deal with Maven particularities. "Why is this jar in my build?", "I can’t see my changes!", "The jar is not included in the distribution!", "The artifact was not found!" are common problems. Learn to tame the Maven Beast and be in complete control of your build to save you countless hours of pain and frustration.
The document discusses the 12 factor app methodology for building software-as-a-service applications. It covers the 12 factors which are codebase, dependencies, configuration, backing services, build/release/run, processes, port binding, concurrency, disposability, development/production parity, logs, and admin processes. The presentation provides examples and explanations of how to design apps adhering to these 12 factors for scalability, maintainability and portability.
Johan Edstrom discussed scaling applications by making them more asynchronous and distributed. He covered several Apache projects like Camel, Karaf, ActiveMQ, Cassandra and CXF that can help achieve this. Specifically, he showed how to:
1. Use Camel and OSGi to build asynchronous microservices that communicate via an enterprise integration pattern like a message queue.
2. Store data in Cassandra for asynchronous and high-performance access across data centers.
3. Expose APIs asynchronously using CXF and handle requests with non-blocking techniques like futures.
4. Offload business logic to an asynchronous process running on a distributed cache like HazelCast for shared data and parallel processing across nodes
This document discusses NoSQL database security issues. It begins by introducing NoSQL and big data concepts. It then covers common NoSQL databases and explains why they are popular. However, it notes that NoSQL solutions are often not designed with security in mind by default. Some key security issues with NoSQL databases include weak authentication, insecure password storage, lack of authorization controls, and vulnerabilities to injection attacks. The document provides examples of these issues and recommends ways to secure NoSQL installations, such as validating inputs, defining a trusted environment, and continuing to sanitize for traditional and NoSQL-specific attacks.
Immutable Server generation: The new App DeploymentAxel Fontaine
From JavaZone 2014
Video: https://vimeo.com/105851488
Abstract:
App deployment and server setup are complex, error-prone and time-consuming. They require OS installers, package managers, configuration recipes, install and deployment scripts, server tuning, hardening and more. But... Is this really necessary? Are we trapped in a mindset of doing things this way just because that's how they've always done?
What if we could start over and radically simplify all this? What if, within seconds, and with a single command, we could wrap our application into the bare minimal machine required to run it? What if this machine could then be transported and run unchanged on our laptop and in the cloud? How do the various tools like Docker and Boxfuse fit into this picture? What are their strengths and weaknesses? When should you use them?
This talk is for developers and architects wishing to radically improve and simplify how they deploy their applications. It takes Continuous Delivery to a level far beyond what you've seen today. Welcome to Immutable Server generation. This is the new black.
When it all goes wrong | PGConf EU 2019 | Will LeinweberCitus Data
This document summarizes a presentation about troubleshooting Postgres performance problems. It discusses how to determine if the issue is with the database, system resources, or the application. It provides examples of common problems like running out of CPU, memory, disk, or parallelism. It also recommends tools to diagnose issues like perf, gdb, iostat, iotop, htop, bwm-ng, and pg_stat_statements. Finally, it discusses setting boundaries around economics, workload, performance, and errors to avoid instability.
When it all goes wrong (with Postgres) | RailsConf 2019 | Will LeinweberCitus Data
You're woken up in the middle of the night to your phone. Your app is down and you're on call to fix it. Eventually you track it down to "something with the db," but what exactly is wrong? And of course, you're sure that nothing changed recently…
Knowing what to fix, and even where to start looking, is a skill that takes a long time to develop. Especially since Postgres normally works very well for months at a time, not letting you get practice!
In this talk, I'll share not only the more common failure cases and how to fix them, but also a general approach to efficiently figuring out what's wrong in the first place.
Best practices say: Software testing should start early in the Software Development Life Cycle. Since automation testing follows SDLC we should start from API (when UI has not ready yet). On practice, most projects start from UI automation. It’s can be quite a hard and painful way especially for manual test engineers who only begin automate testing. API automation is much simple and easier to learn. During the talk, Roman share his knowledge and experience how quickly start API automation testing on the project.
Karate - Web-Service API Testing Made SimpleVodqaBLR
Karate is a DSL for writing web service API acceptance tests that is built on Cucumber and uses BDD syntax. It allows tests to be written in a concise and readable way while enabling faster test execution and better assertion capabilities. Karate tests can be easily integrated into existing CI/CD pipelines.
The document outlines the new features of Java EE 7 including JSF 2.2, CDI 1.1, EJB 3.2, JPA 2.1, Bean Validation 1.1, JMS 2.0, JSON processing, batch applications, JAX-RS 2.0, and the Java API for WebSockets. It also advertises job openings and provides contact information.
While we’re drawing ever closer to Java 9, and even hearing about features in Java 10, it’s also true that many of us are still working with an older version. Even if your project has technically adopted Java 8, and even if you’re using it when coding new features, it’s likely the majority of your code base is still not making the most of what’s available in Java 8 - features like Lambda Expressions, the Streams API, and new Date/Time.
In this presentation, Trisha:
- Highlights some of the benefits of using Java 8 - after all, you’ll probably have to persuade The Management that tampering with existing code is worthwhile
- Demonstrates how to identify areas of code that can be updated to use Java 8 features
- Shows how to automatically refactor your code to make use of features like lambdas and streams.
- Covers some of the pros and cons of using the new features - including suggestions of when refactoring may NOT be the best idea.
Stackato is a PaaS cloud platform from ActiveState that allows developers to easily deploy applications to the cloud. It supports multiple languages including Perl, Ruby, and JavaScript. The presentation demonstrated deploying simple Perl apps to Stackato using the Mojolicious framework. Key benefits of Stackato include minimal differences between development and production environments, one-click deployments, and allowing developers to manage infrastructure. ActiveState is very open and provides documentation, examples, and a community forum to support Stackato users.
Nowadays, TDD, BDD, continuous testing and other methodologies have come into our attention when developing. Yet, we barely know what needs to be tested and why are we testing it? During the talk we will go through a bunch of testing methodologies.
Abstract:
A apresentação centra-se na temática de ter forma de controlar, versionar e actualizar toda a parte de Base de Dados de um projecto. Estamos a falar, desde a produção de modelos ER, a versionamento de scripts, passando pelo deploy dos mesmos e terminado na documentação. A apresentação conta ainda com uma breve demonstração do uso da ferramenta Flyway para versionar e controlar a execução de scripts nos diversos ambientes de um projecto.
Sobre o Nuno Alves:
Chamo-me Nuno Alves nascido em Coimbra, Portugal e vivi maioritariamente em Leiria. Licenciado em Engenharia Informática na ESTG-IPLeiria (Escola Superior de Tecnologia e Gestão) onde o gosto por dados e bases de dados se começou a desenvolver. Daí, profissionalmente a minha área de actuação ser em torno de bases de dados e infra-estruturas. Tenho cerca de 10 anos de experiência repartidos pelas áreas Financeira, Seguros, Governo, Militar em tecnologias que vão desde Oracle, PostgreSQL, MSSQLServer a DB2.
Alberto Maria Angelo Paro - Isomorphic programming in Scala and WebDevelopmen...Codemotion
Scala is the only language that can be used to produce code that can be "trans/compiled" for the JVM, in Javascript and in native binary. This allows to write libraries that are usable in JVM and JS using the power of functional programming (i.e. cats, scalaz), generic programming (i.e. shapeless) and macro/scalameta available in Scala. In this talk, we will see how to write a Scala application backend and a SPA (scala.js/scala-js-react) that share the same code as a business logic, datamodels and transparent API call (JVM/JS) in Scala (via autowire/akka-http/circe).
Eric Lubow discussed key questions to consider when building an infrastructure architecture. He outlined questions around where systems will live, how data will move and be stored, and how to ensure systems can be maintained and scaled. Lubow emphasized the importance of monitoring systems to understand if anything is wrong and highlighted lessons learned from SimpleReach's architecture evolution.
Karate - powerful and simple framework for REST API automation testingRoman Liubun
Karate is a powerful yet simple framework for REST API automation testing. It uses a behavior-driven development style with tests written in a domain-specific language that resembles simple English. Key features include reusable tests, schema validation, data-driven testing with examples, parallel test execution, and built-in reporting. While requiring learning a new language, Karate provides many capabilities out of the box and removes the need for complex programming.
This document discusses strategies for securely operating Docker containers. It begins by acknowledging valid security concerns with Docker and advocating understanding threat models and limitations. The document then explores issues of trust in Docker images and potential solutions like auditing tools and private registries. It outlines opportunities to automate security configuration checks and generate AppArmor profiles. Finally, it presents a vision for a registry of security profiles that could reduce the burden of container hardening. Overall, the document takes an empathetic approach to security concerns while highlighting practical steps and potential technical improvements.
A Deep Dive into the W3C WebDriver SpecificationPeter Thomas
This document provides an overview of WebDriver including:
- It describes WebDriver as a W3C specification for remotely controlling user agents through a REST API.
- It outlines the WebDriver architecture including drivers, bindings, and frameworks.
- It discusses the history and evolution of Selenium to the current WebDriver standard.
This document summarizes several stories about debugging production issues in web applications. It describes common problems like services hanging, exceptions, and high memory usage. It then walks through the troubleshooting process for each story, including the steps taken to identify the issue and how it was fixed. The document highlights various tools used for debugging, such as Performance Monitor, Process Explorer, Fiddler, DebugDiag, and WinDbg. It emphasizes starting with understanding the problem and being able to reproduce it before choosing the appropriate debugging techniques and tools.
Serverless in production, an experience report (LNUG)Yan Cui
AWS Lambda has changed the way we deploy and run software, but this new serverless paradigm has created new challenges to old problems - how do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures?
In this talk Yan and Scott will discuss solutions to these challenges by drawing from real-world experience running Lambda in production and migrating from an existing monolithic architecture.
Love it or hate it (and a lot of people seem to hate it), Maven is a widely used tool. We can consider that Maven has been the de-facto standard build tool for Java over the last 10 years. Most experienced developers already got their share of Maven headaches. Unfortunately, new developers are going through the same hard learning process, because they don't know how to deal with Maven particularities. "Why is this jar in my build?", "I can’t see my changes!", "The jar is not included in the distribution!", "The artifact was not found!" are common problems. Learn to tame the Maven Beast and be in complete control of your build to save you countless hours of pain and frustration.
The document discusses the 12 factor app methodology for building software-as-a-service applications. It covers the 12 factors which are codebase, dependencies, configuration, backing services, build/release/run, processes, port binding, concurrency, disposability, development/production parity, logs, and admin processes. The presentation provides examples and explanations of how to design apps adhering to these 12 factors for scalability, maintainability and portability.
Johan Edstrom discussed scaling applications by making them more asynchronous and distributed. He covered several Apache projects like Camel, Karaf, ActiveMQ, Cassandra and CXF that can help achieve this. Specifically, he showed how to:
1. Use Camel and OSGi to build asynchronous microservices that communicate via an enterprise integration pattern like a message queue.
2. Store data in Cassandra for asynchronous and high-performance access across data centers.
3. Expose APIs asynchronously using CXF and handle requests with non-blocking techniques like futures.
4. Offload business logic to an asynchronous process running on a distributed cache like HazelCast for shared data and parallel processing across nodes
This document discusses NoSQL database security issues. It begins by introducing NoSQL and big data concepts. It then covers common NoSQL databases and explains why they are popular. However, it notes that NoSQL solutions are often not designed with security in mind by default. Some key security issues with NoSQL databases include weak authentication, insecure password storage, lack of authorization controls, and vulnerabilities to injection attacks. The document provides examples of these issues and recommends ways to secure NoSQL installations, such as validating inputs, defining a trusted environment, and continuing to sanitize for traditional and NoSQL-specific attacks.
Test driven development for infrastructure as-a-code, the future trend_Gianfr...Katherine Golovinova
This document discusses testing infrastructure as code (IaaC) using test-driven development (TDD) principles. It recommends applying different types of automated tests for IaaC: unit tests to check for errors, integration tests to validate functionality, and acceptance/security tests on deployed infrastructure. Various tools are mentioned for testing IaaC written in languages like Terraform, Ansible, Chef, and Puppet at the unit, integration, and security levels. Adopting a testing mindset and tools can help catch errors and non-compliance early in development pipelines.
JDBC provides Java APIs to connect to different databases in a standardized way. It defines interfaces that database vendors implement to support connectivity. JDBC drivers act as middleware to translate JDBC calls to vendor-specific APIs. This allows software to connect to any database without being database-specific, unlike earlier approaches. The document explains the JDBC architecture, driver types, and basic steps to connect to a database and perform CRUD operations through examples.
1. The document discusses various optimizations that can be made to an ASP.NET MVC application to improve performance, including compiled LINQ queries, URL caching, and data caching.
2. Benchmark results show that optimizing partial view rendering, LINQ queries, and URL generation improved performance from 8 requests/second to 61.5 requests/second. Additional caching of URLs, statistics, and content improved performance to over 400 requests/second.
3. Turning off ASP.NET debug mode also provided a significant performance boost, showing the importance of running production sites in release mode.
Revolutionizing the Data Abstraction Layer with IBM Optim pureQuery and DB2Vladimir Bacvanski, PhD
Vladimir Bacvanski and Dan Galvin
Looking for a more flexible and efficient way for Java programs to access the database? Join us as we explore how you can
bridge the gap between Java and relational databases. Enhance your Java environment with access layer generation, data
access best practices, traceability between Java packages and SQL statements, improved impact analysis and more. And most importantly, see how new technology can improve not only new development, but existing applications as well. Be prepared to
see designs and code samples!
The document discusses scalable web architectures and common patterns for scaling web applications. It covers key topics like load balancing, caching, database replication, and data federation. The overall goal of application architecture is to scale traffic and data while maintaining high availability and performance. Horizontal scaling by adding more servers is preferable to vertical scaling of buying larger servers.
The document discusses scalable web architectures and common patterns for scaling web applications. It covers key topics like load balancing, caching, database replication and sharding, and asynchronous queuing to distribute workloads across multiple servers. The goal of these patterns is to scale traffic, data size, and maintainability through horizontal expansion rather than just vertical upgrades.
The document discusses scalable web architectures and common patterns for scaling web applications. It covers key topics like load balancing, caching, database replication and sharding, and asynchronous queuing to distribute workloads across multiple servers. The goal of these patterns is to scale traffic, data size, and maintainability through horizontal expansion rather than just vertical upgrades.
Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010Bhupesh Bansal
Jan 22nd, 2010 Hadoop meetup presentation on project voldemort and how it plays well with Hadoop at linkedin. The talk focus on Linkedin Hadoop ecosystem. How linkedin manage complex workflows, data ETL , data storage and online serving of 100GB to TB of data.
The document discusses Project Voldemort, a distributed key-value storage system developed at LinkedIn. It provides an overview of Voldemort's motivation and features, including high availability, horizontal scalability, and consistency guarantees. It also describes LinkedIn's use of Voldemort and Hadoop for applications like event logging, online lookups, and batch processing of large datasets.
A presentation on the selection criteria, testing + evaluation and successful, zero-downtime migration to MongoDB. Additionally details on Wordnik's speed and stability are covered as well as how NoSQL technologies have changed the way Wordnik scales.
My talk in Bessemer VP R&D / CTO yearly event (Jan 2020).
The presentation discusses major concept in resilience testing and MyHeritage's path to Chaos Engineering.
CI/CD with an Idempotent Kafka Producer & Consumer | Kafka Summit London 2022HostedbyConfluent
"Idempotence is a mathematical requirement of particular operations where the operation can be applied multiple times without changing the result beyond the initial application.
The main driver behind the idempotency requirement is often to handle duplicated messages. As developers and architects, we need to pay close attention to how we deal with our production data during new deployments to ensure we are not losing any data, duplicating messages, or introducing malformed data into our system. Furthermore, we need to figure out how to automate the process and add testing guarantees to prevent any potential human error.
In this session, you will learn about the idempotent Kafka Producer & Consumer architecture and how to automate the CI/CD process with open-source tools."
Real World Single Page App - A Knockout Case Studyhousecor
This presentation explores lessons learned from building a highly complex single page application that's used by 100's of automotive dealerships to finance and sell cars. We’ll walk through how to manage a pure client-side application with 1000’s of lines of custom JavaScript and review how Web API, Knockout, Durandal, RequireJS, KendoUI, and surprisingly little jQuery can join forces to make the browser sing. You’ll gain a clear understanding of when a single page app approach makes sense and learn how to pragmatically divide responsibilities between the client and server. This session will give you an appreciation for how far you can push ultra-responsive client-side rendering in the real-world.
Presentation given on NYC software engineering group [http://www.meetup.nycsoftware.org/events/102941112/] on Feb 07 2013.
An overview of the implementation of ElasticSearch as the new search and browse engine at emusic.com.
This talk shows the challenges that the team faced while putting this amazing solution to work.
By replacing a proprietary legacy Oracle Endeca product to ElasticSearch emusic was able to reduce by 400% the number of nodes used by the search engine, response times were down by 200% and with an 80% increase of traffic load.
We serve over 5 million requests a day on our search servers.
UnConference for Georgia Southern Computer Science March 31, 2015Christopher Curtin
I presented to the Georgia Southern Computer Science ACM group. Rather than one topic for 90 minutes, I decided to do an UnConference. I presented them a list of 8-9 topics, let them vote on what to talk about, then repeated.
Each presentation was ~8 minutes, (Except Career) and was by no means an attempt to explain the full concept or technology. Only to wake up their interest.
Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCCal Henderson
The document discusses common patterns and approaches for scaling web architectures. It covers topics like load balancing, caching, database scaling through replication and sharding, high availability, and storing large files across multiple servers and data centers. The overall goal is to discuss how to architect systems that can scale horizontally to handle increasing traffic and data sizes.
Similar to We're going on a bug hunt! Experts Talk Manchester 2018 (20)
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving
Manufacturing custom quality metal nameplates and badges involves several standard operations. Processes include sheet prep, lithography, screening, coating, punch press and inspection. All decoration is completed in the flat sheet with adhesive and tooling operations following. The possibilities for creating unique durable nameplates are endless. How will you create your brand identity? We can help!
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
The Microsoft 365 Migration Tutorial For Beginner.pptxoperationspcvita
This presentation will help you understand the power of Microsoft 365. However, we have mentioned every productivity app included in Office 365. Additionally, we have suggested the migration situation related to Office 365 and how we can help you.
You can also read: https://www.systoolsgroup.com/updates/office-365-tenant-to-tenant-migration-step-by-step-complete-guide/
Conversational agents, or chatbots, are increasingly used to access all sorts of services using natural language. While open-domain chatbots - like ChatGPT - can converse on any topic, task-oriented chatbots - the focus of this paper - are designed for specific tasks, like booking a flight, obtaining customer support, or setting an appointment. Like any other software, task-oriented chatbots need to be properly tested, usually by defining and executing test scenarios (i.e., sequences of user-chatbot interactions). However, there is currently a lack of methods to quantify the completeness and strength of such test scenarios, which can lead to low-quality tests, and hence to buggy chatbots.
To fill this gap, we propose adapting mutation testing (MuT) for task-oriented chatbots. To this end, we introduce a set of mutation operators that emulate faults in chatbot designs, an architecture that enables MuT on chatbots built using heterogeneous technologies, and a practical realisation as an Eclipse plugin. Moreover, we evaluate the applicability, effectiveness and efficiency of our approach on open-source chatbots, with promising results.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
"Choosing proper type of scaling", Olena SyrotaFwdays
Imagine an IoT processing system that is already quite mature and production-ready and for which client coverage is growing and scaling and performance aspects are life and death questions. The system has Redis, MongoDB, and stream processing based on ksqldb. In this talk, firstly, we will analyze scaling approaches and then select the proper ones for our system.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/how-axelera-ai-uses-digital-compute-in-memory-to-deliver-fast-and-energy-efficient-computer-vision-a-presentation-from-axelera-ai/
Bram Verhoef, Head of Machine Learning at Axelera AI, presents the “How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-efficient Computer Vision” tutorial at the May 2024 Embedded Vision Summit.
As artificial intelligence inference transitions from cloud environments to edge locations, computer vision applications achieve heightened responsiveness, reliability and privacy. This migration, however, introduces the challenge of operating within the stringent confines of resource constraints typical at the edge, including small form factors, low energy budgets and diminished memory and computational capacities. Axelera AI addresses these challenges through an innovative approach of performing digital computations within memory itself. This technique facilitates the realization of high-performance, energy-efficient and cost-effective computer vision capabilities at the thin and thick edge, extending the frontier of what is achievable with current technologies.
In this presentation, Verhoef unveils his company’s pioneering chip technology and demonstrates its capacity to deliver exceptional frames-per-second performance across a range of standard computer vision networks typical of applications in security, surveillance and the industrial sector. This shows that advanced computer vision can be accessible and efficient, even at the very edge of our technological ecosystem.
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...Jason Yip
The typical problem in product engineering is not bad strategy, so much as “no strategy”. This leads to confusion, lack of motivation, and incoherent action. The next time you look for a strategy and find an empty space, instead of waiting for it to be filled, I will show you how to fill it in yourself. If you’re wrong, it forces a correction. If you’re right, it helps create focus. I’ll share how I’ve approached this in the past, both what works and lessons for what didn’t work so well.
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
We're going on a bug hunt! Experts Talk Manchester 2018
1. Making Software. Better.
Simple solutions to big business problems.
Equal Experts is a network of talented, experienced, software
consultants, specialising in agile delivery.
2. The network
I’m part of the Equal Experts network,
a community of highly-experienced
software professionals.
At Equal Experts we’re given the
freedom to get on with what’s
important - delivering better software
for our clients’ customers.
3. We’re going on a Bug Hunt
Gerald Benischke
Software Engineer
8. Talking to Oracle Checklist
! Speak with the DBAs to check that the database is up
! Check the DB proxy that traffic is passing through
! Check whether any firewall / VPN changes were made
! Check Logs
! Check Stacktraces
! Check Metrics - and hope we’ve got decent ones!
! Check Logs Again
10. Get sidetracked - What does JMS have to do with a DB Bug?
WAR
Spring
Jms
Template
Active
MQ
Client
ELB
Camel
MQ
Bridge
Active
MQ
VPN
11. Why would that be connected?
jms.brokerUrl: "failover:(tcp://activemq-1:61616,tcp://activemq-2:61616,tcp://activemq-3:61616)?
maxReconnectAttempts=1&startupMaxReconnectAttempts=1"
October 26th 2017, 10:40:39.681 Waiting 30000 ms before attempting connection
October 26th 2017, 10:40:31.470 Attempting 1088th connect to: tcp://ip-10-10-10-217:61616
October 26th 2017, 10:40:09.623 Attempting 1084th connect to: tcp://ip-10-10-10-217:61616
October 26th 2017, 10:40:01.469 Waiting 30000 ms before attempting connection
October 26th 2017, 10:39:39.592 Attempting 1084th connect to: tcp://ip-10-10-10-25:61616
October 26th 2017, 10:39:31.434 Attempting 1087th connect to: tcp://ip-10-10-10-25:61616
October 26th 2017, 10:39:09.592 Waiting 30000 ms before attempting connection
October 26th 2017, 10:39:01.407 Attempting 1087th connect to: tcp://ip-10-10-10-217:61616
12. NO timeout?
! ActiveMQ was restarted - but bound to different DHCP addresses
! App lost connection - and waits forever in sending
15. Work on Sunday
! It works locally - against a DB running in docker
! It is broken on the server
! The settings have not been changed
! The stored procedure is the same
! The server versions are the same
! The driver versions are the same
! What else could it be?
! The logs are definitely showing the production server is
returning wrong results!
17. Oracle JDBC driver
Sets NLS_TERRITORY based on client Locale
Developer workstation was British
Server used Default Locale (US)
Fix the Application Servers settings
-Duser.language: en
-Duser.country: GB
20. Retries out of nowhere
! Backend team reports duplicate submissions
! Looking at the code, we don’t retry submissions
! What could cause it?
WAR
Http
Client
ELB
App
Proxy
VPN
28. Memory Leak Solved
! Due to corrupt data meant duplicate identifiers in a list
! A loop tried to find the “next” item in the list by identifier
and always found the first one - thereby entering an
infinite loop
! As part of the loop a Java object was added to another list
! The infinite loop kept going and adding to the list
! Eventually the JVM would only GC
! Healthcheck would kill the JVM eventually and restart
31. Entropy isn’t what it used to be
! When we moved to AWS - we experienced a dramatic
slow down in performance for SSL
! It turned out that in the old cloud the images had installed
havged package - which was missing from the new
! /dev/random blocked….
35. Thank You
United Kingdom
+44 203 603 7830
helloUK@equalexperts.com
Equal Experts UK Ltd
30 Brock Street
London NW1 3FG
India
+91 20 6607 7763
helloIndia@equalexperts.com
Equal Experts India Private Ltd
Office No. 4-C
Cerebrum IT Park No. B3
Kumar City, Kalyani Nagar
Pune, 411006
Canada
+1 403 775 4861
helloCanada@equalexperts.com
Equal Experts Devices Inc
205 - 279 Midpark way S.E.
T2X 1M2
Calgary, Alberta
Portugal
+351 211 378 414
helloPortugal@equalexperts.com
Equal Experts Portugal
Avenida Dom João II, Nº35
Edificio Infante 11ºA
1990-083 Parque das Nações
Lisboa – Portugal
Thank You
USA
+1 866-943-9737
helloUSA@equalexperts.com
Equal Experts Inc
1460 Broadway
New York
NY 10036
LinkedIn
linkedin.com/company/equal-experts
Twitter
@EqualExperts
Web
www.equalexperts.com