Talk given at CodeWorks PHP Conference in Atlanta on 9/29/2009.
Covers MP "migrations for php" project as well as other best-practices for managing database migrations for PHP projects.
BenchFlow: A Platform for End-to-end Automation of Performance Testing and An...Vincenzo Ferme
BenchFlow is an open-source expert system providing a complete platform for automating performance tests and performance analysis. We know that not all the developers are performance experts, but in nowadays agile environment, they need to deal with performance testing and performance analysis every day. In BenchFlow, the users define objective-driven performance testing using an expressive and SUT-aware DSL implemented in YAML. Then BenchFlow automates the end-to-end process of executing the performance tests and providing performance insights, dealing with system under test deployment relying on Docker technologies, distributing simulated users load on different server, error handling, performance data collection and performance metrics and insights computation.
My talk for SPEC Research Group DevOps (https://research.spec.org/devopswg) about BenchFlow. Discover BenchFlow: https://github.com/benchflow
Towards Holistic Continuous Software Performance AssessmentVincenzo Ferme
In agile, fast and continuous development lifecycles, software performance analysis is fundamental to confidently release continuously improved software versions. Researchers and industry practitioners have identified the importance of integrating performance testing in agile development processes in a timely and efficient way. However, existing techniques are fragmented and not integrated taking into account the heterogeneous skills of the users developing polyglot distributed software, and their need to automate performance practices as they are integrated in the whole lifecycle without breaking its intrinsic velocity. In this paper we present our vision for holistic continuous software performance assessment, which is being implemented in the BenchFlow tool. BenchFlow enables performance testing and analysis practices to be pervasively integrated in continuous development lifecycle activities. Users can specify performance activities (e.g., standard performance tests) by relying on an expressive Domain Specific Language for objective-driven performance analysis. Collected performance knowledge can be thus reused to speed up performance activities throughout the entire process.
My talk from The International Workshop on Quality-aware DevOps (QUDOS 2017). Cite us: http://dl.acm.org/citation.cfm?id=3053636
Collecting and Leveraging a Benchmark of Build System Clones to Aid in Qualit...Shane McIntosh
Build systems specify how sources are transformed into deliverables, and hence must be carefully maintained to ensure that deliverables are assembled correctly. Similar to source code, build systems tend to grow in complexity unless specifications are refactored. This paper describes how clone detection can aid in quality assessments that determine if and where build refactoring effort should be applied. We gauge cloning rates in build systems by collecting and analyzing a benchmark comprising 3,872 build systems. Analysis of the benchmark reveals that: (1) build systems tend to have higher cloning rates than other software artifacts, (2) recent build technologies tend to be more prone to cloning, especially of configuration details like API dependencies, than older technologies, and (3) build systems that have fewer clones achieve higher levels of reuse via mechanisms not offered by build technologies. Our findings aided in refactoring a large industrial build system containing 1.1 million lines.
Learn how to use Continuous Delivery for Puppet Enterprise (CD4PE) in an interactive workshop with hands-on labs. What's CD4PE? CD4PE is the continuous delivery add-on to Puppet Enterprise, aimed at accelerating the speed at which you can get Puppet code changes deployed into production safely. CD4PE facilitates code collaboration across teams, and dramatically improves the release management process for teams that own & maintain individual Puppet modules. CD4PE integrates with both Puppet Enterprise as well as your version control system of choice.
After completing the workshop, you will be able to use CD4PE to perform common code management tasks on your Puppet control repo and modules.
Workflow Engine Performance Benchmarking with BenchFlowVincenzo Ferme
As opposed to databases for which established benchmarks have been driving the advancement of the field since a long time, workflow engines still lack a well-accepted benchmark that allows to give a fair comparison of their performance. In this talk we discuss how BenchFlow addresses the main challenges related to benchmarking these complex middleware systems at the core of business process automation and service composition solutions. In particular, we look at how to define suitable workloads, representative performance metrics, and how to fully automate the execution of the performance experiments. Concerning the automation of experiments execution and analysis, we designed and implemented the BenchFlow framework. The framework automates the deployment of WfMS packaged within Docker containers. This way the initial configuration and conditions for each experiment can be precisely controlled and reproduced. Moreover, the BenchFlow framework supports heterogeneous WfMS APIs by providing an extensible plugin mechanism. During the benchmark execution it automatically deploys a set of BPMN models and invokes them according to parametric load functions specified declaratively. It automatically collects performance and resource consumption data, both on the driver and servers during the experiment as well as extracting them from the WfMS database afterwards. In addition to latency, throughput and resource utilisation, we compute multiple performance metrics that characterise the WfMS performance at the engine level, at the process level and at the BPMN construct level. To ensure reliability and improve usefulness of the obtained results, we automatically compute descriptive statistics and perform statistical tests to asses the homogeneity of results obtained from different repetitions of the same experiment.
The talk will also present experimental results obtained while benchmarking popular open source engines, using workflow patterns as a micro-benchmark.
Virtual Bolt workshop
Learn how to use Bolt in an interactive virtual workshop.
Join us for an interactive virtual Bolt workshop on Wednesday, 11 March 2020. You’ll learn how to install and configure common Bolt activities and leave with your laptops Puppet-ready, with Bolt + PDK + Puppet Agent + VS Code. Plus, you’ll get to speak with experts from Puppet and the community.
What's Bolt? Bolt is an open source, agentless multi-platform automation tool that reduces your time to automation and makes it easier to get started with DevOps. Bolt makes automation much more accessible without requiring any Puppet knowledge, agents, or master. It uses SSH or WinRM to communicate and execute tasks on remote systems.
Your teams can perform various tasks like starting and stopping services, rebooting remote systems, and gathering packages and systems facts from your workstation or laptop on any platform (Linux and Windows).
The web is growing fast and online games and other compute-intensive applications expand their user-bases every day. These trends create a growing demand on performance in JavaScript applications. Even though JavaScript has benefited from huge performance improvements in recent years, there is great potential for an even faster web. Browser technologies such as asm.js or Google’s NativeClient enable near-native performance in browsers. More recent efforts led to the specification of WebAssembly, the future vendor-independent standard for native webapps. This talk takes a closer look at Emscripten, a toolchain that transpiles C++ to asm.js. The talk answers questions regarding performance and about the integration with the popular CMake build system.
BenchFlow: A Platform for End-to-end Automation of Performance Testing and An...Vincenzo Ferme
BenchFlow is an open-source expert system providing a complete platform for automating performance tests and performance analysis. We know that not all the developers are performance experts, but in nowadays agile environment, they need to deal with performance testing and performance analysis every day. In BenchFlow, the users define objective-driven performance testing using an expressive and SUT-aware DSL implemented in YAML. Then BenchFlow automates the end-to-end process of executing the performance tests and providing performance insights, dealing with system under test deployment relying on Docker technologies, distributing simulated users load on different server, error handling, performance data collection and performance metrics and insights computation.
My talk for SPEC Research Group DevOps (https://research.spec.org/devopswg) about BenchFlow. Discover BenchFlow: https://github.com/benchflow
Towards Holistic Continuous Software Performance AssessmentVincenzo Ferme
In agile, fast and continuous development lifecycles, software performance analysis is fundamental to confidently release continuously improved software versions. Researchers and industry practitioners have identified the importance of integrating performance testing in agile development processes in a timely and efficient way. However, existing techniques are fragmented and not integrated taking into account the heterogeneous skills of the users developing polyglot distributed software, and their need to automate performance practices as they are integrated in the whole lifecycle without breaking its intrinsic velocity. In this paper we present our vision for holistic continuous software performance assessment, which is being implemented in the BenchFlow tool. BenchFlow enables performance testing and analysis practices to be pervasively integrated in continuous development lifecycle activities. Users can specify performance activities (e.g., standard performance tests) by relying on an expressive Domain Specific Language for objective-driven performance analysis. Collected performance knowledge can be thus reused to speed up performance activities throughout the entire process.
My talk from The International Workshop on Quality-aware DevOps (QUDOS 2017). Cite us: http://dl.acm.org/citation.cfm?id=3053636
Collecting and Leveraging a Benchmark of Build System Clones to Aid in Qualit...Shane McIntosh
Build systems specify how sources are transformed into deliverables, and hence must be carefully maintained to ensure that deliverables are assembled correctly. Similar to source code, build systems tend to grow in complexity unless specifications are refactored. This paper describes how clone detection can aid in quality assessments that determine if and where build refactoring effort should be applied. We gauge cloning rates in build systems by collecting and analyzing a benchmark comprising 3,872 build systems. Analysis of the benchmark reveals that: (1) build systems tend to have higher cloning rates than other software artifacts, (2) recent build technologies tend to be more prone to cloning, especially of configuration details like API dependencies, than older technologies, and (3) build systems that have fewer clones achieve higher levels of reuse via mechanisms not offered by build technologies. Our findings aided in refactoring a large industrial build system containing 1.1 million lines.
Learn how to use Continuous Delivery for Puppet Enterprise (CD4PE) in an interactive workshop with hands-on labs. What's CD4PE? CD4PE is the continuous delivery add-on to Puppet Enterprise, aimed at accelerating the speed at which you can get Puppet code changes deployed into production safely. CD4PE facilitates code collaboration across teams, and dramatically improves the release management process for teams that own & maintain individual Puppet modules. CD4PE integrates with both Puppet Enterprise as well as your version control system of choice.
After completing the workshop, you will be able to use CD4PE to perform common code management tasks on your Puppet control repo and modules.
Workflow Engine Performance Benchmarking with BenchFlowVincenzo Ferme
As opposed to databases for which established benchmarks have been driving the advancement of the field since a long time, workflow engines still lack a well-accepted benchmark that allows to give a fair comparison of their performance. In this talk we discuss how BenchFlow addresses the main challenges related to benchmarking these complex middleware systems at the core of business process automation and service composition solutions. In particular, we look at how to define suitable workloads, representative performance metrics, and how to fully automate the execution of the performance experiments. Concerning the automation of experiments execution and analysis, we designed and implemented the BenchFlow framework. The framework automates the deployment of WfMS packaged within Docker containers. This way the initial configuration and conditions for each experiment can be precisely controlled and reproduced. Moreover, the BenchFlow framework supports heterogeneous WfMS APIs by providing an extensible plugin mechanism. During the benchmark execution it automatically deploys a set of BPMN models and invokes them according to parametric load functions specified declaratively. It automatically collects performance and resource consumption data, both on the driver and servers during the experiment as well as extracting them from the WfMS database afterwards. In addition to latency, throughput and resource utilisation, we compute multiple performance metrics that characterise the WfMS performance at the engine level, at the process level and at the BPMN construct level. To ensure reliability and improve usefulness of the obtained results, we automatically compute descriptive statistics and perform statistical tests to asses the homogeneity of results obtained from different repetitions of the same experiment.
The talk will also present experimental results obtained while benchmarking popular open source engines, using workflow patterns as a micro-benchmark.
Virtual Bolt workshop
Learn how to use Bolt in an interactive virtual workshop.
Join us for an interactive virtual Bolt workshop on Wednesday, 11 March 2020. You’ll learn how to install and configure common Bolt activities and leave with your laptops Puppet-ready, with Bolt + PDK + Puppet Agent + VS Code. Plus, you’ll get to speak with experts from Puppet and the community.
What's Bolt? Bolt is an open source, agentless multi-platform automation tool that reduces your time to automation and makes it easier to get started with DevOps. Bolt makes automation much more accessible without requiring any Puppet knowledge, agents, or master. It uses SSH or WinRM to communicate and execute tasks on remote systems.
Your teams can perform various tasks like starting and stopping services, rebooting remote systems, and gathering packages and systems facts from your workstation or laptop on any platform (Linux and Windows).
The web is growing fast and online games and other compute-intensive applications expand their user-bases every day. These trends create a growing demand on performance in JavaScript applications. Even though JavaScript has benefited from huge performance improvements in recent years, there is great potential for an even faster web. Browser technologies such as asm.js or Google’s NativeClient enable near-native performance in browsers. More recent efforts led to the specification of WebAssembly, the future vendor-independent standard for native webapps. This talk takes a closer look at Emscripten, a toolchain that transpiles C++ to asm.js. The talk answers questions regarding performance and about the integration with the popular CMake build system.
While CMake has become the de-facto standard buildsystem for C++, it's siblings CTest and CPack are less well known. This talk gives a lightspeed introduction into these three tools and then focuses on best practices on building, testing, and packaging.
Architecting the Future: Abstractions and Metadata - CodeStockDaniel Barker
Kubernetes and Docker are two of the top open source projects, and they’re built around abstractions and metadata. These two concepts are the key to architecting in the future. Come with me as I dig a little deeper into these concepts within k8s and Docker and provide some examples from my own work on Deployment Pipelines.
Jenkins is a Continuous Integration (CI) server or tool which is written in Java. It provides Continuous Integration services for software development, which can be started via command line or web application server. Jenkins Pipeline is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins.
Largely based on Vishnu Gopal's presentation http://www.slideshare.net/vishnu/basic-source-control-with-subversion
Used for a quick SVN introduction in a Software Engineering course at Massey University.
Analysis of merge requests in GitLab using PVS-Studio for C#Andrey Karpov
Do you like GitLab and don't like bugs? Do you want to improve the quality of your source code? Then you've come to the right place. Today we will tell you how to configure the PVS-Studio C# analyzer for checking merge requests. Enjoy the reading and have a nice unicorn mood.
Hyper-pragmatic Pure FP testing with distage-testkit7mind
Having a proper test suite can turn ongoing application maintenance and development into pure joy – the best tests check meaningful properties, not the implementation details, and hold no impliict assumptions about their test environment - every test case must be self-contained and portable. To ensure that tests are free of implementation details and environment dependency, we may simply run them in a different test environment, with different implementations of components. But the boileplate and manual work involved in rewiring components, writing hardcoded fixtures and setting up different test environments make this very hard to do at scale. To tackle this problem we've created distage & distage-testkit, distage-testkit gives you the following superpowers:
* ability to easily swap out individual components or entire test environments
* principled & leak-free control of global resources for integration testing – docker containers, DBs, DDLs
* execute effects or allocate resources per-test, e.g. generate random fixtures per-test
* first-class testing of functional effects
* write tests as lambdas – access test fixtures via parameters or ZIO Environment
...and more! We'll also discuss general testing practices and what really distinguishes good tests from great tests.
Many Scala developers nowadays consider using Dependency Injection frameworks an anti-pattern incompatible with modern FP settings. We argue that it's just a consequence of a bad experience with legacy Java runtime reflection-based implementations that lack features important for modern functional programming, such as a first-class support for higher-kinded types. We argue that as a paradigm for structuring purely functional programs, DI with automatic wiring compares favorably against implicits, monad transformers, free monads, algebraic effects, cake pattern et al, enabling scaling and a degree of modularity unachievable by any manual wiring approach. This talk covers DIStage – a transparent, flexible and efficient DI framework for Scala that enables late binding, testability, effect separation and modular resource management at scale, working with, instead of compromising the Scala type system.
Documentation: https://izumi.7mind.io/latest/release/doc/distage/
Video of the presentation: http://www.youtube.com/watch?v=8z3h4Uv9YbE
At LinkedIn, we have started to use the Play Framework to build front-end and back-end services at massive scale. Play does things a little differently: it's a Java and Scala web framework, but it doesn't follow the servlet spec; it's fairly new, but it runs on top of robust technologies like Akka and Netty; it uses a thread pool, but it's built for non-blocking I/O and reactive programming; most importantly, it's high performance, but also high productivity. We've found that the Play Framework is one of the few frameworks that is able to maintain the delicate balance of performance, reliability, and developer productivity. In the Java and Scala world, nothing even comes close. In this talk, I'll share what we've learned so far, including details of rapid iteration with Java and Scala, the story behind async I/O on the JVM, support for real time web apps (comet, WebSockets), and integrating Play into a large existing codebase.
Introduction to WP-CLI: Manage WordPress from the command lineBehzod Saidov
These are the slides from a talk given at WordCamp Riverside 2017:
https://2017.riverside.wordcamp.org/session/introduction-to-wp-cli-manage-wordpress-from-the-command-line/
The topic is about powerful tool WP-CLI than can help WordPress developers or power users a lot.
If you manage multiple WordPress websites or multiple environments of a WordPress website, you probably have experienced situations where you need to go through exact same steps on multiple websites. Clicking through wp-admin screens to change settings becomes time consuming. WP-CLI can be used to automate those tasks with ease.
During my presentation I’ll introduce the audience with the command line and WP-CLI, then I’ll go through some of the tasks that can be easily done using this tool. I’ll demonstrate some tasks running them on a sample WordPress website running in local development environment. Demonstrations will include tasks like pulling back the database to lower environments (qa/staging) and rewriting it, managing users, plugins, etc.
While CMake has become the de-facto standard buildsystem for C++, it's siblings CTest and CPack are less well known. This talk gives a lightspeed introduction into these three tools and then focuses on best practices on building, testing, and packaging.
Architecting the Future: Abstractions and Metadata - CodeStockDaniel Barker
Kubernetes and Docker are two of the top open source projects, and they’re built around abstractions and metadata. These two concepts are the key to architecting in the future. Come with me as I dig a little deeper into these concepts within k8s and Docker and provide some examples from my own work on Deployment Pipelines.
Jenkins is a Continuous Integration (CI) server or tool which is written in Java. It provides Continuous Integration services for software development, which can be started via command line or web application server. Jenkins Pipeline is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins.
Largely based on Vishnu Gopal's presentation http://www.slideshare.net/vishnu/basic-source-control-with-subversion
Used for a quick SVN introduction in a Software Engineering course at Massey University.
Analysis of merge requests in GitLab using PVS-Studio for C#Andrey Karpov
Do you like GitLab and don't like bugs? Do you want to improve the quality of your source code? Then you've come to the right place. Today we will tell you how to configure the PVS-Studio C# analyzer for checking merge requests. Enjoy the reading and have a nice unicorn mood.
Hyper-pragmatic Pure FP testing with distage-testkit7mind
Having a proper test suite can turn ongoing application maintenance and development into pure joy – the best tests check meaningful properties, not the implementation details, and hold no impliict assumptions about their test environment - every test case must be self-contained and portable. To ensure that tests are free of implementation details and environment dependency, we may simply run them in a different test environment, with different implementations of components. But the boileplate and manual work involved in rewiring components, writing hardcoded fixtures and setting up different test environments make this very hard to do at scale. To tackle this problem we've created distage & distage-testkit, distage-testkit gives you the following superpowers:
* ability to easily swap out individual components or entire test environments
* principled & leak-free control of global resources for integration testing – docker containers, DBs, DDLs
* execute effects or allocate resources per-test, e.g. generate random fixtures per-test
* first-class testing of functional effects
* write tests as lambdas – access test fixtures via parameters or ZIO Environment
...and more! We'll also discuss general testing practices and what really distinguishes good tests from great tests.
Many Scala developers nowadays consider using Dependency Injection frameworks an anti-pattern incompatible with modern FP settings. We argue that it's just a consequence of a bad experience with legacy Java runtime reflection-based implementations that lack features important for modern functional programming, such as a first-class support for higher-kinded types. We argue that as a paradigm for structuring purely functional programs, DI with automatic wiring compares favorably against implicits, monad transformers, free monads, algebraic effects, cake pattern et al, enabling scaling and a degree of modularity unachievable by any manual wiring approach. This talk covers DIStage – a transparent, flexible and efficient DI framework for Scala that enables late binding, testability, effect separation and modular resource management at scale, working with, instead of compromising the Scala type system.
Documentation: https://izumi.7mind.io/latest/release/doc/distage/
Video of the presentation: http://www.youtube.com/watch?v=8z3h4Uv9YbE
At LinkedIn, we have started to use the Play Framework to build front-end and back-end services at massive scale. Play does things a little differently: it's a Java and Scala web framework, but it doesn't follow the servlet spec; it's fairly new, but it runs on top of robust technologies like Akka and Netty; it uses a thread pool, but it's built for non-blocking I/O and reactive programming; most importantly, it's high performance, but also high productivity. We've found that the Play Framework is one of the few frameworks that is able to maintain the delicate balance of performance, reliability, and developer productivity. In the Java and Scala world, nothing even comes close. In this talk, I'll share what we've learned so far, including details of rapid iteration with Java and Scala, the story behind async I/O on the JVM, support for real time web apps (comet, WebSockets), and integrating Play into a large existing codebase.
Introduction to WP-CLI: Manage WordPress from the command lineBehzod Saidov
These are the slides from a talk given at WordCamp Riverside 2017:
https://2017.riverside.wordcamp.org/session/introduction-to-wp-cli-manage-wordpress-from-the-command-line/
The topic is about powerful tool WP-CLI than can help WordPress developers or power users a lot.
If you manage multiple WordPress websites or multiple environments of a WordPress website, you probably have experienced situations where you need to go through exact same steps on multiple websites. Clicking through wp-admin screens to change settings becomes time consuming. WP-CLI can be used to automate those tasks with ease.
During my presentation I’ll introduce the audience with the command line and WP-CLI, then I’ll go through some of the tasks that can be easily done using this tool. I’ll demonstrate some tasks running them on a sample WordPress website running in local development environment. Demonstrations will include tasks like pulling back the database to lower environments (qa/staging) and rewriting it, managing users, plugins, etc.
Advanced Eclipse Workshop (held at IPC2010 -spring edition-)Bastian Feder
So wie sich PHP weiterentwickelt, so entwickelt sich auch die Art der Programmierung weiter. Die Zeiten sind vorbei, in denen PHP nur von Hobbyprogrammierern genutzt wurde. Doch mit dem Anspruch an die Projekte steigt auch der Anspruch bei der Entwicklung. Schnell wird hierbei auf eine leistungsstarke IDE wie Eclipse PDT, Zend Studio oder Netbeans zurückgegriffen. Doch wie sieht eine anspruchsvolle Entwicklung mit solch einer IDE aus? Dieser Workshop wird Ihnen am Beispiel von der IDE Eclipse PDT demonstrieren, wie solch eine Entwicklung aussehen kann. Im Detail wird Ihnen gezeigt, wie Sie mittels SVN und Subversive Ihren Code mit mehreren Leuten gemeinsam pflegen und entwickeln und wie Sie die Entwicklungsumgebung Ihren Bedürfnissen anpassen, um z.B. mittels phing eigene Build-Prozesse anstoßen zu können. Damit Sie direkt eigene Erfahrungen sammeln können, würden wir Ihnen herzlich anraten, Ihren Laptop mitzubringen. Um zeitraubenden Installationen vorzubeugen, wird Ihnen ein Ubuntu in Form einer Live-CD bereitgestellt. Teilnehmer mit bestehender Linux-Installation und entsprechenden Rechten können ihr System während des Workshops direkt für den täglichen Gebrauch einrichten.
So, you know how to deploy your code, what about your database? This talk will go through deploying your database with LiquiBase and DBDeploy a non-framework based approach to handling migrations of DDL and DML.
Devops Day Amsterdam 2015
Arista vEOS lab in Ravello with Opscode Chef server.
Run the lab switches with chef-client and control them with cookbooks and API's
PHP classes in mumbai, Introduction to PHP/MYSQL..
best PHP/MYSQL classes in mumbai with job assistance.
our features are:
expert guidance by IT industry professionals
lowest fees of 5000
practical exposure to handle projects
well equiped lab
after course resume writing guidance
For more Visit: http://vibranttechnologies.co.in/php-classes-in-mumbai.html or http://phptraining.vibranttechnologies.co.in
(ARC402) Deployment Automation: From Developers' Keyboards to End Users' Scre...Amazon Web Services
Some of the best businesses today are deploying their code dozens of times a day. How? By making heavy use of automation, smart tools, and repeatable patterns to get process out of the way and keep the workflow moving. Come to this session to learn how you can do this too, using services such as AWS OpsWorks, AWS CloudFormation, Amazon Simple Workflow Service, and other tools. We'll discuss a number of different deployment patterns, and what aspects you need to focus on when working toward deployment automation yourself.
B-Translator helps to get feedback about l10n (translations of the programs). It tries to collect very small translation contributions from a wide crowd of people and to dilute them into something useful. It is developed as a Drupal7 profile and the code is hosted on GitHub. Here I describe the development setup and process that I use for this project. Most of the the tips are project specific, however some of them can be used on any Drupal project.
Similar to Patterns and Tools for Database Versioning, Migration, Data Loading and Test Data. (20)
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
"Impact of front-end architecture on development cost", Viktor Turskyi
Patterns and Tools for Database Versioning, Migration, Data Loading and Test Data.
1. DB BEST PRACTICES
FOR WEB APPS
Patterns and Tools for Database Versioning, Migration,
Data Loading and Data for Tests
CodeWorks - Atlanta - 9/29/2009
2. WHO AM I?
Alan Pinstein
@apinstein
apinstein@mac.com
Serial Entrepreneur, current project is Neybor.com
PHP Since 3.0 (~1997), C/C++/Obj-C, Cocoa, etc
PHOCOA PHP Framework http://phocoa.com
On GitHub: http://github.com/apinstein
Atlanta Co-working:
IgnitionAlley.com 151Locust.com
3. AGENDA
Common DB management problems that cause
frustration and stress
Benefits of formalized solutions
Case Studies
Tools: MP, fixtures, rake, capistrano
Q&A
4. LIVE Q&A & POLLS
http://codeatl09.eventhive.com/
• Live, crowd-moderated questions & feedback
• Live polls
5. [production-db] update user set password_md5 = md5(password);
[production-db] alter table user drop column password;
6. COMMON DB FRUSTRATIONS
Where should I keep track of schema changes?
What changed since last production push?
Did I apply this patch already to this server?
Did I run the migrations on this copy of the app?
Did I run the update script after updating the
schema?
Someone help the new developer set up the project
on his machine...
7. BENEFITS OF BEST
PRACTICES
Reproducibility
Reduction of complexity and risk
Reduced implementation time for DB changes
Increase in development throughput
Leverage from improvements in third-party tools
Community familiarity with workflows
8. CASE STUDY: BEFORE
$ vi updates.sql
>> alter table add column foo boolean;
$ git commit updates.sql -m “DB change”
$ ssh production@www0
$ git pull && psql -U production mydb
> i updates.sql
$ echo “” > updates.sql
9. CASE STUDY: BEFORE
FRAGILE process.
Quickly falls apart with 2+ devs or 2+ servers.
Branches? Fugghetaboutit.
10. CASE STUDY: AFTER
$ mp -n # create new migration
$ vi migrations/20090723_163051.php
$ mp -m head
$ git commit -a -m “Added new column.”
$ rake db:clone_production
$ rake db:migrate # test on copy of real data
$ cap deploy # push to staging
$ cap production deploy -s tag=staging-2009-09-01.1
11. CASE STUDY: AFTER
Each DB change encapsulated in a single migration
script
Only code what’s unique to your app
Deployment script automatically runs all necessary
migration scripts on production database
immediately after code deploys
Nothing to remember / eliminates human factor
Easy to test, easy to reproduce
12. VERSIONS VS MIGRATIONS
schema schema schema
version migration version migration version
1 2 3
A version is the structure of a database at a point in
time
A migration is code that changes the database
structure between versions
14. TRACKING
SCHEMA CHANGES
Explicit
Expressing changes as SQL or meta DDLs
Implicit
Schema Diffs
15. MP: MIGRATIONS FOR PHP
Generic migrations infrastrructure for any project
Open Source (MIT)
http://github.com/apinstein/mp/tree/master
Within 3 weeks, had:
28 public clones
1 fork
8 watchers
GitHub rocks
16. MP ARCHITECTURE
user: me schema schema
DB Server clean() db: myproj M1->up() version M2->up() version
(no schema) 1 2
M2->down()
clean()
17. MP SETUP
$ mp -f
MP - The PHP Migrator.
No migrations dir found; initializing migrations directory
at ./migrations.
Current version: 0
$ mp -f -n
Created migration 20090805_201842 at ./migrations/
20090805_201842.php.
$ ls -1 ./migrations
20090801_175831.php
clean.php
version.txt
18. BASIC USAGE
$ mp -x “pgsql:dbname=myproject;user=me” -m
=> migrates to latest version
TIP: alias myproj-mp=”mp -x ‘pgsql:dbname=myproject;user=me’”
$ myproj-mp -r -m
=> resets to clean, then migrates schema to latest
$ myproj-mp -m redo
=> run down() then up() on current migration
=> useful when developing migrations
$ mp -f -l
20090717_130521: Initial sql code for project
20090717_145424: Add e-commerce subsystem
20090804_145315: Add payment_method columns.
19. class Migration20090804_145315 extends Migration
{
public function up()
{
$sql = <<<SQL
alter table payment_method
add column account_name text,
add column nickname text,
add column address_id integer,
ADD CONSTRAINT fk_payment_method_address
FOREIGN KEY (address_id)
REFERENCES address(address_id) ON DELETE RESTRICT;
SQL;
$this->migrator->getDbCon()->exec($sql);
}
public function down()
{
$sql = <<<SQL
alter table payment_method
drop column account_name,
drop column nickname,
drop column address_id;
SQL;
$this->migrator->getDbCon()->exec($sql);
}
public function description()
{
return "Add payment_method columns account_name, nickname, and
address_id.";
}
}
20. MP IN PRACTICE
NEVER edit a migration once it’s been relied upon by
anyone else.
Ideally never edit once pushed to Version Control.
ALWAYS update clean.php to deal with each
migration.
ALWAYS test migrations with redo and clean.
ALWAYS test migrations against a production clone
before deployment.
21. MP CAVEATS
Post-schema change update scripts that rely on model code at a
certain version can break
Migration A
Adds blog.someFeatureField
Runs model code to copy data from
blog.oldFeatureField to blog.someFeatureField
Migration B
Removes blog.someFeatureField
Migration A will now fail since latest model code no longer
knows about blog.someFeatureField
Branches with migrations that can run out-of-order could be
problematic.
22. MP CAVEATS
Possible solutions:
Remove migration once it’s run everywhere
Catch errors and fail gracefully
Write upgrade scripts without using model code.
23. $ php externals/mp/mp -h
migrate usage:
MP CLI
VERSION STORAGE (must use one or the other)
-x --dsn <PDO DSN>
NOTE: if you supply a DSN then mp will automatically store the version number
in your database in a table named public.mp_version.
If you do not supply a dsn then you will need to use the -f flag to tell mp
to use the filesystem for storage of the version.
-f --version-in-file Store the version on the filesystem.
COMMANDS
-r --clean Run clean before doing anything else.
-n --create Create a new migration.
-s --show-version Shows the current version.
-V --set-version <version> Set the current version number maintained by mp to
be <version>. Does not run migrations; used to
initialize mp for an existing system.
-m --migrate <TargetVersion> or 0|up|down|head|redo
* redo is an alias for down + up
-l --list List all migrations in VERSION - DESCRIPTION format
OPTIONS
-v --verbose Verbose messages
-q --quiet Supress all message
-h --help Show this help
-d --dir <MigrationDir>
Default command is "show status".
24. MP API
$m = new Migrator(array(
Migrator::OPT_PDO_DSN => $dsn,
Migrator::OPT_VERSION_PROVIDER
=> new MigratorVersionProviderDB()
));
$m->clean();
$m->setVersion(‘xxx’);
$m->migrateToVersion(Migrator::VERSION_HEAD);
Easily Integrate with your own framework, app, build system, etc.
When called programmatically your infrastructure is bootstrapped,
so you have access to all of your codebase for grabbing DB
connections, etc.
Optionally supply a delegate, available to clean(), up(), down()
25. DB BEST PRACTICES
DB Naming Conventions:
Name: project, project_dev, project_staging
User: user
Password: different for each machine
Script to clone production database to local/staging
ssh -C user@host
'pg_dump
--clean --schema-only -U appUser -h 10.0.10.48
myProject'
| psql -U appUser myProject_dev
26. DATA LOADING - FIXTURES
Example - using WFFixture from phocoa php framework
Declarative, model-aware infrastructure makes setup
of test rigs fast and easy
YAML for readability
Useful for loading real baseline data as well as test
data
27. BLOG EXAMPLE
Multi-Tenant Blog App
Blog
name
homePageUrl
Post
name
post
status
Comment
email
comment
28. $fixture = <<<END
Blog:
reasonablyOpinionatedBlog:
name: Reasonably Opinionated
homePageUrl: 'http://reasonablyopinionated.com'
Post:
rant1:
Blog: reasonablyOpinionatedBlog
name: Database Migration is Hard
post: Database Migration is Hard...
status: structureTypeId: <?php BlogStatus::NOT_YET_PUBLISHED ?>
Comment: # to-many
- email: rudecommenter@blah.com
comment: this post sucks
- email: apinstein@mac.com
comment: you suck!
rant2:
Blog: # to-one
name: Some Other Blog
homePageUrl: http://someotherblog.com
name: Cantankerous rant on php 3
post: PHP3 is so old...
status: structureTypeId: <?php BlogStatus::PUBLISHED ?>
END;
$results = WFFixture::WFFixture()->loadFromYaml($fixture);
$this->rant1 = $results['rant1'];
30. RAKE FOR
DB MANAGEMENT
rake db:backup_production # Backup production DB - for safety before
deployment only
rake db:clone_production # Restore Production DB to Local DB
rake db:create # Create Database
rake db:drop # Drop Database
rake db:export:features # Export all feature data for later reloading.
rake db:export:geographies # Export all geography data for later reloading.
rake db:import:development_data # Load useful data for doing development.
rake db:import:features # Load feature data.
rake db:import:geographies # Load geography data.
rake db:migrate # Migrate to latest db version
rake db:migrate:check # Verify project is already set up for MP.
rake db:migrate:clean # Clean database.
rake db:migrate:head # Update to latest version.
rake db:migrate:rebuild # Rebuild db from clean state and update to
latest version.
rake db:model:rebuild # Rebuild model (reverse om convert-conf) from
existing database state.
rake db:rebuild # Rebuild the entire database, reload baseline
data, and rebuild the propel ORM model.
31. MP ROADMAP
Support for pinning migrations to specific VCS tags?
Support for manifesting irreversible migrations?