Schema migration (DB migration) with Phinx.
What is schema migration? Why you should use schema migration? How you do schema migration using phinx, written on PHP.
Delivering Changes for Applications and DatabasesMiguel Alho
Presentation used at PortoData (28/07/2016) about manageing database changes in application projects. Demoed Flyway migrations for script packages, and a strategy for using DBUp and localDb for integration testing at the data layer
Embracing DevOps through database migrations with FlywayRed Gate Software
"Evolutionary Database Design" is the best phrase to describe database migrations. But what do we know about database migrations using PostgreSQL containers?
This session will provide you with answers and guidelines to get you started with Database DevOps practices for your organization. You will learn the aspects, methods, and strategies to build and manage your database deployments through CI/CD pipelines with open source tools like Flyway, Jenkins, and Kubernetes.
You will be able to build your first database migration through a CI/CD pipeline at the end of this session.
The FeatureReader / FeatureWriter RevolutionSafe Software
The FeatureWriter is a revolutionary new transformer that enables you to write data at any point during a workflow. See how to use this transformer in conjunction with the FeatureReader in a variety of ways — like post-processing and integrating with third-party tools.
Using Redgate, AKS and Azure to bring DevOps to your DatabaseRed Gate Software
Join Hamish Watson and Rob Sewell to learn practical solutions on how to bring DevOps to your database, including:
• The importance of getting your database code into source control
• How to test your database changes
• Tools you can use to automate build and test processes
• How to build an automated deployment process for your database with Redgate tools
• How to embrace using Azure Kubernetes Services (AKS) in your deployment pipeline
• Deploying your entire pipeline as and when it is needed from Dev to Prod saving your organisation money
Database upgrades and data in general are often the most complicated part of your deployment process, so having a robust deployment path and checks before getting to production is very important.
The demos will showcase practical solutions that can help you and your team bring DevOps to your database using SQL Source Control, infrastructure as code, docker containers and SQL Change Automation – all leading up to a fully automated test and deployment process.
This will be a fun-filled fast paced hour and you will learn some new skills which will bring immediate benefit to your organization.
When and how to migrate from a relational database to CassandraBen Slater
Many applications are initially developed using relational database technology before “upsizing” to Cassandra as they mature.
This presentation examines the indicators that it is time to start considering a migration away from relational and factors to consider when making the decision to move. It then discusses different approaches for implementing the migration and how to plan, estimate and manage the work along with key risks and gotchas you may encounter.
PASS President Grant Fritchey, and Redgate’s Arneh Eskandari, will show you how Redgate’s Database DevOps solution works to improve your database development and deployment processes.
You'll see a demo of the following Redgate tools - SQL Source Control, DLM Automation and DLM Dashboard - and find out how they plug into Git and Jenkins.
Learn how to:
- Version control your databases from SSMS to track and understand changes
- Include your database in your Continuous Integration process to get quick feedback
- Automate your database deployment process to have safe, reliable, repeatable deployments
- Monitor your databases across environments to understand their versions and get notified of changes
Delivering Changes for Applications and DatabasesMiguel Alho
Presentation used at PortoData (28/07/2016) about manageing database changes in application projects. Demoed Flyway migrations for script packages, and a strategy for using DBUp and localDb for integration testing at the data layer
Embracing DevOps through database migrations with FlywayRed Gate Software
"Evolutionary Database Design" is the best phrase to describe database migrations. But what do we know about database migrations using PostgreSQL containers?
This session will provide you with answers and guidelines to get you started with Database DevOps practices for your organization. You will learn the aspects, methods, and strategies to build and manage your database deployments through CI/CD pipelines with open source tools like Flyway, Jenkins, and Kubernetes.
You will be able to build your first database migration through a CI/CD pipeline at the end of this session.
The FeatureReader / FeatureWriter RevolutionSafe Software
The FeatureWriter is a revolutionary new transformer that enables you to write data at any point during a workflow. See how to use this transformer in conjunction with the FeatureReader in a variety of ways — like post-processing and integrating with third-party tools.
Using Redgate, AKS and Azure to bring DevOps to your DatabaseRed Gate Software
Join Hamish Watson and Rob Sewell to learn practical solutions on how to bring DevOps to your database, including:
• The importance of getting your database code into source control
• How to test your database changes
• Tools you can use to automate build and test processes
• How to build an automated deployment process for your database with Redgate tools
• How to embrace using Azure Kubernetes Services (AKS) in your deployment pipeline
• Deploying your entire pipeline as and when it is needed from Dev to Prod saving your organisation money
Database upgrades and data in general are often the most complicated part of your deployment process, so having a robust deployment path and checks before getting to production is very important.
The demos will showcase practical solutions that can help you and your team bring DevOps to your database using SQL Source Control, infrastructure as code, docker containers and SQL Change Automation – all leading up to a fully automated test and deployment process.
This will be a fun-filled fast paced hour and you will learn some new skills which will bring immediate benefit to your organization.
When and how to migrate from a relational database to CassandraBen Slater
Many applications are initially developed using relational database technology before “upsizing” to Cassandra as they mature.
This presentation examines the indicators that it is time to start considering a migration away from relational and factors to consider when making the decision to move. It then discusses different approaches for implementing the migration and how to plan, estimate and manage the work along with key risks and gotchas you may encounter.
PASS President Grant Fritchey, and Redgate’s Arneh Eskandari, will show you how Redgate’s Database DevOps solution works to improve your database development and deployment processes.
You'll see a demo of the following Redgate tools - SQL Source Control, DLM Automation and DLM Dashboard - and find out how they plug into Git and Jenkins.
Learn how to:
- Version control your databases from SSMS to track and understand changes
- Include your database in your Continuous Integration process to get quick feedback
- Automate your database deployment process to have safe, reliable, repeatable deployments
- Monitor your databases across environments to understand their versions and get notified of changes
WordPress now powers over 25% of sites on the web including big names such as Time, CNN, TechCrunch and more. What changes about a WordPress site when it needs to function on such a large scale? Keanan Koppenhaver will show us how WordPress can be tweaked to serve content to millions of visitors per month, how to keep code modular using custom-built plugins, special considerations for enterprise-sized sites and more! If you've ever wondered what goes into creating and maintaining a high-traffic WordPress site, this will be one you won't want to miss.
Keanan Koppenhaver is a Digital Product Developer and Consulting Engineer with doejo, a WordPress VIP Partner Agency. While at doejo, he has worked on baking WordPress into the publishing workflow of one of the largest investment news publications in the United States, a WordPress-backed real estate investment portal, and many other projects large and small. He is passionate about mentoring other developers as well as teaching people that WordPress can be more than just a blogging platform. Keanan writes about all these topics and more at http://levelupwp.net.
This is an introduction to Spring Batch Framework. After reading this presentation, you will be able to know how Spring Batch works, and you will be able to download a maven project as an example.
Flux is an application architecture for building User Interfaces (UI). A data flow application architecture created and used by Facebook for client-side web applications. React JS is An open source JavaScript library for building user interfaces
What comes to you if you want to have some fun with GraphQL?
Join us for the meetup in Munich and enjoy the interesting talk about ‚GraphQL vs. (the) REST ‘.
In this session our colleagues Tsvetan Nikolov (Senior Developer at coliquio) and Tom Sedlmeier (Senior Developer at coliquio) will discuss the basic concepts and how to use them.
We will present you GraphQL on the backend and on the client side with Apollo.
DDD Domain Relationships can seem very abstract at first. Here are Radars that you can use in Event Storming sessions to present the relationships to your attendees in a digestible way
Moving from application automation to true DevOps by including the databaseRed Gate Software
As software teams face increasing pressure to speed up the delivery cycle, the DevOps approach is gaining momentum. But the database is commonly a bottleneck. This presentation will explore how you can include the database in continuous integration and automated release management. Sharing best practices from high-performing IT organisations, we will discuss how you can extend DevOps practices to the database; increasing productivity, agility and optimizing performance.
So, you know how to deploy your code, what about your database? This talk will go through deploying your database with LiquiBase and DBDeploy a non-framework based approach to handling migrations of DDL and DML.
Building and deploying LLM applications with Apache AirflowKaxil Naik
Behind the growing interest in Generate AI and LLM-based enterprise applications lies an expanded set of requirements for data integrations and ML orchestration. Enterprises want to use proprietary data to power LLM-based applications that create new business value, but they face challenges in moving beyond experimentation. The pipelines that power these models need to run reliably at scale, bringing together data from many sources and reacting continuously to changing conditions.
This talk focuses on the design patterns for using Apache Airflow to support LLM applications created using private enterprise data. We’ll go through a real-world example of what this looks like, as well as a proposal to improve Airflow and to add additional Airflow Providers to make it easier to interact with LLMs such as the ones from OpenAI (such as GPT4) and the ones on HuggingFace, while working with both structured and unstructured data.
In short, this shows how these Airflow patterns enable reliable, traceable, and scalable LLM applications within the enterprise.
https://airflowsummit.org/sessions/2023/keynote-llm/
WordPress now powers over 25% of sites on the web including big names such as Time, CNN, TechCrunch and more. What changes about a WordPress site when it needs to function on such a large scale? Keanan Koppenhaver will show us how WordPress can be tweaked to serve content to millions of visitors per month, how to keep code modular using custom-built plugins, special considerations for enterprise-sized sites and more! If you've ever wondered what goes into creating and maintaining a high-traffic WordPress site, this will be one you won't want to miss.
Keanan Koppenhaver is a Digital Product Developer and Consulting Engineer with doejo, a WordPress VIP Partner Agency. While at doejo, he has worked on baking WordPress into the publishing workflow of one of the largest investment news publications in the United States, a WordPress-backed real estate investment portal, and many other projects large and small. He is passionate about mentoring other developers as well as teaching people that WordPress can be more than just a blogging platform. Keanan writes about all these topics and more at http://levelupwp.net.
This is an introduction to Spring Batch Framework. After reading this presentation, you will be able to know how Spring Batch works, and you will be able to download a maven project as an example.
Flux is an application architecture for building User Interfaces (UI). A data flow application architecture created and used by Facebook for client-side web applications. React JS is An open source JavaScript library for building user interfaces
What comes to you if you want to have some fun with GraphQL?
Join us for the meetup in Munich and enjoy the interesting talk about ‚GraphQL vs. (the) REST ‘.
In this session our colleagues Tsvetan Nikolov (Senior Developer at coliquio) and Tom Sedlmeier (Senior Developer at coliquio) will discuss the basic concepts and how to use them.
We will present you GraphQL on the backend and on the client side with Apollo.
DDD Domain Relationships can seem very abstract at first. Here are Radars that you can use in Event Storming sessions to present the relationships to your attendees in a digestible way
Moving from application automation to true DevOps by including the databaseRed Gate Software
As software teams face increasing pressure to speed up the delivery cycle, the DevOps approach is gaining momentum. But the database is commonly a bottleneck. This presentation will explore how you can include the database in continuous integration and automated release management. Sharing best practices from high-performing IT organisations, we will discuss how you can extend DevOps practices to the database; increasing productivity, agility and optimizing performance.
So, you know how to deploy your code, what about your database? This talk will go through deploying your database with LiquiBase and DBDeploy a non-framework based approach to handling migrations of DDL and DML.
Building and deploying LLM applications with Apache AirflowKaxil Naik
Behind the growing interest in Generate AI and LLM-based enterprise applications lies an expanded set of requirements for data integrations and ML orchestration. Enterprises want to use proprietary data to power LLM-based applications that create new business value, but they face challenges in moving beyond experimentation. The pipelines that power these models need to run reliably at scale, bringing together data from many sources and reacting continuously to changing conditions.
This talk focuses on the design patterns for using Apache Airflow to support LLM applications created using private enterprise data. We’ll go through a real-world example of what this looks like, as well as a proposal to improve Airflow and to add additional Airflow Providers to make it easier to interact with LLMs such as the ones from OpenAI (such as GPT4) and the ones on HuggingFace, while working with both structured and unstructured data.
In short, this shows how these Airflow patterns enable reliable, traceable, and scalable LLM applications within the enterprise.
https://airflowsummit.org/sessions/2023/keynote-llm/
Achieving Full Stack DevOps at Colonial Life DevOps.com
In an ever more competitive marketplace, organizations have turned to Agile and DevOps practices to deliver software innovations to market more quickly and with high quality. Across industries, companies are making heavy investments in tools and process improvements around automated build, test, continuous integration and delivery, and release automation and orchestration. However, despite these investments, many organizations are still struggling to bring the necessary speed and quality to their software delivery. In many cases, this is because Agile and DevOps improvements have not been applied to the entire software stack and are often limited to application code delivery.
This webinar will explore the transformation that Colonial Life made in bringing DevOps to the entire software stack. Specifically, beyond automating and accelerating the validation and delivery of application code, this webinar will focus on the critical role that data and the database play in modern software delivery and the tools and processes that can bring the same automation to database code.
After this webinar, you will understand:
* What holds organizations back despite an Agile application development process
* The benefits of automating the validation and deployment of database changes
* A template for bringing DevOps to the entire software stack
Detailed comparison of two common legacy databases - HP's SQL/MP running on the NonStop Guardian environment and IBM's DB2 running on its z/OS platform, comparing a range of functionalities.
BW Migration to HANA Part 2 - SUM DMO Tool for SAP Upgrade & MigrationLinh Nguyen
This series of publication intends to provide an overview and explanation of major steps and considerations for BW on HANA migrations from anyDB (any database). The complex procedure involves:
1) Preparatory work in the BW system
2) SUM DMO Upgrade and Actual migration
3) Post processing on the migrated systems
This first part focuses on the SUM DMO tool used for the migration, pre-requisites, optimization and the actual migration steps
By OZSoft Consulting for ITConductor.com
Author: Terry Kempis
Editor: Linh Nguyen
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
2. What is Schema migration?
In software engineering, schema migration (also database migration, database change management)
refers to the management of incremental, reversible changes to relational database schemas.
A schema migration is performed on a database whenever it is necessary to update or revert that
database's schema to some newer or older version.
3. Have you ever face these situations?
● Developer A add several columns on table X, developer A commits and push the codes. Developer
B pulls the code and then the app breaks.
● Developer finishes the codes and forgot to give/email the sql alter script to DevOps/Sysadmin guy.
When DevOps deploy to production the app breaks.
● DevOps guy deploying more than 1 DB instances. Can you imagine how long it took to apply DB
changes if the instances are (for example) 10?
● …..
4. Why using Schema migration?
“The rule is simple. You should never tie database migrations to application deploys or vice versa. By
minimising dependencies you enable faster, easier and cleaner deployments.”
http://www.brunton-spall.co.uk/post/2014/05/06/database-migrations-done-right/
5. Benefits of using Schema Migration
● Separated application and database deployment.
● Ensured same DB structure through all the team member.
● Database versioning (keep track of DB structure changes).
● Easier to maintain incremental database changes.
6. Why Phinx?
https://phinx.org/
● Be portable amongst the most popular database vendors.
● Be PHP framework independent.
● Have a simple install process.
● Have an easy to use command-line operation.
● Integrate with various other PHP tools (Phing, PHPUnit) and web frameworks.
7. Why i choose phinx
● Familiar. Because it is written in PHP.
● Command line app. Easy to connect phinx to deployer tool.
● Supports most popular databases (MySQL, PostgreSQL, etc).
● Works independently/standalone. Multi platform and frameworks.
● Helpful for unit testing.
● Their documentation is easy to follow, easy to understand.
8. Installation
Installation is easy. Use composer!
php composer.phar require robmorgan/phinx
execute this after installation finished.
vendor/bin/phinx init
http://docs.phinx.org/en/latest/install.html
9. One thing Before Migrating
There should be a file named phinx.yml after you are done initialized your project.
Phinx.yml file contains your environments and database configurations.
12. Data types? Indexes? Foreign keys?
Check their documentation on this matters.
https://book.cakephp.org/3.0/en/phinx/migrations.html#working-with-columns
They are pretty straightforward and easy to follow.
13. Seeder
● Populate reference data
○ Rarely change data
○ Eg: Country data, User type data, etc
● Populate test data
○ Would you run sql command on PhpMyAdmin 100, 1000, 10000 times?
14. How to create and run a seeder
Create a seeder
$ phinx seed:create MyNewSeeder
Run a seeder
$ phinx seed:run -e development -s MyNewSeeder
15. Seeder Example
Here is an example of seeder.
A seeder for super user account.
This one records will be inserted in users
table.
16. How to generate random data in your seeder?
Use Faker! (https://github.com/fzaninotto/Faker)
17. How to access DB in your seeder?
Often times you encounter cases that
need current data in database in order to
generate other data. This is how you it on
phinx.
Use these 2 methods
● fetchRow()
○ For single row
● fetchAll()
○ For multiple rows
https://book.cakephp.org/3.0/en/phinx/migrations.html#fetching-rows
18. Tips on using Phinx
● Read the documentation thoroughly.
● Write migration wisely. Only some method can be reversed.
● Store it in version control (git) and share it with your team/colleague.
Phinx joined CakePHP family on June 2017
https://bakery.cakephp.org/2017/06/23/welcoming-phinx-to-the-cakephp-family.html
19. That’s All!
Now you know how, what and why you should use schema migration, right?
“Get your database under control”
Hadi Ariawan
@hadiariawan
github.com/hadiariawan