Containers brought new approach for implementation of DevOps workflows. So our CEO, Ruslan Synytsky, devoted a speech to this topic during Madrid meetup and described in details how Java developers can get benefits from Docker containers in Jelastic Cloud.
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...Oleg Shalygin
Â
Kubernetes provides an automated platform to deployment, scaling and operations of applications across a cluster of hosts. Complementing Kubernetes with a series of build scripts in conjunction with Travis-CI, GitHub, Artifactory, and Google Cloud Platform, we can take code from a merged pull request to a deployed environment with no manual intervention on a highly scaleable and robust infrastructure.
Cloud system configurations and their dependencies can quickly grow into the thousands of virtual machine, network and storage components. Once software is included, the number of components can easily rise into six figures.
Frequent releases using continuous integration and deployment tools makes a repository of these components and relationships absolutely critical to cloud system integrity and quality of service no matter what cloud management tools you use.
Systems configurations are more naturally represented using a graph database than the relational representations used by traditional IT management products.
Our talk will explore how we use Neo4J to create a live, active, self-updating repository service, containing nearly all virtual hardware, network and software components and their dependencies, enabling continuous deployment in any cloud environment at scale.
Accelerating Devops via Data Virtualization | DelphixDelphixCorp
Â
“Accelerating DevOps Using Data Virtualization” at the Collaborate 2016 conference in Las Vegas. It discusses the inevitability of data virtualization and its many use cases.
All Things Open : Crash Course in Open Source Cloud Computing Mark Hinkle
Â
Very few trends in IT have generated as much buzz as cloud computing. This session will cut through the hype and quickly clarify the ontology for cloud computing. The bulk of the conversation will focus on the open source software that can be used to build compute clouds (infrastructure-as-a-service) and the complimentary open source management tools that can be combined to automate the management of cloud computing environments.
The session will appeal to anyone who has a good grasp of traditional data center infrastructure but is struggling with the benefits and migration path to a cloud computing environment. Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software.
DCEU 18: Continuous Delivery with Docker Containers and Java: The Good, the B...Docker, Inc.
Â
Daniel Bryant - Product Architect, Datawire
Implementing a continuous delivery (CD) pipeline for Java applications is not trivial, and the introduction of container technology to the development stack can introduce additional challenges and requirements. In this talk we will look at the high-level steps that are essential for creating an effective pipeline for creating and deploying Docker container-based Java applications. Key takeaways include: - The impact of using Docker containers on Java and Continuous Delivery - The benefits and challenges of packaging containerised Java applications - Options for adding metadata to container images - Validating nonfunctional/operational requirement changes imposed by executing Java applications within a container - Lessons learned the hard way (in production, at 3am, with lots of coffee)
Geek Sync | Deployment and Management of Complex Azure EnvironmentsIDERA Software
Â
You can watch the replay of this Geek Sync webinar in the IDERA Resource Center: http://ow.ly/pg7N50A4svf.
Today's data management professional is finding their landscape changing. They have multiple database platforms to manage, multi-OS environments and everyone wants it now.
Join IDERA and Kellyn Pot’Vin-Gorman as she discusses the power of auto deployment in Azure when faced with complex environments and tips to increase the knowledge you need at the speed of light. Kellyn will cover scripting basics, advanced Portal features, opportunities to lessen the learning curve and how multi-platform and tier doesn't have to mean multi-cloud.
Attendees can expect to learn how to build automation scripts efficiently, even if you have little scripting experience, and how to work with Azure automation deployments. This session will allow you to begin building a repository of multi-platform development scripts to use as needed.
About Kellyn: Kellyn Pot’Vin-Gorman is a member of the Oak Table Network and an IDERA ACE and Oracle ACE Director alumnus. She is the newest Technical Solution Professional in Power BI with AI in the EdTech group at Microsoft. Kellyn is known for her extensive work with multi-database platforms, DevOps, cloud migrations, virtualization, visualizations, scripting, environment optimization tuning, automation, and architecture design. She has spoken at numerous technical conferences for Oracle, Big Data, DevOps, Testing and SQL Server. Her blog, http://dbakevlar.com and social media activity under her handle, DBAKevlar is well respected for her insight and content.
Containers brought new approach for implementation of DevOps workflows. So our CEO, Ruslan Synytsky, devoted a speech to this topic during Madrid meetup and described in details how Java developers can get benefits from Docker containers in Jelastic Cloud.
GCP - Continuous Integration and Delivery into Kubernetes with GitHub, Travis...Oleg Shalygin
Â
Kubernetes provides an automated platform to deployment, scaling and operations of applications across a cluster of hosts. Complementing Kubernetes with a series of build scripts in conjunction with Travis-CI, GitHub, Artifactory, and Google Cloud Platform, we can take code from a merged pull request to a deployed environment with no manual intervention on a highly scaleable and robust infrastructure.
Cloud system configurations and their dependencies can quickly grow into the thousands of virtual machine, network and storage components. Once software is included, the number of components can easily rise into six figures.
Frequent releases using continuous integration and deployment tools makes a repository of these components and relationships absolutely critical to cloud system integrity and quality of service no matter what cloud management tools you use.
Systems configurations are more naturally represented using a graph database than the relational representations used by traditional IT management products.
Our talk will explore how we use Neo4J to create a live, active, self-updating repository service, containing nearly all virtual hardware, network and software components and their dependencies, enabling continuous deployment in any cloud environment at scale.
Accelerating Devops via Data Virtualization | DelphixDelphixCorp
Â
“Accelerating DevOps Using Data Virtualization” at the Collaborate 2016 conference in Las Vegas. It discusses the inevitability of data virtualization and its many use cases.
All Things Open : Crash Course in Open Source Cloud Computing Mark Hinkle
Â
Very few trends in IT have generated as much buzz as cloud computing. This session will cut through the hype and quickly clarify the ontology for cloud computing. The bulk of the conversation will focus on the open source software that can be used to build compute clouds (infrastructure-as-a-service) and the complimentary open source management tools that can be combined to automate the management of cloud computing environments.
The session will appeal to anyone who has a good grasp of traditional data center infrastructure but is struggling with the benefits and migration path to a cloud computing environment. Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software.
DCEU 18: Continuous Delivery with Docker Containers and Java: The Good, the B...Docker, Inc.
Â
Daniel Bryant - Product Architect, Datawire
Implementing a continuous delivery (CD) pipeline for Java applications is not trivial, and the introduction of container technology to the development stack can introduce additional challenges and requirements. In this talk we will look at the high-level steps that are essential for creating an effective pipeline for creating and deploying Docker container-based Java applications. Key takeaways include: - The impact of using Docker containers on Java and Continuous Delivery - The benefits and challenges of packaging containerised Java applications - Options for adding metadata to container images - Validating nonfunctional/operational requirement changes imposed by executing Java applications within a container - Lessons learned the hard way (in production, at 3am, with lots of coffee)
Geek Sync | Deployment and Management of Complex Azure EnvironmentsIDERA Software
Â
You can watch the replay of this Geek Sync webinar in the IDERA Resource Center: http://ow.ly/pg7N50A4svf.
Today's data management professional is finding their landscape changing. They have multiple database platforms to manage, multi-OS environments and everyone wants it now.
Join IDERA and Kellyn Pot’Vin-Gorman as she discusses the power of auto deployment in Azure when faced with complex environments and tips to increase the knowledge you need at the speed of light. Kellyn will cover scripting basics, advanced Portal features, opportunities to lessen the learning curve and how multi-platform and tier doesn't have to mean multi-cloud.
Attendees can expect to learn how to build automation scripts efficiently, even if you have little scripting experience, and how to work with Azure automation deployments. This session will allow you to begin building a repository of multi-platform development scripts to use as needed.
About Kellyn: Kellyn Pot’Vin-Gorman is a member of the Oak Table Network and an IDERA ACE and Oracle ACE Director alumnus. She is the newest Technical Solution Professional in Power BI with AI in the EdTech group at Microsoft. Kellyn is known for her extensive work with multi-database platforms, DevOps, cloud migrations, virtualization, visualizations, scripting, environment optimization tuning, automation, and architecture design. She has spoken at numerous technical conferences for Oracle, Big Data, DevOps, Testing and SQL Server. Her blog, http://dbakevlar.com and social media activity under her handle, DBAKevlar is well respected for her insight and content.
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...Amazon Web Services
Â
Organizations around the globe are leveraging the cloud to accomplish world-changing missions. This session will address how AWS can help organizations put more money toward their mission and scale outreach and operations to achieve more with less. Hear some of AWS’s most advanced customers on how their organizations handle DevOps, continuous integration and deployment. Learn how these practices allow them to rapidly develop, iterate, test and deploy highly-scalable web applications and core operational systems on AWS. The discussion will focus on best practices, lessons learned, and the specific technologies and services they use.
Learn how Azure DevOps has empowered Horizons LIMS to streamline their collaboration and CI / CD process to accelerate their enterprise digital transformation. You will also hear about the latest Azure DevOps features and how to integrate DevOps with GetHub, Jenkins, and leverage transformation workloads like Kubernetes and Microsoft Common Data Service to deliver products and services faster.
Are you tired of the ever-increasing complexity in the world of DevOps? Do Docker and Kubernetes scripts, Ansible configurations, and networking woes make your head spin? It's time for a breath of fresh air.
Join us on a transformative journey where we shatter the myth that DevOps has to be overly complicated. Say goodbye to the days of struggling with incomplete scripts and tangled configurations. In this enlightening talk, we'll guide you through the process of rapidly onboarding your new standard microservice into the DevOps and Cloud universe.
We'll unveil the power of GitHub Actions, AWS, OpenAI API, and MS Teams Incoming Web hooks in a way that's both enlightening and entertaining. Additionally, we'll explore how Language Model APIs (LLMs) can be leveraged to enhance and streamline your DevOps workflows. You'll discover that DevOps doesn't have to be a labyrinth of complexity; it can be a streamlined and enjoyable experience.
So, if you're ready to simplify your DevOps journey and embrace a world where AWS, the OpenAI API, and GitHub Actions collaborate seamlessly while harnessing the potential of LLMs, join us and let's make DevOps a breeze!
Slides from Workshop 'Cloud Foundry: Hands-on Deployment Workshop'
http://www.meetup.com/CloudFoundry/events/150601282/
In this workshop you will learn Cloud Foundry fundamental concepts, setup, deployment and operations. We’ll cover a couple of alternatives to deploy CF in a local environment for learning and testing purposes as well as deploying Cloud Foundry atop IaaS production level environment, being able to manage hundreds of components and thousands of applications.
If you did not have a chance to work with Cloud Foundry, it may be useful to test its features locally at first. Deploying this environment on a local machine allows you to get hands-on experience in the solution and, in case you are a contributor, to test some features before you commit them to a production environment.
Cloud and agile software projects: Overview and BenefitsGuillaume Berche
Â
Slides from the session "Cloud and agile software projects: Overview and Benefits" at Agile Grenoble 2014, co presented by Guillaume Berche and Alain Delafosse.
http://agile-grenoble.org/
Continues Integration and Continuous Delivery with Azure DevOps - Deploy Anyt...Janusz Nowak
Â
Continues Integration and Continuous Delivery with Azure DevOps - Deploy Anything to Anywhere with Azure DevOps
Janusz Nowak
@jnowwwak
https://www.linkedin.com/in/janono
https://github.com/janusznowak
https://blog.janono.pl
Strategies and Tips for Building Enterprise Drupal Applications - PNWDS 2013Mack Hardy
Â
Mack Hardy, Dave Tarc, Damien Norris of Affinity Bridge presenting at Pacific Northwest Drupal Summit in Vancouver, October 5th, 2013. The presentation walks through management of releases, deployment strategies and build strategies with drupal features, git, and make files. Performance and caching is also covered, as well as specific tips and tricks for configuring apache and managing private files.
#SPSBrussels 2017 vincent biret #azure #functions microsoft #flowVincent Biret
Â
Slides of the session given at the SharePoint Saturday Brussels 2017 around Microsoft flow and Azure Functions. This session is an introduction to both services and how you can combine them
Similar to DevOps and Decoys How to Build a Successful Microsoft DevOps Including the Data (20)
This are my keynote slides from SQL Saturday Oregon 2023 on AI and the Intersection of AI, Machine Learning and Economnic Challenges as a Technical Specialist
This is the second session of the learning pathway at PASS Summit 2019, which is still a stand alone session to teach you how to write proper Linux BASH scripts
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Â
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Â
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Â
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Â
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Â
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Â
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
Â
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
Â
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Â
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
DevOps and Testing slides at DASA ConnectKari Kakkonen
Â
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
Â
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Â
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
A tale of scale & speed: How the US Navy is enabling software delivery from l...
Â
DevOps and Decoys How to Build a Successful Microsoft DevOps Including the Data
1. Kellyn Pot’Vin-Gorman,
DevOps Engineer and now Data Platform Architect in
Power BI and AI at Microsoft
DevOps and Decoys- How to
Build a Successful Microsoft
DevOps Solution Including
Your Data
3. Free online webinar
events
Free 1-day local
training events
Local user groups
around the world
Online special interest
user groups
Business analytics
training
Get involved
Explore
everything
PASS has
to offer
Free Online Resources
Newsletters
PASS.org
4. Download the GuideBook App and
search: PASS Summit 2018
Follow the QR code link displayed on session
signage throughout the conference venue and
in the program guide
Session
evaluations
Your feedback is
important and valuable.
Go to passSummit.com
3 Ways to Access:
Submit by 5pm Friday, November 16th to win prizes.
5. /kellyngorman
@DBAKevlar
kellyngorman
Kellyn Pot’Vin-Gorman
Data Platform Architect at Microsoft, EDU Team
Former Technical Intelligence Manager, Delphix
• Multi-platform DBA, (Oracle, MSSQL,
MySQL, Sybase, PostgreSQL, Informix…)
• Oracle ACE Director, (Alumni)
• Oak Table Network Member
• Idera ACE Alumni 2018
• STEM education with Raspberry Pi and
Python, including DevOxx4Kids, Oracle
Education Foundation and TechGirls
• Former President, Rocky Mtn Oracle User
Group
• Current President, Denver SQL Server User
Group
• Linux and DevOps author, instructor and
presenter.
• Blogger, (http://dbakevlar.com) Twitter:
@DBAKevlar
6. A Few Facts
1.7Mb of data created
per person, per second
by 2020
44Zb of data by 2020
80% of data isn’t
currently analyzed in
any way.
Even 10% added access
to analyze could result
in +$50 million in
revenue for every
fortune 100 company.
7. Automation is a Key to Much
of the Tech Acceleration
Less intervention from DBAs to maintain,
manage and operate database platforms.
Advancement in tools to eliminate resource
demands.
More tool interconnectivity, allowing less steps
to do more
A resurgence of scripting to enhance
automation and graphical interfaces to
empower those who have wider demands.
10. Our Use Case
Large Percentage of
University Customers in MHE
Searching for same solution
EDU team created a Popular
Solution that allowed for
community involvement
Multi-Tier Deployment- VM,
ADF, Azure Database, Data
and Power BI
Ever-evolving
11. Drawbacks
Teams from universities are made up of varied technical
backgrounds
â—¦ Data Scientists, DBAs, Data Analysts and Educators
â—¦ Spend More time deploying than working with the
solution
◦ Slows down the time the EDU’s limited resources get
to work one-on-one with the customers
â—¦ Discovered some customers lost interest during the
deployment phase or didn’t have the time to deploy
12. Goal
Simplify
Simplify the deployment
with DevOps practices
Remove
Remove demands on
knowledge required to
deploy the solution.
Create
Create more time for
interaction and working
with the solution by the
education teams.
13. Build a Roadmap
Document All Pieces of
Deployment
• Identify any interactive vs. default
entries that will benefit the deployment.
• Update documentation as you go. Don’t
try to do it in the end.
1
Categorize by Physical, Logical
and Interactive
• Build out Physical Deployment first, as it
is the foundation.
• Build in easy restart and clean up steps
to physical deployments
• Test and update any documentation to
reflect the automation
2
Begin to automate logical
slowly and in phases.
• Remove any manual steps or
configurations.
• Take advantage of any plugins that ease
work on end-users side
• Continue to accept feedback and
enhance.
3
14. What is Involved
1. Two SQL Databases- one staging and one data warehouse
2. Azure Data Factory with a SSIS Database
3. Azure Analysis Services
3. Three Power BI Reports
4. CSV files for ongoing data loads, which will need to be configured for ongoing workloads
5. Multiple Solution and project files, some deprecated in VS 2017
* Data loads and configuration via Visual Studio or SSDT solutions already built in.
* Sample data files in Excel could be replaced with the customers own data.
15. The Solution Was Very Repeatable
Most resources and databases could have the same name in every deployment.
Outside of the CSV files containing example data, everything else could be deployed without any
changes by the customer if a few dynamic parameters were pushed to the deployment.
Although official documentation existed, there were numerous versions and the process had
evolved with the introduction and ease of Azure deployment.
17. Visual Studio/SSMS Dev Tools
My predecessor and team members built in some automation already using solution files!
â—¦ Awesome, these can be reused!
18. How Do From Point A to Point
Automate?
Perform the task in the User Interface FIRST.
Gather the information with the CLI to build out your scripts
Test and retest comparing to the UI deployment
Manage and maintain automation.
Don’t go back to manual processing or manual intervention.
Build out in phases- physical to logical, enhancing and improving as
we go along.
19. So Many Choices…
Multiple options to automate-
â—¦Azure CLI
â—¦Terraform
â—¦PowerShell with Azure Commands
â—¦Azure DevOps
20. Terraform
Freemium Product
Came to Azure in 2016
Another robust product that allows for automation of processing via
script.
Supports complex tasks and multi-platform deployments
23. Azure CLI
Allows a command line interface to the Azure cloud
Can be installed locally on Windows, Mac and other OS
platforms
Can be run inside a Docker Container
Can be used with the Azure Cloud Shell without
installation.
Flexible and robust, allows for a CLI solution and
automation via scripting in PowerShell/BASH of Azure
deployments.
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest
25. AZ CLI is Simple to Use and Robust
>C:EDU_Docker>az vm create -n LinuxTstVM -g dba_group --image
UbuntuLTS --generate-ssh-keys
SSH key files 'C:Userskegorman.NORTHAMERICA.sshid_rsa' and
'C:Userskegorman.NORTHAMERICA.sshid_rsa.pub' have been generated
under ~/.ssh to allow SSH access to the VM. If using machines
without permanent storage, back up your keys to a safe location.
- Running ..
C:EDU_Docker>az vm list –g dba_group
C:EDU_Docker>az vm delete -n LinuxTstVM -g dba_group
Are you sure you want to perform this operation? (y/n): Y
26. Locating Information
on ADF
1. Create one in the GUI
2. Inspect the resource facts via
the CLI:
az resource list --
location eastus
27. Walk Before You
Run..
Began to
deploy
individual
resources.
Had a final
merge script,
(aka wrapper)
with a test
script.
Deployed
piece by piece
until phase I
was
completed.
Received
feedback from
peers and
customers as
proceeded.
28. Azure CLI Isn’t Enough – Cloud Shell
Enhanced Azure CLI commands into BASH script to deploy and automate.
A script to enhance automation and set variables to ease customer skill requirements was
required.
From the Azure Portal:
Or Direct: https://shell.azure.com/
29. Initial Build in BASH
Well, I’m a Linux Person
Script is interactive, accepting customer’s requested naming
conventions and requirements.
Builds out the physical resources in Azure
â—¦ SQL Server with a data warehouse and staging database
â—¦ Azure Data Factory
â—¦ Azure Analysis Server
Creates firewall rules for Azure Cloud Shell
Creates all user access
Creates Database objects
32. Living Documentation
Made easier
transition as
redesigned.
Kept track of all
moving parts.
Offered insight to
those who knew
previous, manual
process.
Allowed for
roadmap to be
included.
Allowed for
troubleshooting
section as other
sections shrunk
with automation.
34. PowerShell
Love of the SQL Community
One the roadmap for future, secondary choice
for deployment
Still deciding if a PowerShell version is required.
35. Benefits and Drawbacks
Terraform Azure CLI PowerShell/BASH
Freemium, but great user
community
Newer product, but driven to support Azure PowerShell has a leg up on
BASH in the SQL World
Scripting is proprietary Scripting can be done in Powershell or BASH Scripting is powerful
Can build out anything that
has a command line option
Is very interactive. If you want to script it, must do
more.
Can do more than just
deploy. Scripting can
become more application
based.
Requires Subscription,
tenant, client API and
password to be passed
Good for check commands and settings Can be as much as needed.
Was a bit overkill for what I
was doing
Offers more support than Terraform, that can’t use
some of the defaults
Azure CLI was made to
work with both.
36.
37. Automate the Logical
• Migrate any onsite SSIS pkgs and workflows to Azure
Data Factory
• ADF will build out an SSISDB in Azure SQL database
• Store projects in pipelines
• Schedule, report and check into a Github repository to
automate development cycle.
• SLN and PROJ files are recycled.
38. Next Steps
PowerShell version is in mid-development
â—¦ More likely know PowerShell over BASH if Microsoft professional.
â—¦ Will only require two main scripts to be updated with enhancements and
additions to the repository.
â—¦ Automate the SSIS, (Integration Services) and Data Factory steps, (currently an
unknown, so figuring out.)
◦ Build out scripts to scale up the current “demo” version to an enterprise
version.
â—¦ Script to automatically pause integration and factory services to save on cost.
â—¦ Script out data workload inputs for new features/data model and proprietary
data loads.
â—¦ Build out all steps using Azure DevOps to continue the growth of the
automation.
39. Challenges?
No Reference Material on
How to Create an Azure Data
Factory from the Command
Line
Azure CLI offered
HOPE, but some of it
had to be built out in
the self-service
command of
“resource” and use
json strings.
The motto when
working with new
tools- “Learn to Fish”
•Help menus and examples
online are your best
friend.
•Get a good text editor.
•You will be learning new
scripting languages
•Find support groups,
twitter handles and help
channels.
40. Success
Manual Process takes between two full days of onsite
meetings, to weeks of webmeetings to deploy.
New Automated process deploys in less than 15 minutes
after customer answered questions in interactive script.
Offers extensively more time for customer to work with
solution and Microsoft architects to work with providing
value to the customer on how to use Power BI with
Azure.
In first week, over two dozen customers requested POC
deployment with little assistance from heavily limited
resource team, allowing for more valuable allocation of
resources.
41. /
You Must Evolve
Data WILL come from
more than just SQL
Server and Azure SQL
Databases.
1
You will be expected to
do more with less.
2
Embrace automation
tools inside the database
platform, (backup,
recovery, optimization,
maintenance tasks.)
3
42. Thank You
Learn more from Kellyn Pot’Vin-Gorman
@DBAKevlar kegorman@Microsoft.com