AADays Poland: Perfecting the Craft of Test Automation

•

2 likes•679 views

The document discusses test automation and emphasizes the importance of clean code, collaboration between testers, developers and automators, and exploring diverse approaches beyond basic regression testing. Specifically, it notes that without clean code, test automation will not be effective. It also stresses creating understanding between testers, developers and automators, and that the landscape of test automation involves more than just usual techniques like regression testing.

Software

@maaretp http://maaretp.com
Perfecting the Craft
of Test Automation
by Maaret Pyhäjärvi

@maaretp http://maaretp.com
10 Years has passed….

@maaretp http://maaretp.com
The World of Automation
Today
We have the tools and knowledge to do it well!

@maaretp http://maaretp.com
Testing is done by People.
Our features define how
well testing gets done.

@maaretp http://maaretp.com
1.  Without clean code, all the test
automation in the world will not
save you.
2.  Create a mindshare between
testers, test automators and
developers.
3.  There’s more to test automation
landscape than the usual suspects.

@maaretp http://maaretp.com
Clean Code
Without It, All the test automation
in the world won’t save you

@maaretp http://maaretp.com
Invest in…
Testing vs. Refactoring

@maaretp http://maaretp.com
Technical Debt is not a
decision, it’s like gaining
weight.

@maaretp http://maaretp.com
Messy code is hard to
handle without side-
effects

@maaretp http://maaretp.com
Clean Production and
Automation Code

@maaretp http://maaretp.com
Make your coded tests
smaller and faster

@maaretp http://maaretp.com
Your tests are not
flaky, you are.

@maaretp http://maaretp.com
Continuous
Delivery without
Test Automation
exists.

@maaretp http://maaretp.com
Without clean code, all
the test automation in the
world will not save you.

@maaretp http://maaretp.com
Mindshare
Similar Enough to Get Along,
Diverse Enough for Better Impact

@maaretp http://maaretp.com
(Exploratory)
Tester
(Application)
Programmer
Test Automator
/ Programmer

@maaretp http://maaretp.com
"All the brilliant people working on
the same thing, at the same time, in
the same space, on the same
computer." -- Woody Zuill

@maaretp http://maaretp.com
We raise our collective
competence, allowing
individuals to use
specialized skills

@maaretp http://maaretp.com
Dumbing down
automation from
programming work is
a disservice to all.

@maaretp http://maaretp.com
When you work on
Hard problem è Solution
Easy problem è Innovation

@maaretp http://maaretp.com
Learning Through Osmosis

@maaretp http://maaretp.com
Create a mindshare
between testers, test
automators and
developers.

@maaretp http://maaretp.com
Test Automation
Landscape
More than the Usual Suspect of Regression

@maaretp http://maaretp.com
Automation of the
Environment

@maaretp http://maaretp.com
Disposable
test automation to
aid exploration

@maaretp http://maaretp.com
From Page-Object
Pattern to
Screenplay Pattern

@maaretp http://maaretp.com
Multi-Locators

@maaretp http://maaretp.com
From Asserts to
Approvals

@maaretp http://maaretp.com
Theory Tests and
Partial Oracles
Given Any X
ExpectY

@maaretp http://maaretp.com
Model-based
Test Automation

@maaretp http://maaretp.com
There’s more to test
automation landscape
than the usual suspects.

@maaretp http://maaretp.com
Good Enough of
today is not good
enough of tomorrow.

@maaretp http://maaretp.com
Maaret Pyhäjärvi
Email: maaret@iki.fi
Twitter: @maaretp
Web: maaretp.com
Blog: visible-quality.blogspot.fi
(please connect with me through
Twitter or LinkedIn)

While we draw parallels to Queen's Gambit the series on Netflix and Software Testing, this talk is not about how substance abuse is like testing. Instead, the movie guides us to to its name and figuring out what is this gambit. With exploratory testing, we have the opening moves, the midpart and the end game. The start can be anything but it builds our platform for the end game. And while exploratory testing, automation is part of that way of staying in tune with learning to win in the end game.

Agile2016: Exploratory Testing an API

Maaret Pyhäjärvi

Selenium Summit 21: Contemporary Exploratory Testing

Maaret Pyhäjärvi

MoT Sarajevo: How to get better at Exploratory testing

Maaret Pyhäjärvi

HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale

Maaret Pyhäjärvi

Geekle QA: Automationist's Gambit

Maaret Pyhäjärvi

Agile2016: (Lightning Talk) Breaking illusions

Maaret Pyhäjärvi

DevoxxUK: Learning programming through osmosis

Maaret Pyhäjärvi

Selenium Conference India: Intersection of Automation and Exploratory Testing

Maaret Pyhäjärvi

Agile Serbia: Exploratory Testing Explained and Experienced

Maaret Pyhäjärvi

Mimmit Koodaa - Testiautomaatio on testauksen dokumentointia

Maaret Pyhäjärvi

SLASSCOM: Breaking illusions with Testing

Maaret Pyhäjärvi

Tampere Testing Days: Exploratory Testing an API

Maaret Pyhäjärvi

EuroSTAR Webinar - Mob testing

Maaret Pyhäjärvi

Many different roles contribute to building software: product owners, business specialists, testers. Yet knowledge of programming keeps these roles at a distance. In this talk, we will discuss a new powerful way of removing this distance for the benefit of our teams: mob programming. We’ll walk through my lessons: the pains of joining a mob as a non-programmer, and the gains of what a tester brings into a mob. This talk serves as an inspiration to learning together, immersed in the experience of creating software as we’re learning. Diversity of thoughts and skillsets improved everybody and created a better product.

DDD Europe: Breaking Illusions with Testing (keynote)

Maaret Pyhäjärvi

MoT Sfax: Good Exploratory Testing Tactics for Taking Over Testing

Maaret Pyhäjärvi

Testbash Philly: A Mob Testing Experience

Maaret Pyhäjärvi

AgileTD Scandia: Mob Testing

Maaret Pyhäjärvi

AGRS2019: Breaking illusions with Testing

Maaret Pyhäjärvi

Agile India: Working without Product Owner

Maaret Pyhäjärvi

TestWorksConf: Experience exploratory testing

Maaret Pyhäjärvi

TestWorksConf: Exploratory Testing an API in Mob

Maaret Pyhäjärvi

What's hot

EuroSTAR '21 Keynote: Automationist's Gambit

Maaret Pyhäjärvi

Agile2016: Exploratory Testing an API

Maaret Pyhäjärvi

Selenium Summit 21: Contemporary Exploratory Testing

Maaret Pyhäjärvi

MoT Sarajevo: How to get better at Exploratory testing

Maaret Pyhäjärvi

HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale

Maaret Pyhäjärvi

Geekle QA: Automationist's Gambit

Maaret Pyhäjärvi

Agile2016: (Lightning Talk) Breaking illusions

Maaret Pyhäjärvi

DevoxxUK: Learning programming through osmosis

Maaret Pyhäjärvi

Selenium Conference India: Intersection of Automation and Exploratory Testing

Maaret Pyhäjärvi

Agile Serbia: Exploratory Testing Explained and Experienced

Maaret Pyhäjärvi

Mimmit Koodaa - Testiautomaatio on testauksen dokumentointia

Maaret Pyhäjärvi

SLASSCOM: Breaking illusions with Testing

Maaret Pyhäjärvi

Tampere Testing Days: Exploratory Testing an API

Maaret Pyhäjärvi

EuroSTAR Webinar - Mob testing

Maaret Pyhäjärvi

DDD Europe: Breaking Illusions with Testing (keynote)

Maaret Pyhäjärvi

MoT Sfax: Good Exploratory Testing Tactics for Taking Over Testing

Maaret Pyhäjärvi

Testbash Philly: A Mob Testing Experience

Maaret Pyhäjärvi

AgileTD Scandia: Mob Testing

Maaret Pyhäjärvi

AGRS2019: Breaking illusions with Testing

Maaret Pyhäjärvi

Agile India: Working without Product Owner

Maaret Pyhäjärvi

What's hot (20)

EuroSTAR '21 Keynote: Automationist's Gambit

Agile2016: Exploratory Testing an API

Selenium Summit 21: Contemporary Exploratory Testing

MoT Sarajevo: How to get better at Exploratory testing

HUSTEF '21 Keynote: Hands Off Exploratory Testing - Managing at Scale

Geekle QA: Automationist's Gambit

Agile2016: (Lightning Talk) Breaking illusions

DevoxxUK: Learning programming through osmosis

Selenium Conference India: Intersection of Automation and Exploratory Testing

Agile Serbia: Exploratory Testing Explained and Experienced

Mimmit Koodaa - Testiautomaatio on testauksen dokumentointia

SLASSCOM: Breaking illusions with Testing

Tampere Testing Days: Exploratory Testing an API

EuroSTAR Webinar - Mob testing

DDD Europe: Breaking Illusions with Testing (keynote)

MoT Sfax: Good Exploratory Testing Tactics for Taking Over Testing

Testbash Philly: A Mob Testing Experience

AgileTD Scandia: Mob Testing

AGRS2019: Breaking illusions with Testing

Agile India: Working without Product Owner

Viewers also liked

TestWorksConf: Experience exploratory testing

Maaret Pyhäjärvi

TestWorksConf: Exploratory Testing an API in Mob

Maaret Pyhäjärvi

Agile2015: Explore with Intent - Self-Management

Maaret Pyhäjärvi

Getting existing code under tests

Intentional code

Strong Style Pairing

10x

Mob Testing

Viewers also liked (8)

TestWorksConf: Experience exploratory testing

TestWorksConf: Exploratory Testing an API in Mob

Agile2015: Explore with Intent - Self-Management

Getting existing code under tests

Intentional code

Strong Style Pairing

10x

Mob Testing

Similar to AADays Poland: Perfecting the Craft of Test Automation

Working without a Product Owner by Maaret Pyhajarvi at #AgileIndia2019

Agile Software Community of India

For a decade of software product agile, we had worked in a structure where business responsibility of what to build was allocated to a product owner and the responsibility of how to build it was allocated to a development team. Product owner would maintain a backlog, act as voice of the customers. Until one day we realized that the choice of what to build or fix is hard, and critical to everyone’s success. If we wanted to do it poorly, we delegated it to a single product owner. We started a no product owner experiment. For three months, we experienced the development team delivering multitudes of value to what we had grown to expect, and innovate customer-oriented solutions in direct collaboration with customers. Team satisfaction and happiness bloomed. The experiment turned into a continuous way of working. Customer-focused team directly in touch with their customers performs better without a proxy. Join me to learn how the decision power shared for everyone in the team transformed the ability to deliver, and how collaboration is organized with product experts and business representatives. More details: https://confengine.com/agile-india-2019/proposal/7963/working-without-a-product-owner Conference link: https://2019.agileindia.org

Testbash Australia: Next Level Teamwork - Pairing and Mobbing

Maaret Pyhäjärvi

AgileTD: Mob testing Training Day

Maaret Pyhäjärvi

Practices Change - Moving to Delivering Continuously by Maaret Pyhajarvi at #...

Agile Software Community of India

Delivering continuously - easier said than done. As I joined my team delivering Windows Desktop products two years ago with the aspiration of implementing continuous delivery, I was told it cannot be done. It started from the fact that making a single release was five-day effort, and continued with “no user would allow us to deliver frequently”. Two years later, making a release takes us two hours and the way we work together looks very different. With the principle of fixing maintenance issues in the latest release has resulted in improved flow forward and savings of effort in supporting small number of releases. This talk goes through our lessons learned on the journey of shortening release cycles to a continuous daily flow. We’ve experienced three essentially different sets of practices, with the core difference coming from release frequency. The good place is when we deliver continuously. The soulsucking place is when we deliver on two week cadence. And the insane asylum is when we deliver just once at the end of the project. Think of it like this: a pool is not just a bigger bathtub. The things you can do in a pool are different to those you can do in a bathtub. While both are containers of water, they serve different purposes. It is the same with release frequency. While it is all still releases, the continuous ones enable different practices. More details: https://confengine.com/agile-india-2019/proposal/7961/practices-change-moving-to-delivering-continuously Conference link: https://2019.agileindia.org

Agile India: Practices Change - Moving to Delivering continuously

Maaret Pyhäjärvi

AgileTD: Learning through osmosis

Maaret Pyhäjärvi

Oredev: An Exploratory Tester's Lessons on Security Threat Modeling

Maaret Pyhäjärvi

Session on Contemporary Exploratory Testing by Maaret Pyhäjärvi

Agile Testing Alliance

Maaret Pyhäjärvi delivered a session on "Contemporary Exploratory Testing" at #SeleniumSummit21 Maaret Pyhäjärvi is a feedback fairy with a day job at Vaisala, where she works as Principal Test Engineer. She identifies as empirical technologist, tester and programmer, catalyst for improvement, author and speaker, and community facilitator and conference organizer. She was awarded as Most Influential Agile Testing Professional Person 2016, listed on 100 most influential in ICT in Finland by TiVi 2019, and has spoken at events in 25 countries delivering over 400 sessions. To know more about #SeleniumSummit21 please check : https://seleniumsummit21.agiletestingalliance.org/

Keynote at Sanae Beer.EX: The Automationist's Gambit

Maaret Pyhäjärvi

Manual Tester To Automation Tester

Kalyan Katuru

Selenium Conf: Patterns to Whole Team Test Automation Transformation

Maaret Pyhäjärvi

Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers

Andreas Grabner

I have a Dream that Testers extend their horizon and toolsets and not only test for functional correctness but make a step towards what developers need in order to fix critical issues. I am talking about architectural, scalability and performance metrics such as # of JS Files on a page, Page Size, # of SQL Statements, # of Log Messages Written. If Testers start to capture this information as well and share it with their bug description I am sure it will both increase the value of testers as well as reduce the total time it takes to fix problems.

SEETest: Making Teams Awesome

Maaret Pyhäjärvi

WebTest - Efficient Functional Web Testing with HtmlUnit and Beyond

mguillem

AxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory Testing

Maaret Pyhäjärvi

A new generation of testing is emerging, led by testers who believe test automation is an inseparable part of how we do exploratory testing. Yes, you read that right: exploratory testing with test automation, and on unit, api, ui and e2e levels all at our reach with team collaboration. We work on acceptance criteria and examples in our short increments, yet people are at their most creative on what could go wrong when they use the product or a piece of the product as their external imagination. When we’re creative, we need to do things we can’t do by hand, using programming to extend our reach. When we’re learning, we need to take notes now and to keep for later, again using programming to replenish information we deem worthwhile. We want to work on exploring a category of bugs before implementing, and a category of bugs before releasing, and a category of bugs after releasing. Last year I worked with a team that was transforming from a traditional manual test case + test automation -based approach to contemporary exploratory testing. This year I work with another team that takes this style of testing further than we’ve taken it before with whole-team testing. Let’s learn from a practitioner’s examples of ideas leading into this, changes enabling this, and results we are seeing with contemporary exploratory testing.

ATAGTR '20 Keynote: Building status

Maaret Pyhäjärvi

React Finland: Intersection of Automation and Exploratory Testing

Maaret Pyhäjärvi

Getting Started with Python and Machine Learning for SEO | BrightonSEO Octobe...

Ruth Everett

Provisioning and Capacity Planning Workshop (Dogpatch Labs, September 2015)

Brian Brazil

Machine Learning in Production: Manu Mukerji, Strata CA March 2018

Manu Mukerji

Manu Mukerji walks you through Acme Corporation’s machine learning example for universal catalogs, explaining how the training and test sets are generated and annotated; how they were created when there is no public training data available; how the model is pushed to production, automatically evaluated, and used; how Acme Corporation built a Hadoop/Spark pipeline using different types of models predicting various values; production issues that arise when applying ML at scale in production; and lessons learned along the way.

Similar to AADays Poland: Perfecting the Craft of Test Automation (20)

Working without a Product Owner by Maaret Pyhajarvi at #AgileIndia2019

Testbash Australia: Next Level Teamwork - Pairing and Mobbing

AgileTD: Mob testing Training Day

Practices Change - Moving to Delivering Continuously by Maaret Pyhajarvi at #...

Agile India: Practices Change - Moving to Delivering continuously

AgileTD: Learning through osmosis

Oredev: An Exploratory Tester's Lessons on Security Threat Modeling

Session on Contemporary Exploratory Testing by Maaret Pyhäjärvi

Keynote at Sanae Beer.EX: The Automationist's Gambit

Manual Tester To Automation Tester

Selenium Conf: Patterns to Whole Team Test Automation Transformation

Hugs instead of Bugs: Dreaming of Quality Tools for Devs and Testers

SEETest: Making Teams Awesome

WebTest - Efficient Functional Web Testing with HtmlUnit and Beyond

AxiomSummit 2022: A Practitioner's Guide to Contemporary Exploratory Testing

ATAGTR '20 Keynote: Building status

React Finland: Intersection of Automation and Exploratory Testing

Getting Started with Python and Machine Learning for SEO | BrightonSEO Octobe...

Provisioning and Capacity Planning Workshop (Dogpatch Labs, September 2015)

Machine Learning in Production: Manu Mukerji, Strata CA March 2018

Recently uploaded

Top 7 Unique WhatsApp API Benefits | Saudi Arabia

Yara Milbes

Discover the transformative power of the WhatsApp API in our latest SlideShare presentation, "Top 7 Unique WhatsApp API Benefits." In today's fast-paced digital era, effective communication is crucial for both personal and professional success. Whether you're a small business looking to enhance customer interactions or an individual seeking seamless communication with loved ones, the WhatsApp API offers robust capabilities that can significantly elevate your experience. In this presentation, we delve into the top 7 distinctive benefits of the WhatsApp API, provided by the leading WhatsApp API service provider in Saudi Arabia. Learn how to streamline customer support, automate notifications, leverage rich media messaging, run scalable marketing campaigns, integrate secure payments, synchronize with CRM systems, and ensure enhanced security and privacy.

Enterprise Resource Planning System in Telangana

NYGGS Automation Suite

Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics. To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/

In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...

Juraj Vysvader

Enhancing Research Orchestration Capabilities at ORNL.pdf

Globus

Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.

Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management

Utilocate

Utilocate offers a comprehensive solution for locate ticket management by automating and streamlining the entire process. By integrating with Geospatial Information Systems (GIS), it provides accurate mapping and visualization of utility locations, enhancing decision-making and reducing the risk of errors. The system's advanced data analytics tools help identify trends, predict potential issues, and optimize resource allocation, making the locate ticket management process smarter and more efficient. Additionally, automated ticket management ensures consistency and reduces human error, while real-time notifications keep all relevant personnel informed and ready to respond promptly. The system's ability to streamline workflows and automate ticket routing significantly reduces the time taken to process each ticket, making the process faster and more efficient. Mobile access allows field technicians to update ticket information on the go, ensuring that the latest information is always available and accelerating the locate process. Overall, Utilocate not only enhances the efficiency and accuracy of locate ticket management but also improves safety by minimizing the risk of utility damage through precise and timely locates.

Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...

Mind IT Systems

Introduction to Pygame (Lecture 7 Python Game Development)

abdulrafaychaudhry

Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...

Shahin Sheidaei

Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

Globus

The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.

Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...

informapgpstrackings

Essentials of Automations: The Art of Triggers and Actions in FME

Safe Software

In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation. We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios. Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!

OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam

takuyayamamoto1800

Pro Unity Game Development with C-sharp Book

abdulrafaychaudhry

A Sighting of filterA in Typelevel Rite of Passage

Philip Schwarz

GlobusWorld 2024 Opening Keynote session

Globus

Navigating the Metaverse: A Journey into Virtual Evolution"

Donna Lenk

BoxLang: Review our Visionary Licenses of 2024

Ortus Solutions, Corp

Lecture 1 Introduction to games development

abdulrafaychaudhry

Globus Compute Introduction - GlobusWorld 2024

Globus

Cyaniclab : Software Development Agency Portfolio.pdf

Cyanic lab

CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.

Recently uploaded (20)

Top 7 Unique WhatsApp API Benefits | Saudi Arabia

Enterprise Resource Planning System in Telangana

In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...

Enhancing Research Orchestration Capabilities at ORNL.pdf

Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management

Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...

Introduction to Pygame (Lecture 7 Python Game Development)

Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...

Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...

Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...

Essentials of Automations: The Art of Triggers and Actions in FME

OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam

Pro Unity Game Development with C-sharp Book

A Sighting of filterA in Typelevel Rite of Passage

GlobusWorld 2024 Opening Keynote session

Navigating the Metaverse: A Journey into Virtual Evolution"

BoxLang: Review our Visionary Licenses of 2024

Lecture 1 Introduction to games development

Globus Compute Introduction - GlobusWorld 2024

Cyaniclab : Software Development Agency Portfolio.pdf