The document discusses using ChatGPT as a testing tool. It outlines the author's assumptions and research using ChatGPT to generate code, improvements, and unit tests for Angular applications and APIs. The author notes positives include ChatGPT's ability to suggest code improvements and generate unit tests over 50% of the time. However, the author cautions that evaluating ChatGPT's responses requires knowledge and experience to determine usefulness.

1. Confidential & Proprietary
www.leadingedje.com
ChatGPT as a
Testing Tool
Bob Fornal

2. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
About Me
● Husband and Father
● Senior Solutions Developer at Leading EDJE, Inc.
● 1978 - Started Writing Code
● 1990 - Completed BS CPS
● 1994 - Started Teaching Computers
● 2012 - Select Sires, Inc.
● 2016 - Leading EDJE, Inc.
● May 2023 - Code Squid (https://code-squid.com)

3. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
ASSUMPTIONS (when I started)
1. ChatGPT is a tool.
2. The data is out of date.
3. It’s a natural language processing tool driven by “AI” technology.
4. 100 million active users in January, only two months after its launch.
5. Trained on vast amounts of information from the Internet, including websites, books, news
articles, and more.

4. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
Baseline Research
Senior Developer
In Person
Video
When ChatGPT is confidently wrong
Student Creates App(s)
Fully ChatGPT generated
Interview
Lag before answering

5. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
Inmate Mentality
● If ChatGPT were human, you’d say it’s “lying through its teeth.”
● Inmates always blamed someone else. I often heard, “everyone around me
was doing (this dumb thing), so I did too.”

6. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
My Research
Angular API Code Angular Library Code
GitHub Actions to
GitHub Pages
Angular / React
Email Regular Expressions

7. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
Thoughts …
1. Phrasing a QUERY is as complicated and nuanced as a Google Search.
2. It takes KNOWLEDGE and EXPERIENCE to evaluate the usefulness of a
particular answer.
3. An answer can be beyond our SKILLSET to evaluate.
4. It generates a good list of IMPROVEMENTS to code.
5. It generates a solid set of UNIT TESTS most of the time (50% plus).

8. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
Questions Used (Angular API Code)
1. I'm looking for javascript code to connect to an api endpoint with basic error
handling.
2. Can you do the same thing with Angular and TypeScript?
3. Can the same thing be done with async/await and try/catch blocks?
4. Can you create an angular service with a getData function that gets from an
api endpoint using async/await and try/catch blocks and stored the data in a
behavior subject that can be subscribed to?
5. Can you write the unit tests for this service?
6. Is there a better way to design the service?

9. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
Questions Used (Angular API Service)
Asked, “Is there a better way to design the service?”
It depends on the specific requirements of your application, but here are some ways that you could improve the
design of the service:
1. Return a single instance of the BehaviorSubject: Instead of returning a new Observable from the
getData method every time it is called, you could return a single instance of the BehaviorSubject that is
created in the service. This way, multiple subscribers will receive updates from the same subject.
2. Cache the data: If the data from the API is not expected to change frequently, you could cache the data in
the service after the first successful request and return the cached data on subsequent calls to the getData
method. This will reduce the number of unnecessary HTTP requests and improve the performance of the
application.
3. Return an error state: Instead of logging an error to the console, you could return an error state in the
BehaviorSubject that indicates whether the data was successfully fetched or if an error occurred. This will
allow components to handle the error state and display appropriate error messages to the user.
4. Use a generic type for the BehaviorSubject: You could use a generic type for the BehaviorSubject to
specify the type of the data that it will be storing. This will improve the type safety of the code and make it
easier to catch errors during development.

10. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
Questions Used (Angular Library)
1. Can you generate an input component for an Angular Library?
2. Can you generate the same component with an Event Emitter?
3. Can you show me how to use this component?
4. Can you show me unit tests for the InputComponent?
5. Show me a Storybook story for the InputComponent (that handles the
value change).

11. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
Questions Used (Regular Expressions)
1. Generate a javascript email regular expression.
2. Generate a comprehensive email regular expression.
3. Can you design some unit tests against the comprehensive regular
expression?

12. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
Positives (PSA)
1. Asking “what does this code do” seems OK.
2. Asking “what could be done to improve this code” seems OK.
3. Asking for Unit Tests seems to work in SOME situations.
4. Debugging Code might be OK.

13. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
Thoughts …
1. Phrasing a QUERY is as complicated and nuanced as a Google Search.
2. It takes KNOWLEDGE and EXPERIENCE to evaluate the usefulness of a
particular answer.
3. An answer can be beyond our SKILLSET to evaluate.
4. It generates a good list of IMPROVEMENTS to code.
5. It generates a solid set of UNIT TESTS most of the time (50% plus).

14. Confidential & Proprietary
www.leadingedje.com
Confidential & Proprietary
www.leadingedje.com
What to Avoid When
Writing Unit Tests