PSU 2023 Final Report - ISSIP_AI_Collab.docx

Generative AI Playbook and Workflow Analysis
AI Team
Final Report
Date: 12/01/2023
Submitted to: ISSIP
Jim Spohrer
3561 Homestead Rd, Santa Clara, California, 95051
Submitted by: AI Team
John Pekor, Abigail Moliski, Ahmad Alhabib, and Shiquan Zhang
Advisor: Vittal Prabhu
Y N
Or Intellectual Property Rights Agreement Applies
Or Non-Disclosure Agreement Applies

2
EXECUTIVE SUMMARY
The Generative AI Playbook and Workflow Analysis is designed to benefit the International
Society for Service Innovation Professionals (ISSIP) by creating a comprehensive guide for
leveraging generative AI tools to enhance service innovation. ISSIP, a non-profit organization
based in Santa Clara, California, aims to connect, recognize, and support organizations involved
in service innovation worldwide. With 1600 individual members, over 400 companies, over 200
universities, and a presence in 42 countries, ISSIP is a prominent force in promoting service
innovation. ISSIP’s main goal is to become the undisputed leader in service innovation and through
the Generative AI Playbook and Workflow Analysis ISSIP volunteers will be able to help
volunteers create informative webpages on historic service innovations. The playbook will also
aim to control the input, productivity, and quality control of the content created. Through the
Generative AI Playbook and Workflow Analysis, a proper procedure was laid out for volunteers
to create content with generative AI. The playbook also met the requirements of utilizing free AI
tools, covering benefits and harms of service innovation, and promoting the ethical use of AI tools.
Two other requirements for the playbook were that the playbook result in a satisfactory output
being produced 70% of the time and that the playbook lead to users producing a quality output in
under two attempts. The 70% satisfaction goal was met but the users needing under 2 attempts to
produce a quality output was not met largely due to inconsistency with AI image tools and users
exploring and experimenting with AI tools. To create the playbook, the team first did AI quick
sprints in which team members created webpages with generative AI content. This helped the team
familiarize themselves with AI tools and after that the first playbook prototype was made. After
this, a playbook survey was made and the playbook was continually changed based on the feedback
the team received. Around when the final playbook prototype was made, the team also created an
AI tool selection guide, three case studies, and a quality and productivity framework. From the
final prototype of the playbook, it was found that the playbook was effective and met most of the
requirements set, with the exception of users needing under 2 attempts to reach a quality output.
This project had a total project expenditure of $0 which is not very far from the estimated project
expenditure of $50 from the beginning of the semester. The team did not need to spend any money
for creating the playbook and using AI tools.

3
TABLE OF CONTENTS
Contents
EXECUTIVE SUMMARY …...................................................................................................... 2
TABLE OF CONTENTS ….......................................................................................................... 3
1.0 INTRODUCTION ….............................................................................................................. 4
2.0 PROBLEM STATEMENT …................................................................................................. 4
3.0 PROJECT OBJECTIVES …................................................................................................... 5
4.0 PROJECT MANAGEMENT / APPROACH …..................................................................... 6
4.1 Statement of Work …......................................................................................................... 7
4.2 Schedule …......................................................................................................................... 7
5.0 RESULTS ............................................................................................................................. 12
6.0 PROJECT EXPENDITURES …........................................................................................... 15
7.0 ACKNOWLEDGEMENTS ….............................................................................................. 17
REFERENCES …....................................................................................................................... 18
APPENDIX A: BUDGET AND EXPENDITURES ….............................................................. 19

4
1.0 INTRODUCTION
The company the AI playbook is being made for is ISSIP. ISSIP stands for “International Society
for Service Innovation Professionals” [2]. ISSIP is a non-profit that connects, recognizes, and
supports organizations supporting service innovation that benefits people, business, and society.
ISSIP is based in Santa Clara, California. ISSIP was founded in 2012 by IBM, Cisco, HP, and
highly respected universities. ISSIP is made of 1600 individual members, over 400 companies,
over 200 universities, and is involved in 42 countries.
2.0 PROBLEM STATEMENT
ISSIP is looking for the team to research, design, and create a standard AI playbook for prompt
engineering of generative AI tools including the input, productivity, and quality control of the
output. The primary purpose of the playbook is to help ISSIP volunteers create content that
explains historical service innovations. The content being created are essays, images, videos, and
websites. AI is a very powerful resource and when used properly it can save time and prove to be
a great help. Assuming that a human is solving a problem in five minutes, AI is capable of solving
10 problems in one minute [3]. With AI being capable of being a more efficient problem solver
than humans, AI is a powerful resource that almost everyone should investigate due to its
capabilities. Even with AI’s great capabilities, a large amount of people still do not interact with
AI. According to a survey from Pew Research Center surveying 11,004 adults, 44% of people do
not interact with AI [4]. With 44% of people not interacting with AI, a large amount of people
have not interacted with AI and do not know how to interact with AI. ISSIP having the team
working on an AI standard playbook can prove to be beneficial to those who are new to interacting
with AI. By having new AI users follow along with the standard AI playbook, the new AI users
may be able to get their desired outcomes by following the steps or instructions highlighted in the
AI playbook. Writing a one page paper with in depth research can take up to 1.7 hours [1]. With
writing essays being time consuming, AI can help by creating a quality essay in under a minute.
The playbook will help the users achieve higher productivity by saving volunteers time and by
achieving content that is of higher quality than what the volunteers would produce on their own.
Results of the playbook will be obtained by surveying peers of the team and asking them questions

5
on the quality of the output and how much time they believed that the AI playbook saved them to
get that output. The AI playbook can prove to be very useful by helping inexperienced AI users
harness the powers and capabilities of AI to create content that explains historical service
innovations. Other supplementary deliverables include an AI tool selection guide, three case
studies or example webpages, and a quality and productivity framework were made to meet
requirements from our sponsor.
3.0 PROJECT OBJECTIVES
The project's objectives are multifaceted, designed to create a comprehensive AI playbook that
serves ISSIP volunteers and beyond. Firstly, the playbook's core aim is to empower ISSIP
volunteers, regardless of their prior AI experience, by providing a robust guide to effectively
leverage generative AI tools. This initiative aligns with the broader objective of promoting AI
adoption, addressing the fact that a significant portion of the population remains unfamiliar with
AI. By demystifying AI interactions, this project seeks to encourage volunteers and potential users
to embrace the potential of AI, ultimately enhancing problem-solving efficiency.
Moreover, the playbook plays a pivotal role in facilitating the creation of content that elucidates
historical service innovations, making it both informative and engaging. Standardization is another
key objective, ensuring that AI tool selection, workflow processes, and quality control measures
are consistent and dependable. Skill development is also at the forefront, with the project
committed to fostering the growth of AI newcomers. Clear, user-friendly instructions will
empower individuals to harness AI's capabilities effectively for creating high-quality content.
In addition to facilitating the creation of content for service innovations, the playbook will serve
as a way to result in higher volunteer productivity and increase the quality of the content created.
The playbook will aim to save volunteers time by directing them in a direction in which a quality
output will be created. Productivity will be measured by surveying peers how long it took them
to reach a satisfactory output, how many attempts it took, and how much time the playbook
likely saved them. The team hopes that the playbook results in under 2 attempts being needed to
reach a quality output and that these attempts take under 15 minutes to do.

6
The playbook will also lead to higher quality content being produced. The quality of the content
will be measured by surveying peers and asking them questions on how the quality of the output
was. A scale from 1 to 5 will be used to rate this. The team hopes that over 70% of peers find
their content results satisfactory (3 or higher).
Finally, timely delivery is a fundamental objective, ensuring that the AI playbook becomes
available as per ISSIP's requirements. By uniting these objectives into a cohesive strategy, the
project endeavors to produce a valuable resource that not only empowers ISSIP volunteers but also
advances the broader cause of AI adoption and content creation for historical service innovations.
4.0 APPROACH
The team’s approach for creating an industry-grade AI playbook and workflow analysis that
complies with user-defined constraints involves a series of tasks. Initially, each team member
conducts practice runs with various AI tools in three particular cases: Text, Image, and Video
generation, then documenting each tool’s features, strengths, and weaknesses including output
quality and productivity gain.
To assess the quality of our output, the project will make use of a standardized survey
methodology. This evaluation will allow for a less subjective analysis of each AI tool’s output
quality. Productivity will be evaluated through the amount of time saved with the use of a particular
AI tool in each of the three cases as discussed previously.
The team will share their findings from the AI tools practice runs in order to establish playbook
concepts. Building upon the findings gathered from individual AI practice runs, the team will
conduct a concept review to explore possible formats for the playbook. This phase will be followed
by a preliminary design review to evaluate the pros and cons of each concept.
Following the concept review, the team will select the most promising playbook format and
advance to the creation of an alpha prototype. This prototype will undergo a survey process to

7
gather valuable feedback. Based on the insights obtained from these surveys, a critical design
review will be conducted. Subsequently, a beta prototype will be developed, incorporating
improvements and enhancements. Upon the completion of the beta prototype, a final survey will
be distributed to collect feedback, and using these insights, the team will craft the ultimate
playbook prototype.
The team will then proceed to create a comprehensive final report, design a poster showcasing the
prototype, and prepare for the final showcase presentation.
4.1 Statement of Work
At the beginning of the project timeline, each team member will experiment with AI to find what
works and doesn’t work for the project. All AI tools in this case should be free to use as the
volunteers in ISSIP will require more accessible tools.
Each team member selects certain innovations and collects essays, images, and videos about these
innovations. Once every member finds something worth sharing from their findings, this will be
shared with the team through a meeting or through the teams group chat. With all of the team
members findings a concept review will be held to start creating concepts for the playbook. The
statement of work is to be completed with all team members contributing to develop the statement
of work.
After the concept review, a preliminary design review will be held so the team can go over the
pros and cons of each design for the playbook. Once the design review is done, the team will have
certain metrics to follow to determine the best design to pursue. With the design selected, the team
will begin developing an alpha prototype.
When the alpha prototype is made, a standard testing process will be made, and surveys will be
conducted to receive feedback on the prototype. With the feedback from the surveys, a critical
design review will be conducted by the team to help create a beta prototype. With the beta
prototype in hand, a final survey will be sent out to peers of the team and with this information the
final prototype will be finalized by the team. In the end, when the final prototype is made the team

8
will draft and make a final report, present a poster on the final prototype, and present the final
prototype at the final showcase. Materials may need to be purchased for the final prototype poster.
4.2 Schedule
The project’s timeline and task dependencies can be seen in the following Gantt chart, which
details phases, tasks, milestones, risks, and percent completion of any given part. The tasks are
split into 6 phases: Empathize; Define; Ideate; Prototype; Test; Closeout.

12
5.0 RESULTS
Overall, the Generative AI Playbook and Workflow Analysis met the set requirements for this
project. The playbook itself is a procedure highlighting how to use generative AI to create
content and put all of this content into a webpage. In addition to the playbook, an AI tool
selection guide, three case studies or example webpages, and a quality and productivity
framework were made to meet requirements from our sponsor. The AI tool selection guide
serves as a guide to inform volunteers about AI tools and help them select the right tool to create
content with. The three case studies are example webpages made by the team and have a
procedure that the team followed to create these webpages. The case studies can prove to be
useful as ISSIP volunteers can see what the team did to create their webpages. The quality and
productivity framework goes into detail on how effective the playbook is and displays an
understanding of the goals of the playbook. Regarding the requirements set for the Generative AI
Playbook and Workflow Analysis, from the playbook survey, 100% of users stated that the
playbook was user friendly and organized in a good manner. This indicates that the playbook is
easy to follow and will not be a challenge for ISSIP volunteers to navigate and follow. Another
requirement met by the playbook is that the playbook covers the benefits and harms of service
innovation. In the playbook, there are several examples of content being created to cover the
benefits and harms of service innovation. Another driving requirement for the playbook was that
the playbook utilize free AI tools. The playbook covers the use of free AI tools for all content
created, making the playbook more convenient for volunteers as they will not have to spend
money to create content with AI. The playbook also had a requirement that the playbook cover
proper ethics when using AI such as citing the AI that created the content. The playbook has a
section on how to cite AI and cites AI properly so the volunteers know what to expect when
citing AI. Lastly, there are 2 metrics the playbook was to meet. The first was that 70% of users
reach satisfactory outputs by following the playbook. The playbook led to users reaching a
quality output 92% of the time, with the content not being satisfactory being the webpage. The

13
webpages not being satisfactory can largely be attributed to learning HTML coding for the first
time. The other metric to be met was that users need under two attempts to reach a quality
output. This goal was not met but our sponsor did state that this was a very ambitious goal and
would be difficult to meet. The playbook led to an average of 2.17 attempts being needed to
reach a quality output. For this project, the tools used were all digital with these tools being
Google Forms, Microsoft PowerPoint, generative AI, and a notepad to practice HTML code. To
test the playbook, the team created a Google Form so the team can receive feedback on the
playbook from peers the team found to follow the playbook. Overall, the playbook met most of
the set requirements set for it, likely leading to the playbook being a useful tool for ISSIP
volunteers to follow to create content with AI. The expected benefits of the playbook are that
users will create content of higher quality, create content in less time, and be able to learn more
about generative AI through the use of the playbook.
Figure 1: Verification Cross-Reference Matrix

14
Figure 2: Example #1 of Case Study Created by Team

15

16
6.0 PROJECT EXPENDITURES
Equipment
Equipment Anticipated Costs: $0
Most work will be completed using personal computers through the internet whilst testing AI
APIs and playbook write-up.
Materials
Materials Anticipated Costs: $0
No physical materials will be required due to the nature of this project and its software based
orientation.
Travel
Travel Anticipated Travel Costs: $0
Due to the nature of the project, travel is unnecessary since all work will be conducted
electronically including research and prototyping.
Software
Software Anticipated Costs: $0
It was decided by the sponsor that the focus will be mainly on free AI APIs due to mass
availability for employees among other factors.
Miscellaneous
Miscellaneous Items Anticipated Costs: $0
There are no other items/tools that the team needed to purchase for this project.

17
Summary
Total Anticipated Costs: $50
The budget aligns with the project’s goal of increasing productivity with the use of widely
accessible tools for ISSIP employees around the globe.
7.0 ACKNOWLEDGEMENTS
The team would like to acknowledge our team members Abigail Moliski, Ahmad Alhabib, John
Pekor, and Shiquan Zhang for the creation of the playbook. In addition to the team, the team would
like to acknowledge Dr. Vittal Prabhu who served as the teams faculty advisor, Anshul Balamwar
who is Dr. Prabhus PhD student, Toni Rae who served as the teams alumni coach, and Jim Spohrer
who served as the ISSIP sponsor for this project.

18
REFERENCES
1. Capitilize My Title. (2021, March 29). How long does it take to write 1 pages?.
Capitalize My Title. https://capitalizemytitle.com/writing-time/1-
pages/#:~:text=Writing%201%20pages%20will%20take,can%20grow%20to%201.7%20
hours.
2. ISSIP. (2023, August 7). Welcome to ISSIP. https://issip.org/
3. Kennedy, B. (2023, February 15). Public awareness of artificial intelligence in
Everyday Activities. Pew Research Center Science & Society.
https://www.pewresearch.org/science/2023/02/15/public-awareness-of-artificial-
intelligence-in-everyday-activities/
4. Simplilearn. (2023, June 5). Artificial Intelligence vs. human intelligence: Will AI
replace humans. Simplilearn.com. https://www.simplilearn.com/artificial-intelligence-vs-
human-intelligence
article#:~:text=Computers%20have%20the%20ability%20to,ten%20problems%20in%20
one%20minute.

19
APPENDIX A: BUDGET AND EXPENDITURES
Start Date of Project: 8/21/2023
End Date of Project: 12/8/2023
Budget Item Proposed Interim Actual
Travel $0 $0 $0
Equipment $0 $0 $0
Materials $0 $0 $0
Software $0 $0 $0
Miscellaneous
 Project
poster
 Any
other
expenses
$0 $0 $0
Total Project Costs $0 $0 $0

20
Appendix B: Productivity and Quality
Framework
Generative AI Playbook and Workflow Analysis:
Quality and Productivity Framework
This quality and productivity framework will assess how effective the playbook is at
helping ISSIP create content and how effective the playbook is at helping others understand
historic service innovations. Lots of data referenced in this quality and productivity framework
are from the playbook survey the team sent out for others to try out the playbook. The playbook
the surveyed users used was not the final playbook prototype so it is likely these results can
improve with the changes the team made to the final prototype. This quality and productivity
frame work will assess the playbook in how organized and easy the playbook was to follow, how
effective the playbook is in creating content to understand historic service innovations, the
quality of content created, how many attempts it took to create a quality output, and how long it
took users to create content.
To start the quality and productivity framework, the effectivity of the playbook will be
assessed. The areas that will be looked at are how organized and easy to follow the playbook was
to users, the quality of content created, and how the playbook helped users have higher
productivity. First, from the users surveyed, just under 90% of users gave the playbook a score of
4 or 5 on how easy the playbook was to navigate. This indicated that the playbook is laid out in a
reasonable format and does not lead to issues for users. In addition to this, just under 90% of
users gave the playbook a score of 4 or 5 on if the playbook instructions were easy to follow.
This leads the team to believe that the instructions in the playbook have high quality and make
sense for ISSIP volunteers to follow.
Next, the quality of the content created will now be assessed. With the 4 types of content
created with the playbook (essays, images, videos, and webpages) there were only 2 instances in
which a poor output was produced. Both of these instances happened to be with creating a
webpage. These 2 poor instances can largely be due to the fact that creating a webpage is the
hardest thing to create with the playbook and that users may be inexperienced with HTML,
meaning there is a lot to learn from the playbook with HTML code. Given, that there were only 2
instances in which a users thought a poor output was produced, it can be assumed that the
playbook is effective in generating quality content.
The last metric in which the playbook can be assessed is how the playbook helps users be
more productive. It took an average of 2.32 attempts for users to reach a quality output for all

21
types of content. This number is a little higher than what the team was hoping for but the number
is certainly not terrible. The attempts aspect of productivity can be misleading as some users will
take a lot of attempts just learning and exploring the capabilities of AI, so in the end the team
believes being able to reach a quality output in the end is most important. It took users an
average of around 7.5 minutes to create a quality essay, around 5.6 minutes to create a quality
image, around 21 minutes to create a quality video, and roughly 47 minutes to create an HTML
webpage. The team found that the averages for essays, images, and videos were good but the
average for the HTML was high. Again, the high times for HTML could largely be due to users
not having experience with HTML. Due to these high times, the team will try to supplement as
much information and support as possible to help volunteers with their HTML webpage. Lastly,
we can compare the average times it took to create content to the users estimation on how long it
would take them to find or create content similar to what was created without the playbook. The
users surveyed said it would take them roughly 2.5 hours to create an essay, roughly 4 hours to
get an image like they created, roughly 3.5 hours to create a video similar to what they made, and
roughly 50 minutes to create a webpage with nearly half of users not being able to create a
webpage without the playbook. In the essays, images, and videos there is a clear difference in the
times that can lead to the team saying that the playbook greatly improves users productivity by
essentially making tasks that can take a couple of hours be done in around 20 minutes. The
standout is again creating webpages, this figure was around the same and it can be mostly due to
inexperience with HTML coding and needing a lot of support to learn about coding. The team
also thinks it is worth noting nearly half of users were not able to create a webpage without the
playbook so the team definitely thinks the playbook may not necessarily help users create
webpages from a productivity standpoint but does allow the users to be capable to creating
webpages. Overall, the team can take away that the playbook was set up in a good manner and
had good instructions, led to quality content being made, and led to a large increase in users
productivity.
Next, the quality and productivity framework will assess how effective the playbook is at
helping others understand historic service innovation. This is the main goal of the Generative AI
Playbook and Workflow Analysis, making this an important area to look at. Of the users
surveyed 75% of users gave the playbook a rating of 4 or higher on how effective the playbook is
at creating content to understand historic service innovation. The took away from this that the
playbook can create quality content and that this content is effective in helping others understand
historic service innovations. With the users surveyed indicating that the playbook is effective at
creating content to help understand service innovation, the team thinks that the playbook is
ultimately effective at its main goal.
The final topic this quality and productivity framework will discuss is if the playbook
will help ISSIP be a leader in service innovation. This area is definitely hard to explain with data
as it is largely unknown how effective the ISSIP volunteers webpages will be at making ISSIP a
leader in service innovation. This leads to the team largely making assumptions on this. The
team believes that the webpages that the ISSIP volunteers will make will be effective in
informing others about historic service innovation. However, the webpages being effective at
informing others about service innovation isn’t enough to be a leader in service innovation. The

22
team believes that there has to be a way in which these webpages are going to be visited and
looked at by a large number of people. If these ISSIP volunteers webpages can be viewed by lots
of people the team believes that ISSIP will then be a leader in service innovation. Ultimately, the
team believes that the playbook will be able to lead to quality webpages being made and will
help others understand service innovation. The main concern the team has for these webpages
helping ISSIP be a leader in service innovation is getting these webpages views and recognition
to help ISSIP in its mission to be a leader in service innovation. If ISSIP is to find a way in
which these webpages will come up in a similar way to how people google or search for things,
ISSIP can become a leader in service innovation as the quality of webpages will be seen. In the
end the team believes if the webpages get enough views ISSIP can be a leader in service
innovation if enough people see the webpages and are happy enough with the webpages to go
directly to ISSIP to look up a webpage on a historic service innovation. If ISSIP can find a way
to get the webpages enough views and recognition and becomes the place to go for information
on service innovation, the team strongly believes that ISSIP will be one of the leaders in service
innovation.

PSU 2023 Final Report - ISSIP_AI_Collab.docx

Recommended

Recommended

More Related Content

Similar to PSU 2023 Final Report - ISSIP_AI_Collab.docx

Similar to PSU 2023 Final Report - ISSIP_AI_Collab.docx (20)

More from International Society of Service Innovation Professionals

More from International Society of Service Innovation Professionals (20)

Recently uploaded

Recently uploaded (20)

PSU 2023 Final Report - ISSIP_AI_Collab.docx