Blog post: https://blog.codingoutloud.com/2024/02/24/talk-orlando-code-camp-meet-github-copilot/

1. Bill
Orlando Code Camp
24-Feb-2024
Bill Wilder
Meet GitHub Copilot
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Your AI Coding Assistant!

3. co-author x2 courses
author
founder and co-organizer
MS Cybersecurity
adjunct professor
https://meetup.com/bostonazure
https://bit.ly/billbook
12x recipient

4. William Gibson
author of Neuromancer (1984)
and other works
“The future is already
here — it's just not very
evenly distributed.”

5. 1.Hello, GitHub Copilot
2.Data about AI for devs
3.What the future holds
4.Questions (anytime okay too)
Agenda

6. Hello, GitHub Copilot
Powered by OpenAI LLM

7. GitHub Copilot is an AI Coding Assistant
• Install VS Code extension (or other IDEs)
• Log into github account (probably $)
• Start coding
Need github account, but your code can live anywhere*
because the extension operates on your local project
only (e.g., on your laptop)

9. 1. ```mermaid
2. graph TD
3. A[Developer's Editor] -->|Prompt Assembly|
B[GitHub Copilot Plugin]
4. B -->|Sends Prompt| C[GitHub Copilot Server]
5. C -->|Returns Suggestions| B
6. C -->|Returns Informational Response| B
7. B -->|Accepts or Rejects Suggestions| A
8. B -->|Displays Informational Response| A

10. Let’s look at GitHub Copilot
• Won’t cover all features
• Attempting to show where it shines and where it
struggles
• Demo will use VS Code – which is probably the best
(most advanced) experience among supported IDEs
• LLM based so is non-deterministic
• Conversational
• Error-prone 
• <codex .>

11. https://xkcd.com/2169/
Ghost text! 
The predictive model
behind GitHub Copilot
is the same one backing
ChatGPT: OpenAI LLM

12. Keep in Mind
• GitHub Copilot, not GitHub Pilot, not GitHub
Autopilot
• Accountability is one of Microsoft’s Responsible AI Principles
• https://www.microsoft.com/en-us/ai/our-approach
• You are the accountable party!
• Not always right
• See “Copilot, not Pilot, not Autopilot” above

13. Is my code safe?
Microsoft does not use your (non-public) code for
training
But you can opt in…
• Unless you can use their new service for training from a
github-hosted repo
• But that’s JUST FOR YOUR USE, not going into overall
model

14. Public code matching

15. Which languages are supported?
GitHub Copilot - Visual Studio Marketplace
https://marketplace.visualstudio.com/items?itemName=GitHub.copilot
“
Supported languages, frameworks, and IDEs
GitHub Copilot works on any language, including Java, PHP, Python,
JavaScript, Ruby, Go, C#, or C++. Because it’s been trained on
languages in public repositories, it works for most popular languages,
libraries and frameworks. It’s also integrated into your editor, and fast
enough to use as you type.
”

16. Understanding GH Copilot w/ Scenarios
GH Copilot is…
• Non-deterministic
• Conversional
• Powered by LLM
• Influenced by Prompt
Engineering/Prompt Crafting
• Sends “context window” in
the form of “prompts” from
desktop to server, gets back
”suggestions” (or just
“informational response”)
Example Scenarios…
• “Knows” patterns
• Make me a DTO (stackoverflow)
• “Knows” Syntax
• Generate Unit Tests
• Change Unit Test from X to Y
• e.g., MS Test to xUnit
• Efficiency
• e.g., make prime test faster
• Test data, such as in Spanish
• Explain code
• Look for errors

17. Productivity Tips
1. Prompt Crafting (aka “Prompt Engineering”) matters
• Ask for what you REALLY want
2. Keep updated
• Extension/plugin ecosystem moves fast
3. Productivity features are coming all the time
• Watch the docs, team demos on YouTube, etc.

18. Anti-Patterns to Avoid
1. Blindly taking a suggestion (“let copilot drive”)
• Pilot vs. Copilot
• Style (e.g., naming, param structure)
• Substance (e.g., correctness, efficiency, security)
2. Insufficient Context (”mind reading”)
• Context, prompt engineering, multi-shot
3. Asking Copilot to do it all (“hammer mode”)
• Don’t forget about your other tools, including your keyboard
4. Treating inline as chat (“mode confusion”)
• Chat acts on instructions, Inline acts on coding intent

19. Data about AI for devs

20. 2023 Stack
Overflow
Developer Survey
https://survey.stackoverflow.co/2023/
Dev tool of choice

21. VS Code
Marketplace
https://marketplace.visual
studio.com/items?itemNa
me=GitHub.copilot
13m downloads

22. 2023 Stack
Overflow
Developer Survey
https://survey.stackoverflo
w.co/2023/#ai-developer-
tools
AI dev tool choices

23. https://survey.stackoverflo
w.co/2023/#ai-sentiment-
and-usage
Using AI for dev?
Sentiment
2023 Stack
Overflow
Developer Survey

25. 2023 DORA
Report
⚠️ Less flattering…

26. GitHub research

27. What does the future hold?

28. https://github.blog/2023-11-08-universe-2023-copilot-transforms-github-
into-the-ai-powered-developer-platform/
“Just as GitHub was founded on Git, today we are re-founded
on Copilot. Open source and Git have fundamentally transformed
how we build software. It is now evident that AI is ushering in the
same sweeping change, and at an exponential pace. In just a
short period, GitHub Copilot has expanded and evolved GitHub
into the world’s leading AI-powered developer platform.”
“re-founded on Copilot.”

29. “Let me just kind of give you two anecdotes
and then try to generalize it. On New Year's
Day this year, I just looked in my
newsfeed and I saw this tweet that
Andrej Karpathy, who is the guy at
Tesla who did the autopilot, tweeted
saying 80% of the code he writes today
is being written by GitHub Copilot.”
-Satya Nadella from WSJ interview, Jan 2023
WEDNESDAY, JANUARY 18, 2023
Microsoft CEO Satya Nadella's Big Bet on AI - The Journal. - WSJ Podcasts
https://twitter.com/karpathy/status/1608895189078380544

30. The Copilot Family
• GitHub Copilot
• Mobile GH App
• PowerApps (more later)
• Azure Data Studio
• Defender
• Microsoft 365
• Word, Excel,
• PowerPoint, Teams,
• Outlook, …
• Windows
• Dynamics
• …

31. Is Stack Overflow in trouble?
• https://www.wired.com/story/stack-overflow-will-charge-ai-giants-for-training-data/
• Stack Overflow Will Charge AI Giants for Training Data
• The programmer Q&A site joins Reddit in demanding compensation when its data is used to train
algorithms and ChatGPT-style bots
• https://www.theverge.com/2023/10/16/23919004/stack-overflow-layoff-ai-profitability
• Stack Overflow lays off over 100 people as the AI coding boom continues
• / Stack Overflow has laid off 28 percent of its staff over a year after doubling its employee base in a
massive hiring push.
• https://techcrunch.com/2023/10/17/stack-overflow-cuts-28-of-its-staff/
• https://arstechnica.com/gadgets/2023/10/after-chatgpt-disruption-stack-overflow-
lays-off-28-percent-of-staff/
• https://www.infoworld.com/article/3708738/generative-ai-forces-stack-overflow-to-
lay-off-28-of-its-workforce.html

32. Trends for GitHub Copilot, OpenAI
• Token windows
• GPT-3.5-Turbo = 4k token input window
• GPT-3.5-Turbo-16K = 16k token input window
• GPT 4 – up to 128k token input window?
• Training on your code

33. Thought Experiment #1 – Agile Pressure
• Common Agile (e.g., Scrum)
practice is to build have a
buffer in the backlog
• Sprints of 2-3 weeks are
planned out with stories
• Wash, rinse, repeat
Intra-sprint velocity thrown
out of balance.
What if PO can’t keep up
anymore?
What if QA process can’t keep
up anymore?
Role of Continuous Delivery?

34. Thought Experiment #2 – VSM
• Value stream mapping
defines cycle time and
lead time
• Cycle time: time coding a
feature/story
• Lead time: getting feature
to customers
• Lead time > Cycle time
What happens to lead time
as cycle time is shortened?
Imagine infinitely fast dev
cuts C/T to 0, exerting max
pressure on CD C/T.
Role of GHC in automation.
Role of automation in lead
time. DORA metrics matter
more.

35. Thought Experiment #3 – “the dividend”
• Assume more efficient, but
maybe we don’t ”spend it all” on
more speed…
• How will you spend your
“dividend”?
Do same thing faster?
Do same thing more robustly in
same amount of time?
Do more things in same amount of
time? (More features or pay off
tech debt.)
Max developer happiness per
SPACE metrics?
Or do with fewer developers?

36. Thought Experiment #4 – citizen devs
• Will less skilled coders be
able to build maintainable
software?
• Not clear there’s evidence
GHC will turn a mediocre
dev into a good one (quote
from Andrej Karpathy)
• But what about low/no code
coupled with AI?
Will spaghetti code matter
with AI available to update it?
PowerApps has AI Copilot
• “Build apps through
conversation”
https://learn.microsoft.com/en-us/power-apps/maker/canvas-apps/ai-overview
Echos of Visual Basic in 90s?

37. OWASP Top 10 for LLM Applications
• https://owasp.org/www-project-top-10-for-large-
language-model-applications/

38. Questions?

39. Questions?
Bill Wilder
@codingoutloud
codingoutloud@gmail.com
blog.codingoutloud.com
linkedin.com/in/billwilder
Find this
slide deck
here
See you at
Boston
Azure
bostonazure.org
@bostonazure