The document compares GitHub Copilot, Amazon CodeWhisperer, and ChatGPT for Java developers. It provides an overview of each tool, compares their programming language support, IDE support, and pricing. It demonstrates their abilities for general tasks, simple functions, more complex algorithms, JUnit testing, and Spring Boot web development. It concludes that while the tools provide helpful suggestions, developers are still needed to ensure correctness and efficiency. GitHub Copilot and ChatGPT benefit from OpenAI, while Amazon CodeWhisperer needs quality improvements for Java but may leverage AWS services.

1. GitHub Copilot vs Amazon CodeWhisperer
for Java developers
Vadym Kazulkin, ip.labs, JCON Europe 22 June 2023

2. GitHub Copilot (X) vs Amazon CodeWhisperer
vs ChatGPT for Java developers
Vadym Kazulkin, ip.labs, JCON Europe 22 June 2023

3. Contact
Vadym Kazulkin
ip.labs GmbH Bonn, Germany
Co-Organizer of the Java User Group Bonn
v.kazulkin@gmail.com
@VKazulkin
https://dev.to/vkazulkin
https://github.com/Vadym79/
https://www.linkedin.com/in/vadymkazulkin
https://www.iplabs.de/

4. ip.labs
https://www.iplabs.de/

5. Agenda
• Introduction to GitHub Copilot, AWS CodeWhisperer and
ChatGPT
• Comparison
• IDE Support and Pricing
• General tasks
• Simple functions
• More complex algorithms
• JUnit Testing
• Web Development (with Spring Boot)
• AWS Development (optional)

6. Introduction into GitHub Copilot,
AWS CodeWhisperer and ChatGPT

7. • GitHub Copilot
• AWS CodeWhisperer
• Tabnine
• Bard
Code Auto-Completion Overview
Pre-trained transformer-based language models
GPT-2, GPT-3, GPT-3.5, Codex, GPT-4 and others
https://becominghuman.ai/gpt-2-gpt2-vs-gpt-3-gpt3-the-openai-showdown-71bce28fd431

8. Transformer Architecture
https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
https://daleonai.com/transformers-explained https://arxiv.org/pdf/1706.03762.pdf

9. Generative pre-trained Transformers (GPT)
from OpenAI
GPT-2 GPT- 3 Codex
(Copilot)
GPT-3.5
(ChatGPT)
GPT-4 (
ChatGPT
Copilot X)
Open source Closed source Closed source Closed source Closed source
1.5b params 175b params 12b params ? ?
Answers
questions
about text
GPT-2+ Writes
new text
Writes
boilerplate,
repetitve code &
API foreign calls
Edit or insert
content into
existing text
Image
support,
longer text
Translates
text
Rewriting text,
Refactoring
code
Better
reasoning
https://www.slideshare.net/AndyPiper1/copilot-to-cover-why-ai-cant-replace-developers-with-robots-but-can-make-like-better
https://openai.com/blog/gpt-3-edit-insert/ https://platform.openai.com/docs/models/ https://cdn.openai.com/papers/gpt-4.pdf

10. GPT Overview

11. GitHub Copilot Overview
https://medium.com/analytics-vidhya/github-copilot-all-you-need-to-know-8e6fc1d5ccc

12. AWS CodeWhisperer Overview
https://aws.amazon.com/de/codewhisperer/ https://aws.amazon.com/codewhisperer/faqs/

13. ChatGPT Overview
https://www.linkedin.com/pulse/chatgpt-its-implications-enterprise-ai-debmalya-biswas/ https://lifearchitect.ai/chatgpt/ https://cdn.openai.com/papers/gpt-4.pdf

14. Comparisons

15. Programming Language Support
Langauge
Java x x
JavaScript x x
TypeScript x x
Python x x
C# x x
C/C++ x
PHP x
Go x
Ruby x
Scala x

16. Programming Language Support
https://docs.aws.amazon.com/codewhisperer/latest/userguide/language-ide-support.html

17. IDE Support
Langauge
IntelliJ IDEA x x
Visual Studio x
Visual Studio Code x x
Neovim x
GitHub.com x
AWS Cloud9 x

18. Pricing

19. General Tasks

20. Class Creation

21. Class Creation

22. Class Creation

23. Class Creation

24. General Tasks

25. General Tasks

26. General Tasks

27. Simple Functions

28. Simple Functions/ Calendar Utils

29. Simple Functions/ Calendar Utils

30. Simple Functions/ Calendar Utils

31. More Complex Algorithms and Junit Testing

32. More Complex Algorithms
https://projecteuler.net/archives

33. More Complex Algorithms
Fibonacci Numbers

34. More Complex Algorithms
Fibonacci Numbers

35. More Complex Algorithms
Fibonacci Numbers

36. More Complex Algorithms
Fibonacci Numbers

37. More Complex Algorithms
Fibonacci Numbers

38. More Complex Algorithms
Largest 2-digit Palindrome Number

39. More Complex Algorithms
Largest 2-digit Palindrome
Number

40. More Complex Algorithms
Largest 2-digit Palindrome Number

41. More Complex Algorithms
Largest 2-digit Palindrome
Number

42. More Complex Algorithms
Largest 2-digit Palindrome Number

43. Web Development (with Spring Boot)

44. Spring Boot Product Entity

45. Spring Boot Product Repository

46. Spring Boot Product Application

47. Spring Boot Product Controller

48. Spring Boot Product Controller Test

49. Spring Boot Product Entity

50. Spring Boot Product Repository

51. Spring Boot Product Application

52. Spring Boot Product Controller

53. Spring Boot Product Application
Description 4

54. Spring Boot Product Application
Description 4

55. Spring Boot Product Application
Entity 4

56. Spring Boot Product Application
Repository 4

57. Spring Boot Product Application
Service 4

58. Spring Boot Product Application
Controller 4

59. Spring Boot Product Application
Controller 4

60. Spring Boot Product Application
Configuration 4

61. Spring Boot Product Application
Configuration 4

62. Spring Boot Product Application
Deployment 4

63. Spring Boot Product Application
Deployment 4

64. AWS Development

65. Serverless Example Application

66. GitHub Copilot X Overview
GPT-4 Powered Chat
https://github.com/features/preview/copilot-x

67. GitHub Copilot X Overview
AI generated PR-Requests %
Automate automated testing
https://github.com/features/preview/copilot-x

68. Conclusions GitHub
• GitHub Copilot provides lots of meaningful suggestions
• Low number of compilation and algorithmic errors (most are easy
to correct)
• Only small steps required to complete the task or correct the errors
• Access to OpenAI technologies make it very promising
• Bringing GPT-4 capabilities like Chat as Copilot X is a huge step
forward
• GPT-4 has only knowledge as of September 2021
https://twitter.com/karpathy/status/1608895189078380544

69. Conclusions GitHub
https://twitter.com/karpathy/status/1608895189078380544

70. Conclusions AWS CodeWhisperer
• AWS CodeWhisperer UX and suggestion quality needs to be
improved drastically (currently in Preview)
• Many Alt+C on several levels required to get some result
• Especially during the import suggestions
• Many imports statements required in advance in order to get
more or less correct suggestions
• Some compilation and algorithmic errors (most are easy to
correct)
• Drastically improved the quality of the suggestions between
preview and GA

71. Conclusions AWS CodeWhisperer
• Working with AWS Services should become the USP
• Glue together AWS Serverless Services (API Gateway, Lambda,
DynamoDB, SQS, EventBridge)
• Generate Infrastructure-as-a-Code (SAM, CDK) for it
• Integration with Amazon CodeGuru should become out of the
box feature
• My expectation: no CodeWhisperer code completion suggestion
shouldn’t violate best practices highlighted by CodeGuru
• Code Security Scans as an add-on feature

72. Conclusions ChatGPT
• Many suggestions are right or can be fixed quickly
• Sometimes missing pieces that need to be additionally requested
• Sometimes ChatGPT forgets the previous context
• Not all the suggestions are correct though
• Not existing classes or methods are suggested esp. with GPT-3.5
• Each new try with the same “command” may suggest totally different
(correct and sometimes wrong) solution
• ChatGPT-4 is a huge improvement over 3.5 for developers
• GPT-4 has only knowledge as of September 2021

73. Additional benefits of ChatGPT and
CodePilot X GPT-4 powered Chat
• Can explain the existing code
• Can provide suggestions for improving the existing code
• Can search for bugs in the existing code and try to fix it
• Can document the existing code
• Can more easily generate automated tests
• Capable of generating the whole application (Code, Maven/Gradle
dependency file, Deployment (IaaC))

74. Conclusions
• Lot of room the general improvements for the code completion tools
• Suggestion quality, UI/UX
• Developers who effectively use the AI tools will increase their
development speed and productivity
• Your profound expertise is still required to supervise the tool (correctness,
efficiency)
• Main target group -> senior developers
• GitHub Copilot and ChatGPT both powered by OpenAI APIs, so the
improvements and synergies are expected in both
• Already happened with the preview release of the Copilot X based on GPT-4
• Quality of Amazon CodeWhisperer should be improved (at least for Java)

75. Final Thoughts
• Code completion alone are not enough
• need full cycle support: dependency management, code, configuration,
deployment (cloud -> IaaC), testing, observability (logging, monitoring, alerting),
debugging applications.
• Context and environment understanding are crucial (and currently constrained)
• Microsoft will bring products for the conversational programming into the
Azure Cloud
• Copilot Suite (base on GPT-4) is everywhere in the Microsoft world: MS Office,
Bing, Azure Cloud
• Azure OpenAI service (currently limited) is already there.

76. Final Thoughts
• Put Amazon/AWS and Google as a Cloud Provider on pressure to answer
• Amazon partner and Hugging Face partner to make AI more accessible
• Amazon announced Bedrock
• Google answered with Bard

77. Announcing New Tools for Building with
Generative AI on AWS
https://aws.amazon.com/blogs/machine-learning/announcing-new-tools-for-building-with-generative-ai-on-aws/

78. Amazon Bedrock Use Cases
https://aws.amazon.com/bedrock/

79. Amazon Bedrock: Choice of foundation
models from leading AI startups and
Amazon
https://aws.amazon.com/bedrock/

80. Conversational Programming
https://swardley.medium.com/why-the-fuss-about-conversational-programming-60c8d1908237
https://swardley.medium.com/why-the-fuss-about-conversational-programming-70a8b7ca0d2b

82. www.iplabs.de
Accelerate Your Photo Business
Get in Touch