I will introduce two AWS services: CodeGuru and DevOps Guru. CodeGuru Reviewer uses ML and automated reasoning to automatically identify critical issues, security vulnerabilities, and hard-to-find bugs during application development. DevOps Guru analyzes data like application metrics, logs, events, and traces to establish baseline operational behavior and then uses ML to detect anomalies. It does this by having the ability to correlate and group metrics together to understand the relationships between those metrics, so it knows when to alert.

1. Revolutionize DevOps with ML capabilities
Deep dive into DevOps Guru and CodeGuru
Vadym Kazulkin, ip.labs, 14 June 2022

2. Contact
Vadym Kazulkin
ip.labs GmbH Bonn, Germany
Co-Organizer of the Java User Group Bonn
v.kazulkin@gmail.com
@VKazulkin
https://www.linkedin.com/in/vadymkazulkin
https://www.iplabs.de/

3. ip.labs
https://www.iplabs.de/

4. AWS DevOps Guru

5. What is AWS DevOps Guru
Amazon DevOps Guru is a service powered by machine learning
(ML) that is designed to make it easy to improve an application’s
operational performance and availability
DevOps Guru helps detect behaviors that deviate from normal
operating patterns so you can identify operational issues long
before they impact your customers
• increased latency
• error rates (timeouts, throttles)
• resource constraints (exceeding AWS account limits)
https://aws.amazon.com/devops-guru

6. Benefits of DevOps Guru
https://aws.amazon.com/devops-guru

7. How DevOps Guru work
https://aws.amazon.com/devops-guru

8. DevOps Guru integration in 3rd party incident
management tools
https://aws.amazon.com/devops-guru
• PagerDuty
• Atlassian Opsgenie

9. DevOps Guru Example Application

10. DevOps Guru Set Up

11. DevOps Guru Integration with PagerDuty
https://www.pagerduty.com/docs/guides/amazon-devops-guru-integration-guide/

12. DevOps Guru Integration with PagerDuty
Enter „Integration
URL“ generated by
PagerDuty

13. DevOps Guru Dashboard

14. DevOps Guru Dashboard

15. DevOps Guru Insights

16. DevOps Guru PagerDuty Incidents

17. • Design test experiment to provoke errors and latency increase
• Reduce the service quote of the AWS service (API
Gateway, Lambda, DynamoDB)
• Set very low service quotas for the sake of
reducing AWS costs only
• Add latency artificially
• Stress test with JMeter to run into the operational issues
• See if the DevOps Guru recognized the operational issues
• Remediate the operational issues by increasing service quote
or removing the artificial latency
• See If DevOps Guru closes the incident
DevOps Guru Examples
| CONFIDENTIAL
21

18. DevOps Guru: Recognize operational issues
in DynamoDB

19. DevOps Guru Examples: DynamoDB Throttling

20. DevOps Guru Examples: DynamoDB Throttling
stress test and empty burst credits

21. DevOps Guru Examples: DynamoDB Throttling

22. DevOps Guru: Recognize operational issues
in API Gateway

23. DevOps Guru Examples: API Gateway Throttling

24. DevOps Guru Examples: API Gateway Throttling

25. DevOps Guru: Recognize operational issues
in Lambda

26. DevOps Guru Examples: Lambda Throttling

27. DevOps Guru Examples: Lambda Throttling

28. DevOps Guru Examples: Lambda Timeout Error
Add 11 sec
sleep in the
code of the
Lambda
function

29. DevOps Guru Examples: Lambda Timeout Error

30. DevOps Guru Examples: Lambda Increased
Latency
Temporary add
9.5 sec sleep in
the code of the
Lambda function

31. DevOps Guru Examples: Lambda Increased
Latency

32. • Lambda concurrent executions reaching account limit
• Triggered when concurrent executions reach an account limit for a continuous period. Reduce the
service quote of the AWS service (API Gateway, Lambda, DynamoDB)
• Lambda Provisioned Concurrency function limit breached
• Triggered when the reserved amount of provisioned concurrency is not enough over a period.
Stress test with JMeter to run into the operational issues
• Lambda timeout high compared to SQS’s visibility timeout
• Triggered when the duration of the lambda function exceeds the visibility timeout for the event
source Amazon Simple Queue Service (Amazon SQS).
Other operational issues and the proactive
insights 1/2
| CONFIDENTIAL
38
https://aws.amazon.com/de/blogs/aws/automatically-detect-operational-issues-in-lambda-functions-with-amazon-devops-guru-for-serverless/

33. • Account read/write capacity for DynamoDB consumption reaching account limit
• Triggered when the account consumed capacity is approaching account-level limits during a
period of time.
• DynamoDB table consumed capacity reaching AutoScaling Maximum parameter limit
• Triggered when table consumed capacity is reaching AutoScaling Max parameters limit over a
period.
• DynamoDB read/write consumption lower than expected
• Triggered when the value for ProvisionedWriteCapacityUnits or ProvisionedReadCapacityUnits is
far from what is being consumed during a period of time
Other operational issues and the proactive
insights 2/2
| CONFIDENTIAL
39
https://aws.amazon.com/de/blogs/aws/automatically-detect-operational-issues-in-lambda-functions-with-amazon-devops-guru-for-serverless/

34. DevOps Guru Conclusions
• All errors have been correctly recognized so far
• It took several minutes to create an incident after anomaly
appeared
• Tested mainly in the context of AWS Serverless stack
• AWS is responsible for monitoring those managed services
• Not all PagerDuty alarms have been automatically closed after
the incident resolution
• Especially in case there have been several anomalies within
one incident

35. DevOps Guru for RDS
https://aws.amazon.com/devops-guru/features/devops-guru-for-rds/ https://aws.amazon.com/de/blogs/devops/leverage-devops-guru-for-rds-to-detect-anomalies-and-resolve-operational-issues/

36. AWS CodeGuru

37. What is AWS CodeGuru
Amazon CodeGuru is a developer tool that provides intelligent
recommendations to improve code quality and identify an
application’s most expensive lines of code
• CodeGuru Reviewer uses machine learning and automated
reasoning to identify critical issues, security vulnerabilities, and
hard-to-find bugs during application development and provides
recommendations to improve code quality
• CodeGuru Profiler helps developers find an application’s most
expensive lines of code by helping them understand the runtime
behavior of their applications, identify and remove code
inefficiencies and improve performance
https://aws.amazon.com/codeguru

38. Benefits of CodeGuru
• CodeGuru Reviewer benefits
• Catch code problems before they hit production
• Proactively improve code quality with continuous monitoring
• CodeGuru Profiler benefits
• Troubleshoot performance issues
• Discover anomalies and common issues in your application
performance
• Catch your most expensive line of code
https://aws.amazon.com/codeguru

39. CodeGuru Programming Language Support
• Java
• Python

40. CodeGuru
CodeGuru Reviewer in Java

41. CodeGuru Setup

42. CodeGuru Setup

43. CodeGuru Reviewer Scans
• Full repository analysis
• Incremental code reviews (pull requests)

44. Java Code for CodeGuru Analysis

45. CodeGuru Reviewer Recommendation
The recommendations for Java fall into the following categories:
• AWS best practices
• Security
• Resource leaks
• Concurrency
• Integration with Infer (https://fbinfer.com/)
• detect null pointer dereferences, thread safety violations and
improper use of synchronization locks
• General best practices on data structures, control flow, exception
handling, and more
https://aws.amazon.com/de/blogs/devops/improving-aws-java-applications-with-amazon-codeguru-reviewer/

46. CodeGuru Review Full Repository Analysis

47. What's different between the AWS SDK for Java
1.x and 2.x
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/migration-whats-different.html

48. CodeGuru Review AWS Best Practices with
Java SDK V1

49. CodeGuru Review AWS Best Practices with
Java SDK V1

50. CodeGuru Review AWS Best Practices with
Java SDK V1

51. CodeGuru Review AWS Best Practices with
Java SDK V2

52. CodeGuru Review other AWS Best Practices

53. CodeGuru Review Concurrency

54. CodeGuru Review Concurrency

55. CodeGuru Review Resource Leak

56. CodeGuru Review Security

57. CodeGuru Review Security

58. CodeGuru Incremental Review
Occurs automatically when creating a pull request with CodeGuru associated
with CodeCommit repository

59. CodeGuru Review Expected, but No Findings
https://aws.amazon.com/de/blogs/devops/tightening-application-security-with-amazon-codeguru/

60. CodeGuru vs SonarQube

61. CodeGuru vs SonarQube
• CodeGuru currently support only 2 languages vs SonarQube
supporting 20+
• CodeGuru is much powerful in detecting AWS best practices
(including AWS security best practices)
• SonarQube is much more powerful detecting common Java
issues
• SonarQube is better at detecting OWASP Top 10-related issues

62. CodeGuru vs SonarQube
• Code Repositories
• CodeGuru
• SonarQube
• CI Integration
• CodeGuru
• SonarQube

63. CodeGuru vs SonarQube
• SonarQube plugin eco system is much more powerful
• SonarLint alternative on the CodeGuru side is currently missing
• Use CodeGuru in conjunction with SonarQube

64. CodeGuru Conclusions
• Very good findings for AWS best practices when using Java SDK
V1
• Many missing findings with Java SDK V2 compared to 1
• Many officially described security findings are not detected in my
examples
• Full repository scans are very expensive
• Use incremental code reviews (pull requests) scan as much
as possible

66. www.iplabs.de
Accelerate Your Photo Business
Get in Touch