aws system management

1. AWS SYSTEMS
MANAGEMENT AND
MONITORING OPTIONS
DR.RAJAPRAVEEN

3. AWS SYSTEMS MANAGER FEATURES
• AWS Systems Manager allows you to centralize operational data from multiple AWS
services and automate tasks across your AWS resources. You can create logical groups of
resources such as applications, different layers of an application stack, or production
versus development environments. With Systems Manager, you can select a resource
group and view its recent API activity, resource configuration changes, related
notifications, operational alerts, software inventory, and patch compliance status. You can
also take action on each resource group depending on your operational needs. Systems
Manager provides a central place to view and manage your AWS resources, so you can
have complete visibility and control over your operations.

4. EXPLORER
• AWS Systems Manager Explorer is a customizable dashboard, providing key insights and
analysis into the operational health and performance of your AWS environment. Explorer
aggregates operational data from across AWS accounts and AWS Regions to help you
prioritize and identify where action may be required.

5. OPSCENTER
• OpsCenter provides a central location where operations engineers and IT professionals
can view, investigate, and resolve operational issues related to any AWS
resource. OpsCenter aggregates and standardizes operational issues, referred to as
OpsItems, while providing contextually relevant data that helps with diagnosis and
remediation. Engineers working on an OpsItem get access to information such as:

6. OPSCENTER
 Event, resource and account details
 Past OpsItems with similar characteristics
 Related AWS Config changes
 AWS CloudTrail logs
 Amazon CloudWatch alarms
 Stack information
 Other quick-links to access logs and metrics

7. OPSCENTER
 List of runbooks and recommended runbooks
 Other information passed to OpsCenter through AWS services
• This information helps engineers to investigate and remediate operational issues faster.
Engineers can use OpsCenter to view and address issues using the Systems Manager
console or via the Systems Manager OpsCenter APIs.

8. APPLICATION MANAGER
• AWS Systems Manager Application Manager helps you investigate and remediate issues
with your AWS resources in the context of your applications. With Application Manager,
you can discover and/or define your application components, view operations data (e.g.
deployment status, Amazon CloudWatch alarms, resource configurations, and operational
issues) in the context of an application, and perform remedial actions such as patching and
running Automation runbooks. This streamlines operational workflows for your
applications, avoiding the need to use different consoles to investigate and remediate
operational issues. In addition, you can use Application Manager to view operational data
and CloudWatch alarms and take action on your existing container clusters in Amazon
EKS environments.

9. APPCONFIG
• AWS AppConfig helps you deploy application configuration in a managed and a
monitored way just like code deployments, but without the need to deploy the code if a
configuration value changes. AWS AppConfig scales with your infrastructure so you can
deploy configurations to any number of Amazon EC2 instances, containers, AWS Lambda
functions, mobile apps, IoT devices or on-premises instances. AWS AppConfig enables
you to update configurations by entering changes through the API or Console. AWS
AppConfig allows you to validate those changes semantically and syntactically to ensure
configurations are aligned to their respective applications’ expectation, thus enabling you
to help prevent potential outages. You can deploy your application configurations with
similar best practices as code deployments, including staging roll-outs, monitoring alarms,
and roll back changes should an error occur.

10. CHANGE MANAGER
• AWS Systems Manager Change Manager simplifies the way you request, approve,
implement, and report on operational changes to your application configuration and
infrastructure on AWS and on-premises. With Change Manager, you can use pre-approved
change workflows to help avoid unintentional results when making operational changes.
Change Manager helps you safely implement changes, while detecting schedule conflicts
with important business events and automatically notifying impacted approvers. Using
Change Manager’s change reports, you can monitor progress and audit operational
changes across your organization, providing improved visibility and accountability.

11. AUTOMATION
• AWS Systems Manager allows you to safely automate common and repetitive IT
operations and management tasks. With Systems Manager Automation, you can use
predefined playbooks, or you can build, run, and share wiki-style automated playbooks to
enable AWS resource management across multiple accounts and AWS Regions. You can
execute Python or PowerShell scripts as part of a playbook in combination with other
automation actions such as approvals, AWS API calls, or running commands on your EC2
instances. These playbooks can be scheduled in a maintenance window, triggered based
on changes to AWS resources through Amazon CloudWatch Events, or executed directly
through the AWS Management Console, CLIs, and SDKs. You can track the execution of
each step in a playbook, require approvals, incrementally roll out changes, and
automatically halt the roll out if errors occur.

12. MAINTENANCE WINDOWS
• AWS Systems Manager lets you schedule windows of time to run administrative and
maintenance tasks across your instances. This ensures that you can select a convenient
and safe time to install patches and updates or make other configuration changes,
improving the availability and reliability of your services and applications.

13. FLEET MANAGER
• AWS Systems Manager Fleet Manager streamlines your remote server management
process. With Fleet Manager, you can easily manage and troubleshoot your fleet of
servers running on AWS and on-premises. You can drill down to individual servers to
perform common troubleshooting and management tasks using a centralized graphical
user interface. Perform a variety of system administration tasks, including disk and file
exploration, log management, Windows Registry operations, and user management,
without needing to remotely connect to your virtual machines, saving your administrators
time and effort.

14. COMPLIANCE
• AWS Systems Manager automatically aggregates and displays operational data
for each resource group through a dashboard. Systems Manager eliminates the
need for you to navigate across multiple AWS consoles to view your operational
data. With Systems Manager you can view API call logs from AWS CloudTrail,
resource configuration changes from AWS Config, software inventory, and patch
compliance status by resource group. You can also easily integrate your AWS
CloudWatch Dashboards, AWS Trusted Advisor notifications, and AWS Personal
Health Dashboard performance and availability alerts into your Systems Manager
dashboard. Systems Manager centralizes all relevant operational data, so you
can have a clear view of your infrastructure compliance and performance.

16. INVENTORY
• AWS Systems Manager collects information about your instances and the software
installed on them, helping you to understand your system configurations and installed
applications. You can collect data about applications, files, network configurations,
Windows services, registries, server roles, updates, and any other system properties. The
gathered data enables you to manage application assets, track licenses, monitor file
integrity, discover applications not installed by a traditional installer, and more.

18. SESSION MANAGER
• AWS Systems Manager provides a browser-based interactive shell and CLI for managing
Windows and Linux EC2 instances, without the need to open inbound ports, manage SSH
keys, or use bastion hosts. Administrators can grant and revoke access to instances
through a central location by using AWS Identity and Access Management
(IAM) policies. This allows you to control which users can access each instance,
including the option to provide non-root access to specified users. Once access is
provided, you can audit which user accessed an instance and log each command
to Amazon S3 or Amazon Cloud Watch Logs using AWS CloudTrail.

19. RUN COMMAND
• AWS Systems Manager provides you safe, secure remote management of your instances at
scale without logging into your servers, replacing the need for bastion hosts, SSH, or remote
PowerShell. It provides a simple way of automating common administrative tasks across
groups of instances such as registry edits, user management, and software and patch
installations. Through integration with AWS Identity and Access Management (IAM), you
can apply granular permissions to control the actions users can perform on instances. All
actions taken with Systems Manager are recorded by AWS CloudTrail, allowing you to audit
changes throughout your environment.

20. STATE MANAGER
• AWS Systems Manager provides configuration management, which helps you maintain
consistent configuration of your Amazon EC2 or on-premises instances. With Systems
Manager, you can control configuration details such as server configurations, anti-virus
definitions, firewall settings, and more. You can define configuration policies for your
servers through the AWS Management Console or use existing scripts, PowerShell
modules, or Ansible playbooks directly from GitHub or Amazon S3 buckets. Systems
Manager automatically applies your configurations across your instances at a time and
frequency that you define. You can query Systems Manager at any time to view the status
of your instance configurations, giving you on-demand visibility into your compliance
status.

21. PATCH MANAGER
• AWS Systems Manager helps you select and deploy operating system and software
patches automatically across large groups of Amazon EC2 or on-premises instances.
Through patch baselines, you can set rules to auto-approve select categories of patches to
be installed, such as operating system or high severity patches, and you can specify a list
of patches that override these rules and are automatically approved or rejected. You can
also schedule maintenance windows for your patches so that they are only applied during
preset times. Systems Manager helps ensure that your software is up-to-date and meets
your compliance policies.

22. DISTRIBUTOR
• AWS Systems Manager helps you securely distribute and install software packages, such
as software agents. Systems Manager Distributor allows you to centrally store and
systematically distribute software packages while you maintain control over versioning.
You can use Distributor to create and distribute software packages and then install them
using Systems Manager Run Command and State Manager. Distributor can also use
Identity and Access Management (IAM) policies to control who can create or update
packages in your account. You can use the existing IAM policy support for Systems
Manager Run Command and State Manager to define who can install packages on your
hosts.

23. CONNECT WITH ITSM / ITOM SOFTWARE
• IT Service Management (ITSM) tools, such as Jira Service Desk, can connect with AWS Systems
Manager to make it easier for ITSM platform users to manage AWS resources. These AWS Service
Management Connectors provide Jira Service Desk administrators governance and oversight over
AWS products.

24. WHAT IS AWS MONITORING?
• Amazon Web Services (AWS) monitoring is a set of practices you can use to verify the
security and performance of your AWS resources and data. These practices rely on various
tools and services to collect, analyze, and present data insights. You can then use these
insights to identify vulnerabilities and issues, predict performance, and optimize
configurations.

25. AWS FIRST-PARTY MONITORING TOOLS
• There are multiple services and utilities available from AWS that you can use to monitor
your systems and access. Some of these tools are included in existing services, while
others are available for additional costs.

26. AWS CLOUDTRAIL
• CloudTrail is a service that you can use to track events across your account. The service
automatically records event logs and activity logs for your services and stores the data in S3.
Collected data includes user identities, traffic origin IPs, and timestamps. You can view all
management events for free for the most recent 90 days. Data events and insights based on your
data are also available for an additional fee.

27. AWS CLOUDWATCH
• CloudWatch is a service you can use to aggregate, visualize, and respond to service
metrics. CloudWatch has two main components: alarms, which create alerts according to
thresholds for single metrics, and events, which can automate responses to metric values
or system changes.

28. AWS CERTIFICATE MANAGER
• Certificate Manager is a tool you can use to provision, manage, and apply transport layer
security (TLS) and secure sockets layer (SSL) certificates. These certificates are used to
prove your services or devices' authenticity and enable you to secure network
connections.

29. AMAZON EC2 DASHBOARD
• EC2 Dashboard is a monitoring tool for the Amazon EC2 virtual machine service. You
can use this dashboard to monitor and maintain your EC2 instances and infrastructure.
The dashboard lets you view instance states and service health, manage alarms and status
reports, view scheduled events, and assess volume and instance metrics

30. AWS THIRD-PARTY MONITORING TOOLS
• In addition to native tools, many AWS users also adopt third-party tools. These tools are
useful for separating monitoring operations from your primary resources and can often
provide support for hybrid or on-premises resources as well.

31. NETAPP CLOUD INSIGHTS
• NetApp Cloud Insights is a tool for monitoring that you can use to visualize your
infrastructure.It enables you to monitor, optimize, and troubleshoot resources in public
and private clouds and on-premises. Cloud Insights includes features for conditional
alerting, optimization recommendations, predictive analytics, machine learning based
anomaly detection, and compliance auditing.

32. SOLARWINDS APPOPTICS
• AppOptics is a tool that you can use to supplement metrics collected by CloudWatch. It
enables you to track performance statistics, log trends, and capacity limits. You can
integrate AppOptics with other AWS services and generate automatic analyses of your
operations. AppOptics also includes features that enable you to monitor multiple AWS
accounts from a single interface.

33. ZENOSS ZENPACK
• ZenPack is an open source tool you can use to aggregate CloudWatch metrics and external resource
metrics data. It includes an easy to use graphical user interface (GUI) and is compatible with a
variety of AWS services. These services include S3, Amazon Virtual Private Cloud (VPC), and
Amazon Suite.

34. • Zabbix
• Zabbix is an open source tool for collecting metrics from AWS and a variety of other
applications, services, and databases. It includes features for dashboards, alert escalation,
and a robust online community of support. The downside of Zabbix is that it cannot
import data or generate performance reports.
• Weave Scope
• Weave Scope is an open source tool you can use to monitor and visualize your
microservices. It includes features for service discovery and is compatible with Elastic
Container Services (ECS). Weave Scope is based on three components (an interface, an
app, and a probe) and enables you to troubleshoot service performance in real time.

35. STEPS FOR SUCCESSFULAWS RESOURCE MONITORING
• Phase A: Assess Your AWS Monitoring Needs
• Before introducing monitoring into your pipeline or making changes to your
existing workflow, you should carefully assess your existing infrastructure,
tooling, resources, and skillset. Taking the time to assess your situation can help
you develop a strategy that suits your needs.

36. STEP 1: ASSESSMENT QUESTIONS
• Here are key questions to ask when assessing your AWS monitoring needs:
 Infrastructure—where is your network located? Is it on-premise? Do you want a dedicated monitoring
system for each environment or do you want to do on-premise monitoring with cloud monitoring using
one tool?
 Compliance—what are your current compliance policies? What legal percussions do you need to take in
order to comply with industry standards? Can you introduce a SaaS monitoring and logging solution into
your ecosystem and remain compliant?
 Inventory—do you need a new tool for AWS monitoring or can your current stack perform this task?
 Complexity—what are the complexities and costs involved in removing any and all legacy agents from
all servers, to clear space for the installation of new agents?
 Metrics—do you know which metrics you absolutely need to monitor, and which metrics might be
redundant?

37. STEP 2: DEVELOP A STRATEGY TO TAG AWS
RESOURCES
Once you gain insight into your current monitoring needs and prioritize metrics, you can start
developing a strategy for tagging AWS resources. Tags help you keep track of your resources,
and monitor usage and behavior.
• If you don’t have a tagging system in place, it can take some time to figure out how to organize
resources. While every project and organization is unique, it is important to create a tagging
system that can be used by a wide variety of professionals and collaborators. This way, all
relevant parties can gain access to monitoring insights when needed.

38. PHASE B: SELECT THE RIGHT SOLUTION FOR
YOUR ORGANIZATION
• After assessing your needs and setting up a tagging system for AWS resources, you can
look for the solution that suits your needs. Often, it is effective to start with a simple
solution and then expand as needed. However, if you know in advance you need a robust
set of features, it’s best to go with a solution that either fits your needs, can be scaled
easily, or meets all criteria.

39. STEP 3: START SIMPLE WITH AMAZON
CLOUDWATCH
•
CloudWatch metrics can help you monitor practically any AWS resource. CloudWatch provides a
wide range of pre-built counters like DiskQueueLength and CPUUtilization. Some AWS services,
such as RDS and EC2, can provide additional counters when integrated with CloudWatch.
• CloudWatch counters enable you to create dashboards, which you can leverage when you need
visualized data. In addition to counters and dashboards, CloudWatch offers an alerting system,
which lets you know when incidents occur. If you are not using a dedicated monitoring system, and
you need simple features, you can use CloudWatch.

40. STEP 4: LEVERAGE BEST-OF-BREED
SOLUTIONS
•
When it comes to visibility, the more resource types you monitor, the more you can ensure
the performance and safety of your assets. However, not all monitoring systems can
provide visibility for all resources. Some monitoring solutions are designed for
infrastructure while others are built for network traffic.
• To avoid losing visibility over parts of your environment, you can either use a stack of
tools or you can extend the capabilities of existing systems. If you opt to use a stack of
monitoring, you might want to first check that the tools provide the features you require
and are compatible with each other and your existing stack.

41. STEP 4: LEVERAGE BEST-OF-BREED
SOLUTIONS
• Additionally, you should consider adding a tool to centralize the stack, to ensure
productivity remains effective. If you choose to extend existing systems by installing
plugins or integrating with APIs, you should enable AWS integration and ensure that each
extension is compliant with any regulatory requirements you are legally required to
uphold.

42. PHASE C: CAPTURE LOGS
• Once you set up your monitoring solution or stack, you should decide which logs you
want to capture and how you want to set this up. Logs are highly effective for keeping
track of compliance requirements and troubleshooting issues.

43. PHASE C: CAPTURE LOGS
• Here is a list of logs you might want to capture:
 Database logs—help you detect queries that are slow to run.
 Application logs—point out application failures.
 AWS CloudTrail—detects API calls made to AWS.
 Elastic Load Balancing and host logs—might indicate availability or latency changes.
 OS logs—can identify host failure reasons.
 Web server logs—as well as firewall logs and VPC flow logs can detect patterns of access and attacks.
• The majority of monitoring systems are either suited for metrics or logs, rather than prioritizing both of these tasks
equally. To ensure full coverage, you should either use a stack or find a solution that enables you to capture both
metrics and logs from AWS.

44. AWS MONITORING BEST PRACTICES
• When monitoring your AWS resources, the following best practices can help you ensure that no
resources are overlooked and that you can troubleshoot efficiently.

45. USE AUTOMATION WHERE POSSIBLE
• Production deployments in AWS are typically too large and dynamic to monitor manually. The
volume of metrics and log data that is generated is too large for humans to efficiently analyze. To
ensure that critical data is not missed and responses are timely, you should use automation to handle
most of your monitoring tasks.

46. CREATE POLICIES TO DEFINE PRIORITY LEVELS
• Prioritizing monitoring tasks helps ensure that critical services remain operational and that data
remains protected. Additionally, prioritizing alerts or alert categories helps ensure that IT teams
effectively distribute their time and efforts.

47. RESOLVE PROBLEMS EARLY ON
• Monitoring data should be used to respond to issues like potential service interruptions
proactively. It is much easier to scale resources or throttle traffic in advance than manage
a service outage. Additionally, addressing potential issues early on can help you avoid
wasted resources and costs.

48. USE THE CLOUD TO YOUR ADVANTAGE
• Cloud environments are flexible and can enable you to experiment with configuration changes
without affecting services. When optimizing based on metrics, take time to test your configurations.
This way, you can verify if changes are more efficient before implementing them in production.

49. AWS MONITORING WITH NETAPP CLOUD INSIGHTS
• NetApp Cloud Insights is an infrastructure monitoring tool that gives you visibility into
your complete infrastructure. With Cloud Insights, you can monitor, troubleshoot and
optimize all your resources including your public clouds and your private data centers.
• Cloud Insights helps you find problems fast before they impact your business. Optimize
usage so you can defer spend, do more with your limited budgets, detect ransomware
attacks before it’s too late and easily report on data access for security compliance
auditing.
• In particular, NetApp Cloud Insights lets you automatically build topologies, correlate
metrics, detect greedy or degraded resources, and alert on anomalous user behavior

50. AWS MONITORING BEST PRACTICES
•
Monitoring cloud environments can be quite different than on-premises ones. These environments
are dynamic, highly distributed, and inherently more vulnerable to cyber threats. To ensure that you
are applying the proper strategies when monitoring your cloud resources, it is important to make
sure you are following best practices.
• This article explains what AWS monitoring best practices are, how monitoring in AWS works, and
highlights 6 best practices for ensuring effective monitoring in AWS.

51. AWS MONITORING DASHBOARD
•
Dashboards are an effective way to centralize your metrics monitoring and provide
information to teams quickly. You can use these tools to ensure that your entire team is
working from reliable information or to share the status of your operations with executive
and shareholders.
• This article explains what AWS monitoring dashboards are, the components of a
dashboard, provides two tutorials for creating dashboards, and highlights some best
practices.

52. CLOUDWATCH MONITORING
•
Monitoring your AWS resources is one of the best ways to ensure that your services and
applications remain performant and cost effective. To make this monitoring easier, AWS
offers a service called CloudWatch which you can use to collect and visualize metrics
across your services.
• This article explains what CloudWatch monitoring is, how CloudWatch works, some key
concepts to know in CloudWatch, and highlights a few metrics to watch for EBS and EC2.

53. CLOUDWATCH LOG INSIGHTS
•
The ability to query and interpret logs enables you to derive greater insights from your data. It also
enables you to diagnose and identify issues or opportunities for improvement faster. In AWS, you
can use CloudWatch Logs Insights to perform these tasks and ensure your operations continue
smoothly.
• This article explains what CloudWatch Logs Insights is, how to get log data to the service, what the
syntax for queries is, and how to perform a sample query.

54. MONITORING THE COSTS OF UNDERUTILIZED
EBS VOLUMES
•
Overprovisioning your resources can eat away at your carefully planned cloud resource budgets and
limit the amount of value you gain from services. To prevent wasted costs, it’s important to make
sure that your resources are right-sized for your operations and are being used efficiently.
• In this article you’ll learn how to find underperforming resources in EBS, how to evaluate your
resource use, and how to apply metrics to improve your resource efficiency.