Ravindra BagaleCourses & study guides Track your progress

Guides

How to Install the CloudWatch Agent on EC2 for Memory, Disk Metrics and Logs

EC2 does not send memory or disk space metrics by default, because they are inside the operating system. To get them, attach an IAM role with CloudWatchAgentServerPolicy to the instance, install the CloudWatch agent (sudo dnf install -y amazon-cloudwatch-agent on Amazon Linux 2023), run the config wizard, and start it with amazon-cloudwatch-agent-ctl. After a few minutes, mem_used_percent and disk_used_percent appear under CWAgent, and you can set alarms.

Quick answer (Amazon Linux 2023)

sudo dnf install -y amazon-cloudwatch-agent
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
  -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a status

A live example first

Our WordPress server on a t3.small crashed twice from low memory, and CloudWatch showed nothing unusual, because only CPU was visible. Let us add memory, disk and the nginx error log.

  1. IAM: create a role for EC2 with the managed policy CloudWatchAgentServerPolicy, then EC2 → instance → Actions → Security → Modify IAM role → attach it.
  2. Install:
sudo dnf install -y amazon-cloudwatch-agent
  1. Instead of the wizard, write a small config by hand at /opt/aws/amazon-cloudwatch-agent/etc/config.json:
{
  "metrics": {
    "append_dimensions": { "InstanceId": "${aws:InstanceId}" },
    "metrics_collected": {
      "mem":  { "measurement": ["mem_used_percent"] },
      "disk": { "measurement": ["used_percent"], "resources": ["/"] }
    }
  },
  "logs": {
    "logs_collected": { "files": { "collect_list": [
      { "file_path": "/var/log/nginx/error.log",
        "log_group_name": "/rb/wordpress/nginx-error", "retention_in_days": 14 }
    ] } }
  }
}
  1. Start:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
  -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/etc/config.json
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a status
{ "status": "running", "starttime": "2026-10-16T19:20:11+00:00", "version": "1.300..." }
  1. After about 5 minutes, CloudWatch → Metrics → CWAgent → InstanceId shows mem_used_percent and disk_used_percent. We create an alarm at 85% memory for 5 minutes that emails us through SNS.

The next time memory climbed, we got the email before the crash. Friends, you cannot fix what you cannot see.

How does the agent work?

CloudWatch agent on EC2 The instance role lets the agent send memory, disk and log data from inside the OS to CloudWatch, where alarms watch them. CloudWatch agent on EC2 IAM roleagent policyInstalldnf / aptWizardpick metricsStartfetch-configAlarmsmem / disk you Memory and disk space live inside the OS, so the agent must report them

Memory and disk space live inside the OS. The agent, with an IAM role, sends them to CloudWatch.

The basic EC2 metrics are collected by the hypervisor from outside the virtual machine. It can see CPU and network, but not how the OS uses its memory or how full each filesystem is. The agent runs inside the OS, reads those numbers (and log files), and sends them to CloudWatch using the instance role's temporary credentials.

How do I set it up step by step?

  1. Create and attach an IAM role with CloudWatchAgentServerPolicy.
  2. Install the agent for your OS (below).
  3. Create a config with the wizard or by hand.
  4. Start the agent with fetch-config.
  5. Check -a status and the agent log if needed.
  6. Wait 5 minutes and find the metrics under CWAgent.
  7. Create alarms and set log retention so costs stay low.

Install on each OS

OS Install
Amazon Linux 2023 / 2 sudo dnf install -​y amazon-​cloudwatch-​agent (yum on AL2)
Ubuntu / Debian download the .deb from the AWS S3 download link in the docs, then sudo dpkg -​i -​E ./​amazon-​cloudwatch-​agent.​deb
RHEL / CentOS download the .rpm and sudo rpm -​U ./​amazon-​cloudwatch-​agent.​rpm
Windows Server download the .msi, install, then run amazon-​cloudwatch-​agent-​ctl.​ps1 -​a fetch-​config in PowerShell
Many instances Systems Manager → Run Command → AWS-​ConfigureAWSPackage (name AmazonCloudWatchAgent)

Which logs to send, by stack

Stack Log file to collect
nginx /​var/​log/​nginx/​error.​log, access.log
Apache /​var/​log/​httpd/​error_​log (Amazon Linux), /​var/​log/​apache2/​error.​log (Ubuntu)
Tomcat / Java /​opt/​tomcat/​logs/​catalina.​out or your app log
Node.js (PM2) ~/.​pm2/​logs/*-​error.​log
Python (Gunicorn) the --error-logfile path
PHP-FPM /​var/​log/​php-​fpm/​www-​error.​log
MySQL /​var/​log/​mysqld.​log or the slow query log
System /​var/​log/​messages or /var/log/syslog, /var/log/secure

Send error logs first. Full access logs can be large, so think about cost.

Common errors

Problem Cause Fix
No CWAgent metrics No IAM role, or wrong policy Attach role with CloudWatchAgentServerPolicy
AccessDeniedException in agent log Role lacks permission Same as above
Agent status "stopped" Config JSON error Check the agent log, validate the JSON
Logs missing Agent user cannot read the file Run as root (default) or fix file permissions
Instance in private subnet, no data No route to CloudWatch NAT gateway or VPC endpoints for monitoring and logs

The same idea on other clouds

  • Azure: Azure Monitor Agent with data collection rules.
  • Google Cloud: the Ops Agent.
  • Open source: Prometheus node_exporter + Grafana, which many teams also run on AWS.

Real example

Many outage write-ups start with "the disk filled up" or "the server ran out of memory", and both are invisible in default EC2 metrics. For example, Elasticsearch clusters stop accepting writes when disk use reaches the flood-stage watermark (95% by default), and teams without disk metrics often learn about it from angry users. A simple disk_used_percent > 80% alarm from the CloudWatch agent turns that surprise into a routine ticket.

Mistakes to avoid

  1. Collecting every metric every 10 seconds. Custom metrics cost money; start with memory and root disk at 60 seconds.
  2. No log retention. Set retention_in_days, or logs are kept forever.
  3. Putting access keys in the config. Use the instance role.

Summary table

Step Action
Permission Role with CloudWatchAgentServerPolicy
Install dnf install amazon-​cloudwatch-​agent / deb / rpm / msi
Configure wizard or config.json (mem, disk, logs)
Start amazon-​cloudwatch-​agent-​ctl -​a fetch-​config -​m ec2 -​s -​c file:...
See data CloudWatch → Metrics → CWAgent
Alert Alarms at 85% memory / 80% disk

Quick quiz: check what you learned

  1. Why is EC2 memory missing from default metrics?

    1. It is inside the operating system
    2. AWS hides it for free tier users
    3. Memory cannot be measured at all
    4. It appears only in us-east-1
    Show answer

    A. The hypervisor cannot see memory use inside the guest OS.

  2. Which managed policy should the instance role have?

    1. CloudWatchAgentServerPolicy
    2. AmazonS3FullAccess policy
    3. AdministratorAccess policy
    4. AmazonEC2ReadOnlyAccess
    Show answer

    A. This policy lets the agent put metrics and logs.

  3. Where do agent metrics appear in CloudWatch?

    1. The CWAgent namespace
    2. The AWS/S3 namespace
    3. The AWS/Billing namespace
    4. The AWS/Lambda namespace
    Show answer

    A. By default the agent publishes under CWAgent.

  4. Which tool installs the agent on many instances?

    1. Systems Manager Run Command
    2. A CloudFront distribution
    3. An S3 lifecycle rule
    4. A Route 53 health check
    Show answer

    A. SSM can install and configure the agent across a fleet.

Learn it properly

These pages go deeper, with labs:

Frequently asked questions

Why do I need the CloudWatch agent?

EC2 sends CPU, network and disk I/O metrics by default, but not memory or disk space, because those are inside the operating system. The agent reads them and sends them.

Which IAM policy does the agent need?

The AWS managed policy CloudWatchAgentServerPolicy, attached to the instance role.

Where are the agent logs?

On Linux: /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log.

How much does it cost?

The agent is free. You pay for custom metrics, log ingestion and storage in CloudWatch. Choose only the metrics and logs you need.

Can I send nginx or app logs too?

Yes. Add the file paths in the logs section of the config, for example /var/log/nginx/access.log, with a log group name.

Can I install it on many instances at once?

Yes, with AWS Systems Manager Run Command (AWS-ConfigureAWSPackage) and a config stored in Parameter Store.