Amazon S3 Retention Settings Not Removing Backups

We goofed! Forgot to add a wildcard (*): resources.ARN startsWith arn:aws:s3:::<our-bucket>/<our-subdirectory>/*. CloudTrail was only capturing the bucket not the subdirectory in the bucket. Based on our backup schedule and retention settings, here’s when we should expect to see data events in CloudTrail:

Expected Data Events Timeline:

1. Next Upload Event:
   • When: September 5, 2025 at 7:20 AM (about 3 hours from now)
   • Event Type: PutObject
   • Visible in CloudTrail: By 7:35 AM (15 minutes after backup)
2. First Deletion Events:
   • When: After the 7:20 AM backup completes
   • Why: We currently have 22 objects, but retention is set to keep only 7 most recent
   • Expected: We should see DeleteObject or DeleteObjects events removing ~15-16 old backups
   • Visible in CloudTrail: By 7:35-7:40 AM
3. Age-Based Deletions:
   • When: Your oldest backup is from August 31 (5 days old)
   • 14-day rule: Won’t trigger deletions until backups are 14+ days old
   • First age-based deletion: Around September 14, 2025 (when August 31 backups turn 14 days old)

Immediate (next few hours):

• 1 PutObject event around 7:20 AM
• Multiple DeleteObject/DeleteObjects events shortly after (removing excess backups beyond the 7 most recent)

Your retention policy should delete ~15 backups because:

• Current: 22 objects
• After new backup: 23 objects
• Keep most recent 7: Delete 16 objects
• Final count: 7 objects

We’ll run our CloudTrail query around 8:00 AM to see both the upload and the expected cleanup deletions. If we don’t see deletion events by then, our retention policy isn’t working correctly.

The “keep 7 most recent” rule should trigger immediately after the next backup, so we should see significant deletion activity very soon, correct?

P.S. Don’t you just love AI/ML? I’m like 100 employees now. Yay! lol

Hi Yani and Team,
It ran all weekend and we now have 76 total objects in S3. To recap, our retention settings are:

Keep the most recent 7 backups. Default: 0 unlimited
Limit the total size of backups to 0 MB Default: 0 unlimited
Remove backups older than 14 days. Default: 0 off

Our <iam_user> permissions are:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:DeleteObjectVersion",
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:DeleteBucket"
],
"Resource": [
"arn:aws:s3:::<bucket_name>",
"arn:aws:s3:::<bucket_name>/" ] }, { "Effect": "Allow", "Action": "s3:ListAllMyBuckets", "Resource": ""
}
]
}

Our CloudTrail Event data store event selectors are:

eventSource equals s3.amazon.com
resource.ARN starts with arn:aws:s3:::<bucket_name>/<object_name>/*
readOnly equals false
eventType equals AwsApiCall

Our basic CloudTrail query is:

SELECT eventName,
       count(*) as event_count
FROM <event_data_store_ID>
WHERE userIdentity.userName = '<iam_user>'
    AND eventtime >= timestamp '2025-09-05 00:00:00'
GROUP BY eventName
ORDER BY event_count DESC;

Result:

eventName		event_count
UploadPart		794
CompleteMultipartUpload	8
CreateMultipartUpload	8

Our deep dive CloudTrail query-picky query:

SELECT eventTime,
       eventName,
       JSON_EXTRACT_SCALAR(CAST(requestParameters AS JSON), '$.bucketName') AS bucketName,
       JSON_EXTRACT_SCALAR(CAST(requestParameters AS JSON), '$.key') AS objectKey,
       JSON_EXTRACT(CAST(requestParameters AS JSON), '$.delete.objects') AS deletedObjects,
       userIdentity.arn AS callerArn,
       r.ARN AS resourceArn
FROM <event_data_store_ID>
CROSS JOIN UNNEST(resources) AS r (ARN, accountId, type, name)
WHERE eventSource = 's3.amazonaws.com'
  AND eventTime >= TIMESTAMP '2025-09-05 00:00:00'
  AND JSON_EXTRACT_SCALAR(CAST(requestParameters AS JSON), '$.bucketName') = '<bucket_name>'
  AND eventName IN ('PutObject','UploadPart','CompleteMultipartUpload','DeleteObject','DeleteObjects')
  AND r.ARN LIKE 'arn:aws:s3:::<bucket_name>/<object_name>/%'
ORDER BY eventTime DESC;

Result:

0 records matched | 810 records (335.8 kB) scanned in 1.2s @ 651.1 records/s (269.9 kB/s)

Shall we debug WordPress?

I cannot find anything on our end that would prevent object deletions. Our service control policy (SCP) where MFA is required to delete objects in S3 is for a completely different Organizational Unit (OU) than the WordPress websites. We’re unable to find anything else in AWS that would be restricting it, and the plugin is not even attempting the DeleteObject API call it seems.

Side Note: Using a terminal inside VSCode (Virtual Studio Code) for CLI.

We rotated keys, meaning I tossed this guy’s secret key on creation, created a profile using the new key, and it worked:

aws configure set aws_access_key_id <new_access_key> --profile wp-migration-test
aws configure set aws_secret_access_key <new_secret_key> --profile wp-migration-test
aws configure set region us-east-1 --profile wp-migration-test

Ensure I was connected to the correct AWS account:

aws sts get-caller-identity --profile wp-migration-test

{
    "UserId": "<user_id>",
    "Account": "<account_id>",
    "Arn": "arn:aws:iam::<account_id>:user/<iam_user_name>"
}

aws s3 rm s3://<bucket_name>/<object_name>/<oldest_filename> --profile wp-migration-test

delete: s3://<bucket_name>/<object_name>/<oldest_filename>

The plugin now has the new key, and the next backup date: September 8, 2025 10:20 am. Stay tuned…

How thoroughly have you guys tested this? Just because you can AWS CLI delete with access keys does not mean the plugin works:

// The order is very important - we delete files by date, by size, and finally by total count
$this->delete_backups_older_than();
$this->delete_backups_when_total_size_over();
$this->delete_backups_when_total_count_over();

The get_limit() method in the cleanup code returns 0 instead of 7, causing the entire deletion process to be skipped. The database is correct: ai1wmve_schedule_events contains: “retention”;a:4:{s:7:”backups”;i:7;…}

Tag, you’re it…