Unable to delete bucket / delete objects recursively with s3cmd
Hi, I would like to delete a non-empty bucket in Linode Object Storage with s3cmd rb -r -f but failed to do so.
Could you please help?
Expected result
- Objects in the bucket are first deleted recursively. Bucket is then deleted.
Actual result
- No object has been deleted
- Failed to delete bucket:
ERROR: S3 error: 409 (BucketNotEmpty): failed to delete
Example output
$ s3cmd ls -r s3://dchan-dev-bucket/
2025-02-05 22:53 209623 s3://dchan-dev-bucket/DASH/DASH_0_481_240000_Segment-1.m4v
2025-02-05 22:52 87106 s3://dchan-dev-bucket/DASH/DASH_0_481_240000_Segment-2.m4v
2025-02-05 22:53 731 s3://dchan-dev-bucket/DASH/DASH_0_481_240000_init.m4i
... < omit for brevity > ...
2025-02-05 22:53 2250 s3://dchan-dev-bucket/sample.txt
$ s3cmd rb -r -f s3://dchan-dev-bucket/
WARNING: Bucket is not empty. Removing all the objects from it first. This may take some time...
delete: 's3://dchan-dev-bucket/DASH/DASH_0_481_240000_Segment-1.m4v'
delete: 's3://dchan-dev-bucket/DASH/DASH_0_481_240000_Segment-2.m4v'
delete: 's3://dchan-dev-bucket/DASH/DASH_0_481_240000_init.m4i'
... < omit for brevity > ...
delete: 's3://dchan-dev-bucket/sample.txt'
ERROR: S3 error: 409 (BucketNotEmpty): failed to delete
PS. similarly, s3cmd rm -r -f also failed to delete objects.
$ s3cmd rm -r -f s3://dchan-dev-bucket/
delete: 's3://dchan-dev-bucket/DASH/DASH_0_481_240000_Segment-1.m4v'
delete: 's3://dchan-dev-bucket/DASH/DASH_0_481_240000_Segment-2.m4v'
delete: 's3://dchan-dev-bucket/DASH/DASH_0_481_240000_init.m4i'
... < omit for brevity > ...
delete: 's3://dchan-dev-bucket/sample.txt'
$ s3cmd ls -r s3://dchan-dev-bucket/
2025-02-05 22:53 209623 s3://dchan-dev-bucket/DASH/DASH_0_481_240000_Segment-1.m4v
2025-02-05 22:52 87106 s3://dchan-dev-bucket/DASH/DASH_0_481_240000_Segment-2.m4v
2025-02-05 22:53 731 s3://dchan-dev-bucket/DASH/DASH_0_481_240000_init.m4i
... < omit for brevity > ...
2025-02-05 22:53 2250 s3://dchan-dev-bucket/sample.txt
Note
- Version:
s3cmd version 2.4.0 - Confirmed
s3cmdclient has permission to delete object in the bucket - Single object can be successfully deleted via
s3cmd rm
$ s3cmd get s3://dchan-dev-bucket/HLS/master.m3u8
download: 's3://dchan-dev-bucket/HLS/master.m3u8' -> './master.m3u8' [1 of 1]
433 of 433 100% in 0s 6.63 KB/s done
$ s3cmd rm -f s3://dchan-dev-bucket/HLS/master.m3u8
delete: 's3://dchan-dev-bucket/HLS/master.m3u8'
$ s3cmd get s3://dchan-dev-bucket/HLS/master.m3u8
ERROR: Parameter problem: Source object 's3://dchan-dev-bucket/HLS/master.m3u8' does not exist.
1 Reply
hdiep
Linode Staff
If you're trying this and things are still not working, you'll want to try a few additional methods such as deleting the bucket contents specifically in a step before removal, "canceling" any incomplete multipart uploads (which would remove them), or adding a lifecycle policy to the bucket to automatically remove failed multipart uploads after a certain number of days (which should also remove them right away, unless an incomplete multipart upload is more recent than the timeframe you set). I'll share some resources for these other options below:
How can I remove a bucket that is not empty? | S3cmd: FAQ and Knowledge Base
s3cmd - delete failed multipart uploads taking up space and being charged
Configuring a bucket lifecycle configuration to delete incomplete multipart uploads
All these resources or discussions contain links to other related topics, so you can explore those as well to help get more insight into the bucket file management process.