How to do the equivalent of 's3cmd ls s3://some_bucket/foo/bar' in Ruby?

How do I do the equivalent of 's3cmd ls s3://some_bucket/foo/bar' in开发者_StackOverflow Ruby?

I found the Amazon S3 gem for Ruby and also the Right AWS S3 library, but somehow it's not immediately obvious how to do a simple 'ls' like command on an S3 'folder' like location.

Using the aws gem this should do the trick:

s3 = Aws::S3.new(YOUR_ID, YOUR_SECTRET_KEY)
bucket = s3.bucket('some_bucket')
bucket.keys('prefix' => 'foo/bar')

I found a similar question here: Listing directories at a given level in Amazon S3

Based on that I created a method that behaves as much as possible as 's3cmd ls <path>':

require 'right_aws'

module RightAws
  class S3
    class Bucket
      def list(prefix, delimiter = '/')
        list = []
        @s3.interface.incrementally_list_bucket(@name, {'prefix' => prefix, 'delimiter' => delimiter}) do |item|
          if item[:contents].empty?
            list << item[:common_prefixes]
          else
            list << item[:contents].map{|n| n[:key]}
          end
        end
        list.flatten
      end
    end
  end
end

s3 = RightAws::S3.new(ID, SECRET_KEY)
bucket = s3.bucket('some_bucket')

puts bucket.list('foo/bar/').inspect

In case some looks for the answer to this question for the aws-sdk version 2, you can very easily do this this way:

creds = Aws::SharedCredentials.new(profile_name: 'my_credentials')

s3_client = Aws::S3::Client.new(region: 'us-east-1',
                                credentials: creds)

response = s3_client.list_objects(bucket: "mybucket",
                                  delimiter: "/")

Now, if you do

response.common_prefixes

It will give you the "Folders" of that particular subdirectory, and if you do

response.contents

It will have the files of that particular directory

The official Ruby AWS SDK now supports this: http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/S3/Tree.html

You can also add the following convenience method:

class AWS::S3::Bucket
  def ls(path)
    as_tree(:prefix => path).children.select(&:branch?).map(&:prefix)
  end
end

Then use it like this:

mybucket.ls 'foo/bar' # => ["/foo/bar/dir1/", "/foo/bar/dir2/"]

a quick and simple method to list files in a bucket folder using the ruby aws-sdk:

require 'aws-sdk'

  s3 = AWS::S3.new
  your_bucket = s3.buckets['bucket_o_files']
  your_bucket.objects.with_prefix('lots/of/files/in/2014/09/03/').each do |file|
    puts file.key
  end

Notice the '/' at the end of the key, it is important.

I like the Idea of opening the Bucket class and adding a 'ls' method. I would have done it like this...

class AWS::S3::Bucket
  def ls(path)
    objects.with_prefix("#{path}").as_tree.children.select(&:leaf?).collect(&:member).collect(&:key)
  end
end

s3 = AWS::S3.new
your_bucket = s3.buckets['bucket_o_files']
your_bucket.ls('lots/of/files/in/2014/09/03/')