python - How to move/copy all files directory from local machine to s3 bucket

Question

I am trying to move/copy the particular extension file(.xlsx) move to my s3 bucket.

I am using below command for this.

cmd= 'aws s3 cp myfolder/*.xlsx s3://mybucket/myfolder --recursive'

where:
**myfolder** is my local system directory where the file is present

**s3://mybucket/myfolder** : folder location where i am placing all the .xlsx file in s3.

But when I am running this command using python

os.system(cmd)

Getting Error : 255 

can anyone suggest me any change or any other way to achieve the goal?

Note: I already configured my system cmd whit my s3 key and all commands are working fine also.

UPDATE:

cmd = '$aws s3 cp myfolder/*.xlsx s3://mybucket/myfolder --recursive'

getting error:1 on interactive python.

question from:https://stackoverflow.com/questions/65841187/how-to-move-copy-all-files-directory-from-local-machine-to-s3-bucket

Answer 1

There is no support for the use of UNIX style wildcards in a command's path arguments. Use --exclude and --include filters.

aws s3 cp myfolder s3://mybucket/myfolder/ --recursive --exclude "*" --include "*.xlsx"

The order of the parameters matters.

Note that running help command aws s3 help, brings same command reference as linked below. From AWS docs:

Currently, there is no support for the use of UNIX style wildcards in a command's path arguments. However, most commands have --exclude "<value>" and --include "<value>" parameters that can achieve the desired result. These parameters perform pattern matching to either exclude or include a particular file or object. The following pattern symbols are supported.

• "*": Matches everything
• "?": Matches any single character
• "[sequence]": Matches any character in "sequence"
• "[!sequence]": Matches any character not in "sequence"

Note that, by default, all files are included. This means that providing only an --include filter will not change what files are transferred. --include will only re-include files that have been excluded from an --exclude filter. If you only want to upload files with a particular extension, you need to first exclude all files, then re-include the files with the particular extension.