AWS CLI: Using the AWS Command Line Interface
For more information on the AWS CLI, visit aws.amazon.com
Description
There are many ways to authenticate to AWS in order to launch new services, or query an existing one. Beyond using the AWS console, and available SDK's there is also a command line tool that allows a user an easy way to fire off an API request, and receive the response using just a simple command line argument in the AWS CLI tool from your local shell environment. Once installed, simply issue an aws configure command to set an Access Key and Secret Key both of which are stored in the .aws directory found within your home directory path.
Pre-Requisites
The first pre-requisite required is to have Python and Pip installed on the machine from which you want to use the AWS CLI. The AWS CLI tool is written and distributed in python, and as such, available for install via the PIP package manager. Once python has been installed, we can then use pip to install our second pre-requisite the awscli package, which will install all of the AWS Cli Tools.
1. Download the Python installer package:
Open a browser window and navigate to the python.org download page. From the download page, ensure that you have selected the proper platform of Windows, Linux, Unix, Mac OS X, or Other and download the latest version of Python 3
2. Install the Python installer package:
Once your download has completed, install the downloaded package. Installing the Python package should result in having both Python, and Pip installed on your machine. You can test both by issuing the appropriate version command from both command line utilities.
iMac27$ python3 --version Python 3.6.3
iMac27$ pip3 --version pip 10.0.1 from /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip (python 3.6)
3. Install the AWS CLI PIP package:
Once Python has been installed and verified, we can now use pip to install the aws cli tools from pypi.
Verify awscli isn't currently installed:
iMac27$ pip3 list Package Version ---------- ------- pip 10.0.1 setuptools 28.8.0 six 1.11.0 virtualenv 15.1.0
Install the awscli packages:
iMac27$ pip3 install awscli Collecting awscli Using cached https://files.pythonhosted.org/packages/b3/0e/8b55f31cbe931cf2f59f85f1c6a2a2ba47933ae9c4778339f5628d46224d/awscli-1.15.40-py2.py3-none-any.whl Collecting s3transfer<0.2.0,>=0.1.12 (from awscli) Using cached https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl Collecting PyYAML<=3.12,>=3.10 (from awscli) Collecting colorama<=0.3.9,>=0.2.5 (from awscli) Downloading https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl Collecting docutils>=0.10 (from awscli) Downloading https://files.pythonhosted.org/packages/36/fa/08e9e6e0e3cbd1d362c3bbee8d01d0aedb2155c4ac112b19ef3cae8eed8d/docutils-0.14-py3-none-any.whl (543kB) 100% |████████████████████████████████| 552kB 5.1MB/s Collecting botocore==1.10.40 (from awscli) Using cached https://files.pythonhosted.org/packages/fe/6f/e9c3981f8b7e93bfa4461b754563b0e917968947920d0bdcf2a7dcf77da2/botocore-1.10.40-py2.py3-none-any.whl Collecting rsa<=3.5.0,>=3.1.2 (from awscli) Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl Collecting python-dateutil<3.0.0,>=2.1; python_version >= "2.7" (from botocore==1.10.40->awscli) Downloading https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl (211kB) 100% |████████████████████████████████| 215kB 4.1MB/s Collecting jmespath<1.0.0,>=0.7.1 (from botocore==1.10.40->awscli) Downloading https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl Collecting pyasn1>=0.1.3 (from rsa<=3.5.0,>=3.1.2->awscli) Downloading https://files.pythonhosted.org/packages/a0/70/2c27740f08e477499ce19eefe05dbcae6f19fdc49e9e82ce4768be0643b9/pyasn1-0.4.3-py2.py3-none-any.whl (72kB) 100% |████████████████████████████████| 81kB 6.9MB/s Requirement already satisfied: six>=1.5 in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore==1.10.40->awscli) (1.11.0) Installing collected packages: python-dateutil, jmespath, docutils, botocore, s3transfer, PyYAML, colorama, pyasn1, rsa, awscli Successfully installed PyYAML-3.12 awscli-1.15.40 botocore-1.10.40 colorama-0.3.9 docutils-0.14 jmespath-0.9.3 pyasn1-0.4.3 python-dateutil-2.7.3 rsa-3.4.2 s3transfer-0.1.13
Verify the awscli install:
iMac27$ pip3 list Package Version --------------- ------- awscli 1.15.40 botocore 1.10.40 colorama 0.3.9 docutils 0.14 jmespath 0.9.3 pip 10.0.1 pyasn1 0.4.3 python-dateutil 2.7.3 PyYAML 3.12 rsa 3.4.2 s3transfer 0.1.13 setuptools 28.8.0 six 1.11.0 virtualenv 15.1.0 iMac27:beta.awsdocs.com rnason$
Configuring an IAM User
In order to authenticate to AWS through the command line, we need to set credentials that the AWS CLI tool will use. This can be accomplished by setting up an IAM user that has programmatic access to AWS Services via an Access Key and Secret Key. To obtain these credentials we must first, from the AWS console create an IAM user and corresponding key pair.
1. Log into your AWS account:
Open a browser window and visit the AWS Console Page
2. Locate and navigate to the IAM Service:
From the top left side of the navigational menu bar, click on the Services menu, and then choose IAM by either navigating to the 
section of the listed services, or by typing the first few letters of the service name in the search box, and then choosing it from the filtered list.
3. Create a new IAM user:
From the IAM console dashboard, click on Users to see a current list of users. From the Users view, click the Add user button to create a new user.
4. Set IAM User Access:
In the Add User page, type a user name that you would like to assign to the new user (1), and then click Programmatic access for the Access Type (2). Once complete, click on Next:Permissions (3).
5. Set User Permissions:
Next, we need to set the permissions that the user will have. For the purpose of this walk through, we want to allow the CLI user access to the S3 service, so we will set a policy to allow the user access to S3. From the Set permissions for {User} page, click on Attach existing policies directly to pull up a list of existing polices. Use the filter to shift through the list of available policies by typing the service name to see the available options. In this example, we will use the AmazonS3ReadOnlyAccess policy to grant the new user read only access to objects in our S3 buckets. Once you have selected the policies that you want to apply to the user, click on the Next:Review button.
Account Limits
You can create up to 5000 users, 300 groups, and 1000 roles in your account, each with up to maximum of 10 policies applied to each user, group, or role
Custom IAM Policies
If none of the available policies meet the intended requirements for the user, then you can create a custom policy from the IAM dashboard->Policies section. Once a custom policy has been created, it will be available in the Attach existing policies directly section of the Permissions page.
6. Review User:
Next, simply review the user details, and click on the Create user button.
7. Copy User Credentials:
Last, click the expander arrow next to the new user to see the process flow for the user, and ensure that no errors were encountered. Once your satisfied that the user is correct, click on the Show link next to the users Secret access key to get the secret. Copy both the Access Key ID and the Secret Access Key as they will be used to configure the CLI in the next step. If you want to save the credentials longer term, you can also optionally click on the Download.csv button above the newly created user entry. Click the Close button once finished.
Save New Credentials
Ensure that you save the new user credentials when creating your user or issuing a new programmatic key. Once you click the Close, you will not be able to retrieve the Secret Access Key again, and will need to create a new key if it was not saved during this step.
Configuring AWS CLI Credentials
Now that we have an IAM user with the required permissions setup, next we need to configure the AWS CLI tool to use those credentials when sending an API request to the AWS service for any of its related commands. In order to do this we will use the aws configure command.
1. Set CLI Credentials:
Open your command shell, and then type in the aws configure command, pasting the Access Key ID, and Secret Access Key that we obtained earlier setting up the IAM user. Choose the default region that you want to use, and leave the default output format blank.
iMac27$ aws configure AWS Access Key ID [****************F4MA]: ABCDEFGHIJKLMNOPQRS AWS Secret Access Key [****************pyW5]: a1b2c3d4e5f6g7h8i9j0k9l8m7n6o5p4q3r2s1t0 Default region name [us-east-2]: us-east-1 Default output format [None]:
2. Verify Credentials:
You have the ability to both modify and add additional credentials via profiles with the AWS CLI Tool. In order to view or modify an existing profile, open the credentials file and search for the profile that you want to verify or modify. The credentials that we just set will be set under the [default] profile. If credentials previously existed in the [default] profile, using the aws configure command will automatically replace credentials that are in the [default] profile. If you would like to specify additional profiles, create a new [profile_name] block and then paste the aws_access_key_id, and aws_secret_access_key under the profile. If multiple profiles are configured, you can toggle the profile used when issuing any aws cli command by simply using the --profile profile_name into the command syntax.
iMac27$ cat ~/.aws/credentials [default] aws_access_key_id = ABCDEFGHIJKLMNOPQRS aws_secret_access_key = a1b2c3d4e5f6g7h8i9j0k9l8m7n6o5p4q3r2s1t0 [S3_RO] aws_access_key_id = ABCDEFGHIJKLMNOPQRS aws_secret_access_key = a1b2c3d4e5f6g7h8i9j0k9l8m7n6o5p4q3r2s1t0 [RDS_RW] aws_access_key_id = ABCDEFGHIJKLMNOPQRS aws_secret_access_key = a1b2c3d4e5f6g7h8i9j0k9l8m7n6o5p4q3r2s1t0 [AMZ_RO] aws_access_key_id = ABCDEFGHIJKLMNOPQRS aws_secret_access_key = a1b2c3d4e5f6g7h8i9j0k9l8m7n6o5p4q3r2s1t0
Using the AWS CLI
Now that everything is configured, we should now be able to issue commands against our AWS account using the AWS CLI
iMac27$ aws s3 ls 2018-04-19 10:18:09 example_bucket.backups 2017-10-25 14:50:11 awsdocs.com 2018-06-03 18:07:46 beta.awsdocs.com 2018-05-21 16:13:52 cf-templates-us-east-1 2018-05-23 11:44:11 cf-templates-us-east-2 2016-05-12 19:19:32 example_bucket.etc iMac27$ aws s3 ls --profile default --output text 2018-04-19 10:18:09 example_bucket.backups 2017-10-25 14:50:11 awsdocs.com 2018-06-03 18:07:46 beta.awsdocs.com 2018-05-21 16:13:52 cf-templates-us-east-1 2018-05-23 11:44:11 cf-templates-us-east-2 2016-05-12 19:19:32 example_bucket.etc
Cleanup
To uninstall the AWS CLI tools, simply use the pip uninstall command, and then ensure that your .aws home diretory is removed.
Uninstall AWS CLI Command:
pip uninstall awscli
Remove .aws Directory Command:
rm -fr ~/.aws
Conclusion
You should now have have the AWS CLI tool installed on your workstation, and can use them to interact with AWS in various ways using the myriad of API endpoints available on the AWS platform !!
Additional Resources
No Additional Resources.
Site/Information References
AWS Command Line Interface
AWS CLI Github Project
AWS CLI Command Reference
AWS CLI Command Output