Secrets Manager: Using the AWS Secrets Manager

For more information on AWS Secrets Manager, visit aws.amazon.com

Secrets Manager Description

---

AWS Secrets Manager is a service that allows a user to store confidential information such as traditional username & password connection credentials that a number of applications require to connect to resources such as a Database, in a secure encrypted vault. Credentials and other key value pairs that are stored in the secrets manager are easily retrieved by your application ensuring that those credentials aren't accidentally hard coded into your application or accidentally committed to your git repository, but instead safely kept in an encrypted store which not only averts accidental human error, but also provides a mechanism to allow rotation of those credentials to be easy and seamless for your application in the future.

Navigating to Secrets Manager

---

1.    Log into your AWS account:
Open a browser window and visit the AWS Console Page

2.    Locate and navigate to Secrets Manager: From the top left side of the navigational menu bar, click on the Services menu, and then choose Secrets Manager by either navigating to the section of the listed services, or by typing the first few letters of the service name in the search box, and then choosing it from the filtered list.

Storing a Secret from the Console

---

In the Secrets Manager console, once opened if never used will present you with a Get Started section at the very top of the page that describes what the Secrets Manager service does and how to use it. To get started saving credentials in the secrets manager, start by clicking the Store a new secret button on the top of the page to navigate to the section of the service where we can actually store our credentials.

1.    Choose the Secret Type:
The first thing we need to do is to select the type of secret that we are going to store. There is a default option that is geared to store RDS type credentials, an option for other types of Databases, and then Other types for any other custom key:value storage. The primary difference between these options is simply the pre-populated key:value pairs that are defined within the option. For this example, we are going to store Active Directory Service Account credentials that will be used by a custom Windows AMI that will both retrieve and use the stored credentials to automatically join new Windows based instances to an Active Directory Domain. To store this type of credential, choose the Other type of secrets type, and then under the Secret key/value section, lets add a key value pair with the key set to ServiceAccount with a value of {your_service_account_user_name}, followed by a second key set to Password with a value consisting of the {service_account_password}.. Once you have created the key:value pairs, then ensure that DefaultEncryptionKey is selected under the encryption section, and click the Next button.

2.    Name your Secret:
Next we need to select a name, and type a description for the new secret. Name your secret based on your naming conventions, and type a description on what the secret will be used for, once complete, click the Next button.

3.    Define Secrets Rotation:
Next define the rotation policy that will be applied to this secret. To keep things simple, we will just ensure that Disable automatic rotation is selected, and the click the Next button.

4.    Review the Secret:
The last thing we need to do is to simply Review the Secret we are about to save. Once you have verified that all of the information is correct, then scroll down the page to the Sample Code section. Here you will find examples of how we can retrieve the secret in a variety of languages such as Java, JavaScript, C#, and Python 3. These code samples can be used directly or built into custom script that your application can utilize in order to retrieve and use the saved secret. Once you have reviewed the secret, and copied any code snippets that you want to preserve, click on the Store button to save the secret to the Secrets Manager.

Code Samples

If you need to reference the given code samples after the secret has been stored, you can do so by going to the Secrets Manager Console, selecting the desired secret, and then on the secret details page, scroll down to the Sample code section.

Storing a Secret from the CLI

---

Now that we have walked through creating and storing a secret via the console. Lets quickly walk through creating a secret from within the AWS CLI.

1.    Create the secret from the CLI:
Use the syntax example below to store a new secret into the Secrets Manager Service.

Syntax:

aws secretsmanager create-secret --region {REGION} --name {SECRET_NAME} \
--description {SECRET_DESCRIPTION} \
--secret-string {JSON Key Value Pairs containing the key and value pairs you want stored in the secret}

Example Request:

aws secretsmanager create-secret --region us-east-1 --name TEST/MySuperSecret \
--description "This Secret will contain some test data sent from the command line" \
--secret-string "{\"username\":\"me\",\"password\":\"abc123xyz456\"}"

Example Response:

{
    "ARN": "arn:aws:secretsmanager:us-east-1:123456789012:secret:TEST/MySuperSecret-AlFzzx",
    "Name": "TEST/MySuperSecret",
    "VersionId": "71e8a466-9d85-406a-a6a6-ba2da713be61"
}

--secret-string vs --secret-binary

Either --secret-string or --secret-binary must have a value, but not both. They cannot both be empty, nor can they both be populated

2.    Verify the Secret from the CLI:
Now that we have created the secret, lets use the CLI to retrieve it to ensure it was written properly to the SM Service.

Syntax:

aws secretsmanager get-secret-value --region us-east-1 --secret-id {SECRET_NAME}

Example Request:

aws secretsmanager get-secret-value --region us-east-1 --secret-id TEST/MySuperSecret

Example Response:

{
    "ARN": "arn:aws:secretsmanager:us-east-1:123456789012:secret:TEST/MySuperSecret-AlFzzx",
    "Name": "TEST/MySuperSecret",
    "VersionId": "71e8a466-9d85-406a-a6a6-ba2da713be61",
    "SecretString": "{\"username\":\"me\",\"password\":\"abc123xyz456\"}",
    "VersionStages": [
        "AWSCURRENT"
    ],
    "CreatedDate": 1533790094.174
}

Permissions Required to Retrieve a Secret

---

Prior to attempting to make a call from a resource such as an EC2 instance or Lambda function to retrieve our stored secret, we must ensure that the resource that will be making the call has the appropriate permissions to do so. In order to ensure this, lets quickly follow tutorial steps found in the Creating a Custom Policy and Creating a Custom Role articles respectively to create a custom Role that will be applied to our resource, be it EC2 or Lambda, etc.. that will simply contain a custom policy that will allow ReadOnly access to the Secrets Manager.

1.    IAM Custom Policy:
Construct a custom policy to allow all List and Read operations on the Secrets Manager service.

2.    IAM Custom Role:
Choose EC2/Lambda/Etc as the service, then choose the new custom policy that we created in the last step as the policy to be applied.

Retrieving a Secret

---

Once we have created and stored our secret, we were given examples on how to get the secret in a variety of languages such as Java, JavaScript, C# and Python 3. We now must come up with a methodology to retrieve the secret from an EC2 instance or Service such as Lambda depending on our desired use case.

1.    Python3 Example:
The following example outlines code written in Python3 that can make the call to the Secrets Manager Service to retrieve the value of a given secret.

import boto3  # Required to interact with AWS
import json   # Required for return object parsing
from botocore.exceptions import ClientError

# Set required variables
secret_name = "Wordpress/Stage"
endpoint_url = "https://secretsmanager.us-east-2.amazonaws.com"
region_name = "us-east-2"

session = boto3.session.Session()

client = session.client(
    service_name='secretsmanager',
    region_name=region_name,
    endpoint_url=endpoint_url
)

try:
    get_secret_value_response = client.get_secret_value(
        SecretId=secret_name
    )
except ClientError as e:
    if e.response['Error']['Code'] == 'ResourceNotFoundException':
        print("The requested secret " + secret_name + " was not found")
    elif e.response['Error']['Code'] == 'InvalidRequestException':
        print("The request was invalid due to:", e)
    elif e.response['Error']['Code'] == 'InvalidParameterException':
        print("The request had invalid params:", e)
else:
    # Decrypted secret using the associated KMS CMK
    # Depending on whether the secret was a string or binary, one of these fields will be populated
    if 'SecretString' in get_secret_value_response:
        secret = json.loads(get_secret_value_response['SecretString'])
    else:
        binary_secret_data = get_secret_value_response['SecretBinary']

user = secret['username']
password = secret['password']
db_name = secret['dbname']
host = secret['host']

2.    Powershell Example:
The following example outlines code written in PowerShell that can make the call to the Secrets Manager Service to retrieve the value of a given secret.

# Make a request to the secret manager
$secret_manager = Get-SECSecretValue -SecretId Windows/ServiceAccounts/DomainJoin

# Parse the response and convert the Secret String JSON into an object
$secret = $secret_manager.SecretString | ConvertFrom-Json

# Construct the domain credentials
$username = $secret.ServiceAccount
$password = $secret.Password | ConvertTo-SecureString -asPlainText -Force

Secret Conclusion

---

Using the CLI, or various SDK's it's very easy to store and retrieve a secret in the AWS Secrets Manager. Functions like the examples above, can easily be configured on application server instances to grab application, or database credentials from the encrypted Secrets Manager, allowing you to keep credentials secure, while still being easily accessible to your application or service.

Secrets Manager Additional Resources

---

No Additional Resources.

Secrets Manager Site/Information References

---

PowerShell CMD Reference
Creating a Basic Secret
Create-Secret Command Reference
Get-Secret-Value Command Reference

Comments