When you're developing cloud and serverless applications, you need to grant access to various AWS resources like S3 buckets and RDS databases. To handle this, you create IAM roles and assign permissions through policies. However, configuring these policies can be challenging, especially if you want to ensure minimal access of all principals to your resources.

LocalStack's IAM Policy Stream automates the generation of IAM policies for your AWS API requests on your local machine. This stream helps you identify the necessary permissions for your cloud application and allows you to detect logical errors, such as unexpected actions in your policies.

In this blog, we'll guide you through setting up IAM Policy Stream for a locally running AWS application. We'll use a basic example involving an SNS topic, an SQS queue, and a subscription of the queue to the SNS topic. You'll be able to generate and insert the policy without manual effort, adhering to the principle of least privilege.

Why use IAM Policy Stream?

LocalStack is a tool that lets you simulate the AWS cloud on your local machine, allowing you to run your AWS cloud and serverless applications locally. It enables you to create and enforce local IAM roles and policies using the ENFORCE_IAM feature. However, users often struggle to figure out the necessary permissions for different actions. It's important to find a balance, avoiding giving too many permissions while making sure the right ones are granted.

This challenge becomes more complex when dealing with AWS services that make requests not directly visible to users. For instance, if an SNS topic sends a message to an SQS queue and the underlying call fails, there might be no clear error message, causing confusion, especially for those less familiar with the services.

IAM Policy Stream simplifies this by automatically generating the needed policies and showing them to users. This makes it easier to integrate with resources, roles, and users, streamlining the development process. Additionally, it serves as a useful learning tool, helping users understand the permissions linked to various AWS calls and improving the onboarding experience for newcomers to AWS.

Prerequisite

• LocalStack CLI with LOCALSTACK_AUTH_TOKEN

• Docker

• AWS CLI with awslocal wrapper

• LocalStack Web Application account

• jq

Subscribing a SQS queue to a SNS topic

We've got a basic demo app featuring an SNS topic named test-topic and an SQS queue named test-queue. There's also a subscription in place. The procedure includes sending a message to SNS, which, thanks to the subscription, gets pushed into the SQS queue. With LocalStack's IAM enforcement enabled, you can thoroughly test your policy and address the IAM violations by auto-generating your policies through the IAM Policy Stream.

Start your LocalStack container

Launch the LocalStack container on your local machine using the specified command:

export LOCALSTACK_AUTH_TOKEN=...
DEBUG=1 ENFORCE_IAM=1 localstack start

Once initiated, you'll receive a confirmation output indicating that the LocalStack container is up and running.

     __                     _______ __             __
    / /   ____  _________ _/ / ___// /_____ ______/ /__
   / /   / __ \/ ___/ __ `/ /\__ \/ __/ __ `/ ___/ //_/
  / /___/ /_/ / /__/ /_/ / /___/ / /_/ /_/ / /__/ ,<
 /_____/\____/\___/\__,_/_//____/\__/\__,_/\___/_/|_|

 💻 LocalStack CLI 3.1.0
 👤 Profile: default

[11:44:39] starting LocalStack in      localstack.py:494
           Docker mode 🐳

...

──── LocalStack Runtime Log (press CTRL-C to quit) ─────
LocalStack supervisor: starting
LocalStack supervisor: localstack process (PID 18) starting

LocalStack version: 3.1.1.dev20240131022456
LocalStack Docker container id: 931fae5c27d2
LocalStack build date: 2024-02-01
LocalStack build git hash: 616ef31

Navigate to IAM Policy Stream

Access the LocalStack Web Application and go to the IAM Policy Stream dashboard. This feature enables you to directly examine the generated policies, displaying the precise permissions required for each API call.

Upon the successful launch of your LocalStack container, you'll observe the Stream active status icon, indicating that making any local AWS API request will trigger the generation of an IAM Policy. Now, let's proceed to create the SNS topic and the SQS queue.

Create the AWS resources

Create a local SNS topic with the command:

awslocal sns create-topic --name test-topic

The output will be:

{
    "TopicArn": "arn:aws:sns:us-east-1:000000000000:test-topic"
}

Go to the IAM Policy Stream dashboard, and you'll see the generated policy.

The AWS API call created an identity-based policy for the root user; If you do not set any credentials returned by IAM or STS, LocalStack will identify the request as being made by the root user. The policy is for the CreateTopic API action, allowing the specified resource.

Now, create the SQS queue with:

awslocal sqs create-queue --queue-name test-queue

The output will be:

{
    "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/test-queue"
}

On the IAM Policy Stream dashboard, you'll notice the generated policy.

Create a subscription with the topic ARN of the SNS topic, the protocol, and the notification endpoint:

awslocal sns subscribe \
    --topic-arn arn:aws:sns:us-east-1:000000000000:test-topic \
    --protocol sqs \
    --notification-endpoint arn:aws:sqs:us-east-1:000000000000:test-queue

The output will be:

{
    "SubscriptionArn": "arn:aws:sns:us-east-1:000000000000:test-topic:4283d647-18b6-4aeb-b283-19d9327a963a"
}

Testing the subscription

To test the subscription, publish a message to the SNS topic:

awslocal sns publish \
    --topic-arn arn:aws:sns:us-east-1:000000000000:test-topic \
    --message '{"some": "event"}'

The output will be:

{
    "MessageId": "63317413-c40b-41ed-982b-db722337eb5b"
}

Check if your SQS queue received the message:

awslocal sqs receive-message \
    --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/test-queue

Unfortunately, there is no message due to a lack of proper IAM policy, preventing the SNS service from publishing to the SQS queue. Let's use IAM Policy Stream to resolve this issue.

Analyzing IAM Policies

Navigate to the IAM Policy Stream dashboard and observe various API calls like Publish, SendMessage, ReceiveMessage. Note that the SendMessage call was rejected due to an IAM violation.

Click on SQS.SendMessage to view the request parameters and the required resource-based policy.

LocalStack automatically suggests a resource-based policy for the arn:aws:sqs:us-east-1:000000000000:test-queue SQS queue. Copy and paste the policy into a new JSON file named policy.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Test432a8c7b",
      "Effect": "Allow",
      "Action": "sqs:SendMessage",
      "Resource": "arn:aws:sqs:us-east-1:000000000000:test-queue",
      "Principal": {
        "Service": [
          "sns.amazonaws.com"
        ]
      },
      "Condition": {
        "ArnEquals": {
          "aws:SourceArn": "arn:aws:sns:us-east-1:000000000000:test-topic"
        }
      }
    }
  ]
}

Encode this policy as a string using jq, required by the AWS CLI:

jq @json < policy.json
"{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"Test432a8c7b\",\"Effect\":\"Allow\",\"Action\":\"sqs:SendMessage\",\"Resource\":\"arn:aws:sqs:us-east-1:000000000000:test-queue\",\"Principal\":{\"Service\":[\"sns.amazonaws.com\"]},\"Condition\":{\"ArnEquals\":{\"aws:SourceArn\":\"arn:aws:sns:us-east-1:000000000000:test-topic\"}}}]}"

Create a new file named sqs-queue-attributes.json and paste the generated policy:

{
    "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"Test432a8c7b\",\"Effect\":\"Allow\",\"Action\":\"sqs:SendMessage\",\"Resource\":\"arn:aws:sqs:us-east-1:000000000000:test-queue\",\"Principal\":{\"Service\":[\"sns.amazonaws.com\"]},\"Condition\":{\"ArnEquals\":{\"aws:SourceArn\":\"arn:aws:sns:us-east-1:000000000000:test-topic\"}}}]}"
}

Set the queue attributes for the SQS queue:

awslocal sqs set-queue-attributes \
    --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/test-queue \
    --attributes file://sqs-queue-attributes.json

Send another message to the SNS topic to be received by the SQS queue:

awslocal sns publish \
    --topic-arn arn:aws:sns:us-east-1:000000000000:test-topic \
    --message '{"some": "event"}'
awslocal sqs receive-message \
    --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/test-queue

Verify that you have received the message:

{
    "Messages": [
        {
            "MessageId": "84a24f67-995b-45db-87d7-f6fca8891f4e",
            "ReceiptHandle": "MDUwNTAwYzQtMTJiNS00NzdiLTg2OTYtODA5MjAzMWQ3YzY1IGFybjphd3M6c3FzOnVzLWVhc3QtMTowMDAwMDAwMDAwMDA6dGVzdC1xdWV1ZSA4NGEyNGY2Ny05OTViLTQ1ZGItODdkNy1mNmZjYTg4OTFmNGUgMTcwNzcyMDg2My45OTEwMDQ=",
            "MD5OfBody": "a85e018e2a7d2d06866b7da00268fcc9",
            "Body": "{\"Type\": \"Notification\", \"MessageId\": \"4dbec68e-2489-4e73-bee4-90d02ffe691f\", \"TopicArn\": \"arn:aws:sns:us-east-1:000000000000:test-topic\", \"Message\": \"{\\\"some\\\": \\\"event\\\"}\", \"Timestamp\": \"2024-02-12T06:54:20.269Z\", \"UnsubscribeURL\": \"http://localhost.localstack.cloud:4566/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:000000000000:test-topic:4283d647-18b6-4aeb-b283-19d9327a963a\", \"SignatureVersion\": \"1\", \"Signature\": \"ymKIdXa+KzAy5aZ6XAA7P1TM7azzCounaFv4etpm7GL7qHawdn86aeM6q7VhDgTzCBdI3iGEjOaoAzaWCnB1RdQd9rt8Gfwckk0QtlGefEJBVdiH1DCNyGD+A48hSGUPtAk22d0Ar1AzBWtQ49DFTbfgEqfGGNlPdrri+JJmfztgg7hb0tUZoeWM3p7wfNuj7+nXGtS4JuVf5yC0f/v6ryo0IbNiGEjfsXyAU5++Lx2V3o2aZK/WfUWa5EkIqjAc6RzmnIE60IUYvn/7mVEgdl5CZLvJ0hPGrLaxFwdMd04LiYJCE4bLMfWxDyVQFysKH8GwFqRfZjrYmpdiVtI9Rw==\", \"SigningCertURL\": \"http://localhost.localstack.cloud:4566/_aws/sns/SimpleNotificationService-6c6f63616c737461636b69736e696365.pem\"}"
        }
    ]

You can now successfully verify on the IAM Policy Stream dashboard that no violation has been noticed, and your AWS API requests have successfully executed with the right IAM policies.

Generating a comprehensive policy

In scenarios where there are many AWS services, and every AWS API request generates a policy it might be cumbersome to analyze every policy. In such cases, you can generate one comprehensive policy for all your AWS resources together.

You can navigate to the Summary Policy tab on the IAM Policy Stream dashboard. This concatenates the policy per principle which the policy should be attached to. For the example above, you would be able to see the Identity Policy for the root user which has all the actions and resources inside one single policy file for the operations we performed.

On the other hand, you have the Resource Policy for the SQS queue, where you can see the permission necessary for the subscription. For larger AWS applications, you would be able to find multiple roles and multiple resource-based policies depending on your scenario.

Conclusion

IAM Policy Stream streamlines your development process by minimizing the manual creation of policies and confirming the necessity of granted permissions. However, it is advisable to manually confirm that your policy aligns with your intended actions. Your code may unintentionally make requests, and LocalStack considers all requests made during policy generation as valid.

A practical scenario is automating tests like integration or end-to-end testing against your application, allowing LocalStack to automatically generate policies with required permissions. You can then review and customize them to meet your needs, ensuring that overly permissive policies don't find their way into production environments.

We are actively working on expanding this feature and offering more advantages for developers, such as automatically analyzing deployed policies for unused permissions and more! Stay tuned for updates on our IAM feature set!