Using Aws Iam Role to Upload Files to S3 Example
Option two: Configuring an AWS IAM Part to Access Amazon S3 — Deprecated¶
This section describes how to configure an S3 bucket, IAM role, and policies for Snowflake to access an external phase in a secure fashion on behalf of one or more individual users in your Snowflake account.
Equally a all-time practice, limit S3 bucket admission to a specific IAM part with the minimum required permissions. The IAM function is created in your AWS account along with the permissions to access your S3 saucepan and the trust policy to allow Snowflake to assume the IAM role.
-
An AWS IAM user created for your Snowflake business relationship is associated with an IAM role yous configure via a trust relationship.
-
The role is granted limited admission to an S3 bucket through IAM policies y'all configure.
Note
Completing the instructions in this topic requires administrative access to AWS. If you are not an AWS administrator, ask your AWS administrator to perform these tasks.
In this Topic:
-
Footstep ane: Configure S3 Bucket Access Permissions
-
Stride 2: Create an AWS IAM Role
-
Step 3: Create an External Stage
-
Step 4: Configure the AWS IAM Function to Allow Admission to the Stage
Footstep 1: Configure S3 Bucket Access Permissions¶
AWS Access Control Requirements¶
Snowflake requires the following permissions on an S3 saucepan and folder to exist able to access files in the folder (and any sub-folders):
-
s3:GetBucketLocation -
s3:GetObject -
s3:GetObjectVersion -
s3:ListBucket
Note
The following additional permissions are required to perform additional SQL actions:
| Permission | SQL Action |
|---|---|
| | Unload files to the bucket. |
| | Either automatically purge files from the stage later a successful load or execute REMOVE statements to manually remove files. |
As a all-time do, Snowflake recommends creating an IAM policy for Snowflake access to the S3 saucepan. You can then adhere the policy to the role and use the security credentials generated by AWS for the role to access files in the bucket.
Creating an IAM Policy¶
The following step-past-step instructions describe how to configure admission permissions for Snowflake in your AWS Management Console and so that you can use an S3 saucepan to load and unload data:
-
Log into the AWS Management Console.
-
From the dwelling house dashboard, choose Identity & Admission Management (IAM):
-
Choose Account settings from the left-manus navigation pane.
-
Expand the Security Token Service Regions listing, observe the AWS region corresponding to the region where your account is located, and cull Actuate if the status is Inactive.
-
Choose Policies from the left-hand navigation pane.
-
Click Create Policy:
-
Click the JSON tab.
-
Add a policy document that will allow Snowflake to admission the S3 bucket and binder.
The following policy (in JSON format) provides Snowflake with the required permissions to load or unload data using a single bucket and folder path. Y'all can also purge information files using the PURGE copy option.
Copy and paste the text into the policy editor:
Note
Brand certain to replace
bucketandprefixwith your actual bucket proper noun and folder path prefix.{ "Version": "2012-ten-17", "Argument": [ { "Effect": "Allow" , "Action": [ "s3:PutObject", "s3:GetObject", "s3:GetObjectVersion", "s3:DeleteObject", "s3:DeleteObjectVersion" ] , "Resource": "arn:aws:s3::: <bucket> / <prefix> /*" } , { "Consequence": "Permit" , "Action": [ "s3:ListBucket", "s3:GetBucketLocation" ] , "Resources": "arn:aws:s3::: <saucepan> ", "Condition": { "StringLike": { "s3:prefix": [ " <prefix> /*" ] } } } ] }
Note
Setting the
"s3:prefix":status to either["*"]or["<path>/*"]grants access to all prefixes in the specified bucket or path in the saucepan, respectively.Notation that AWS policies support a variety of different security use cases.
The post-obit policy provides Snowflake with the required permissions to load data from a unmarried read-simply bucket and folder path. The policy includes the
s3:GetBucketLocation,s3:GetObject,s3:GetObjectVersion, ands3:ListBucketpermissions:Alternative policy: Load from a read-only S3 bucket
{ "Version": "2012-x-17", "Statement": [ { "Effect": "Permit" , "Action": [ "s3:GetObject", "s3:GetObjectVersion" ] , "Resource": "arn:aws:s3::: <bucket> / <prefix> /*" } , { "Outcome": "Allow" , "Activeness": [ "s3:ListBucket", "s3:GetBucketLocation" ] , "Resource": "arn:aws:s3::: <bucket> ", "Condition": { "StringLike": { "s3:prefix": [ " <prefix> /*" ] } } } ] }
-
Click Review policy.
-
Enter the policy name (e.g.
snowflake_access) and an optional description. And then, click Create policy to create the policy.
Step 2: Create an AWS IAM Function¶
In the AWS Management Console, create an AWS IAM role that grants privileges on the S3 saucepan containing your data files.
-
Log into the AWS Management Console.
-
From the home dashboard, choose Identity & Admission Direction (IAM):
-
Choose Roles from the left-hand navigation pane.
-
Click the Create role button.
-
Select Another AWS account every bit the trusted entity type.
-
In the Business relationship ID field, enter your own AWS account ID. Later, y'all volition change the trusted human relationship and grant access to Snowflake. An external ID is required to grant access to your AWS resources (i.eastward. S3) to a third party (i.e. Snowflake in this case) later in these instructions.
-
Select the Crave external ID choice. Enter a dummy ID such as
0000. Later, y'all will alter the trusted relationship and specify the external ID for your Snowflake stage. -
Click the Side by side button.
-
Locate the policy yous created in Step 1: Configure S3 Bucket Access Permissions (in this topic), and select this policy.
-
Click the Next button.
-
Enter a name and description for the office, and click the Create role push button.
You accept now created an IAM policy for a saucepan, created an IAM function, and attached the policy to the role.
-
Record the Role ARN value located on the function summary folio. In the next step, you lot will create a Snowflake stage that references this role every bit the security credentials.
Step three: Create an External Phase¶
Create an external (i.e. S3) stage that references the AWS role you created.
-
Create an external stage using the CREATE STAGE control, or yous can cull to change an existing external stage and set the CREDENTIALS option.
Annotation
-
Credentials are handled separately from other phase parameters such every bit ENCRYPTION and FILE_FORMAT. Support for these other parameters is the same regardless of the credentials used to access your external S3 bucket.
-
Append a forrad slash (
/) to the URL value to filter to the specified folder path. If the forwards slash is omitted, all files and folders starting with the prefix for the specified path are included.Note that the forward slash is required to access and call back unstructured data files in the stage.
For instance, set
mydb.publicevery bit the current database and schema for the user session, and and so create a stage namedmy_S3_stage. In this case, the stage references the S3 bucket and pathmybucket/load/files. Files in the S3 bucket are encrypted with server-side encryption (AWS_SSE_KMS):Utilise SCHEMA mydb . public ; CREATE STAGE my_s3_stage URL = 's3://mybucket/load/files' CREDENTIALS = ( AWS_ROLE = 'arn:aws:iam::001234567890:part/mysnowflakerole' ) ENCRYPTION =( TYPE = 'AWS_SSE_KMS' KMS_KEY_ID = 'aws/key' );
-
-
Execute the Draw STAGE control to view the phase properties:
DESC STAGE my_S3_stage ; + --------------------+--------------------------------+---------------+----------------------------------------------------------------+------------------+ | parent_property | property | property_type | property_value | property_default | |--------------------+--------------------------------+---------------+----------------------------------------------------------------+------------------| ... | STAGE_CREDENTIALS | AWS_ROLE | Cord | arn:aws:iam::001234567890:role/mysnowflakerole | | | STAGE_CREDENTIALS | AWS_EXTERNAL_ID | String | MYACCOUNT_SFCRole=2_jYfRf+gT0xSH7G2q0RAODp00Cqw= | | | STAGE_CREDENTIALS | SNOWFLAKE_IAM_USER | String | arn:aws:iam::123456789001:user/vj4g-a-abcd1234 | | + --------------------+--------------------------------+---------------+----------------------------------------------------------------+------------------+
-
Tape the values for the SNOWFLAKE_IAM_USER and AWS_EXTERNAL_ID properties, where:
- SNOWFLAKE_IAM_USER
-
An AWS IAM user created for your Snowflake business relationship. This user is the same for every external S3 stage created in your account.
- AWS_EXTERNAL_ID
-
A unique ID assigned to the specific phase. The ID has the following format:
snowflakeAccount _SFCRole= snowflakeRoleId _ randomId
Note that the AWS_ROLE, AWS_EXTERNAL_ID, and SNOWFLAKE_IAM_USER values used in this example are for analogy purposes only.
In the side by side step, you will configure your AWS IAM function to grant access to the Snowflake IAM user using the generated AWS external ID.
Pace 4: Configure the AWS IAM Function to Allow Access to the Phase¶
In the AWS Management Console, configure the IAM office using the phase properties yous recorded in Stride 3: Create an External Stage (in this topic):
-
Log into the AWS Management Console.
-
From the dwelling house dashboard, choose Identity & Access Direction (IAM):
-
Cull Roles from the left-hand navigation pane, and click on the role you created in Step ii: Create an AWS IAM Role (in this topic).
-
Click the Trust relationships tab, and click the Edit trust relationship push.
-
In the Policy Document field, update the policy with the property values for the phase:
-
AWS: Enter the ARN for the SNOWFLAKE_IAM_USER phase property, i.e.
arn:aws:iam::123456789001:user/vj4g-a-abcd1234in this example. -
sts:ExternalId: Enter the generated external ID, i.east.
MYACCOUNT_SFCRole=2_jYfRf+gT0xSH7G2q0RAODp00Cqw=in this example.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Let" , "Principal": { "AWS": [ "arn:aws:iam::123456789001:user/vj4g-a-abcd1234" ] } , "Activeness": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "MYACCOUNT_SFCRole=2_jYfRf+gT0xSH7G2q0RAODp00Cqw=" } } } ] }
Note
The above trust policy allows a single external stage in your Snowflake account to assume your IAM role. Information technology is the most restrictive trust policy and is therefore the about secure.
The permission to assume the IAM part is associated with the external ID. An external ID has the following format:
snowflake_account _SFCRole= snowflake_role_id _ random_idWhere:
-
snowflake_accountis the proper name assigned to your Snowflake account. -
snowflake_role_idis an ID assigned to the Snowflake role that created the stage in Footstep 3: Create an External Stage (in this topic).In the current example, the
snowflake_role_idvalue isii. This ID is associated with a unmarried role in your Snowflake account. The purpose of this ID is limited to the trust policies for external stages; equally such, a mapping of Snowflake roles to IDs is non available. The role ID for a given role is merely exposed in the AWS_EXTERNAL_ID value in the DESCRIBE STAGE output. As a best do, restrict the ability to create external S3 stages to a single Snowflake role.Note that the role that creates a stage is not necessarily the same equally the stage owner (i.due east. the function that has the OWNERSHIP privilege on the stage). Ownership of the stage can be transferred to a different office later with no respective change required to the trust policy.
For security reasons, if you create a new storage integration (or recreate an existing storage integration using the CREATE OR REPLACE STORAGE INTEGRATION syntax), the resulting integration has a different external ID then it cannot assume the IAM role unless the trust policy is modified.
If you require a trust policy with a less secure set of restrictions (i.due east. a policy that supports all external stages in your account), supersede
random_idin the external ID with a wildcard graphic symbol (*):snowflake_account _SFCRole= snowflake_role_id _*, e.g.MYACCOUNT_SFCRole=2_*in the current example.This form of the external ID allows any external S3 stage created past a user in your account with the same Snowflake role (i.due east. SYSADMIN) to assume the IAM role, and in plow whatsoever S3 bucket the IAM role has access to. Note that if you implement this less secure type of trust policy, y'all must alter the
ConditionfromStringEqualstoStringLike. -
-
-
Click the Update Trust Policy button.
You accept now completed the one-fourth dimension setup to access your S3 bucket using an AWS role.
Next: AWS Information File Encryption
Source: https://docs.snowflake.com/en/user-guide/data-load-s3-config-aws-iam-role.html
Post a Comment for "Using Aws Iam Role to Upload Files to S3 Example"