AWS Glue Connector

AWS Glue is a serverless data integration service from Amazon Web Services.
The AWS Glue component allows listing, starting, and stopping jobs, triggers, and crawlers in an AWS Glue account.

API Documentation

This component was built using the AWS Glue Developer Guide.

Connections

AWS Glue Access Key and Secret

Authenticates requests to AWS Glue using an API Key and API Secret.

An AWS IAM access key pair is required to interact with AWS Glue.
Ensure the key pair generated in AWS has proper permissions to the AWS Glue resources to access.
Read more about Glue IAM actions in the AWS documentation.

Prerequisites

• An AWS account with IAM access
• Appropriate permissions to create IAM access keys

Setup Steps

To create an IAM access key pair:

1. Sign in to the AWS Console and navigate to Identity and Access Management (IAM)
2. Select the IAM user that will be used for the integration
3. Navigate to the Security credentials tab
4. Under the Access keys section, select Create access key
5. Choose the appropriate use case (e.g., Third-party service or Application running outside AWS)
6. Copy both the Access key ID and Secret access key when displayed

Configure the Connection

• Enter the Access Key ID into the connection configuration
• Enter the Secret Access Key into the connection configuration

AWS Role ARN

Connect to AWS using an assumed role

AWS Assume Role authentication allows assuming an IAM role using temporary security credentials.
This method is useful for cross account access or when implementing principle of least privilege.

Refer to the AWS documentation on assuming roles for detailed information.

Prerequisites

• Completed Access Key and Secret connection setup (access key pair required)
• Appropriate permissions to create IAM roles

Setup Steps

An IAM user with access keys is required to assume a role. If access keys have not been created, follow the Access Key and Secret setup steps first, then return here to create the IAM role.

Create an IAM Role with Trust Policy

1. From the IAM Console, navigate to Roles and select Create Role
2. Select Custom trust policy as the trusted entity type
3. Enter the trust policy below, replacing USER_ARN with the IAM user ARN:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "USER_ARN"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

1. Click Next and attach the appropriate Glue permissions policy (e.g., AWSGlueConsoleFullAccess)
2. Complete the remaining steps and select Create Role
3. Copy the Role ARN from the role summary (format: arn:aws:iam::123456789012:role/role-name)

Configure the Connection

• Role ARN: The ARN of the IAM role to assume
• Access Key ID: From the IAM user (see Access Key and Secret)
• Secret Access Key: From the IAM user
• External ID (optional): Shared secret for enhanced security

Verify Connection

The IAM user credentials assume the role, which provides temporary credentials with the role's attached permissions. Ensure the trust policy correctly references the IAM user ARN.

Actions

Get Job Run

Retrieves the metadata for a given job run.

List Crawlers

List Crawlers available in AWS Glue

List Jobs

List job schemas available in AWS Glue

List Triggers

List the names of all triggers in the account.

Start Crawler

Starts an existing crawler in AWS Glue.

Start Job Run

Starts a job run using a AWS Glue job definition.

Start Trigger

Starts an existing trigger in AWS Glue.

Stop Crawler

If the specified crawler is running, stops the crawl

Stop Job Run

Stops one or more job runs for a specified job definition

Stop trigger

Stops a specified trigger