Josh Kahn - Solutions architect, speaker, technology guy, dad

A few months ago, I wrote a post on the AWS Mobile Blog that demonstrated how to invoke AWS service directly from AWS AppSync. In that case, I specifically focused on AWS Step Functions, but it is possible to integrate AppSync with many other AWS services. The goal of this post is to document the integration of AppSync with services beyond Step Functions.

AWS AppSync uses GraphQL resolvers to define the interaction between AppSync and each data source. These are Apache Velocity Templates that will be unique per GraphQL operation. By moving integration of AWS services to resolvers, we can minimize the maintenance of integration code. Velocity Templates generally require little upkeep over time. This approach can be even more maintainable than relying on a thin layer of Lambda code for integration, which would still require dependency management and updates.

Adding an AWS data source

The first step in integrating AppSync with an AWS service is to create an HTTP data source for the service. For an AWS service, the data source endpoint is set to the service API endpoint for the given AWS region and we configure SigV4 signing.

Currently, to configure signing of HTTP data sources, we can use either the AWS CLI or CloudFormation. My preference is to utilize CloudFormation for these needs, though my earlier blog post demonstrates the CLI approach as well. For example, we can add a new data source to invoke Amazon SQS using CloudFormation as follows

SQSHttpDataSource:
  Type: AWS::AppSync::DataSource
  Properties:
    ApiId: !GetAtt MyApi.ApiId
    Name: SQSDataSource
    Description: SQS Data Source
    Type: HTTP
    # IAM role defined elsewhere in AWS CloudFormation template
    # Note: this role needs a policy with 'sqs:SendMessage' permission
    ServiceRoleArn: !GetAtt AppSyncServiceRole.Arn
    HttpConfig:
      Endpoint: !Sub https://sqs.${AWS::Region}.amazonaws.com/
      AuthorizationConfig:
        AuthorizationType: AWS_IAM
        AwsIamConfig:
          SigningRegion: !Ref AWS::Region
          SigningServiceName: sqs

AppSync will require an AWS IAM role that allows the service to perform the desired action. For example, SQS would require sqs:SendMessage permission and Step Functions states:StartExecution.

After creating the data source, we can implement the resolvers that allow interaction with the AWS service. Let's step through a few example integrations.

Amazon SQS

For each service reviewed in this post, we will start with a sample GraphQL schema that defines the interactions and data associated with the resolver.

We will begin with Amazon Simple Queue Service (SQS), one of the earliest AWS services and one of my favorites for building reliable, decoupled workloads. Here, we send a messsage via a queue, though other operations are available as well. For example, we could list all messages on a particular queue.

input SendMessageInput {
  email: String!
  message: String!
}

type Message {
  id: ID!
  email: String!
  message: String!
}

type Mutation {
  sendMessage(input: SendMessageInput!): Message
}

AppSync can be used to deliver a message to an SQS Queue using the following resolver. While SQS also supports a POST API as well, I have implemented here using a GET. Note that the endpoint resource path includes the AWS Account ID and name of the queue. Infrastructure as code approaches, such as CloudFormation, would make wiring this information easy.

sendMessage - Request Mapping

{
  "version": "2018-05-29",
  "method": "GET",
  "resourcePath": "/<ACCOUNT_ID>/<QUEUE_NAME>",
  "params": {
    "query": {
      "Action": "SendMessage",
      "Version": "2012-11-05",
      "MessageBody": "$util.urlEncode($util.toJson($ctx.args.input))"
    }
  }
}

The response from SQS is encoded in XML and can be easily transformed to a JSON payload using the $util functions provided by AppSync.

sendMessage - Response Mapping

#set( $result = $utils.xml.toMap($ctx.result.body) )
{
  "id": "$result.SendMessageResponse.SendMessageResult.MessageId",
  "email": "$ctx.args.input.email",
  "message": "$ctx.args.input.message",
}

AWS Secrets Manager

AWS Secrets Manager allows customers to protect sensitive information such as API keys. In a future blog post (UPDATE: blog post), I will share how I incorporated Secrets Manager in an AWS AppSync Pipeline Resolver to first retrieve a secret API key before querying a third-party web service for data, all via AppSync resolvers. For now, we can simply retrieve and return a secret from Secrets Manager.

type Query {
  getSecret(SecretName: String!): String
}

getSecret - Request Mapping

{
  "version": "2018-05-29",
  "method": "POST",
  "resourcePath": "/",
  "params": {
    "headers": {
      "content-type": "application/x-amz-json-1.1",
      "x-amz-target": "secretsmanager.GetSecretValue"
    },
    "body": {
      "SecretId": "$ctx.args.SecretName"
    }
  }
}

getSecret - Response Mapping

#set( $result = $util.parseJson($ctx.result.body) )
$util.toJson($result.SecretString)

AWS Step Functions

Although I discussed Step Functions in the earlier mentioned blog, I will include starting a Step Functions state machine here for completeness and to document handling more complex input.

type Mutation {
  submitOrder(input: OrderInput!): Order
}

submitOrder - Request Mapping

$util.qr($ctx.stash.put("orderId", $util.autoId()))
{
  "version": "2018-05-29",
  "method": "POST",
  "resourcePath": "/",
  "params": {
    "headers": {
      "content-type": "application/x-amz-json-1.0",
      "x-amz-target":"AWSStepFunctions.StartExecution"
    },
    "body": {
      "stateMachineArn": "<STATE_MACHINE_ARN>",
      "input": "$util.escapeJavaScript($util.toJson($input))"
    }
  }
}

sumitOrder - Response Mapping

{
  "id": "${ctx.stash.orderId}"
}

Amazon S3

Inspired by a customer, we can also read and write objects from Amazon S3 via AppSync. Like the other examples included here, we rely on an HTTP Data Source configured with AWS IAM authorization. Unlike other service endpoints, S3 requires us to include the name of the target bucket in the endpoint. In CloudFormation, configuration of an S3 Data Source is as follows:

StorageDataSource:
  Type: AWS::AppSync::DataSource
  Properties:
    ApiId: !GetAtt MyApi.ApiId
    Name: S3DataSource
    Description: Amazon S3 Bucket
    Type: HTTP
    ServiceRoleArn: !GetAtt AppSyncServiceRole.Arn
    HttpConfig:
      Endpoint: !Sub "https://${MyBucket}.s3.${AWS::Region}.amazonaws.com"
      AuthorizationConfig:
        AuthorizationType: AWS_IAM
        AwsIamConfig:
          SigningRegion: !Sub "${AWS::Region}"
          SigningServiceName: s3

For this use case, the customer asked to read and write JSON payloads to S3 and automatically generate a unique identifier on write. The GraphQL schema for these operations is as follows:

type Mutation {
  putPayload(input: Payload): Response
}
input Payload {
  payload: AWSJSON!
}
type Query {
  getPayload(payloadId: ID!): Response!
}
type Response {
  payloadId: String!
  payload: AWSJSON
}

Remember, each data source works with a single S3 Bucket. If your application interacts with multiple S3 Buckets, create a data source for each.

putPayload - Request Mapping

To write an object to S3, we use the putObject REST API. We assume the AppSync service role has appropriate permissions to perform this operation on the S3 bucket. Note that other content types may require changes to the body of the request or content type header.

## generate a unique identifier for the payload
$util.qr($ctx.stash.put("payloadId", $util.autoId()))
{
  "version": "2018-05-29",
  ## put the object in the bucket
  "method": "PUT",
  ## specify the name of the object in the resource path
  "resourcePath": "/${ctx.stash.payloadId}.json",
  "params": {
    "headers": {
      ## specify the content type of the object, e.g. JSON
      "Content-Type" : "application/json"
    },
    "body": $util.toJson($ctx.args.input.payload)
  }
}

putPayload - Response Mapping

The response to putting the payload in S3 is the unique identifier for that object in S3:

{
  "payloadId": "${ctx.stash.payloadId}"
}

getPayload - Request Mapping

To retrieve an object from S3, we can use the 'getObject` REST API. Again, the AppSync service role referenced for this data source will need appropriate permissions.

{
  "version": "2018-05-29",
  "method": "GET",
  ## specify name of the object to retrieve
  "resourcePath": "/${ctx.args.payloadId}.json"
}

getPayload - Response Mapping

For the response, we include both the payload JSON (string encoded) as well as the payload identifier. As noted earlier, this can be modified to support different use cases.

{
  "payloadId": "${ctx.args.payloadId}",
  "payload": $ctx.result.body
}

AWS EventBridge

Thanks to Ed Lima for posting a solution on integrating AppSync with AWS EventBridge. A typical operation for interacting with EventBridge is to put an event on an event bus, this is a mutation.

type Mutation {
  putEvent(event: String!): Event
}

putEvent - Request Mapping

Like other services documented here, the x-amz-target header is important in specifying the intended action. The body also must be formatted as the REST API for the service expects.

{
  "version": "2018-05-29",
  "method": "POST",
  "resourcePath": "/",
  "params": {
    "headers": {
      "content-type": "application/x-amz-json-1.1",
      "x-amz-target":"AWSEvents.PutEvents"
    },
    "body": {
      "Entries":[ 
        {
          "Source":"appsync",
          "EventBusName": "default",
          "Detail":"{ \\\"event\\\": \\\"$ctx.arguments.event\\\"}",
          "DetailType":"Event Bridge via GraphQL"
          }
      ]
    }
  }
}

putEvent - Response Mapping

Ed provides a nice response mapping that will raise an error if EventBridge responds with an error message. See service documentation for details on how each AWS service handles errors.

#if($ctx.error)
  $util.error($ctx.error.message, $ctx.error.type)
#end
## if the response status code is not 200, then return an error. Else return the body **
#if($ctx.result.statusCode == 200)
  ## If response is 200, return the body.
  {
    "result": "$util.parseJson($ctx.result.body)"
  }
#else
  ## If response is not 200, append the response to error block.
  $utils.appendError($ctx.result.body, $ctx.result.statusCode)
#end

Please let me know if there are other integrations you would be interested in seeing.

Photo by Patrick Perkins on Unsplash