AWS Lambda

Manage Functions as a Service with the CLI

An engineer has a few tasks to perform when it comes to working with Lambda -- writing the code, packaging it for deployment, and finally monitoring its execution and fine-tuning. We're going to look at basic packaging and deployment with the aws cli.

Check User or Role Credentials

# check the current aws credentials used, similar to whoami
aws iam get-user
# check the user policies that are attached to the user name
aws iam list-attached-user-policies --user-name myawsusername
# check the group policies that are attached to the user name
aws iam list-groups-for-user --user-name myawsusername
# check for access keys granted to a user name
aws iam list-access-keys --user-name myawsusername
# check for roles within an account
aws iam list-roles

IAM Role for Lambda to Assume

In order for the lambda function to execute, we need to create an minimalist IAM Role that Lambda can assume
for executing the function on our behalf.

// save this file as policy.json  
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
         "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

#  Create AWS IAM role
aws iam create-role --role-name basic-lambda-role --assume-role-policy-document file://policy.json

# output
{
    "Role": {
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17", 
            "Statement": [
                {
                    "Action": "sts:AssumeRole", 
                    "Effect": "Allow", 
                    "Principal": {
                        "Service": "lambda.amazonaws.com"
                    }
                }
            ]
        }, 
        "RoleId": "AROAIFLLG5YTUYNT6GKGA", 
        "CreateDate": "2017-12-04T02:54:51.623Z", 
        "RoleName": "basic-lambda-role", 
        "Path": "/", 
        "Arn": "arn:aws:iam::1234567891012:role/basic-lambda-role"
    }
}

# List functions aws lambda list-functions --output table

Execute (Invoke) a Lambda function

Each function can be invoked from the command line, with the event data passed in from a local file.

# read request data into a local variable
request=$(< request.json)
# invoke the function with the data, and write to local file
aws lambda invoke --function-name myFunction --payload "$request" response.json
# read response file into local variable then print to the console
responseOutput=$(< response.json )
echo $responseOutput

Move files between a local host and S3

# copy a local file to a s3 bucket and folder
aws s3 cp foo.json s3://mybucket/myfolder
# retrieve a local file from a s3 bucket
aws s3 cp s3://mybucket/myfolder foo.json
# list all buckets within an account
aws s3 ls
# list all of the objects and folder within a bucket
aws s3 ls s3://mybucket
# test removal (aka dry run) of an object from a s3 bucket
aws s3 rm s3://mybucket/myfolder/foo.json --dryrun
# remove an object from a S3 bucket
aws s3 rm s3://mybucket/myfolder/foo.json
See S3 docs

Lambda Function reads DynamoDB table: Books

// nodejs4
// index.js
var AWS = require("aws-sdk");
var doClient = new AWS.DynamoDB.DocumentClient();

exports.handler = (event, context, callback) => {

   console.log("PARAMS---" +parseInt(event.params.querystring.bookid));
   var params = {
      TableName: "Books",
      Key: {
           "bookid": parseInt(event.params.querystring.bookid)
      }
    }; // end params
    
    doClient.get(params, function(err, data) {
    
       if(err) {
           console.error("Unable to read item. Error JSON:", JSON.string(err, null, 2));
       } else {
           callback(null, JSON.parse( JSON.stringify(data, null, 2)));
       }
       
    }); // end doClient

}; // end of exports.handler

// json input test
{
  "body-json": {},
  "params" : {
     "path" : {},
     "querystring" : {
       "bookid" : "1"
     }
  }
}

Update an existing Lambda function

# first create a zip file containing all elements in the package
# note lambda_dir/ is what includes libraries in the zip
zip -r myFunction.zip index.js lambda_dir/ package.json
# then copy the zip file to S3
aws s3 cp myfunction.zip s3://mybucket/myfolder
# finally deploy the package to the runtime environment
aws lambda update-function-code --function-name myFunction --s3-bucket mybucket --s3-key myfunction.zip

Create a new Lambda function

# first create a zip file containing all elements in the package
# note lambda_dir/ is what includes libraries in the zip
zip -r myFunction.zip index.js lambda_dir/ package.json
# create a new function based on the parameters and zip package
aws lambda create-function --function-name newFunction --region us-east-1 --memory-size 128 --runtime nodejs6.10 --role arn:aws:iam::1234567891012:role/basic-lambda-role --handler index.handler --zip-file "fileb://myfunction.zip"
# note: runtime options include nodejs6.10, java8, python2.7

CloudWatch Event triggers a Lambda to check if site is up

Github repo for testing the code locally




calling code locally:

$> expected=Univrs.io site=http://univrs.io python checkSite.py

from __future__ import print_function

import os
from datetime import datetime
from urllib2 import urlopen

SITE = os.environ['site']  # URL of the site to check, stored in the site environment variable
EXPECTED = os.environ['expected']  # String expected to be on the page, stored in the expected environment variable


def validate(res):
    '''Return False to trigger the canary

    Currently this simply checks whether the EXPECTED string is present.
    However, you could modify this to perform any number of arbitrary
    checks on the contents of SITE.
    '''
    return EXPECTED in res


def lambda_handler(event, context):
    print('Checking {} at {}...'.format(SITE, event['time']))
    try:
        if not validate(urlopen(SITE).read()):
            raise Exception('Validation failed')
    except:
        print('Check failed!')
        raise
    else:
        print('Check passed!')
        return event['time']
    finally:
        print('Check complete at {}'.format(str(datetime.now())))

Using CloudFormation's cfn-response

AWSTemplateFormatVersion: '2010-09-09'
Parameters:
  ExistingSecurityGroups:
    Type: List
  ExistingVPC:
    Type: AWS::EC2::VPC::Id
    Description: The VPC ID that includes the security groups in the ExistingSecurityGroups
      parameter.
  InstanceType:
    Type: String
    Default: t2.micro
    AllowedValues:
    - t2.micro
    - m1.small
Mappings:
  AWSInstanceType2Arch:
    t2.micro:
      Arch: HVM64
    m1.small:
      Arch: PV64
  AWSRegionArch2AMI:
    us-east-1:
      PV64: ami-1ccae774
      HVM64: ami-1ecae776
    us-west-2:
      PV64: ami-ff527ecf
      HVM64: ami-e7527ed7
    us-west-1:
      PV64: ami-d514f291
      HVM64: ami-d114f295
    eu-west-1:
      PV64: ami-bf0897c8
      HVM64: ami-a10897d6
    eu-central-1:
      PV64: ami-ac221fb1
      HVM64: ami-a8221fb5
    ap-northeast-1:
      PV64: ami-27f90e27
      HVM64: ami-cbf90ecb
    ap-southeast-1:
      PV64: ami-acd9e8fe
      HVM64: ami-68d8e93a
    ap-southeast-2:
      PV64: ami-ff9cecc5
      HVM64: ami-fd9cecc7
    sa-east-1:
      PV64: ami-bb2890a6
      HVM64: ami-b52890a8
    cn-north-1:
      PV64: ami-fa39abc3
      HVM64: ami-f239abcb
Resources:
  SecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Allow HTTP traffic to the host
      VpcId:
        Ref: ExistingVPC
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: '80'
        ToPort: '80'
        CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
      - IpProtocol: tcp
        FromPort: '80'
        ToPort: '80'
        CidrIp: 0.0.0.0/0
  AllSecurityGroups:
    Type: Custom::Split
    Properties:
      ServiceToken: !GetAtt AppendItemToListFunction.Arn
      List:
        Ref: ExistingSecurityGroups
      AppendedItem:
        Ref: SecurityGroup
  AppendItemToListFunction:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          var response = require('cfn-response');
          exports.handler = function(event, context) {
             var responseData = {Value: event.ResourceProperties.List};
             responseData.Value.push(event.ResourceProperties.AppendedItem);
             response.send(event, context, response.SUCCESS, responseData);
          };
      Runtime: nodejs4.3
  MyEC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId:
        Fn::FindInMap:
        - AWSRegionArch2AMI
        - Ref: AWS::Region
        - Fn::FindInMap:
          - AWSInstanceType2Arch
          - Ref: InstanceType
          - Arch
      SecurityGroupIds: !GetAtt AllSecurityGroups.Value
      InstanceType:
        Ref: InstanceType
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: root
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - logs:*
            Resource: arn:aws:logs:*:*:*
Outputs:
  AllSecurityGroups:
    Description: Security Groups that are associated with the EC2 instance
    Value:
      Fn::Join:
      - ", "
      - Fn::GetAtt:
        - AllSecurityGroups
        - Value

CloudFormation WAF & Lambda

---
AWSTemplateFormatVersion: '2010-09-09'
Description: 'This template helps setup a WAF ACL to block IPs listed on the SANS
  Bad IP list.  It also sets up a Lambda function to help keep the WAF up-to-date
  with the current SANS list.   After creating a stack with this template you can
  manually initiate an update by performing a test run on the Lambda function.'
Parameters: {}
Resources:
  LambdaRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
          Action: sts:AssumeRole
      Policies:
      - PolicyName: CloudWatchLogs
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action: logs:*
            Resource: "*"
      - PolicyName: WAFGetChangeToken
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action:
            - waf:GetChangeToken
            - waf:GetChangeTokenStatus
            Resource: "*"
      - PolicyName: WAFGetAndUpdateIPSet
        PolicyDocument:
          Statement:
          - Effect: Allow
            Action:
            - waf:GetIPSet
            - waf:UpdateIPSet
            Resource:
            - Fn::Join:
              - ''
              - - 'arn:aws:waf::'
                - Ref: AWS::AccountId
                - ":ipset/"
                - Ref: IPSet
  IPSet:
    Type: AWS::WAF::IPSet
    Properties:
      Name: SANS IPs
  Rule:
    Type: AWS::WAF::Rule
    Properties:
      Name: SANS Rule
      MetricName: sansRule
      Predicates:
      - DataId:
          Ref: IPSet
        Type: IPMatch
        Negated: 'false'
  WebACL:
    Type: AWS::WAF::WebACL
    Properties:
      Name: WebACL
      DefaultAction:
        Type: ALLOW
      MetricName: WebACL
      Rules:
      - Action:
          Type: BLOCK
        Priority: 1
        RuleId:
          Ref: Rule
  LambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.handler
      Role:
        Fn::GetAtt:
        - LambdaRole
        - Arn
      Runtime: python2.7
      MemorySize: '512'
      Timeout: '60'
      Code:
        ZipFile: !Sub |
          #!/usr/bin/python
          import urllib2
          import re
          import boto3
          from StringIO import StringIO
          import gzip

          ip_set_id = "7ca34005-4af0-40df-b378-25dfa8719f63"
          url = "https://isc.sans.edu/block.txt"

          waf_client = boto3.client('waf')

          #
          # Source the bad IPs from the SANS service.  
          #
          def getSansBadIps(url):
          
            # Uncomment for testing...
            #return ['142.77.69.0/24', '182.100.27.0/24', '61.240.144.0/24', '222.174.5.0/24', 
            #'222.186.21.0/24', '117.34.74.0/24', '62.138.6.0/24', '209.126.127.0/24', 
            #'69.64.57.0/24', '62.138.3.0/24', '209.126.111.0/24', '172.93.97.0/24', 
            #'91.213.33.0/24', '119.189.108.0/24', '83.220.172.0/24', '14.32.80.0/24', 
            #'14.43.137.0/24', '118.39.182.0/24', '210.218.188.0/24', '39.67.160.0/24']
          
            ret = []
            headers = {'User-Agent': 'lambda-python-sec-script',
                   'Accept-encoding': 'gzip'}
            
            try:
              request = urllib2.Request(url, headers=headers)
              response = urllib2.urlopen(request)
              
            except urllib2.HTTPError, e:
              print("Failed to get get web resource - {}.".format(e.code))
              return False
              
            else:
            
              if (response.info().get('Content-Encoding') == 'gzip'):
                buf = StringIO( response.read())
                f = gzip.GzipFile(fileobj=buf)
                contents = f.read() 
              else:
                contents = response.read()
              
              #print(contents)
              lines = contents.split("\n")
          
              for line in lines:
                if (len(line) > 0):
                  if (line[0] != "#"):
                    parts = line.split("  ")
                    if re.match("(?:\d{1,3}\.){3}\d{1,3}(?:/\d\d?)?", parts[0]): 
                      ret.append("{}/{}".format(parts[0], parts[2]))
              
              return ret
          #
          # Source the values from the current WAF IPSet  
          #
          def getCurrentIPSet(ip_set_id):
          
            ret = []
            
            try:
              ip_set = waf_client.get_ip_set(
                IPSetId=ip_set_id
              )
              
            except:
              print("Failed to get get IPSet")
              return False
              
            else:
              for item in  ip_set['IPSet']['IPSetDescriptors']:
                ret.append(item['Value'])
            
            return ret
          
          #
          # Format a dict for the update statment  
          #
          def createUpdatesList( cidr_to_remove, cidr_to_add ):
            
            ret = []
            for cidr in cidr_to_remove:
              ret.append( {'Action': 'DELETE','IPSetDescriptor': {'Type': 'IPV4', 'Value': cidr}} )
          
            for cidr in cidr_to_add:
              ret.append( {'Action': 'INSERT','IPSetDescriptor': {'Type': 'IPV4', 'Value': cidr}} )    
            
            return ret
             
          #
          # Send update to AWS WAF  
          #
          def updateIPSet( IPSet, updatesList ):
          
            change_token = waf_client.get_change_token()
          
            print("Change token: {}".format(change_token['ChangeToken']))
            
            response_token = waf_client.update_ip_set(
              IPSetId=IPSet,
              ChangeToken=change_token['ChangeToken'],
              Updates=updatesList
            )
            
            return waf_client.get_change_token_status(ChangeToken=response_token['ChangeToken'])
          
          #
          # Lambda Handler  
          #
          def handler( event, context ):  
          
            print("Starting update...")
            
            sans_bad_ips = getSansBadIps(url)
            print("Found {} CIDRs from SANS".format( len(sans_bad_ips) ))
            print(sans_bad_ips)
          
            current_ip_set = getCurrentIPSet(ip_set_id)
            print("Found {} CIDRs from current IPSet".format( len(current_ip_set) ))
            print(current_ip_set)
          
            cidr_to_remove = [ i for i in current_ip_set if i not in sans_bad_ips]
            print("There are {} CIDRs to REMOVE from current IPSet".format( len(cidr_to_remove) ))
            print(cidr_to_remove)
          
            cidr_to_add = [ i for i in sans_bad_ips if i not in current_ip_set]
            print("There are {} CIDRs to add ADD to current IPSet".format( len(cidr_to_add) ))
            print(cidr_to_add)
          
            updatesList = (createUpdatesList(cidr_to_remove, cidr_to_add))
          
            if len(updatesList):
              print("Sending update to WAF...")
              return(updateIPSet( ip_set_id, updatesList ))
            else:
              return("No changes to be made.")
            print("Done.")
            
          handler(False, False)


  EventsRule:
    Type: AWS::Events::Rule
    Properties:
      Description: WAF Reputation Lists
      ScheduleExpression: rate(1 hour)
      Targets:
      - Arn:
          Fn::GetAtt:
          - LambdaFunction
          - Arn
        Id: LambdaFunction
  LambdaPermission:
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName:
        Ref: LambdaFunction
      Action: lambda:InvokeFunction
      Principal: events.amazonaws.com
      SourceArn:
        Fn::GetAtt:
        - EventsRule
        - Arn

CloudFormation Backend Photo Rekognition

AWSTemplateFormatVersion: '2010-09-09'
Description: Backend for photo sharing reference architecture.
Outputs:
  CognitoIdentityPool:
    Value:
      Ref: TestClientIdentityPool
  DDBAlbumMetadataTable:
    Value:
      Ref: AlbumMetadataDDBTable
  DDBImageMetadataTable:
    Value:
      Ref: ImageMetadataDDBTable
  DescribeExecutionLambda:
    Value:
      Ref: DescribeExecutionFunction
  Region:
    Value:
      Ref: AWS::Region
  S3PhotoRepoBucket:
    Value:
      Ref: PhotoRepoS3Bucket
Resources:
  AlbumMetadataDDBTable:
    Properties:
      AttributeDefinitions:
      - AttributeName: albumID
        AttributeType: S
      - AttributeName: creationTime
        AttributeType: N
      - AttributeName: userID
        AttributeType: S
      GlobalSecondaryIndexes:
      - IndexName: userID-creationTime-index
        KeySchema:
        - AttributeName: userID
          KeyType: HASH
        - AttributeName: creationTime
          KeyType: RANGE
        Projection:
          ProjectionType: ALL
        ProvisionedThroughput:
          ReadCapacityUnits: '2'
          WriteCapacityUnits: '1'
      KeySchema:
      - AttributeName: albumID
        KeyType: HASH
      ProvisionedThroughput:
        ReadCapacityUnits: '2'
        WriteCapacityUnits: '1'
    Type: AWS::DynamoDB::Table
  BackendProcessingLambdaRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
        Version: '2012-10-17'
      Path: /MediaSharingRefarch/
      Policies:
      - PolicyDocument:
          Statement:
          - Action:
            - logs:CreateLogGroup
            - logs:CreateLogStream
            - logs:PutLogEvents
            Effect: Allow
            Resource: '*'
            Sid: AllowLogging
          Version: '2012-10-17'
        PolicyName: LambdaWriteCWLogs
      - PolicyDocument:
          Statement:
          - Action:
            - s3:Get*
            Effect: Allow
            Resource:
              Fn::Sub: arn:aws:s3:::${PhotoRepoS3Bucket}/*
            Sid: ReadFromPhotoRepoS3Bucket
          Version: '2012-10-17'
        PolicyName: ReadFromPhotoRepoS3Bucket
      - PolicyDocument:
          Statement:
          - Action:
            - s3:PutObject
            Effect: Allow
            Resource:
              Fn::Sub: arn:aws:s3:::${PhotoRepoS3Bucket}/*
            Sid: WriteToPhotoRepoS3Bucket
          Version: '2012-10-17'
        PolicyName: WriteToPhotoRepoS3Bucket
      - PolicyDocument:
          Statement:
          - Action:
            - dynamodb:UpdateItem
            - dynamodb:PutItem
            Effect: Allow
            Resource:
              Fn::Sub: arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${ImageMetadataDDBTable}
            Sid: WriteToImageMetadataDDBTable
          Version: '2012-10-17'
        PolicyName: WriteToImageMetadataDDBTable
      - PolicyDocument:
          Statement:
          - Action:
            - rekognition:DetectLabels
            Effect: Allow
            Resource: '*'
            Sid: RekognitionDetectLabels
          Version: '2012-10-17'
        PolicyName: RekognitionDetectLabels
      - PolicyDocument:
          Statement:
          - Action:
            - states:StartExecution
            Effect: Allow
            Resource: '*'
            Sid: StepFunctionStartExecution
          Version: '2012-10-17'
        PolicyName: StepFunctionStartExecution
    Type: AWS::IAM::Role
  CreateS3EventTriggerFunction:
    Properties:
      CodeUri: s3://media-sharing-refarch/8c7e2179b5bc3480407509396c78b95e
      Description: Used with CloudFormation as a custom resource helper to enable
        S3 event trigger to invoke the start step function Lambda function.
      Handler: index.handler
      MemorySize: 1024
      Role:
        Fn::GetAtt:
        - CustomResourceHelperRole
        - Arn
      Runtime: nodejs4.3
      Timeout: 200
    Type: AWS::Serverless::Function
  CustomResourceHelperRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
        Version: '2012-10-17'
      Path: /MediaSharingRefarch/
      Policies:
      - PolicyDocument:
          Statement:
          - Action:
            - logs:CreateLogGroup
            - logs:CreateLogStream
            - logs:PutLogEvents
            Effect: Allow
            Resource: '*'
            Sid: AllowLogging
          Version: '2012-10-17'
        PolicyName: LambdaWriteCWLogs
      - PolicyDocument:
          Statement:
          - Action:
            - s3:PutBucketNotification
            Effect: Allow
            Resource:
              Fn::Sub: arn:aws:s3:::${PhotoRepoS3Bucket}
            Sid: PutS3EventNofication
          - Action:
            - lambda:AddPermission
            Effect: Allow
            Resource: '*'
            Sid: AddPermissionToLambda
          Version: '2012-10-17'
        PolicyName: AddS3EventTrigger
    Type: AWS::IAM::Role
  DescribeExecutionFunction:
    Properties:
      CodeUri: s3://media-sharing-refarch/1f00cdd048caae4e89d5ce2a890ebe76
      Description: Calls DescribeExecution on a state machine execution.
      Handler: index.handler
      MemorySize: 1024
      Role:
        Fn::GetAtt:
        - DescribeExecutionFunctionRole
        - Arn
      Runtime: nodejs4.3
      Timeout: 200
    Type: AWS::Serverless::Function
  DescribeExecutionFunctionRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
        Version: '2012-10-17'
      Path: /MediaSharingRefarch/
      Policies:
      - PolicyDocument:
          Statement:
          - Action:
            - logs:CreateLogGroup
            - logs:CreateLogStream
            - logs:PutLogEvents
            Effect: Allow
            Resource: '*'
            Sid: AllowLogging
          Version: '2012-10-17'
        PolicyName: LambdaWriteCWLogs
      - PolicyDocument:
          Statement:
          - Action:
            - states:DescribeExecution
            Effect: Allow
            Resource: '*'
            Sid: DescribeStepFunction
          Version: '2012-10-17'
        PolicyName: DescribeStepFunction
    Type: AWS::IAM::Role
  ExtractImageMetadataFunction:
    Properties:
      CodeUri: s3://media-sharing-refarch/85cfd440512536a13ea58fe42d838984
      Description: Extract image metadata such as format, size, geolocation, etc.
      Handler: index.handler
      MemorySize: 1024
      Role:
        Fn::GetAtt:
        - BackendProcessingLambdaRole
        - Arn
      Runtime: nodejs4.3
      Timeout: 200
    Type: AWS::Serverless::Function
  GenerateThumbnailFunction:
    Properties:
      CodeUri: s3://media-sharing-refarch/bf7eda74763ba43004262c3a9867bbc9
      Description: Generate thumbnails for images
      Handler: index.handler
      MemorySize: 1536
      Role:
        Fn::GetAtt:
        - BackendProcessingLambdaRole
        - Arn
      Runtime: nodejs4.3
      Timeout: 300
    Type: AWS::Serverless::Function
  ImageMetadataDDBTable:
    Properties:
      AttributeDefinitions:
      - AttributeName: albumID
        AttributeType: S
      - AttributeName: imageID
        AttributeType: S
      - AttributeName: uploadTime
        AttributeType: N
      GlobalSecondaryIndexes:
      - IndexName: albumID-uploadTime-index
        KeySchema:
        - AttributeName: albumID
          KeyType: HASH
        - AttributeName: uploadTime
          KeyType: RANGE
        Projection:
          ProjectionType: ALL
        ProvisionedThroughput:
          ReadCapacityUnits: '3'
          WriteCapacityUnits: '3'
      KeySchema:
      - AttributeName: imageID
        KeyType: HASH
      ProvisionedThroughput:
        ReadCapacityUnits: '3'
        WriteCapacityUnits: '3'
    Type: AWS::DynamoDB::Table
  ImageProcStartExecutionFunction:
    DependsOn: PhotoRepoS3Bucket
    Properties:
      CodeUri: s3://media-sharing-refarch/2db09bf6931a36fae77883ec8c32f740
      Description: Triggered by S3 image upload to the repo bucket and start the image
        processing step function workflow
      Environment:
        Variables:
          IMAGE_METADATA_DDB_TABLE:
            Ref: ImageMetadataDDBTable
          STATE_MACHINE_ARN:
            Ref: ImageProcStateMachine
      Handler: index.handler
      MemorySize: 256
      Role:
        Fn::GetAtt:
        - BackendProcessingLambdaRole
        - Arn
      Runtime: nodejs4.3
      Timeout: 60
    Type: AWS::Serverless::Function
  ImageProcStateMachine:
    Properties:
      DefinitionString:
        Fn::Sub:
        - "{\n  \"Comment\": \"Image Processing workflow\",\n  \"StartAt\": \"ExtractImageMetadata\"\
          ,\n  \"States\": {\n    \"ExtractImageMetadata\": {\n      \"Type\": \"\
          Task\",\n      \"Resource\": \"${ExtractImageMetadataLambdaArn}\",\n   \
          \   \"InputPath\": \"$\",\n      \"ResultPath\": \"$.extractedMetadata\"\
          ,\n      \"Next\": \"ImageTypeCheck\",\n      \"Catch\": [\n        {\n\
          \          \"ErrorEquals\": [\n            \"ImageIdentifyError\"\n    \
          \      ],\n          \"Next\": \"NotSupportedImageType\"\n        }\n  \
          \    ],\n      \"Retry\": [\n        {\n          \"ErrorEquals\": [\n \
          \           \"ImageIdentifyError\"\n          ],\n          \"MaxAttempts\"\
          : 0\n        },\n        {\n          \"ErrorEquals\": [\n            \"\
          States.ALL\"\n          ],\n          \"IntervalSeconds\": 1,\n        \
          \  \"MaxAttempts\": 2,\n          \"BackoffRate\": 1.5\n        }\n    \
          \  ]\n    },\n    \"ImageTypeCheck\": {\n      \"Type\": \"Choice\",\n \
          \     \"Choices\": [\n        {\n          \"Or\": [\n            {\n  \
          \            \"Variable\": \"$.extractedMetadata.format\",\n           \
          \   \"StringEquals\": \"JPEG\"\n            },\n            {\n        \
          \      \"Variable\": \"$.extractedMetadata.format\",\n              \"StringEquals\"\
          : \"PNG\"\n            }\n          ],\n          \"Next\": \"TransformMetadata\"\
          \n        }\n      ],\n      \"Default\": \"NotSupportedImageType\"\n  \
          \  },\n    \"TransformMetadata\": {\n      \"Type\": \"Task\",\n      \"\
          Resource\": \"${TransformMetadataLambdaArn}\",\n      \"InputPath\": \"\
          $.extractedMetadata\",\n      \"ResultPath\": \"$.extractedMetadata\",\n\
          \      \"Retry\": [\n        {\n          \"ErrorEquals\": [\n         \
          \   \"States.ALL\"\n          ],\n          \"IntervalSeconds\": 1,\n  \
          \        \"MaxAttempts\": 2,\n          \"BackoffRate\": 1.5\n        }\n\
          \      ],\n      \"Next\": \"ParallelProcessing\"\n    },\n    \"NotSupportedImageType\"\
          : {\n      \"Type\": \"Fail\",\n      \"Cause\": \"Image type not supported!\"\
          ,\n      \"Error\": \"FileTypeNotSupported\"\n    },\n    \"ParallelProcessing\"\
          : {\n      \"Type\": \"Parallel\",\n      \"Branches\": [\n        {\n \
          \         \"StartAt\": \"Rekognition\",\n          \"States\": {\n     \
          \       \"Rekognition\": {\n              \"Type\": \"Task\",\n        \
          \      \"Resource\": \"${RekognitionLambdaArn}\",\n              \"Retry\"\
          : [\n                {\n                  \"ErrorEquals\": [\n         \
          \           \"States.ALL\"\n                  ],\n                  \"IntervalSeconds\"\
          : 1,\n                  \"MaxAttempts\": 2,\n                  \"BackoffRate\"\
          : 1.5\n                }\n              ],\n              \"End\": true\n\
          \            }\n          }\n        },\n        {\n          \"StartAt\"\
          : \"Thumbnail\",\n          \"States\": {\n            \"Thumbnail\": {\n\
          \              \"Type\": \"Task\",\n              \"Resource\": \"${GenerateThumbnailLambdaArn}\"\
          ,\n              \"Retry\": [\n                {\n                  \"ErrorEquals\"\
          : [\n                    \"States.ALL\"\n                  ],\n        \
          \          \"IntervalSeconds\": 1,\n                  \"MaxAttempts\": 2,\n\
          \                  \"BackoffRate\": 1.5\n                }\n           \
          \   ],\n              \"End\": true\n            }\n          }\n      \
          \  }\n      ],\n      \"ResultPath\": \"$.parallelResults\",\n      \"Next\"\
          : \"StoreImageMetadata\"\n    },\n    \"StoreImageMetadata\": {\n      \"\
          Type\": \"Task\",\n      \"Resource\": \"${StoreImageMetadataLambdaArn}\"\
          ,\n      \"InputPath\": \"$\",\n      \"ResultPath\": \"$.storeResult\"\
          ,\n      \"Retry\": [\n        {\n          \"ErrorEquals\": [\n       \
          \     \"States.ALL\"\n          ],\n          \"IntervalSeconds\": 1,\n\
          \          \"MaxAttempts\": 2,\n          \"BackoffRate\": 1.5\n       \
          \ }\n      ],\n      \"End\": true\n    }\n  }\n}"
        - ExtractImageMetadataLambdaArn:
            Fn::GetAtt:
            - ExtractImageMetadataFunction
            - Arn
          GenerateThumbnailLambdaArn:
            Fn::GetAtt:
            - GenerateThumbnailFunction
            - Arn
          RekognitionLambdaArn:
            Fn::GetAtt:
            - RekognitionFunction
            - Arn
          StoreImageMetadataLambdaArn:
            Fn::GetAtt:
            - StoreImageMetadataFunction
            - Arn
          TransformMetadataLambdaArn:
            Fn::GetAtt:
            - TransformMetadataFunction
            - Arn
      RoleArn:
        Fn::GetAtt:
        - StateMachineRole
        - Arn
    Type: AWS::StepFunctions::StateMachine
  PhotoRepoS3Bucket:
    Properties:
      CorsConfiguration:
        CorsRules:
        - AllowedHeaders:
          - '*'
          AllowedMethods:
          - PUT
          - GET
          - POST
          - HEAD
          AllowedOrigins:
          - '*'
          ExposedHeaders:
          - ETag
    Type: AWS::S3::Bucket
  RekognitionFunction:
    Properties:
      CodeUri: s3://media-sharing-refarch/31d08688ac4c72609539eba52cd520f8
      Description: Use Amazon Rekognition to detect labels from image
      Handler: index.handler
      MemorySize: 256
      Role:
        Fn::GetAtt:
        - BackendProcessingLambdaRole
        - Arn
      Runtime: nodejs4.3
      Timeout: 60
    Type: AWS::Serverless::Function
  S3EventTrigger:
    DependsOn:
    - PhotoRepoS3Bucket
    - ImageProcStartExecutionFunction
    Properties:
      PhotoRepoS3Bucket:
        Ref: PhotoRepoS3Bucket
      ServiceToken:
        Fn::GetAtt:
        - CreateS3EventTriggerFunction
        - Arn
      StartExecutionFunction:
        Ref: ImageProcStartExecutionFunction
      StartExecutionFunctionArn:
        Fn::GetAtt:
        - ImageProcStartExecutionFunction
        - Arn
      accountId:
        Ref: AWS::AccountId
    Type: Custom::S3EventTrigger
    Version: '1.0'
  StateMachineRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
              Fn::Sub: states.${AWS::Region}.amazonaws.com
        Version: '2012-10-17'
      Path: /MediaSharingRefarch/
      Policies:
      - PolicyDocument:
          Statement:
          - Action:
            - lambda:InvokeFunction
            Effect: Allow
            Resource: '*'
            Sid: InvokeLambda
          Version: '2012-10-17'
        PolicyName: InvokeLambda
    Type: AWS::IAM::Role
  StoreImageMetadataFunction:
    Properties:
      CodeUri: s3://media-sharing-refarch/1a8335158b144e084df0e1aa485faeb5
      Description: Store image metadata into database
      Environment:
        Variables:
          IMAGE_METADATA_DDB_TABLE:
            Ref: ImageMetadataDDBTable
      Handler: index.handler
      MemorySize: 256
      Role:
        Fn::GetAtt:
        - BackendProcessingLambdaRole
        - Arn
      Runtime: nodejs4.3
      Timeout: 60
    Type: AWS::Serverless::Function
  TestClientIAMRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          - sts:AssumeRoleWithWebIdentity
          Effect: Allow
          Principal:
            Federated:
            - cognito-identity.amazonaws.com
        Version: '2012-10-17'
      Policies:
      - PolicyDocument:
          Statement:
          - Action:
            - s3:*
            Effect: Allow
            Resource:
              Fn::Sub: arn:aws:s3:::${PhotoRepoS3Bucket}/*
            Sid: S3ReadWrite
          Version: '2012-10-17'
        PolicyName: S3PhotoRepoBucketAccess
      - PolicyDocument:
          Statement:
          - Action:
            - dynamodb:*
            Effect: Allow
            Resource:
            - Fn::Sub: arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${AlbumMetadataDDBTable}
            - Fn::Sub: arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${AlbumMetadataDDBTable}/*
            - Fn::Sub: arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${ImageMetadataDDBTable}
            - Fn::Sub: arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${ImageMetadataDDBTable}/*
            Sid: DynamoTableAccess
          Version: '2012-10-17'
        PolicyName: DynamoTableAccess
      - PolicyDocument:
          Statement:
          - Action:
            - lambda:InvokeFunction
            Effect: Allow
            Resource:
            - Fn::Sub: arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:${DescribeExecutionFunction}
            Sid: InvokeDescribeExecutionLambda
          Version: '2012-10-17'
        PolicyName: InvokeDescribeExecutionLambda
    Type: AWS::IAM::Role
  TestClientIdentityPool:
    Properties:
      AllowUnauthenticatedIdentities: true
      IdentityPoolName: TestWebApp
    Type: AWS::Cognito::IdentityPool
  TestClientIdentityPoolRoles:
    Properties:
      IdentityPoolId:
        Ref: TestClientIdentityPool
      Roles:
        authenticated:
          Fn::GetAtt:
          - TestClientIAMRole
          - Arn
        unauthenticated:
          Fn::GetAtt:
          - TestClientIAMRole
          - Arn
    Type: AWS::Cognito::IdentityPoolRoleAttachment
  TransformMetadataFunction:
    Properties:
      CodeUri: s3://media-sharing-refarch/7761cb906913bffe76391119cff090f0
      Description: massages JSON of extracted image metadata
      Handler: index.handler
      MemorySize: 256
      Role:
        Fn::GetAtt:
        - BackendProcessingLambdaRole
        - Arn
      Runtime: nodejs4.3
      Timeout: 60
    Type: AWS::Serverless::Function
Transform: AWS::Serverless-2016-10-31