Aws - Click Event Rest API (Capture user behavior)

Card Puncher Data Processing


Adapted from the Aws tutorial - Building a Modern app (module 5).


Click Event records will be processed in real-time by a serverless code function, aggregated, and stored for any future analysis that you may want to perform.

Aws User Click Event Processing Architecture


Create the code and save it to a repository

aws codecommit create-repository --repository-name MythicalMysfitsStreamingService-Repository
    "repositoryMetadata": {
        "accountId": "REPLACE_ME_ACCOUNT_ID",
        "repositoryId": "32941d10-66e1-4230-b3d6-eeb55e64a60b",
        "repositoryName": "MythicalMysfitsStreamingService-Repository",
        "lastModifiedDate": 1554965738.429,
        "creationDate": 1554965738.429,
        "cloneUrlHttp": "",
        "cloneUrlSsh": "ssh://",
        "Arn": "arn:aws:codecommit:eu-central-1:REPLACE_ME_ACCOUNT_ID:MythicalMysfitsStreamingService-Repository"

git clone
Cloning into 'MythicalMysfitsStreamingService-Repository'...
warning: You appear to have cloned an empty repository.

  • Copy the source
cd MythicalMysfitsStreamingService-Repository
cp ..\aws-modern-application-workshop\module-5\cfn .
cp ..\aws-modern-application-workshop\module-5\app\streaming .
:: A CFN template for creating the full stack.
        1 file(s) copied.
:: A Python script that contains the code for our Lambda function:
        1 file(s) copied.

  • package all of library dependencies together with the Lambda code function
:: Install the request package into the local directory
pip install requests -t .
Collecting requests
  Using cached
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Using cached
Collecting certifi>=2017.4.17 (from requests)
  Downloading (158kB)
    100% |                                        | 163kB 163kB/s
Collecting urllib3<1.25,>=1.21.1 (from requests)
  Using cached
Collecting idna<2.9,>=2.5 (from requests)
  Using cached
Installing collected packages: chardet, certifi, urllib3, idna, requests
Successfully installed certifi-2019.3.9 chardet-3.0.4 idna-2.8 requests-2.21.0 urllib3-1.24.1

  • List
ls -A1

  • Replace REPLACE_ME_API_ENDPOINT in the Lambda Function Code by the ApiEndpoint of the Mysfits service API (this is the same service ApiEndpoint that you created for the website frontend).
def retrieveMysfit(mysfitId):
    apiEndpoint = 'REPLACE_ME_API_ENDPOINT' + '/mysfits/' + str(
        mysfitId)  # eg: ''
    mysfit = requests.get(apiEndpoint).json()
    return mysfit
  • Package
git add .
git commit -m "New stream processing service."
git push

Create the streaming service stack with SAM

The creation of the streaming stack is done using the AWS SAM CLI to:

  • package all of our function code,
  • upload it to S3,
  • and create the deployable CloudFormation template

Create a S3 bucket

Create a bucket named real-time-streaming

aws s3 mb s3://real-time-streaming

Package the code with SAM

With the Sam cli

cd aws-modern-application-workshop\module-5
sam package ^
  --template-file .\cfn\real-time-streaming.yml ^
  --output-template-file .\cfn\transformed-streaming.yml ^
  --s3-bucket real-time-streaming
Uploading to 31ae6f83bb35657cc2aad3d168246ada  2629 / 2629.0  (100.00%)
Successfully packaged artifacts and wrote output template to file .\cfn\transformed-streaming.yml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file d:\code-external\aws-modern-application-workshop\module-5\cfn\transformed-streaming.yml --stack-name <YOUR STACK NAME>

  • The CodeUri parameter of the serverless Lambda function has been updated in the transformed-streaming.yml file with the object location where the SAM CLI has uploaded your packaged code
    Type: AWS::Serverless::Function
      Handler: streamProcessor.processRecord
      Runtime: python3.6
      CodeUri: s3://real-time-streaming/31ae6f83bb35657cc2aad3d168246ada

S3 Lambda Sam Uploaded Code

Deploy The Stack Using AWS CloudFormation

Returned by the SAM CLI command is the CloudFormation command needed to be executed to create our new full stack. But because our stack creates IAM resources, you'll need to add one additional parameter to the command.

  • deploy the streaming stack (deploy the function)
aws cloudformation deploy ^
  --template-file .\cfn\transformed-streaming.yml ^
  --stack-name MythicalMysfitsStreamingStack ^
  --capabilities CAPABILITY_IAM

Once this stack creation is complete, the full real-time processing microservice will be created.

Documentation / Reference


AccessDenied when calling the CreateChangeSet operation

An error occurred (AccessDenied) when calling the CreateChangeSet operation: User: arn:aws:iam::09000999845148:user/cli is not authorized to perform: cloudformation:CreateChangeSet on resource: arn:aws:cloudformation:eu-central-1:094473333148:stack/MythicalMysfitsStreamingStack/*

Discover More
Aws Moder App Mysfits Home Page
Aws - Modern Web App Workshop

Note the Aws project Build a Modern Web Application Code: aws-samples/aws-modern-application-workshop/tree/python...

Share this page:
Follow us:
Task Runner