Day 28 was about migrating HTML tooltips to Svelte for easier maintenance and fun. On Day 29, we continue that journey and connect tooltips with API Gateway using AWS SigV4 and temporary credentials from the Cognito Identity Pool.

Loading Data with the Cognito Identity Pool credentials and AWS SigV4

Registered users generate tooltips, so we always know who created which tooltip, how many tooltips each user generated, etc. However, tooltips are used by their end users, which are anonymous for Knowlo. We do not care about end users’ identity. We do not want to know anything about them because of their privacy. However, we need to show some basic analytics, such as the number of users interacting with tooltips, and we need to be able to limit each user to a certain number of follow-up questions.

To be able to do that, we need to tag each anonymous user with some anonymous user ID. This is a standard procedure for analytics apps and many other types of applications, so we do not want to develop this system from scratch.

What are our options? We can use some system that we can plug into our AWS resources to check anonymous user IDs when they send a request, or we can use AWS built-in functionality.

The AWS built-in functionality sounds better because of the direct integration with AWS resources and the less code we’ll need to write.

An AWS service with a built-in solution to track anonymous users is part of Amazon Cognito service, and it’s called Amazon Cognito Identity Pool.

Amazon Cognito Identity Pool is an important component of Amazon’s Cognito service that manages user identities and authorizes access to AWS resources. The Identity Pool is flexible, supporting both authenticated (logged in) and unauthenticated (guest) users, with adjustable access controls for each identity.

An Identity Pool is especially useful when your application requires direct access to AWS resources or when multiple authentication providers are necessary. It integrates with external identity providers and Cognito User Pools, allowing you to manage various levels of user interaction and permissions.

A unique feature of Cognito Identity Pool is its ability to authorize unauthenticated users, or guests, to perform certain actions, such as making requests to API Gateway. By granting appropriate IAM roles to these unauthenticated identities, you can, for instance, provide read-only access to certain APIs for users who haven’t logged in yet. While powerful, this feature should be used judiciously to prevent unintended access to your AWS resources.

To allow anonymous users to make API Gateway requests from the front end application while still identifying and authorizing them based on their anonymous user identities, we need to follow these steps:

1. Create a Cognito Identity Pool (we already have it)
2. Give an appropriate IAM role to the unauthenticated users so that they can send an API Gateway request
3. Get temporary credentials in the front end application
4. Send a signed API request with temporary credentials

To make API Gateway requests with temporary Cognito Identity Pool credentials, we need to sign these requests using Signature Version 4, or SigV4, a protocol used by AWS for securing HTTP requests. SigV4 signs our API requests, which provides a way to verify the request sender’s identity and protects the integrity of the request data.

In our React application, AWS Amplify uses SigV4 under the hood, so we do not need to think about it. However, we do not want to import the entire AWS Amplify library to render tooltips, so we’ll need to find a different way to sign our requests.

Backend: API Gateway, IAM, and Lambda functions

We already have a Cognito Identity Pool, but we need to connect it to API Gateway. But before we do that, let’s move the auth part of our CDK stack to a separate file. I created the “stack” folder in the “lib” folder of our CDK project and a file named “auth-service.ts” inside it.

Then I pasted the following code to the new “auth-service.ts” file:

import { Construct } from 'constructs'
import { Duration, RemovalPolicy } from 'aws-cdk-lib'
import * as cognito from 'aws-cdk-lib/aws-cognito'
import { IdentityPool, UserPoolAuthenticationProvider } from '@aws-cdk/aws-cognito-identitypool-alpha'
import { Effect, Policy, PolicyStatement, Role } from 'aws-cdk-lib/aws-iam'
import { IAuthServiceProps } from './types'

export class AuthService extends Construct {
  readonly knowloUserPool: cognito.IUserPool
  readonly knowloUserPoolClient: cognito.IUserPoolClient
  readonly knowloIdentityPool: IdentityPool
  readonly unAuthRole: Role

  constructor(scope: Construct, id: string, { environment }: IAuthServiceProps) {
    super(scope, id)

    const knowloUserPool = new cognito.UserPool(this, 'KnowloUserPool', {
      selfSignUpEnabled: true,
      passwordPolicy: {
        minLength: 8,
        requireLowercase: true,
        requireDigits: true,
        requireSymbols: true,
        requireUppercase: false,
      },
      mfa: cognito.Mfa.OPTIONAL,
      mfaSecondFactor: {
        sms: false,
        otp: true,
      },
      signInAliases: {
        username: true,
        email: true,
      },
      signInCaseSensitive: false,
      accountRecovery: cognito.AccountRecovery.EMAIL_ONLY,
      removalPolicy: environment === 'production' ? RemovalPolicy.RETAIN : RemovalPolicy.DESTROY,
      deletionProtection: environment === 'production' ? true : false,
      email: environment === 'production' ? cognito.UserPoolEmail.withSES({
        fromEmail: '', // TODO: pass email via parameter later
      }) : cognito.UserPoolEmail.withCognito(),
    })

    const knowloUserPoolClient = knowloUserPool.addClient('KnowloUserPoolClient', {
      accessTokenValidity: Duration.minutes(15),
      idTokenValidity: Duration.minutes(15),
      refreshTokenValidity: Duration.days(365),
      supportedIdentityProviders: [cognito.UserPoolClientIdentityProvider.COGNITO],
      preventUserExistenceErrors: true,
      oAuth: {
        flows: {
          authorizationCodeGrant: true,
          implicitCodeGrant: true,
        },
        scopes: [cognito.OAuthScope.EMAIL, cognito.OAuthScope.OPENID, cognito.OAuthScope.PROFILE],
      },
      authFlows: {
        userPassword: true,
        userSrp: true,
      },
      enableTokenRevocation: true,
    })

    const knowloIdentityPool = new IdentityPool(this, 'KnowloIdentityPool', {
      allowClassicFlow: false,
      allowUnauthenticatedIdentities: true,
      authenticationProviders: {
        userPools: [new UserPoolAuthenticationProvider({
          userPool: knowloUserPool,
        })],
      },
    })

    this.knowloUserPool = knowloUserPool
    this.knowloUserPoolClient = knowloUserPoolClient
    this.knowloIdentityPool = knowloIdentityPool
  }
}

This code was part of our main CDK stack, and we used it to create Cognito User Pool, User Pool Client, and Identity Pool.

The main CDK stack now needs to do the following to get references to our auth resources:

const { knowloUserPool, knowloIdentityPool, knowloUserPoolClient } = new AuthService(this, 'AuthService', {
  environment: environmentParameter.valueAsString,
  apiGatewayArn: api.arnForExecuteApi(), // We'll pass this because we need to allow anonymous users to send API Gateway requests
})

As you can see in the snippet above, we pass the API Gateway ARN. We need it to allow anonymous users to send API Gateway requests.

To add required permissions to anonymous users, we add the following to the bottom of the “auth-service.ts” file:

knowloIdentityPool.unauthenticatedRole.attachInlinePolicy(new Policy(this, 'AllowApiGatewayInvoke', {
  policyName: 'AllowApiGatewayInvoke',
  statements: [
    new PolicyStatement({
      effect: Effect.ALLOW,
      actions: ['execute-api:Invoke'],
      resources: [`${apiGatewayArn}/*`],
    })
  ],
}))

This code adds an IAM policy to the unauthenticated users’ IAM role that allows these users to invoke API Gateway API endpoints for the specific API Gateway in our app. We can replace the asterisk (*) with the exact routes we want to allow, but I’ll leave that as part of the security-related improvements after we onboard our first beta customers.

Now that the permission part is done, we need to create a new API Gateway endpoint and a Lambda function that handles it.

If you ask, “Why don’t you use GraphQL?” that’s a valid question. The only reason for using an HTTP request instead of GraphQL is that we want to keep the tooltip component as simple as possible and with the least amount of dependencies possible.

We add the following code to create a new API Gateway endpoint and a Lambda function in our main CDK stack:

const getTooltipFunction = new lambda.NodejsFunction(this, 'GetTooltipFunction', {
  entry: './lib/functions/get-tooltip/lambda.ts',
  handler: 'handler',
  runtime: Runtime.NODEJS_18_X,
  timeout: Duration.seconds(30),
  memorySize: 256,
  environment: {
    NODE_OPTIONS: '--enable-source-maps',
    TABLE_NAME: coreDbTable.tableName,
    POWERTOOLS_LOGGER_LOG_EVENT: 'true',
  },
  logRetention: environmentParameter.valueAsString === 'production' ? RetentionDays.INFINITE : RetentionDays.ONE_WEEK,
  bundling: {
    sourceMap: true,
  }
})

coreDbTable.grantReadData(getTooltipFunction)

const getTooltipLambdaIntegration = new apiGateway.LambdaIntegration(getTooltipFunction, {
  proxy: true,
})

const getTooltipApiResource = api.root
  .addResource('project')
  .addResource('{projectId}')
  .addResource('tooltip')
  .addResource('{tooltipId}')

getTooltipApiResource.addCorsPreflight({
  allowOrigins: ['*'],
  allowMethods: ['OPTIONS', 'GET', 'POST'],
  allowHeaders: ['x-apigateway-header', 'x-amz-content-sha256', 'Authorization', 'Content-Type', 'x-amz-date', 'x-amz-security-token'],
  maxAge: Duration.seconds(600),
})

getTooltipApiResource.addMethod('GET', getTooltipLambdaIntegration, {
  authorizationType: apiGateway.AuthorizationType.IAM, // <-- Adds a new authorization type
})

In the code snippet above:

• An AWS Lambda function, named GetTooltipFunction, is created using the AWS CDK lambda.NodejsFunction construct. This function is configured with certain parameters, such as runtime, timeout, memory size, and environment variables.
• coreDbTable.grantReadData(getTooltipFunction) allows the Lambda function to read data from a DynamoDB table represented by coreDbTable.
• The Lambda function is then integrated with AWS API Gateway using apiGateway.LambdaIntegration.
• A nested API resource path /project/{projectId}/tooltip/{tooltipId} is created on API Gateway.
• CORS preflight settings are configured on this API resource, specifying allowed origins, methods, headers, and the max age for a preflight request to be cached.
• A ‘GET’ method is added to the API resource, using the Lambda function as the backend. This API endpoint uses IAM for authorization (we need this because to be able to send a request using Cognito Identity Pool credentials).

The initial version of the Lambda function business logic looks similar to the following code snippet:

import { cosineSimilarity } from './cosine-similarity'
import { IGetTooltipSuggestionParams, IGetTooltipSuggestionResponse, IGenerateTooltipResponse } from '../types'

export async function getTooltipSuggestion<T>({
  event, parser, repositories, logger
}: IGetTooltipSuggestionParams<T>): Promise<IGetTooltipSuggestionResponse> {
  try {
    const { projectId, description, tooltipStyle, previousTooltip, comment } = parser(event)

    logger.debug('projectId', projectId)
    logger.debug('description', description)

    const { knowledgeBaseS3Path, knowledgeBaseUrl } = await repositories.dbRepository.getProject(projectId)

    logger.debug('knowledgeBaseS3Path', knowledgeBaseS3Path)
    logger.debug('knowledgeBaseUrl', knowledgeBaseUrl)

    const helpdeskEmbeddingsRaw = await repositories.fileRepository.getFile(knowledgeBaseS3Path)

    logger.debug('helpdeskEmbeddingsRaw', `${helpdeskEmbeddingsRaw.length}`)

    const helpdeskEmbeddings = JSON.parse(helpdeskEmbeddingsRaw)

    const questionEmbeddings = await repositories.aiRepository.createEmbeddings(description)

    logger.debug('questionEmbeddings', questionEmbeddings)

    const similarities = helpdeskEmbeddings.map((item: any, idx: number) => ({
      idx: idx,
      similarity: cosineSimilarity(questionEmbeddings.data[0].embedding, item.embedding.data[0].embedding),
    }))

    logger.debug('similarities', similarities)

    const mostSimilarArticleIndex = similarities.reduce((maxIdx: number, curr: any, idx: number) => {
      return curr.similarity > similarities[maxIdx].similarity ? idx : maxIdx;
    }, 0)

    const selectedArticle = helpdeskEmbeddings[mostSimilarArticleIndex]

    logger.debug('selectedArticle', selectedArticle)

    let generatedTooltip: IGenerateTooltipResponse

    if (previousTooltip && comment) {
      generatedTooltip = await repositories.aiRepository.regenerateTooltip({
        id: selectedArticle.id,
        title: selectedArticle.title,
        slug: selectedArticle.slug,
        content: selectedArticle.content,
        style: tooltipStyle,
        previousTooltip,
        comment,
      }, description)
    } else {
      generatedTooltip = await repositories.aiRepository.generateTooltip({
        id: selectedArticle.id,
        title: selectedArticle.title,
        slug: selectedArticle.slug,
        content: selectedArticle.content,
        style: tooltipStyle,
      }, description)
    }

    logger.debug('generatedTooltip', JSON.stringify(generatedTooltip, null, 2))

    return {
      answer: generatedTooltip.answer,
      articleUrl: `${knowledgeBaseUrl}/${selectedArticle.slug}`,
      articleTitle: selectedArticle.title,
    }
  } catch(err) {
    logger.error(err)
    throw err
  }
}

The code above defines an async function named getTooltipSuggestion, which carries out several steps:

• It first deconstructs and logs various parameters from the event input, including projectId and description.
• Next, it retrieves the knowledgeBaseS3Path and knowledgeBaseUrl from a database repository using the provided projectId.
• It fetches raw helpdesk embeddings from an S3 file (located at knowledgeBaseS3Path) and parses it into JSON format.
• The function then creates embeddings for the description using an AI repository.
• It calculates the cosine similarity between the description embeddings and each of the helpdesk embeddings. The similarity scores and their corresponding indices are stored in an array.
• The function identifies the index of the most similar helpdesk article to the description.
• Depending on the presence of previousTooltip and comment, it either regenerates or generates a tooltip using the most similar helpdesk article and the provided description. This is used to generate a new tooltip or refine the existing one.
• Finally, the function logs the generatedTooltip and returns an object containing the answer from the generated tooltip, a URL to the most similar helpdesk article, and the article’s title.

If any error occurs during the process, the function logs the error and throws it.

We’ll need to add a few things to this function later, but this is enough for the initial integration. Once we saved everything, I run the npm run cdk deploy command to redeploy the project.

Loading the Tooltip Data in a Web Component

There are many ways to sign the request using SigV4. But whatever you do, you often end up in a rabbit hole trying to debug why the request doesn’t work. And trust me, that part is not fun.

By working with serverless for many years now, I learned that if you can use a tool or library that Michael Hart created, you should do it because it’ll save you from many painful moments. Luckily, Michael made an AWS SigV4 npm package called aws4. So let’s use it!

After installing the aws4 npm package, we added the following function to our Svelte Web Component:

const loadTooltipData = async ({projectId, tooltipId, credentials, apiUrl}: ILoadTooltipParams) => {
  const { hostname, path } = url.parse(apiUrl)
  let signedUrl = `https://${hostname}${path}project/${projectId}/tooltip/${tooltipId}`
  const opts: any = {
    method: 'GET',
    host: hostname,
    url: signedUrl,
    path: `${path}project/${projectId}/tooltip/${tooltipId}`,
  }

  const signedRequest = aws4.sign(opts, {
    secretAccessKey: credentials.secretAccessKey,
    accessKeyId: credentials.accessKeyId,
    sessionToken: credentials.sessionToken,
  })

  delete signedRequest.headers['Host']
  delete signedRequest.headers['Content-Length']

  if (signedRequest.query) {
    signedUrl += `?${signedRequest.query}`
  }

  const result = await axios(signedRequest)

  return result.data.tooltip
}

This function does the following:

• Given parameters like projectId, tooltipId, credentials, and apiUrl, it initially constructs a URL to access a specific tooltip belonging to a project.
• It then sets up an AWS signed request using these credentials, which include secretAccessKey, accessKeyId, and sessionToken. This is important as it authenticates the request made to the AWS service.
• The ‘Host’ and ‘Content-Length’ headers are removed from the signed request as they are not needed.
• If the signedRequest contains a query, it’s appended to the signedUrl.
• Finally, the function sends a GET request using axios with the signedRequest. The response from this call, which is expected to be the desired tooltip data, is then returned.

I decided to use axios because there was a working example somewhere in the Github issues of the aws4 library. Using fetch is probably better because we remove one dependency, but we’ll add that as an improvement later because the initial setup didn’t work.

In addition to that, we also installed the svelte-markdown npm module to be able to render markdown in the tooltip body. With that plugin, we added the following changes to our Web Component:

<svelte:options customElement="knowlo-tooltip" />

<script lang="ts">
  import { onMount, onDestroy } from 'svelte'
  import SvelteMarkdown from 'svelte-markdown'
  import aws4 from 'aws4'
  import url from 'url'
  import 'iconify-icon'
  import axios from 'axios'
  import { fromCognitoIdentityPool } from '@aws-sdk/credential-provider-cognito-identity'
  import { CognitoIdentityClient } from '@aws-sdk/client-cognito-identity'
  import type { ILoadTooltipParams } from './types'

  let source: string | null = null

  // Rest of the TS code

  onMount(async () => {
    // Rest of the "onMount" function

    tooltipData = await loadTooltipData({
      apiUrl,
      credentials: {
        accessKeyId: credentials.accessKeyId,
        secretAccessKey: credentials.secretAccessKey,
        sessionToken: credentials.sessionToken,
      },
      projectId,
      region,
      tooltipId,
    })

    source = tooltipData.tooltipText as string
    
    // Rest of the "onMount" function
  })

  onDestroy(() => {
    // ...
  })
</script>

<div class="tooltip" data-knowlo-tooltip bind:this={tooltip}>
  <div class="tooltip-container">
    <div class="tooltip-content">
      <div class="tooltip-text">
        { #if source }
          <SvelteMarkdown {source} isInline options={{ breaks: true }}/>
        {/if}
      </div>
      <!-- Rest of the template -->
  </div>
</div>

<style lang="scss">
  // styles
</style>

<slot />

A big part of the code is removed to highlight the important changes.

Ah, I almost forgot the most important part – getting the temporary credentials! Here’s the code:

const getCredentials = async (identityPoolId: string, region: string) => {
  const cognitoidentity = new CognitoIdentityClient({
    region: region,
    credentials : fromCognitoIdentityPool({
      client : new CognitoIdentityClient({
        region: region,
      }),
      identityPoolId: identityPoolId,
    }),
  })
  return await cognitoidentity.config.credentials()
}

The Cognito Identity Pool ID and region are passed as environment variables. And we use the following packages to get the credentials:

import { fromCognitoIdentityPool } from '@aws-sdk/credential-provider-cognito-identity'
import { CognitoIdentityClient } from '@aws-sdk/client-cognito-identity'

Results and issues

Do you want to see a quick demo? Here it is:

This demo shows the new tooltip creation flow. I’ll create more demos for the following articles. The tooltip preview shows users the mockup of the real tooltip they are about to create.

However, not everything has worked great so far. We had many issues generating the answer using markdown syntax and returning the escaped answer in the JSON response. The main problem was escaping new lines (\n).

To solve that issue, we’ll need to ask ChatGPT to generate the answer in plain markdown instead of JSON.

Along with that problem, there were many smaller problems, such as passing the project ID in data loaders in React Router v6, timeouts with some ChatGPT models (because of the API Gateway 30-second timeout), etc. All these issues are easily solvable, but they require some time and debugging.

Scoreboard

Time spent today: 8h

Total time spent: 194h

Investment today: $47 USD

Total investment: $1,331.45 USD

Beta list subscribers: 91

Paying customers: 0

Revenue: $0

What’s Next?

The important part is finally ready, but CofounderGPT and I still have some work to do. We’ll need to extract prompts to separate files, fix the markdown issue, and then add the invitation code functionality and basic analytics and deploy the production version of the app.