Deploying a Vert.x Native function with AWS Lambda

This how-to shows you how to deploy a Vert.x-based native image function served by AWS Lambda.

What you will build

You will write a function that accepts returns a Quote Of The Day (similar to the UNIX tool).
This function will be written in Java.
The code will be compiled to native with the help of GraalVM.
A function will be created with AWS Command Line Interface.
This function will be deployed with AWS Lambda.

What you need

A text editor or IDE
GraalVM (>= 1.0.0-rc13)
Maven
AWS CLI tools
A AWS account to deploy your function.

What is a AWS Lambda function and why is Vert.x a good match?

AWS Lambda is a service which takes care of computing your code without any knowledge on the server environment. It is said to be serverless compute. The code is executed based on the response of events in AWS services like adding /removing files in S3 bucket, updating Amazon DynamoDB tables, HTTP request from Amazon Api gateway etc.

AWS Lambda code can be written in Java using the Amazon SDK. However due to the nature of the JVM and typical Java frameworks startup time can become a huge bottleneck to the performance of the function. To workaround this usually Java functions would be run on a Java server container, waiting for requests (defeating the whole serverless aspect).

Enter Vert.x and GraalVM. Vert.x is ideal for writing functions on GraalVM native-images, because:

Vert.x applications start very fast since there is no magic happening at run time,
GraalVM is used compilation to further reduce the startup and memory footprint,
Vert.x applications are resource-efficient and remain responsive even under heavy load,
Vert.x offers a large ecosystem of reactive clients to other middlewares (databases, messaging, etc),
a single functional interface implementation suffices to bootstrap a Vert.x application!

Create a project

Start by cloning the following github project: https://github.com/pmlopes/aws-lambda-native-vertx

After that let’s walk over the important parts of the project. Here is the content of the pom.xml file that you should be using:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>myapp</artifactId>
  <version>0.0.1-SNAPSHOT</version>

  <name>aws-lambda-vertx-native</name>

  <properties>
    <vertx.verticle>vertx.lambda.LambdaBootstrap</vertx.verticle>

    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.target>1.8</maven.compiler.target>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.testSource>1.8</maven.compiler.testSource>
    <maven.compiler.testTarget>1.8</maven.compiler.testTarget>
    <graal.version>1.0.0-rc13</graal.version>
  </properties>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>io.vertx</groupId>
        <artifactId>vertx-stack-depchain</artifactId>
        <version>3.6.3</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>
    <dependency>   <!--(1)-->
      <groupId>com.oracle.substratevm</groupId>
      <artifactId>svm-driver</artifactId>
      <version>${graal.version}</version>
      <scope>provided</scope>
    </dependency>
    <dependency>
      <groupId>io.vertx</groupId>
      <artifactId>vertx-core</artifactId>
    </dependency>
    <dependency>   <!--(2)-->
      <groupId>io.vertx</groupId>
      <artifactId>vertx-web-client</artifactId>
    </dependency>
    <dependency>
      <groupId>io.vertx</groupId>
      <artifactId>vertx-unit</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.13.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins> <!--(3)-->
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <version>3.0.2</version>
        <configuration>
          <archive>
            <manifest>
              <mainClass>${vertx.verticle}</mainClass>
            </manifest>
          </archive>
        </configuration>
      </plugin>
      <plugin>
        <groupId>com.oracle.substratevm</groupId>
        <artifactId>native-image-maven-plugin</artifactId>
        <version>${graal.version}</version>
        <executions>
          <execution>
            <goals>
              <goal>native-image</goal>
            </goals>
            <phase>package</phase>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

We need svm-driver to handle code incompatibilities with substrateVM.
We need vertx-webclient to interact with the lambda environment.
GraalVM configuration to produce an image.

Writing the function

The function receives the incoming HTTP headers and the HTTP body. A random QOTD is selected. For each request, the random quote is sent:

/*
 * Copyright 2019 Paulo Lopes.
 *
 *  All rights reserved. This program and the accompanying materials
 *  are made available under the terms of the Eclipse Public License v1.0
 *  and Apache License v2.0 which accompanies this distribution.
 *
 *  The Eclipse Public License is available at
 *  http://www.eclipse.org/legal/epl-v10.html
 *
 *  The Apache License v2.0 is available at
 *  http://www.opensource.org/licenses/apache2.0.php
 *
 *  You may elect to redistribute this code under either of these licenses.
 */
package lambda;

import io.vertx.core.Future;
import io.vertx.core.MultiMap;
import io.vertx.core.Vertx;
import io.vertx.core.buffer.Buffer;
import vertx.lambda.Lambda;

import java.util.Random;

/**
 * A simple QOTD Lambda
 */
public class QOTDLambda implements Lambda {

  private static final Random RANDOM = new Random();

  private static final String[] QUOTES = {    // (1)
    "If you find a random command on the internet, try running it as root. What's the worse that could happen?",
    "Can root create a process that even root can't kill?",
    "There's no place like ~",
    "Thou shalt not kill -9.",
    "The 'n' in unmount is missing, leaving it as umount! If you find it, please write to Bell Labs, 600 Mountain Avenue Murray Hill, NJ.",
    "echo \"Just another useless use of cat.\" | cat",
    "`echo \"Just another useless use of backticks.\"`",
    "We've just created a special character device for handling complaints, conveniently located at /dev/null.",
    "false - do nothing, unsuccessfully"
  };

  @Override
  public
  Future<Buffer>  // (2)
  call(Vertx vertx, MultiMap headers, Buffer body) {    // (3)
    // return a random qotd
    return Future.succeededFuture(Buffer.buffer(QUOTES[RANDOM.nextInt(QUOTES.length)]));  // (4)
  }
}

We declare a array of quotes.
Implement the Lambda interface which returns a Future to allow asynchronous processing.
The implementation receives the current Vertx instance and the request headers and body.
For each request we return a resolved future with a random quote.

Testing the function

/*
 * Copyright 2019 Paulo Lopes.
 *
 *  All rights reserved. This program and the accompanying materials
 *  are made available under the terms of the Eclipse Public License v1.0
 *  and Apache License v2.0 which accompanies this distribution.
 *
 *  The Eclipse Public License is available at
 *  http://www.eclipse.org/legal/epl-v10.html
 *
 *  The Apache License v2.0 is available at
 *  http://www.opensource.org/licenses/apache2.0.php
 *
 *  You may elect to redistribute this code under either of these licenses.
 */
package lambda;

import io.vertx.core.Future;
import io.vertx.core.MultiMap;
import io.vertx.core.buffer.Buffer;
import io.vertx.ext.unit.Async;
import io.vertx.ext.unit.TestContext;
import io.vertx.ext.unit.junit.RunTestOnContext;
import io.vertx.ext.unit.junit.VertxUnitRunner;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;

@RunWith(VertxUnitRunner.class)
public class QOTDLambdaTest {

  private final QOTDLambda fn = new QOTDLambda();

  @Rule
  public RunTestOnContext rule = new RunTestOnContext();

  @Test
  public void shouldGetAQOTD(TestContext should) {
    final Async test = should.async();

    Future<Buffer> fut = fn.call(rule.vertx(), MultiMap.caseInsensitiveMultiMap(), null);   // (1)

    fut.setHandler(call -> {
      if (call.failed()) {
        should.fail(call.cause());  // (2)
      } else {
        should.assertNotNull(call.result());  // (3)
        should.assertTrue(call.result().length() > 0);
        test.complete();  // (4)
      }
    });
  }

}

Call the function with empty headers and body.
Fail the test if the call fails.
Verify that there’s a quote.
Terminate the test.

We can easily test that the function works:

from your IDE, run the junit test, or
with Maven: mvn test.

Preparing your AWS dev environment

Create a Lambda role

Bofore being able to deploy to AWS you need to have a role capable of accessing the service. In order to do this create a temp json file (/tmp/trust-policy.json) with:

And deploy it with:

$ aws iam create-role \
    --role-name lambda-role \
    --path "/service-role/" \
    --assume-role-policy-document file:///tmp/trust-policy.json

Building your function

Building the function is running the usual maven command:

$ mvn clean package
$ zip -r function.zip bootstrap target/lambda

The final zip is the base layer used by the functions.

Deploying the runtime layer

This layer will be installed on top of the Amazon Linux in the /opt directory.

$ aws lambda publish-layer-version \
    --layer-name vertx-native-example \
    --zip-file fileb://function.zip

This will install the lambda entrypoint /opt/bootstrap and the native image ELF containing all the functions.

Deploying the function

You will need to provide a function zip file, in our case this is not useful as all code lives in the base layer so we can upload "again" the same zip or upload an empty file.

$ aws lambda create-function --function-name vertxNativeTester \
    --zip-file fileb://function.zip --handler lambda.EchoLambda --runtime provided \
    --role arn:aws:iam::<YOUR-ARN>:role/service-role/lambda-role

This could be useful if your function requires file resources. In this case you could package the resources for each function on a separate zip file.

Since the deployment was done with a custom runtime we need to link the 2:

$ aws lambda update-function-configuration --function-name vertxNativeTester \
    --layers arn:aws:lambda:eu-central-1:<YOUR-ARN>:layer:vertx-native-example:1

Testing the function

Congratulations you just published your first native function. In order to quickly test it use the toolkit:

$ aws lambda invoke --function-name vertxNativeTester \
    --payload '{"message":"Hello World"}' \
    --log-type Tail response.txt | grep "LogResult" | awk -F'"' '{print $4}' | base64 --decode

Summary

We wrote a native function with Vert.x that returns a QOTD.
We built a AWS lambda package.
We deployed this function with AWS CLI.

Deploying a Vert.x Native function with AWS Lambda

What you will build

What you need

What is a AWS Lambda function and why is Vert.x a good match?

Create a project

Writing the function

Testing the function

Preparing your AWS dev environment

Create a Lambda role

Building your function

Deploying the runtime layer

Deploying the function

Testing the function

Summary

See also