Introduction

This document describes the Funcatron architecture and development strategy.

Funcatron is a Serverless framework. It’s dead simple: associate a "function" with an event. Each time the event happens, the function is applied and the function’s return value is returned to the event source. An event can be an HTTP(S) request, something on an event queue, whatever.

Functions are ephemeral. They exist for the duration of the function call. Once the function returns a value, all of its state and scope and everything else about it is assumed to go away.

Scaling this kind of architecture is simple: the more frequently a function gets applied, the more compute resources are allocated to support the function.

Funcatron, where possible, abstracts away "switched" connections into events. What does this mean?

Think of TCP/IP. To a developer, data in and out of a TCP/IP socket appears to be a switched connection to the machine on the other side. But, the protocol is implemented as a series of packets that may not be reliably delivered.

Funcatron uses a message queue, where possible, to transmit requests. Each request is stateless in that it may be serviced by a different system than the previous request. All state associated with request is pushed to databases and caches outside of the address space where the code that’s handling the request lives. And there’s always state.

There are numerous advantages to event-based architecture:

  • Events can be converted into “switched” or blocking calls easily, but the inverse is not true. Switch architectures rely on polling to check for updates or new information.

  • Message Queues are mature and well understood. Switch service routers like linkerd are newer and less well understood by ops.

  • There are many kinds of events that do not require an "answer". Switched systems imply an answer beyond an ACK that a message was reliably enqueued.

  • Event-based architectures are easier to test. Each block in the architecture can be tested by sending events and capturing the resulting events. There’s no need to set up harnesses that simulate or provide the other end of a switch.

Funcatron Pieces and Packaging

In Funcatron, user code is referred to as Funcs. Funcs are bundled together along with routing descriptors in a single file that’s called a Func Bundle.

Initially, Funcatron is focused on associating HTTP endpoints with code. The routing description for HTTP-focused Func Bundles is written in Swagger.

Func Bundles contain all the information necessary to create routes and run code based on incoming requests. Funcatron supports the following languages:

  • Java/JVM (including Scala, Kotlin, Clojure, etc). JVM Func Bundles are packages as Assemblies or an UberJar.

  • Python. Python code is packaged up into a PEX

Note

Python support is scheduled for Funcatron in the future

  • JavaScript. JavaScript code is bundled using Webpack

Note

JavaScript support is scheduled for Funcatron in the future

  • CLR (Common Language Runtime). Run .Net assemblies in Mono (scheduled for Funcatron in the future)

Network

Funcatron has a series of discrete components, all of which are packaged as Docker components:

  • Frontend — the code the recieves the incoming request.

  • Message Queue — the medium of communication among the system components.

  • Tron — the control system that knows what

  • Runner — The code that dequeues requests and runs the function associated with the request.

The components communicate via network as follows:

diag 7de02e942de8e578254a0d5d4c2090aa

In the current implementation, here is the actual technology used:

  • Frontend — OpenResty with Lua scripts that enqueue the requests and dequeue the response

  • Message Queue — RabbitMQ

  • Tron — JVM code (Clojure and Java)

  • Runner — JVM code that uses classloaders to load JVM Func Bundles. For other languages, a new process will be kicked off for each Func Bundle

Request Sequence Diagram

Incoming HTTP requests are handled as follows:

diag db374371816c728d4ecac64fb7ec6b2c

Directly proxied requests (bypass message queue)

Note, the specific route may be marked “direct” because the request or response payload is too big to be reasonably handled by a message queue. In the case of a direct request, the sequence is:

diag 5c3a8cd813b46fa036147919cc809933

In the above example, the message queue is used so that the Runner that is available to handle the request is the Runner that has the HTTP request proxied to it. This avoids the Frontend systems needing an up to date list of available Runners and avoids having different logic for routing direct and normal requests.

Note

The Frontend code may, in the future, look at the Content-Length header and opt to request a direct connection for large request bodies.

diag 6410a0942a9c4620c2ba3110642f959e

Runner/Func Bundle JVM lifecycle

Funcatron will support Func Bundles from various languages. JVM-language Func Bundles are handled differently than any other Func Bundles.

The Java Virtual Machine has a facility known as Classloaders. A Classloader allows loading of a collection of Java classes from a source (JAR file, class file, etc.) and have that class isolated from other classes or sets of classes. Most application servers make use of Classloaders to allow loading different “applications” into the same Java virtual machine where each application is linked to the libraries and library versions that it was packaged with.

Funcatron loads JVM-based Func Bundles via a separate Classloader into a Funcatron Runner instance.

In order to service a request, Funcatron needs two basic facilities:

  • The Swagger file that defines the REST endpoints

  • The ability to dispatch a specific request to the associated operationId for the endpoint/method combination

By default, Funcatron looks through an UberJar for the funcatron.yaml or funcatron.json file for the Swagger information. However, if you’ve got a Spring Boot application, the Swagger information is derived from annotations and dispatch is not on a class basis, but based on the annotations.

Note

We deliberately chose a name different than swagger.yaml such that if there was a Swagger file in the JAR, it would not conflict with what Funcatron looks for.

Additionally, there may be Func Bundle-level initialization that needs to take place (e.g., Clojure apps need to load the RT class before any dispatch takes place.)

So, Funcatron needs a plugable mechanism for initializing and destroying Classloader-based contexts. Additionally, Funcatron needs to delegate classloading to special “code weaving” style classloaders that exist in Spring-based applications. And Funcatron needs to support alternative mechanisms for delivering Swagger files. And Funcatron needs to support alternative operationId to function-level dispatch. And while we’re at it, why not support full Middleware facilities to wrap the request/response stack?

The Java Virtual Machine has a facility for discovering “Providers” of specific types of services: the ServiceLoader.

Funcatron makes extensive use of the ServiceLoader to support changing default behaviors based on the contents of a Func Bundle.

Loading a JVM Func Bundle

When a Func Bundle bundle is loaded, a new URLClassloader is created with no parent classloader and just the Func Bundle JAR as a set of classes to load. This results in a Classlaoder that only has access to the classes in the JAR and the Java Virtual Machine base library. The classes in the Java Virtual Machine base library include things like String and Map.

The new classloader is asked for the funcatron.intf.impl.ContextImpl class. The only communication between Funcatron and the Func Bundle is via this class and specifically, the initContext static method.

Funcatron invokes the initContext method with the Execution Properties, the newly created Classloader and Logger.

Bootstrapping the ClassLoader: ClassloaderProvider

initContext uses the ServiceLoader to find all the ClassloaderProvider instances. These instances are sorted in reverse order based on the order() method. This allows providers that need to be executed early to be executed first by returning a high value.

The initial ClassLoader is passed to the first ClassloaderProvider which builds a new ClassLoader from the initial one and returns it. That process continues for all the ClassloaderProviders.

The ClassloaderProvider is an ideal place to initialize a runtime like Clojure. In this case, the Clojure RT class would be loaded into the provided ClassLoader and the original ClassLoader would be returned.

The addEndOfLife funcation is passed to the buildFrom method in ClassloaderProvider. If the provider allocates any resources that will not be automatically garbage collected, the provider can register a function that will be applied when the Func Bundle goes out of scope.

Adding Operations: OperationProvider

After the ClassLoader is computed and we’ve initialized the execution environment, the next phase is to add or change “operations”.

Operations are a set of functions that can be invoked by the Runner or other facilities to either get information from or impact the operation of the Func Bundle.

Operations have a type signature BiFunction<Map<Object, Object>, Logger, Object>. They are functions that take two parameters: the parameter map and a logger and return something.

Here are the built in operations:

  • operations — returns a Set<String> of all the named operations. The parameter map and logger are ignored.

  • getClassloader — returns the computed ClassLoader for the Func Bundle. The parameter map and logger are ignored.

  • getSwagger — returns the Swagger file or information in a Map where the keys are swagger which is an object that contains the Swagger information and type which is the type of the file. Replace this function if your Func Bundle computes the Swagger based on something other than a file. By default, the /swagger.yaml or /swagger.json resource is loaded from the JAR file. Understood types are:

    • yaml — the swagger should be a String and it’s parsed using a YAML parser.

    • json — the swagger should be a String and it’s parsed using a JSON parser.

    • map — the swagger should be a Map and it’s passed unchanged.

  • getVersion — queries the funcatron/intf/MANIFEST.MF file for the version. Returns a String. If the version is greater than or equal to the Runner’s version, the Runner assumes all capabilities that the Runner knows about. If the version is less than the Runner’s version, the Runner may change behavior to compensate for older versions. This allows Funcatron clusters to run various different Func Bundles without having to have version equality.

  • dispatcherFor — based on an operationId and other Swagger information, return a BiFunction<InputStream, Map<Object, Object>, Map<Object, Object>> that will service the request. The $operationId field in the parameter map must be set. Replace this operation if dispatching will happen by mechanism other than Funcatron’s built-in class-based dispatcher. For example, a Clojure dispatcher and a Spring Boot dispatcher would replace this operation.

  • getSerializer — Return a BiFunction<Object, String, byte[]> if there’s a special serializer. This is a handy mechanism for creating a generic serializer. The Object is the thing to serialize, String is the mime type or blank for JSON, and byte[] is the returned serialized bytes. If this operation returns null, then use the built in serialization logic.

  • getDeserializer — Return a BiFunction<InputStream, List<Object>, Object> where the InputStream is the body of the thing to deserialize. The first element in List<Object> is the content type which may be null (assume application/json) and the second element is the Class of the thing to deserializer. Return the deserialized value. If this operation returns null, then use the built in deserialization logic.

  • wrapWithMiddleware — take a BiFunction<InputStream, Map<Object, Object>, Map<Object, Object>> and wrap middleware around it, returning a BiFunction<InputStream, Map<Object, Object>, Map<Object, Object>>. The function property in the incoming Map must be the original BiFunction<InputStream, Map<Object, Object>, Map<Object, Object>>.

Adding Services: ServiceVendorProvider

Finally, the all ServiceVendorProvider services are loaded and each ServiceVendorProvider registers services, such as a Redis provider, with the context.

Registering the Middleware

In order to service an incoming request, Funcatron applies the incoming body (an InputStream) and the request parameters, headers, etc. (the Map<Object, Object>) to a BiFunction<InputStream, Map<Object, Object>, Map<Object, Object>>. The returned Map<Object, Object> is the thing that’s returned…​ that’s the answer. It’s possible to wrap “middleware” around the function to either modify the incoming stuff or modify the return value.

Middleware is implemented as a MiddlewareProvider and loaded into the Context. Then each of the functions generated by dispatchFor is wrapped by the middleware.

And back again

Once the ContextImpl.initContext method is complete, it returns a Function<String, BiFunction<Map<Object, Object>, Logger, Object>> to the caller. This allows the caller to look up and apply operations. This provides the bridge between the Func Runner and the Func Bundle.