Introduction

Funcatron lets you focus on the business logic of your code. What?

Much of the code you write deals with a cross-section of concerns…​ “How do I serialize/deserialize the data?” “What are the security rules associated with this REST endpoint?” “How does this code scale?” etc. Note that none of the above is the actual business logic of what the code is supposed to do.

Funcatron separates the concerns in your code so that you can focus on the business logic and declare the rules for the other concerns.

Funcatron uses technology that you’re familiar with (Java, Scala, JSON, JavaScript, etc.) as well as some newer, but very popular technology: Swagger.

With Funcation you:

  1. Define your endpoints including data shapes and access control in a Swagger document.

  2. Write methods/functions to implement those endpoints.

  3. Package the pieces together in an Assembly, PEX, or other “collection of libraries and code” bundle. These are known as “Func Bundles”.

  4. Deploy to Mesos, Kubernetes, or Docker Swarm. These are known as “Container Substrates”.

What Funcatron does:

  1. Wires up HTTP endpoints.

  2. Routes requests.

  3. Serializes/Deserializes data.

  4. Handles access control.

  5. Auto-scales.

Initially, Funcatron will handle HTTP-based REST endpoints. But an HTTP request is an event. Funcatron will route events…​ so the same code that may service an HTTP endpoint, may also service a “new customer added” event.

Basic Coding Concepts

So, what do you have to care about? Writing simple classes:

public class SimpleFunc implements Func<Data> {
    @Override
    public Object apply(Data data, Context context) {
        Number cnt = (Number) context.getRequestParams().get("path").get("cnt");

        List<Data> ret = new ArrayList<>();
        for (int i = 1; i <= cnt.intValue(); i++) {
            ret.add(new Data(data.getName() + i, data.getAge() + i));
        }

        return ret;
    }
}

The above code is your business logic. The first parameter to the apply method is the value that came from the “caller” (e.g., the HTTP POST), and the second parameter is the “Context”

Each Funcatron Func extends the funcatron.intf.Func interface. The type parameter tells Funcatron what class to deserialize the parameter into.

The Context parameter contains the raw request information as well as access to Logger and other resources.

The apply method is invoked (applied) when the event occurs and the return value of the method is serialized and returned as a response to the event.

To wire the Func to an HTTP endpoint, a Swagger file named funcatron.yml defines the relationship:

  /change/{cnt}:
    post:
      description: Returns a user based on a single ID, if the user does not have access to the pet
      operationId: funcatron.java_sample.PostOrDelete
      parameters:
        - name: cnt
          in: path
          description: number of Data to return
          required: true
          type: integer
          format: int64
        - name: data
          in: body
          description: The data
          required: true
          schema:
            $ref: '#/definitions/Data'
      responses:
        "200":
          description: Repeats the posted data cnt times
          type: array
          items:
            $ref: '#/definitions/Data'
...
  Data:
    required:
      - name
      - age

The operationId field contains the class of the Func. Note the cnt path parameter is defined as an integer. The cnt parameter is required. Funcatron will coerce the parameter to a Number before the apply method is called. The contract allows the developer to focus on the business logic without having to test all the parameters.

Getting Started

Okay, we’ve taken a look at the basic concepts in Funcatron. Now, let’s start a new project.

To get started, you will need to install the following:

  • Docker — Docker allows you to run Funcatron on your development box so that you can do live debugging.

  • Java — Install the Java Development Kit (JDK) so you can run and compile Java code

  • Maven — You can use the build tool of your choice with Funcatron. However, for this tutorial, we are using Maven. There are sample Funcatron projects using Maven, Gradle, sbt, and lein.

  • Your IDE of choice.

Start Funcatron

First, let’s start a local version of Funcatron running in a Docker container:

docker run -ti --rm -e TRON_1=--devmode -p 3001:3001 -p 54657:54657 funcatron/tron:latest

That command tells docker to run the funcatron/tron:latest container.

We want an -ti interactive terminal so we can see the logs from Funcatron.

--rm removes the instance at the end of execution.

-e TRON_1=--devmode tells Funcatron to run in developer mode where HTTP requests to port 3001 are run through the Funcatron code and forwarded to a developer “shim” connected via port 54657.

-p 3001:3001 -p 54657:54657 exposes the container’s port on localhost.

A couple of notes. This is a stripped down version of Funcatron that only routes HTTP requests to the development time code. It does not host Func Bundles. It does not have any statistics about usage. It’s just a dumb pipe of HTTP request to the “shim” port. Also, the “shim” port speaks a very dumb protocol. Don’t try to curl to it or do anything else with it. Why a dumb protocol? Because we wanted to have as small a footprint as possible for the code that runs in your application.

Test to see if Funcation is running by pointing your browser to http://localhost:3001 . You should see a message like: No Swagger Defined. Unable to route request. This is because there’s no application connected to Funcatron. So…​ let’s create an app.

Create a new project

The first thing we do is create a new project using Maven’s Archetype feature:

mvn org.apache.maven.plugins:maven-archetype-plugin:2.4:generate -X -B  \
    -DarchetypeGroupId=funcatron \
    -DarchetypeArtifactId=starter \
    -DarchetypeVersion=0.3.0-SNAPSHOT \
    -DgroupId=my.stellar \
    -DartifactId=thang \
    -DarchetypeRepository=https://oss.sonatype.org/content/repositories/snapshots

Things you’ll change for your project: -DgroupId= and -DartifactId.

Once you have the project created, cd into the project directory and type mvn compile exec:java.

Once the code is running, you’ll be able to browse to http://localhost:3001/api/sample and see data.

Yay!

You’ve got your new Funcatron project up and running.

Pieces Parts

We’ve created a running project. Now, let’s go through the parts of the project.

The Java Stuff

The actual code that’s executed is the Java code.

Data Shapes

The data is in a PoJo in the MyPojo.java file. The code is pretty normal:

public class MyPojo implements java.io.Serializable {
     private String name;
     private int age;

     public String getName(){
         return this.name;
     }

     public void setName(String name){
         this.name = name;
     }

     public Integer getAge(){
         return this.age;
     }

     public void setAge(Integer age){
         this.age = age;
     }
 }

MyPojo is a Java class with getters and setters.

Funcatron converts incoming requests into a parameter for the Func application (call to the apply method on the Func implementation) and serializes the return value. By default, Funcatron uses Jackson to serialize and deserialize values. Having PoJos that represent the data shapes for your application makes it super simple to do serialization.

Logic

In the MyFunction.java file, there are a bunch of different pieces: the apply method that Funcatron applies, the database access code, and the “dev-time” code that connects to the Funcatron instance.

Let’s start by looking at the dev-time code:

    public static void main(String[] args) throws Exception {
        System.out.println("Starting connection to Funcatron dev server");
        System.out.println("run the Funcatron dev server with: docker run -ti --rm  -e TRON_1=--devmode -p 3001:3001 -p 54657:54657 funcatron/tron:latest");
        System.out.println("Then point your browser to http://localhost:3001/api/sample");

        Register.register(funcatronDevHost(), funcatronDevPort(),
                new File("src/main/resources/funcatron.yaml"),
                new File("src/main/resources/exec_props.json"));
    }

The code prints some messages and connects to the development-time Funcatron instance in the Docker container.

This code is useful for you to set up your IDE to do debugging, etc. while you run a mini version of Funcatron in a Docker container. What does it do?

It makes a connection to the Docker container running mini-Funcatron. When mini-Func gets an HTTP request, it packages the request up and forwards the request to the app which is likely running in your IDE. You can see output, set breakpoints, and generally rapidly update your app.

Also, given that your Funcatron apps should be small, recompile times should be short so you can quickly cycle and quickly build your app.

If you’re using a language or a development environment that allows dynamic code reloading (e.g., Clojure or JRebel) the funcatron.yaml (Swagger file) and the exec_props (runtime properties) will reflect the current values…​ update them at will.

Next, let’s look at the database code:

    /**
     * Add the pojo to the database
     * @param pojo the Pojo to add
     * @param c the context
     */
    private void addToDatabase(MyPojo pojo, Context c) {
        try {
            // get the DB connection
            c.vendForName("db", Connection.class).
                    map((Connection db) -> {
                        try {
                            // db stuff here
                        } catch (SQLException se) {
                            c.getLogger().log(Level.WARNING, "Failed to insert pojo", se);
                        }
                        return null;
                    });
        } catch (Exception e) {
            c.getLogger().log(Level.WARNING, "Failed to add pojo to db", e);
        }
    }

The key takeaways are:

  • The Context allows access to logging via the getLogger() method.

  • Access to the database and other services is done via the vendForName(name, class) method which returns an Optional<class>. These items are defined in the exec_props.json file.

  • The map method on the Optional accesses the vended instance.

  • If objects vended during a request are transactional (e.g., JDBC connections), the transactions will be automatically committed if the function returns successfully, but will be rolled back if the function throws an exception.

Finally, let’s take a look at the apply method (the heart of the business logic for the Func):

    public Object apply(MyPojo pojo, final Context context) throws Exception {
        if (null == pojo) {
            pojo = new MyPojo();
            pojo.setName("Example");
            pojo.setAge(42);
        }

        // if we have a Redis driver, let the world know
        context.vendForName("cache", Jedis.class).map(a ->
        {
            context.getLogger().log(Level.INFO, "Yay!. Got Redis Driver");
            return null;
        });

        pojo.setName("Hello: " + pojo.getName() + " at " + (new Date()));
        pojo.setAge(pojo.getAge() + 1);

        // put the pojo in the DB
        addToDatabase(pojo, context);

        return pojo;
    }

If the pojo is passed as a parameter (i.e., the function was invoked via a POST or PUT), it will be populated in the pojo parameter.

The method contains plain old Java code, which is exactly what you want: focus on the business logic.

Oh…​ and we print a message if we’ve got a Redis driver…​ so…​ how did we get a Redis driver?

“Why does Funcatron use the Java logger?” Well…​ it’s like this…​ there are 18 billion logging libraries in Java-land and we needed to choose one, so we chose the one built into Java.

Under the covers, we do lots of fun things with logging including associating each log line with the Git SHA (unique code version) of the code that generated the log line as well as having a unique id for each request that’s propagated across the cluster so you can see all the places where a request fanned out to.

Let’s take a gander at RedisDriver.java:

public class RedisProvider implements ServiceVendorBuilder {
    /**
     * What's the name of this driver?
     * @return the unique name
     */
    @Override
    public String forType() {
        return "redis";
    }

    /**
     * Some fancy null testing
     * @param o an object
     * @param clz a class to test
     * @param <T> the type of the class
     * @return null if o is not an instance of the class or null
     */
    private <T> T ofType(Object o, Class<T> clz) {
        if (null != o &&
                clz.isInstance(o)) return (T) o;
        return null;
    }

    /**
     * Build something that will vend the named service based on the property map
     * @param name the name of the item
     * @param properties the properties
     * @param logger if something needs logging
     * @return If the properties are valid, return a ServiceVendor that will do the right thing
     */
    @Override
    public Optional<ServiceVendor<?>> buildVendor(String name, Map<String, Object> properties, Logger logger) {
        final String host = ofType(properties.get("host"), String.class);

        if (null == host) return Optional.empty();

        return Optional.of(new ServiceVendor<Jedis>() {
            @Override
            public String name() {
                return name;
            }

            @Override
            public Class<Jedis> type() {
                return Jedis.class;
            }

            @Override
            public Jedis vend(Accumulator acc) throws Exception {
                Jedis ret = new Jedis(host);
                // make sure we are notified of release
                acc.accumulate(ret, this);
                return ret;
            }

            @Override
            public void endLife() {

            }

            @Override
            public void release(Jedis item, boolean success) throws Exception {
                item.close();
            }
        });
    }
}

The above code associates Execution Properties with code that will vend connections to databases, caches, and other services. How does it work?

Take a look at exec_props.json. There’s an entry:

  "cache": {
    "type": "redis",
    "host": "localhost"
  }

This entry says “there’s a service named cache that has a driver type redis that connects to a host named localhost.” To access the service, we invoke context.vendForName("cache", Jedis.class) and get an Optional<Jedis> back.

You can create ServiceVendorBuilder instances for any type and, boom, have access to those services based on Execution Properties.

Serializers

By default, Funcatron uses Jackson to serialize and deserialized JSON data. This is fine for Java PoJos that have getters/setters. But if you are using immutable pojos, Scala case classes, etc., you may have more complex serialization needs.

The Func interface allows you to write custom serializers.

To deserialize incoming data using special rules, override the jsonDecoder method in Func and to serialize returned data using special rules, override the jsonEncoder method:

    public Function<InputStream, Data> jsonDecoder() {
        return m -> {
            try {
                return jackson.readValue(m, Data.class);
            } catch (Exception e) {
                throw new RuntimeException("Failed to deserialize", e);
            }
        };
    }

The method returns a Function that takes an InputStream and returns an instance of the type matching the first parameter of apply method.

The jsonEncoder method does the opposite. Here’s a Scala example:

trait DecoderOMatic[T] {

  protected def ct: Class[T]

  def jsonDecoder(): Function[InputStream, T] = {
    new Function[InputStream, T] {
      def apply(t: InputStream): T = DecoderOMatic.jackson.readValue(t, ct)
    }
  }

  def jsonEncoder(): Function[Object, Array[Byte]] =
    new Function[Object, Array[Byte]] {
      def apply(o: Object) = DecoderOMatic.jackson.writer().writeValueAsBytes(o)
    }
}

object DecoderOMatic {
  val jackson: ObjectMapper = {
    val mapper = new ObjectMapper()
    mapper.registerModule(DefaultScalaModule)
    mapper
  }
}

The above code creates an instance of the Jackson ObjectMapper and adds the Scala module.

For complex data types, build your own serializers.

Funcatron Swagger File

So…​ how does Funcatron associate code with HTTP endpoints? How does Funcatron ensure that the functions are called with properly shaped data?

Funcatron endpoints are defined in a Swagger file named either funcatron.yaml or funcatron.json. Define the endpoints, associate them with the class that implements the Func interface via the operationId field and Funcatron does the rest.

What’s "the rest"?

  • Funcatron ensures the incoming data is shaped correctly and will not attempt to deserialize the JSON data if it’s not properly shaped.

  • Funcatron ensures all the rules defined in the Swagger file (e.g., OAuth rules, etc.) are properly enforced.

  • In production, wires up the front end web servers to respond to requests.

  • In development mode, presents a UI to test out the API endpoints at http://localhost:3001/ui/

So, let’s see some of the Swagger magic in action. Point your browser to http://localhost:3001/ui/ then click through "default" and "POST". Cool. You can try out a POST from your browser.

Next, let’s update the funcatron.yaml file. Replace the post: line and all subsequent lines in the file with:

    post:
      description: Creates new sample data
      operationId: my.stellar.MyFunction
      parameters:
        - name: body
          in: body
          required: true
          schema:
            $ref: '#/definitions/Data'
      responses:
        "200":
          description: sample response
        default:
          description: unexpected error

definitions:
  Data:
    required:
      - name
      - age
    properties:
      name:
        type: string
      age:
        type: number

Now, reload the browser and you can see the UI presents you with the option of entering JSON data. Enter some and click the Try it out! button.

Well…​ you get the idea. Basically, model your REST endpoints in Swagger, test stuff out in the browser. Associate your classes with the operationId field, and the code will Just Work™.

Execution Properties

The static relationship between REST endpoints and your code as well as access control rules, etc. are defined in your code and the funcatron.yaml Swagger file.

However, there will be runtime properties that are different among different environments. For example, you’ll have database access credentials for development time, for testing time, staging, and production.

Those items bundled together in “Execution Properties”.

For the development-time project, we have the exec_props.json file which contains Execution Properties that are used during development-time.

We’ve seen if the Execution Properties have a type field and a named Service Vendor is associated with the type, the service is available in the Context.

The entire contents of the Execution Properties information is available in the Context.properties() method. This returns a Map of key/value pairs. Put any old information associated with the execution of the code in the Execution Properties and it’s available to the Func.

Execution properties are defined via JSON when a Func Bundle is deployed. See Deploying.

Packaging your Func Bundle

Once you’re happy with your code and want to create a “Func Bundle” and upload it to Funcatron:

mvn clean test package

Once the packaging is done, you’ll have a bundle that you can upload to the Funcatron cluster:

dpp@octopus:~/tmp/thang$ ls -l target/thang-0.1.0-jar-with-dependencies.jar
-rw-rw-r-- 1 dpp dpp 4254796 Jan  2 17:42 target/thang-0.1.0-jar-with-dependencies.jar

Deploying

To upload:

wget -q -O - --post-file=target/thang-0.1.0-jar-with-dependencies.jar http://<SERVER>:<PORT>/api/v1/add_func

StatsD

Funcatron supports logging out to StatsD.

To enable logging to a host (in this case, 127.0.0.1):

curl -H "Content-Type: application/json" -d '{"enable": true, "host": "127.0.0.1", "port": 8125}' -X POST http://FUNCATRON_SERVER:FUNCATRON_PORT/api/v1/stats

To disable statsd logging:

curl -H "Content-Type: application/json" -d '{"enable": false}' -X POST http://FUNCATRON_SERVER:FUNCATRON_PORT/api/v1/stats

The Build File