Matthew Gilliard's blog || Flow 102

Important note: This blog was correct at the time of posting but Fn has changed since. An up-to-date version of this document is maintained at https://github.com/fnproject/tutorials/tree/master/Flow102

If you haven’t read Flow 101 yet, I encourage you to start there and get grounded in what Flow is, what it’s for and how it works.

In this post I’ll go through how to build a more complex Flow with parallelism and asynchronous chaining. I’ll assume you have set up the services as described in the Flow 101.

–

The demo Flow
Installing helper functions
Creating our Flow function
Visualising the Flow
Further reading

The demo Flow

An app which:

reads some text
greps for a given keyword
counts the matching lines
prints the count
prints the file header

In your shell, it might look something like:

⇒ cat my_file | grep -i love | wc -l | xargs -n1 echo result:
⇒ head -n10 my_file

Installing helper functions

One of the cool things about Fn is that because it’s based on Docker, functions can be written in any language - even Bash!

Clone this repo of simple Bash functions and deploy them all:

⇒ git clone https://github.com/mjg123/fnproject-text-functions.git
⇒ cd fnproject-text-functions
⇒ fn deploy --local --all

You can test all of these individually, for example:

⇒ curl -H "Word: bar" -d $' foo \n bar \n baz' http://localhost:8080/r/flow102/grep
 bar

Creating our Flow function

In a new directory called word-flow:

⇒ fn init --runtime=java
⇒ rm -rf src/test  ## yolo, again

And, make HelloFunction.java look like this:

package com.example.fn;

import com.fnproject.fn.api.flow.Flow;
import com.fnproject.fn.api.flow.FlowFuture;
import com.fnproject.fn.api.flow.Flows;
import com.fnproject.fn.api.flow.HttpResponse;

import static com.fnproject.fn.api.Headers.emptyHeaders;
import static com.fnproject.fn.api.flow.HttpMethod.POST;

public class HelloFunction {

    public String handleRequest(String input) {
        Flow flow = Flows.currentFlow();

        // Get the first ten lines of the file
        FlowFuture<byte[]> headText = flow.invokeFunction( "./head", POST,
                    emptyHeaders().withHeader("LINES", "10"), input.getBytes() )
                .thenApply(HttpResponse::getBodyAsBytes);

        // Grep for "love"
        FlowFuture<byte[]> wordCountResult =
                flow.invokeFunction( "./grep", POST,
                                     emptyHeaders().withHeader("WORD", "love"),
                                     input.getBytes())
                .thenApply(HttpResponse::getBodyAsBytes)

        // and count the hits
                .thenCompose( grepResponse ->
                        flow.invokeFunction("./linecount", POST,
                                            emptyHeaders(),
                                            grepResponse ))
                .thenApply(HttpResponse::getBodyAsBytes);


        return "Number of times I found 'love': " + new String(wordCountResult.get()) + "\n" +
               "The first ten lines are: \n" + new String(headText.get());
    }
}

It’s worth reading this code carefully, remembering that anything returning a FlowFuture object is an asynchronous call, which can be chained with thenApply, thenCompose and the other Flow API methods.

We’ll want some test data:

⇒ curl http://www.gutenberg.org/cache/epub/1524/pg1524.txt > hamlet.txt

Deploy the function, and remember to configure the app with the location of the completer:

⇒ export DOCKER_LOCALHOST=$(docker network inspect bridge -f '{{range .IPAM.Config}}{{.Gateway}}{{end}}')
⇒ fn apps config set flow102 COMPLETER_BASE_URL "http://$DOCKER_LOCALHOST:8081"
⇒ fn deploy --app flow102 --local

And… send in the Shakespeare:

⇒ curl --data-binary @hamlet.txt http://localhost:8080/r/flow102/word-flow
Number of times I found 'love': 76

The first ten lines are: 
...etc etc...

Visualising the Flow

Check the UI on http://localhost:3002 and you should see something like this:

flow-ui

As you could see from the code above, the head and grep are executed in parallel, the linecount has to wait for the grep, and the main has to wait till everything else is finished.

The demo Flow

Installing helper functions

Creating our Flow function

Visualising the Flow

Further reading