flow-engine

KO | EN

gitlite — search

#python #react #typescript #llm #go #golang #android #machine-learning #macos #nodejs #cli #kubernetes #vue #ios

(★ 19)

No description available.

flow-engine Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# flow-engine

The flow-engine is a node.js module specializing in processing of a sequence of
tasks. The tasks execution sequence is defined in JSON object's `assembly`
property and passed to flow-engine for execution.
In The following example, flow-engine executes three tasks - `if`, `if`, and
`response`.

```
{
  "assembly": {
    "execute": [
      {
        "if": {
          "condition": "'$(request.verb)' === 'POST'",
          "execute": [
            {
              "message-mediation": {
                "target": "context.response",
                "value": "{ status: 200, message: 'This is a POST response' }"
              }
            }
          ]
        }
      },
      {
        "if": {
          "condition": "'$(request.verb)' === 'GET'",
          "execute": [
            {
              "message-mediation": {
                "target": "context.response",
                "value": "{ status: 200, message: 'This is a GET response' }"
              }
            }
          ]
        }
      },
      {
        "response": {
          "payload": "context.response"
        }
      }
    ]
  }
}
```

Each task is a JavaScript function that can
  - execute any code
  - make change to the `context` object
  - instruct flow-engine to continue or stop the remaining tasks execution

The flow-engine provides some prebuilt tasks at [lib/task](lib/task).
You can use these prebuilt tasks to compose your flow or use them as references
to build customized ones. Putting the customized tasks to the `lib/task`
directory, and flow-engine can pick up the new tasks without additional
configuration.

# flow-engine and IBM microgateway
The flow-engine is designed to be the heart of a gateway or reverse proxy.
It is included and used as part of the microgateway module that
serves as API policies enforcement point.

Each API policy on the microgateway is implemented as a flow-engine
task. And what and when API policies need to be enforced is describe as the
assembly flow to be executed by the flow-engine.

#Installation
`npm install flow-engine`

#APIs
To use the FlowEngine, one must `require('flow-engine')`.

- setup function `require('flow-engine')(configObj)`:  
  configObj supports the follow options:
  - flow : the flow definition file and it shall be a YAML file
  - paramResolver: a module's **location** that is used to resolve the
    placeholder inside the flow assembly
  - tasks: an object that contains all custom task modules' name and
    **location**
  - baseDir: the base directory that will be used to load modules

and the return value is a middleware handler
  
- `Flow(assembly, optionObj)`:  
  Create a flow instance with the flow assembly
  - assembly: the flow's assembly and it's JSON object
  - optionObj: supports the following options:
    - paramResolver: a function that is used to resolve the placeholder inside
      the flow assembly
    - tasks: an object that contains all custom task modules' name and their
      handler functions. Use this option to control the creation/lifecycle of
      task handlers by yourself. 
    - tid: transaction id. if it doesn't present, flow-engine uses tid() to get
      tid that is maintained in flow-engine's scope.
    - logger: a bunyan logger obj that flow-engine uses it for logging. if it
      doesn't present, flow-engine creates its own logger.
    - logLevel: flow-engine uses it to create its own logger if `logger` is not
       there.

- `Flow.prototype.prepare(context, next)`:  
   Pass the `context` object and `next` function to the flow instance that you
   create by the Flow ctor

- `Flow.prototype.run()`:    
   Start to execute the flow assembly

- `Flow.prototype.subscribe(event, handler)`:    
   Subscribe an event with an event handler. 
   - event: 'START', 'FINISH', 'ERROR' or 'pre|post:task-name'
   - handler: a handler should be:  
     `function(event, next)`  
     A handler function would get the event name and a done callback(`next`). 
     The handler must call `next()` when it finishes.

- `Flow.prototype.unsubscribe(event, handler)`:    
   Remove the handler from the event's subscriber list

- `tid()`:    
   get a flow-engine scope transaction Id. Basically, it's a global counter
   that start with 1 and increase 1 by each call.

To execute a flow described in YAML:

```
var flow = require('flow-engine')({ config: "flow.yaml" });
//the flow is a middleware handler and ready to be used

```

Or directly create a Flow instance with a specific JSON config
```
const Flow = require('flow-engine').Flow;
var flow = new Flow( json, optionObj );
//you have to manually pass the context and next into the flow and execute it
//get context obj from somewhere or create it by yourself
//and request and response object should be attached to the context obj
flow.prepare(context, next);
flow.run();
```

#APIs for Task/Policy developer
A task is the unit of the flow assembly. flow-engine leverages tasks to fulfill
a flow assembly. When developing a custom task, flow-engine provides the
following APIs:

- `invoke(assembly, options, next)`:  
  flow-engine runs the given `assembly`. When the assembly finishes. the `next`
  callback will be invoked. the `options` here is the same as the options in
  `Flow(assembly, optionObj)`
  - assembly: the flow's assembly and it's JSON object
  - next: this callback will be invoked after the assembly finishes
  - options: supports the following options:
    - paramResolver: a function that is used to resolve the placeholder inside
      the flow assembly
    - tasks: an object that contains all custom task modules' name and handler
      function
    If these properties don't exist, the parent's will be used.
- `subscribe(event, handler)`:    
   Subscribe an event with an event handler. 
   - event: 'START', 'FINISH', 'ERROR' or 'pre|post:task-name'.
   - handler: a handler should be:  
     `function(event, next)`  
     A handler function would get the event name and a done callback(`next`).
     The handler must call `next()` when it finishes.

- `unsubscribe(event, handler)`:    
   Remove the handler from the event's subscriber list

- `proceed()`:    
   When a task finishes its job, it must call this API to tell flow-engine to
   run the next task.

- `fail(error)`:    
   When a task finishes its job with error situation, it must call this API to
   tell flow-engine to run the Error handing to process the error.

- `stop()`:    
   When a task finishes its job and the flow should also end, it must call this
   API to tell flow-engine to finish the assembly execution.

- `logger`:    
   A bunyan logger object which is used for logging.

These APIs are attached to the third param of task handler interface and could
be used by task developer. Here is a simple task implementation:

```
'use strict';

module.exports = function ( config ) {

return function (props, context, flow) {
        var logger = flow.logger;
        logger.info('ENTER mypolicy, params:', JSON.stringify(props));
        //some business logic
        
        //and register a FINISH event handler
        flow.subscribe('FINISH', function(event, next) {
            //this would be executed when a flow finishes
            next();
        });
        
        //call next task
        flow.proceed();
    };
};
```

#Sample
Use the setup function to create a flow-engine middleware and register this
middleware to handle all of the requests:
```
var createFlow = require('flow-engine');

var flow = createFlow({
        flow: "flow.yaml",
        tasks: {'activity-log': './myactivity-log.js'},
        baseDir: __dirname});

var app = express();
app.post('/*', [ flow ]);

```

#The Design
### The middleware interface
- The interface is `function(request, response, next)`
- The flow-engine module supports the middleware interface. It returns the
  middleware handler:

```
  var middlewareHandler = require('flow-engine')(some_config);
  middlewareHandler(request, response, next);
  ```

### The Task/Policy interface
- Every task that is defined inside the assembly should have a corresponding
  task module.And the task module should provide a factory function - 
  `function(config)` and return a task handler function - 
  `function(props, context, flow)`. Therefore, flow-engine will use the task
  module like this:

```
  var taskHandler = require('taskModule')(task_config);
  taskHandler(props, context, flow);
  ```
  
- **Finding task/policy handlers to execute the tasks/policies that defined in
  the assembly**:
  - tasks under `<flow-engine>/lib/task` are loaded when require('flow-engine').
    flow-engine calls these tasks' factory functions with empty config object
    and keeps the returned task handler functions
  - use task's name to search the task module and see if there is a task module
    under options.tasks[taskName].
    The options.tasks here is the options that you pass to Flow's constructor or
    the setup function.
    The difference is the options.tasks that you pass to the setup function will
    be processed via the `require()`, call its factory function by passing empty
    config object and get the task handler function. And the options.tasks that
    you pass to Flow's constructor function shall be task handler functions
    already.
  - searching the task module under `<flow-engine>/lib/task.
  - directly use `require` against the task's name, call its factory function
    with empty config and get the handler function

- **Execute a task/policy**:
  - use the `Finding task handlers procedure` above to get task's handler
    function
  - get the properties/values that defined in the assemble for the task and use
    paramResolver to replace the placeholder into corresponding value if exists
  - call the task handler function we get from step 1 and pass the
    properties/values, context obj and flow obj into it. the `flow` here is an
    object which contains a set of APIs:
    - .proceed(): when a task finishes its job successfully, it must call this
      function to tell flow-engine that its job is done, please continue with
      next task
    - .fail(error): when a task couldn't finish it job, it must call this
      function to tell flow-engine that an error is found and ask flow-engine
      to handle the error. flow-engine will perform the `Error Handling`
      procedure then.
    - .subscribe(event, cb): a task could subscribe some specific events.
      flow-engine would invoke the `cb` when the specific events are triggered.
      The `cb` shall be a `function(event, next)`. Inside the callback, next()
      must be called when the callback finishes. Currently, the following events
      are supported:
      - FINISH: a flow finishes. flow-engine finishes an assembly execution
      - ERROR: an error is threw from an assembly execution
      - pre|post:taskName: before or after a task's execution
    - .unsubscribe(event, cb): unsubscribe an event
    - .logger: a bunyan logger object and could be used for logging
  - if the task finishes without any error, then the flow engine goes to the
    next task

- **Error Handling**:
  - check if there is local error handling for the task:
    - if yes, then execute the tasks defined in the error handling:
      - when the local error handling finishes and the local handling throws
        another error, then go to global error handling
      - then the flow ends
    - if no, then see if there is any global error handling:
      - no global error handling : use default error handling and then ends the
        flow
      - global error handling exists: execute the global error handling and then
        ends the flow

### Event
flow-engine supports observable interface and provides `subscribe()` and
`unsubscribe()` APIs. Currently, the following events are supported:

- START: flow-engine starts to execute an assembly and none of the policy is
  executed yet
- FINISH: all of the policies in an assembly are executed successfully
- ERROR: the flow execution ends with an error
- 'pre|post:task-name': before or after the execution of a task/policy.
  The post event is **only** triggered when a task/policy finishes without any
  error, even if the 'catch' clause recovers the error.

In order to keep the flow execution is clean, any error that is thrown/created
by an event handler would only be logged and then ignored. Therefore, an event
handler wouldn't impact the flow execution as well as other event handlers.

You can access the `subscribe()` and `unsubscribe()` from a Flow instance if
you use the Flow ctor to create a flow instance. In side a task, the third
param of the task handler function is an flow object which contains
`subscribe()` and `unsubscribe()` APIs.

### The `context` object
The `context` object should be created before the flow-engine, probably by
using a context middleware before the flow-engine middleware. flow-engine uses
the `context` object as one of the arguments when invoking every task function.
A task could access `context` object, including retrieving and populating
properties. When flow-engine finishes, all the information should be stored into
the `context` object. Having a middleware after the flow-engine middleware is a
typical approach to produce the output content and maybe flush/write to the
response object at the same time.

Currently, flow-engine provides a context module in `./lib/context` which could
be used to create the context object. It provides the following APIs:
- createContext([namespace]):  
  create a context object with the specified namespace. namespace is optional.

- The context object provides getter, setter and dot notation to access its
  variables:
  - get(name):  
    get the variable by name.

- set(name, value[, readOnly]):    
    add or update a variable with the specified name and value. default value
    of readOnly is 'false'.
    Use readonly=true to add read-only variable.  
    Note: updating a read-only variable will cause the exception.

- define(name, getter[,setter, configurable]):
    - `name`: variable name
    - `getter`: getter function
    - `setter`: setter function
    - `configurable`: allow re-define or not. (true|false)

add or update a variable with the specified getter, setter and configurable.
    Use configurable=false to avoid variable re-define.  
    Note: set new value to a getter-only variable will cause the exception.

- del(name):  
    delete the variable by name.

- subscribe(event, handler):  
    subscribe an event with the handler function : `function(event, next)`.

- unsubscribe(event, handler):  
    unsubscribe the event

- notify(event, callback):  
    fire the event and all of the handlers are invoked sequentially (based on
    the registration sequence). Then `callback` is called with undefined or an
    array of errors that subscriber handlers return.

- dot notation:  
    You can also use dot notation to access the variables of a context object.
```javascript
        //If a context is created with a namespace - 'fool',
        //you can access its variables like this:    
        var ctx1 = require('./lib/context').createContext('fool');
        ctx1.set('myvar', 'value');
        ctx1.fool.myvar === 'value';
        ctx1.fool.myvar = 'new-value';
        ctx1.get('myvar') === 'new-value';
        //
        //If there is no namespace, then all of the variables are
        //directly attached to the context object:
        var ctx2 = require('./lib/context').createContext();
        ctx2.set('myvar', 'value');
        ctx2.myvar === 'value';
        ctx2.myvar = 'new-value';
        ctx2.get('myvar') === 'new-value';
```

flow-engine

flow-engine Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

flow-engine

flow-engine Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README