Plugins

Overview

Lumbar may be extended through plugins that can inject or modify the behavior at numerous different places in the build process. A chaining pattern is used so each plugin can either return, modify or replace the response from plugins that are later in the chain.

Installation

Plugins can be used in 2 ways. The first allows passing configuration options to the plugin which can be accessed as the first parameter if the plugin exports a function.

module.exports = function(options) {
  return {
    // plugin methods
  };
};

For plugins that do not require instance options, a singleton exports pattern can be used

module.exports = {
  // plugin methods
};

Modes

A mode is an operating context or filter so that only plugins that are registered for a given mode are allowed to operate - otherwise they are ignored.

The plugin can contribute new modes (or bind to existing modes) by exporting a mode value which can either be a string or an array of strings. Lumber operates with 3 defined modes by default:

• scripts: Compile and copy all javascript artifacts to build
• styles: Compile and copy all stylus and css artifacts to build
• static: Copy all static resources to build

Lumbar will iterate a build lifecycle for each unique platform, module and mode combination. This allows the plugins to filter build resources and operations to only what is meaningful for their purpose.

module.exports = {
  mode: 'scripts' // operate within the scripts mode along with other core plugins
  mode: ['scripts', 'foo'] // scripts mode and add a new mode called 'foo'
  mode: ['foo', 'all'] // add a new 'foo' mode and also operate under all modes
  // if no mode is defined, the plugin will operate under all modes
}

It is recommended that, unless necessary, a mode be explicitly defined.

Plugin API

Plugins primary method of interaction with Lumbar it through the moduleResources, resourceList, file, module, and resource callbacks.

Each of these callbacks are implemented as a chain, allowing for a plugin to modify the current context object and determine if subsequent plugins are allowed to operate, via the next parameter passed to the callback.

Almost all plugin methods are asynchronous and have the same signature - (context, next, complete)

• context: provides access to all data that a plugin should need.
• next: the chaining callback used to execute the remaining plugins
• complete: the completion callback

All parameters are described in more detail after the method documentation.

moduleResources moduleResources(context, next, complete)

Called when generating a list of all resources that a given module will need. This method may be used to add additional content to the module such as the router declaration for router modules.

The expected return value is an array. The contents of the array can be whatever is meaningful to the plugin. Other plugin methods can be used to take action on individual entries in the returned list.

Resource expansion

Each value in the returned array will be expanded if that value represents a directory structure. This is done by either using a simple string or using the src attribute. In this case, every child directory and file will automatically be added as a resource entry.

For example, if the application structure is:

app
  - lumbar.json
  - files
  --- file1.txt
  --- sub-files
  ----- file2.txt

And a resource entry is returned with the value of

{src: "files", foo: "bar"}

or

"files"

The resource entries will be converted to

[
  {dir: "files", foo: "bar"},
  {src: "files/file1.txt", srcDir: "files", foo: "bar"}
  {dir: "files/sub-files", srcDir: "files", foo: "bar"}
  {src: "files/sub-files/file2.txt", srcDir: "files", foo: "bar"}
]

The existence of srcDir to determine if the resource was auto-generated from a resource entry representing a directory.

Any additional attributes that were provided will be added to all created entries as you can see with the foo attribute.

Note: the foo attribute would not be present if the resource entry was just files - just {src: "files", foo: "bar"}.

Current behavior

Without implementing this method, the resources retrieved will be the serialized JSON value referenced by the mode key on the module.

For example, if the plugin has defined a mode called foo and a lumbar.json file of:

{
  "modules": {
    "myModule": {

      "foo": [
        "abc", "def"
      ],

      "bar": [
        "ghi", "jkl"
      ]
    }
  }
}

The resources available to the plugin would be:

["abc", "def"]

Example

If the plugin intends to use the 'bar' value (disregarding the fact that maybe the mode should be 'bar'), a sample moduleResources would be:

module.exports = {
  moduleResources: function(context, next, complete) {
    complete(undefined, context.module.bar);
  }
}

It is also possible to add to the module resources when multiple plugins operate within the same mode. Here is an example of the router plugin:

moduleResources: function(context, next, complete) {
  next(function(err, ret) {
    if (err) {
      return complete(err);
    }

    // Generate the router if we have the info for it
    var module = context.module;
    if (module.routes) {
      ret.unshift({ routes: module.routes });
    }

    complete(undefined, ret);
  });

resourceList resourceList(context, next, complete)

Allows plugins to create multiple resources from a single resource. This is called once for each resource generated from the moduleResources callback.

This is useful for plugins that expand on specific resources.

The expected return value is an array of resource objects. The data associated with these objects may be anything the plugin or other plugins will operate on.

Strings will be treated as file or directory includes as will object that define a src field. Resources that define a platform or platforms fields will be filtered based on the current platform being executed.

For example, the scope plugin wraps the returned resources add a execution scope.

resourceList: function(context, next, complete) {
  next(function(err, resources) {
    if (err) {
      return complete(err);
    }

    if (context.config.attributes.scope === 'resource'
        && !context.resource.global
        && !context.resource.dir) {
      resources.unshift(generator('(function() {\n'));
      resources.push(generator('}).call(this);\n'));
    }
    complete(undefined, resources);
  });
}

file file(context, next, complete)

Allows plugins to apply file-level changes to the resources. Called once for each file generated, just prior to resources being combined. May alter the context.resources field to change the resource list.

This could be used, for example, to append JSONP callbacks to a file.

fileName fileName(context, next, complete)

Allows for plugins to override the default file name used for output file creation.

The return value should be an object with the following attributes:

• path: the file path relative to the output directory (minus the extension)
• extension: the file extension

For example, the script plugin uses the platform path and module name to create the file name:

fileName: function(context, next, complete) {
  var name = context.module ? context.module.name : context.package;
  complete(undefined, {path: name, extension: 'js'});
}

module module(context, next, complete)

Allow plugins to apply module-level changes to the resources. Called once for each module. May alter the resource list associated with the module by altering the context.moduleResources field.

This can be useful for writing resources to the output directory. For example, this is how the static-output plugin adds the static files to the output directory:

module: function(context, next, complete) {
  next(function(err) {
    async.forEach(context.moduleResources, function(resource, callback) {
        var fileContext = context.clone();
        fileContext.resource = resource;
        var fileInfo = fu.loadResource(resource, function(err, data) {
          if (err || !data || !data.content) {
            return callback(err);
          }

          fileContext.outputFile(function(callback) {
            var ret = {
              fileName: fileContext.fileName,
              inputs: fileInfo.inputs || [ fileInfo.name ],
              fileConfig: context.fileConfig,
              platform: context.platform,
              package: context.package,
              mode: context.mode
            };

            fu.writeFile(fileContext.fileName, data.content, function(err) {
              callback(err, ret);
            });
          },
          callback);
        });
      },
      complete);
  });
}

resource resource(context, next, complete)

Allows plugins to include content other than direct file references as well as chain resource modifications.

The current resource can be referenced using context.resource.

In general, the plugin should have one of the following return values:

Return: callback function

This function is used for asynchronous data loading. The callback has the standard (err, data) signature

• err is used to indicate an error
• data can be a string or buffer representing file contents or a hash with the following values:
  • data: the string or buffer file contents
  • noSeparator: truthy - adds ';;' separator for when content is known to be validated javascript or css, Resources that always end in a complete statement should utilize this field.
  • inputs: a list of files that, if in watch mode, impact the generation of this file

For example, this is how the async callback function can be used to write "Hello World!"

resource: function(context, next, complete) {
  complete(undefined, function(context, complete) {
    if ( *simple* ) {
      complete(undefined, "Hello World!");
    } else {
      var dependantFiles = [...];
      complete(undefined, {data: "Hello World!", inputs: dependantFiles}
    }
  });
}

Return: An object

This object should have the following attributes: src: file path relative to the lumbar.json file dest: only applicable for static resources - the destination path relative to the platform * sourceFile: file path that, if in watch mode, should be watched to trigger a rebuild. This is not needed if src is defined.

Method parameters

Context

Each plugin method is passed a context parameter which describes the entire state of the build at the point of the call. Plugins are free to modify this structure as they please.

The context is cloned at various times during the lumbar lifecycle so any modifications to the context can not be guaranteed to exist outside of the plugin method that made the modification.

• package : The name of the package currently being operated on.
• platform : The name of the platform currently being operated on.
• module : The module currently being operated on, as defined in the JSON file.
• resource : The resource currently being operated on, if applicable. Definition depends on plugins.
• moduleResources : All resources associated with the current module, if available.
• resources : All resources associated with a file. For non-combined cases this is identical to moduleResources
• platformPath : Path to the output path for the current platform
• options : Options passed to the lumbar initialize call
• config : Current lumbar configuration. See config.js
• combined : Truthy if the output content is intended to be combined when possible

Some utility functions are also available:

• loadResource(resource, callback): Async method for retrieving file contents of a resource.

  • resource: the resource that is provided as context.resource in the resource method

  • callback: async callback method with the following parameters:

    • err: error if anything went wrong

    • data: buffer or what was returned if the resource provided was a function

• outputFile(writer, callback): write content to a file

  • writer:
  • callback: FIXME: Kevin, can you document the parameter usage?

Next and Complete

Each plugin is responsible for completing the plugin chain by calling next() or compete(). Next is called to let the other plugins respond while complete is used to stop the plugin chain and directly return a result.

The complete callback can be provided as a parameter to next if desired but not necessary.

see examples below:

module.exports = {
  moduleResources: function(context, next, complete) {
    if ( *continue with chain* ) {
      next();

    } else if ( *modify plugin result* ) {
      // define a new complete function
      function _complete (err, data) {
        if (err) {
          // something bad happened
          complete(err, data);
        } else {
          data.push("something new");
          complete(undefined, data);
        }
      }
      // call next and override the existing complete function
      next(_complete);

    } else if ( *stop the plugin chain and return something* ) {
      var something = [...];
      complete(undefined, something);

    } else {
      // we're asyncronous - *always* make sure to call next or complete!
      next();
    }
  }
}

Lifecycle Pseudocode

For an understanding of how these methods work together, see the following extremely simplified pseudocode:

for each defined platform
  for each mode {added by `plugin.mode`}
    for each module in platform {as determined by package}
      resources = `plugin.moduleResources`
      for each resource in resources
        if resource matches `plugin.fileFilter`
          replace/expand resource if it matches a directory
        else
          remove from the list of resources

      for each resource in resources
        replace/flatten resource with `plugin.resourceList`

      call `plugin.module`
      for each resource in resources
        resource = 'plugin.resource'

Caches

Each context object defines a variety of caches that are reset at specific points through the build process. This allows plugins to cache any relevant data for specific timeframes. Note that these objects are shared across all plugins so proper naming conventions should be followed to prevent conflicts.

• configCache : Reset when the configuration file changes
• fileCache : Reset when the current file processing completes
• moduleCache : Reset when the current module processing completes

Warnings

As most Lumbar projects are dealing with a large number of files it is quite susceptible to EMFILE exceptions under OSX. The current recovery method for this is to utilize async methods and retry methods that fail due to this error. A variety of file methods that are protected from this case have been made available on the lumbar.fileUtil object. It is recommended that these methods are used whenever possible while dealing with files throughout the system.

FileUtils

With respect to the previous warning about EMFILE, all file access should be done using fileUtils (fileUtils.js). This should be accessed from the context using the fileUtil key. This wraps much of the functionality of fs with handling of EMFILE errors.

FileUtils also caches files that are referenced to optimize build time.

resetCache resetCache(path)

Clear all cached file content

• path: Clear all or clear for a specific path. falsy or missing input for path will clear all.

resolvePath resolvePath(path)

Return a file path that, if relative, is appropriatly qualitied with the build output path based on the 'lookupPath'

• path: the file path

readFileSync readFileSync(path)

Same as fs.readFileSync but uses resolvePath

• path: the file path

makeRelative makeRelative(path)

The opposite of resolvePath. This will remove the lookup path if the path has that as a prefix.

stat stat(file, callback)

Same as fs.stat but with EMFILE handling

• file: the file path
• callback: the asynchronous callback

readFile readFile(file, callback)

Same as fs.readFile cacheing. A buffer is returned.

• file: the file path
• callback: the asynchronous callback

readdir readdir(dir, callback)

same as fs.readdir with cacheing.

• dir: the directory path
• callback: the asynchronous callback

ensureDirs ensureDirs(pathname, callback)

Ensure that the parent directories for the provided file path exist and create otherwise.

• pathname: the file path
• callback: the asynchronous callback

writeFile writeFile(file, data, callback)

Same as fs.writefile but will also ensure directories, cache file contents, and handle EMFILE errors gracefully.

• file: the file path
• data: the file contents
• callback: the asynchronous callback

loadResource loadResource(resource, callback)

Specifically designed to load a lumbar resource (see the lumbar API resource method).

• resource: the lumbar resource
• callback: the asynchronous callback