KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #ai-agents #developer-tools #codex #claude-code #mcp #rust #deepseek-harness #typescript #ai-agent #cli

serverless-ephemeral

★ 67 Open GitHub ↗

This is a Serverless Framework plugin that helps bundling any stateless zipped library to AWS Lambda.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Serverless Ephemeral

[![NPM Version][npm-image]][npm-url]
[![Build Status][travis-image]][travis-url]

![Serephem](https://raw.githubusercontent.com/Accenture/serverless-ephemeral/master/img/logo192.png)

Serverless Ephemeral (or Serephem) is a [Serverless Framework plugin](https://serverless.com/framework/docs/providers/aws/guide/plugins/) that helps bundle any stateless library into a Lambda deployment artifact.

## Pre-requirements

- Node >= 6.9
- Serverless Framework >= 1.12.0
- Docker (with docker compose)

## Examples

- [TensorFlow Lambda (CPU only, Python 2.7)](examples/tensorflow-lambda): Pulls in or builds (via Docker) a TensorFlow package. For reference on how to manually build a TensorFlow package for Lambdas, see [docs/build-tensorflow-package.md](docs/build-tensorflow-package.md).

- [Image Processor](examples/image-processor): Builds and adds a Python [Pillow](https://python-pillow.org/) package in order to resize an image uploaded to an S3 bucket.

## Add the plugin

1. Install it

```bash
   npm i --save-dev serverless-ephemeral
   ```

1. Add it to your `serverless.yml` file and exclude the `.ephemeral` directory

```yaml
   plugins:
     - serverless-ephemeral

package:
     exclude:
       - package.json
       - package-lock.json
       - node_modules/**
       - .ephemeral/**
   ```

1. Add the `.ephemeral` directory to `.gitignore`

```text
# Serverless Framework
.serverless
.ephemeral
```

## Configuration

The configuration for the Ephemeral plugin is set inside the `custom` section of the `serverless.yml` file. In it, you can define the list of stateless libraries you wish to pull into the final Lambda artifact.

There are two types of configuration:

- [Build a library during runtime](#build-a-library)
- [Download a library](#download-a-library)

Both can be enhanced with [global configuration options](#global-options).

### Build a library

You can build a specific library during runtime. This is achieved via a Docker container that outputs a zip library.

The Serepehm plugin provides some useful packagers out of the box. However, you can create your own packager via Docker files.

#### Serephem packagers

You can use one of the Docker packagers provided with the Serephem plugin.

##### TensorFlow

```yaml
custom:
  ephemeral:
    libraries:
      - packager:
          name: tensorflow
          version: 1.8.0
```

- **packager.name** is required. This is the packager name identifier for TensorFlow: **tensorflow**
- **packager.version** is required. This will determine which TensorFlow version you want to build.

#### Build your own packager

You can create your own packager via Docker. To do so:

1. Create a directory where you will store all your Docker files:

```bash
   mkdir my-packager
   cd my-packager
   ```

1. Create a `docker-compose.yml` file. For example:

```yaml
   version: "3"
   services:
     packager:
       build: .
   ```

Keep note of the name of your packager service, in this case `packager`.

1. Create a `Dockerfile` and any other support files. For example:

**`Dockerfile`**

```yaml
   FROM amazonlinux

COPY scripts/build.sh scripts/build.sh
   RUN yum -y install zip && \
   chmod +x scripts/build.sh

CMD [ "scripts/build.sh" ]
   ```

**`scripts/build.sh`**

```bash
   # create zip destination directory
   mkdir -p /tmp/lambda-libraries

# download library files
   mkdir /tmp/files
   cd /tmp/files
   curl http://example.com/file-1.py --output file-1.py
   curl http://example.com/file-2.py --output file-2.py
   zip -9rq /tmp/lambda-libraries/library-a.zip *
   ```

**IMPORTANT**: the container must generate a zip file containing the stateless library files. Thus:

- Your container must zip the stateless library files.

- You must create a directory where the final zip(s) will be stored. This directory will be mounted to the Serephem's libraries directory, so add only the necessary zip files.

- It is recommended that your Docker container extends from `amazonlinux` image to maximize compatibility with the Lambda environment.

1. Add this configuration to your `serverless.yml`:

```yaml
   custom:
     ephemeral:
       libraries:
         - packager:
             compose: my-packager/docker-compose.yml
             service: packager
             output: /tmp/lambda-libraries/library-a.zip
   ```

Notice how each of the values correspond to a setting previously created:

- `compose`: points to your Docker compose file, inside the directory you created

- `service`: the name of the service inside the `docker-compose.yml` file

- `output`: the output path for the zip file in the Docker container

### Download a library

You can download a previously zipped library that contains all the necessary files and automatically add it to your Lambda deployment.

```yaml
custom:
  ephemeral:
    libraries:
      - url: https://xxxxx.s3.amazonaws.com/my-library.zip
```

- **url** is required. This is the packaged library you want to include. The library must be a zip file.

> Documentation explaining how to create a deployable TensorFlow zipped package can be found here: [docs/build-tensorflow-package.md](docs/build-tensorflow-package.md). This approach can be used as a base to create other stateless libraries.

### Global options

```yaml
custom:
  ephemeral:
    libraries:
      - packager:
          name: tensorflow
          version: 1.8.0
        directory: tfpackage
      - url: https://xxxxx.s3.amazonaws.com/my-library.zip
        nocache: true
```

- **directory** is optional. When ommitted, the package contents will be unzipped at service root level. If entered, a new folder will be created at that level using the specified name and everything will be unzipped there. The folder can only be named using alphanumeric characters and the symbols `. _ -`

- **nocache** is optional. When ommitted or set to _false_, it will use the locally cached copy of the library. Otherwise, if set to _true_, it will re-fetch (download or build) the library every time the service is packaged.

> Note: the **forceDownload** option has been deprecated in favor of **nocache**

## Deploy

Deploy your service normally with the `serverless deploy` (or `sls deploy`) command. If you use the `-v` option, Ephemeral will show more information about the process.

```bash
   sls deploy -v
   ```

> If the Serverless deployment is timing out, use the `AWS_CLIENT_TIMEOUT` environment variable: <https://github.com/serverless/serverless/issues/490#issuecomment-204976134>

### The .ephemeral directory

During the deployment process, the `.ephemeral` directory will be created. The purpose of this directory is:

- Saving the libraries' zip files inside the `.ephemeral/lib` folder
- Bundling the libraries and the Serverless Lambda function file(s) inside the `.ephemeral/pkg` folder

---

## Development

This plugin is created with Node and uses the Serverless Framework hooks to execute the necessary actions.

### Getting started

1. Clone this repo
2. Install dependencies via `npm i`

### Running Lint

The plugin code uses the AirBnB ESLint rule set with some enhancements (see `.eslintrc` file). To run the linter:

```bash
npm run lint
```

### Tests

The unit tests are coded with [Ava](https://github.com/avajs/ava) and [SinonJS](http://sinonjs.org/docs/). They can be found inside the `spec` folder.

To run the tests:

```bash
npm test
```

To run tests on "watch" mode and add verbosity:

```bash
npm test -- --watch -v
```

[npm-image]: https://img.shields.io/npm/v/serverless-ephemeral.svg
[npm-url]: https://npmjs.org/package/serverless-ephemeral
[travis-image]: https://img.shields.io/travis/Accenture/serverless-ephemeral/master.svg
[travis-url]: https://travis-ci.org/Accenture/serverless-ephemeral

### Test via examples

Refer to the [`examples`](examples) directory, for instance the [`TensorFlow example`](examples/tensorflow-lambda/README.md).

serverless-ephemeral

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README