KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#typescript #ai-agents #developer-tools #cli #codex #claude-code #react #mcp #open-source #deepseek-harness #dsh-plugin #ai

docker-clamav-malware-scanner

★ 92 Open GitHub ↗

No description available.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Malware Scanner Service

This repository contains the code to build a pipeline that scans objects
uploaded to GCS for malware, moving the documents to a clean or quarantined
bucket depending on the malware scan status.

It illustrates how to use Cloud Run and Eventarc to build such a pipeline.

![Architecture diagram](https://raw.githubusercontent.com/GoogleCloudPlatform/docker-clamav-malware-scanner/main/architecture.svg)

## How to use this example

Use the
[tutorial](https://cloud.google.com/solutions/automating-malware-scanning-for-documents-uploaded-to-cloud-storage)
to understand how to configure your Google Cloud Platform project to use Cloud
Run and Eventarc.

## Using Environment variables in the configuration

The tutorial above uses a configuration file `config.json` built into the Docker
container for the configuration of the unscanned, clean, quarantined and CVD
updater cloud storage buckets.

Environment variables can be used to vary the deployment in 2 ways:

### Expansion of environment variables

Any environment variables specified using shell-format within the `config.json`
file will be expanded using
[`envsubst`](https://manpages.debian.org/bookworm/gettext-base/envsubst.1.en.html).

### Passing entire configuration as environment variable

An alternative to building the configuration file into the container is to use
environmental variables to contain the configuration of the service, so that
multiple deployments can use the same container, and configuration updates do
not need a container rebuild.

This can be done by setting the environmental variable `CONFIG_JSON` containing
the JSON configuration, which will override any config in the `config.json`
file.

If using the `gcloud run deploy` command line, this environment variable must be
set using the
[`--env-vars-file`](https://cloud.google.com/sdk/gcloud/reference/run/deploy#--env-vars-file)
argument, specifying a YAML file containing the environment variable definitions
(This is because the commas in JSON would break the parsing of `--set-env-vars`)

Take care when embedding JSON in YAML - it is recommended to use the
[Literal Block Scalar style](https://yaml-multiline.info/) using `|`, as this
preserves newlines and quotes

For example, the `CONFIG_JSON` environment variable could be defined in a file
`config-env.yaml` as follows:

```yaml
CONFIG_JSON: |
  {
    "buckets": [
      {
        "unscanned": "unscanned-bucket-name",
        "clean": "clean-bucket-name",
        "quarantined": "quarantined-bucket-name"
      }
    ],
    "ClamCvdMirrorBucket": "cvd-mirror-bucket-name",
    "fileExclusionPatterns": [],
    ignoreZeroLengthFiles: false
  }
```

An example commandline using this file to specify the environment:

```sh
gcloud beta run deploy "${SERVICE_NAME}" \
  --source . \
  --region "${REGION}" \
  --no-allow-unauthenticated \
  --memory 4Gi \
  --cpu 1 \
  --concurrency 20 \
  --min-instances 1 \
  --max-instances 5 \
  --no-cpu-throttling \
  --cpu-boost \
  --service-account="${SERVICE_ACCOUNT}" \
  --env-vars-file=config-env.yaml
```

If you are using Terraform to deploy, then the equivalent way to specify the
environment variable using the
[`google_cloud_run_v2_service`](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_v2_service)
resource is by using the
[env](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_v2_service#env)
block and
[jsonencode](https://developer.hashicorp.com/terraform/language/functions/jsonencode):

```tf
resource "google_cloud_run_v2_service" "malware-scanner" {
  name = "malware-scanner"
  // other service parameters...
  template {
    // other template parameters...
    containers {
      // other container parameters...
      env {
        name = "CONFIG_JSON"
        value = jsonencode({
          buckets = [
            {
              unscanned   = "unscanned-bucket-name",
              clean       = "clean-bucket-name",
              quarantined = "quarantined-bucket-name"
            }
          ]
          ClamCvdMirrorBucket = "cvd-mirror-bucket-name",
          fileExclusionPatterns = [],
          ignoreZeroLengthFiles = false
        })
      }
    }
  }
}
```

## Notes on `fileExclusionPatterns`

The `fileExclusionPatterns` array in the config file can be used to ignore any
uploaded files matching a
[Regular Expression](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Regular_expressions).

This can be used for example if you have an upload system that creates temporary
files, then renames them once the files are fully uploaded.

The elements in the `fileExclusionPatterns` array can either be simple strings,
for example:

```json
"fileExclusionPatterns": [
  "\\.tmp$",
  "^ignore_me.*\\.txt$"
]
```

or they can be an array of 2 string values, allowing regular expression flags to
be specified, for example `"i"` for case-insensitive matches:

```json
"fileExclusionPatterns": [
  [ "\\.tmp$", "i" ],
  [ "tempfile.*.upload$", "i" ]
]
```

Files matching these patterns will be ignored by the scanner, and left in the
`unscanned` bucket, and an `ignored-files` counter incremented.

Helpful tools for regular expressions include the
[Regular Expression Cheatsheet](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions/Cheatsheet),
and the [Regex101](https://regex101.com/r/QK47Hp/1) playground (ensure
ECMAScript flavor is selected).

Note that when adding regular expressions into the config file, care must be
taken with `\` and `"` characters -- any of these characters in the regular
expression must be escaped with another `\`.

## Notes on `quarantine` config block

The `quarantine` block in the config configures auto-quarantine of files which
do not have any malware detected, depending on certain factors:

### `quarantine.encryptedFiles`

(Default: `true`)

This enables ClamAV's encrypted file detection
([`AlertEncrypted` setting](https://manpages.debian.org/testing/clamav-daemon/clamd.conf.5.en.html#AlertEncrypted)),
which treats encrypted archive or docuemnt files as containing malware. This is
enabled by default as it is impossible to scan the contents of encyrpted files
for malware.

Encrypted files will be quarantined as if they were malware with a log line
indicating that they were infected with `Heuristics.Encrypted.xxx`.

### `quarantine.fileExtensionAllowList`

(Default: `[]` -- allow all.)

A list of allowed file extensions. Files not having these extensions will be
quarantined as if they were malware with a log line indicating that they were
infected with `Config.AllowList.Blocked`.

If no extensions are specified all files are allowed.

File extensions are case-insensitive. An empty string: `""` matches files with
no extension. Specifying double-extensions (eg `"tar.gz"` is supported).

Example: `[ "doc", "pdf", "jpg" ]`

### `quarantine.fileExtensionDenyList`

(Default: `[]` -- deny nothing.)

A list of denied file extensions. Files having these extensions will be
quarantined as if they were malware with a log line indicating that they were
infected with `Config.DenyList.Blocked`.

If no extensions are specified all files are allowed.

File extensions are case-insensitive. An empty string: `""` matches files with
no extension. Specifying double-extensions (eg `"tar.gz"` is supported).

Example: `[ "zip", "tar.gz", "jpg" ]`

## Change history

See [CHANGELOG.md](cloudrun-malware-scanner/CHANGELOG.md)

## Upgrading from v2.x to v3.x

In Version 3.x, the metrics reporting was changed to OpenTelemetry which uses a
different naming convention for metrics, so the metric names have changed from:

```text
custom.googleapis.com/opencensus/malware-scanning/METRIC-NAME
```

```text
workload.googleapis.com/googlecloudplatform/gcs-malware-scanning/METRIC-NAME
```

Any dashboards or alerts using these metrics must be updated

## Upgrading from v1.x to v2.x

Version 2 has a different way of handling ClamAV updates to avoid issues with
the ClamAV content distribution network.

See [upgrade_from_v1.md](upgrade_from_v1.md) for upgrading instructions.

## License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use
this file except in compliance with the License. You may obtain a copy of the
License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
```

docker-clamav-malware-scanner

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README