nextflow-test-pipelines

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #typescript #llm #go #android #machine-learning #deep-learning #cli #kubernetes #vue #ios

nextflow-test-pipelines

(★ 11)

Nextflow, WDL and other bioinformatics pipelines for testing our pipeline monitoring platform

nextflow-test-pipelines Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center" style="border-bottom: none">
 <a href="https://tracer.cloud" target="_blank"><img alt="Tracer" src="https://github.com/user-attachments/assets/5bbbdcee-11ca-4f09-b042-a5259309b7e4"></a> Nextflow Pipelines with Tracer Observability
</h1>

Try <a href="https://sandbox.tracer.cloud" target="_blank">tracer.cloud</a> instantly in the sandbox environment.

---

Tracer is a system-level observability platform purpose-built for scientific computing. With a one-line Linux client install and instant dashboards, Tracer gives you deep insights into pipeline performance and cost optimization.

---

## Why Tracer?

- **Reliable Nextflow pipelines:** Fully-tested examples that run seamlessly across all environments
- **Automated CI/CD:** Regularly validated pipelines for consistent functionality
- **Easy Installation:** Automated scripts for quick setup
- **Fast Community Support:** Open an issue and get a response within 24 hours
- **GitHub Codespaces Compatible:** Run examples directly in Codespaces

## Quickstart (Sandbox)

Get started instantly:

1. **Visit the [Tracer Sandbox](https://sandbox.tracer.cloud/)**
2. Follow the onboarding instructions to launch your first monitored pipeline

---

## Getting Started

### 1. Install Tracer

Install Tracer on your system (one-time setup):

> **Note:** Root privileges required

```bash
curl -sSL https://install.tracer.cloud | sh
```

### 2. Initialize the Tracer Client (requires token)

Start the Tracer client to initialize a pipeline and enable monitoring. You must provide your Tracer token every time you initialize.

- Get your token from the [Tracer Sandbox](https://sandbox.tracer.cloud/)
- Then run:

> **Note:** Root privileges required

```bash
sudo tracer init --token <paste-your-token-here> --watch-dir "/tmp/tracer"
```

### 3. Run a Pipeline

You can use your own pipelines or try our ready-to-run examples. For the smoothest onboarding, follow the sandbox instructions or pick a pipeline for your OS below:

- **[Codespaces](./pipelines/codespaces/)**
- **[Linux x86_64](./pipelines/)**
  - [Ubuntu](./pipelines/linux-x86-ubuntu/)
  - [Amazon Linux](./pipelines/linux-arm-amazon-linux/)
- **[Linux aarch64](./pipelines/)**
  - [Ubuntu](./pipelines/linux-arm-ubuntu/)
  - [Amazon Linux](./pipelines/linux-arm-amazon-linux/)
- **[macOS](./pipelines/)**
  - [Apple Silicon / arm64](./pipelines/macos-arm64/)
  - [Apple Intel / x86](./pipelines/macos-intel-x86/)
- **[AWS Batch](./pipelines/aws-batch/)**

We provide a variety of examples for different compute environments, written in Bash, Python, Nextflow, WDL, and CWL.

**Recommended:** Start with a simple Nextflow fastquorum pipeline.

---

## Monitor Your Pipeline

Track your pipeline's progress in real time via the Tracer dashboard. Access it through the 'Open Grafana Dashboard' button during onboarding.

The dashboard provides:

- Execution metrics
- Pipeline stages
- Status updates

---

## What Makes Tracer Different?

Tracer is a cutting-edge observability platform designed for scientific computing. Unlike general-purpose monitoring tools, Tracer is purpose-built for scientific pipelines, offering:

- Intelligent organization and labeling of pipelines, runs, and steps
- Zero code changes: runs directly on Linux
- Support for any programming language
- Seamless integration with diverse environments (AlphaFold, Slurm, Airflow, Nextflow, Bash, and more)
- Enterprise-grade security: your data never leaves your infrastructure

This is especially valuable for environments like AWS Batch, where tracking processes across containers is challenging and logs can be lost.

---

## Key Features

- Time and cost per dataset processed
- Step-level execution duration and bottleneck identification
- Cost attribution across pipelines, teams, and environments (dev, CI/CD, prod)

These insights help you optimize complex scientific toolchains that traditionally lack proper observability.

---

## Mission

> "_The goal of Tracer's Rust client is to equip scientists and engineers with DevOps intelligence to efficiently harness massive computational power for humanity's most critical challenges._"

---

For more details, see the [tracer client](https://github.com/Tracer-Cloud/tracer-client) and our [website](https://tracer.cloud).

nextflow-test-pipelines

nextflow-test-pipelines Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

nextflow-test-pipelines

nextflow-test-pipelines Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README