probitas

KO | EN

gitlite — search

#javascript #python #hacktoberfest #golang #android #machine-learning #deep-learning #macos #java #windows #kubernetes #awesome-list #vue #open-source

(★ 21)

Scenario-based testing & workflow execution framework for programmers.

probitas Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<h1 align="center">Probitas</h1>

[![JSR](https://jsr.io/badges/@probitas/probitas)](https://jsr.io/@probitas/probitas)
[![Test](https://github.com/probitas-test/probitas/actions/workflows/test.yml/badge.svg)](https://github.com/probitas-test/probitas/actions/workflows/test.yml)
[![codecov](https://codecov.io/github/probitas-test/probitas/graph/badge.svg?token=Yu0GPZAMv6)](https://codecov.io/github/probitas-test/probitas)
[![Docs](https://img.shields.io/badge/docs-probitas--test.github.io-blue)](https://probitas-test.github.io)

Scenario-based testing & workflow execution framework.

## Overview

Probitas is a comprehensive framework for scenario-based testing and workflow
execution, providing:

- **Type-safe scenario builder** - Fluent API for defining test workflows with
  automatic type inference
- **Powerful CLI** - Command-line interface for running, listing, and managing
  scenarios
- **Rich client ecosystem** - Pre-configured clients for HTTP, databases,
  message queues, and more
- **Flexible execution** - Run scenarios in parallel or sequentially with
  configurable concurrency
- **Smart assertions** - Client-specific expectation utilities for comprehensive
  testing

## Installation

### Local Installation

#### Using Install Script

```bash
curl -fsSL https://raw.githubusercontent.com/probitas-test/probitas/main/install.sh | bash
```

Options via environment variables:

```bash
# Install specific version
curl -fsSL https://raw.githubusercontent.com/probitas-test/probitas/main/install.sh | PROBITAS_VERSION=0.7.1 bash

# Install to custom directory
curl -fsSL https://raw.githubusercontent.com/probitas-test/probitas/main/install.sh | PROBITAS_INSTALL_DIR=/usr/local/bin bash
```

#### Using Homebrew

```bash
# Add the tap
brew tap probitas-test/tap

# Install probitas
brew install probitas

# Upgrade to latest version
brew upgrade probitas
```

See [probitas-test/homebrew-tap](https://github.com/probitas-test/homebrew-tap)
for more details.

#### Using Nix

```bash
# Run without installing
nix run github:probitas-test/probitas

# Install into your profile
nix profile install github:probitas-test/probitas
```

Or add to your project's `flake.nix`:

```nix
{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    probitas-flake.url = "github:probitas-test/probitas";
    flake-utils.url = "github:numtide/flake-utils";
  };

outputs = { nixpkgs, probitas-flake, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ probitas-flake.overlays.default ];
        };
      in {
        devShells.default = pkgs.mkShell {
          packages = with pkgs; [ probitas ];
        };
      });
}
```

Then enter the development shell:

```bash
nix develop
probitas run
```

### GitHub Actions

#### Using setup-probitas Action

The recommended way to use Probitas in GitHub Actions:

```yaml
- uses: probitas-test/setup-probitas@v1
  with:
    version: latest # or specific version like '0.7.1'

- name: Run scenarios
  run: probitas run
```

See
[probitas-test/setup-probitas](https://github.com/probitas-test/setup-probitas)
for full documentation.

#### Using Nix in GitHub Actions

For projects using Nix flakes with
[nixbuild/nix-quick-install-action](https://github.com/nixbuild/nix-quick-install-action):

```yaml
- uses: nixbuild/nix-quick-install-action@v34

- name: Run scenarios
  run: nix run github:probitas-test/probitas -- run
```

Or within a Nix development shell:

```yaml
- uses: nixbuild/nix-quick-install-action@v34

- name: Run scenarios
  run: nix develop -c probitas run
```

## Quick Start

### 1. Create Your First Scenario

Create `probitas/hello.probitas.ts`:

```typescript
import { client, expect, scenario, Skip } from "jsr:@probitas/probitas@^0";

const apiUrl = Deno.env.get("API_URL");

export default scenario("User API Test", { tags: ["api", "user"] })
  .step("Check API availability", () => {
    if (!apiUrl) {
      throw new Skip("API_URL not configured");
    }
  })
  .resource("http", () => client.http.createHttpClient({ url: apiUrl! }))
  .step("Create user", async (ctx) => {
    const response = await ctx.resources.http.post("/users", {
      body: { name: "Alice", email: "alice@example.com" },
    });
    expect(response).toBeOk().toHaveStatus(201);
    return response.json as { id: string };
  })
  .step("Verify user exists", async (ctx) => {
    const { id } = ctx.previous!;
    const response = await ctx.resources.http.get(`/users/${id}`);
    expect(response).toBeOk().toHaveJsonProperty("name", "Alice");
  })
  .build();
```

> [!NOTE]
> **Enable Deno LSP in your editor** for the best development experience. With
> Deno LSP enabled, you'll get:
>
> - Auto-completion for Probitas APIs (`scenario`, `expect`, `client`, etc.)
> - Type checking and inline error detection
> - Jump to definition for functions and types
> - Hover documentation for all APIs
>
> See
> [Deno's editor setup guide](https://docs.deno.com/runtime/getting_started/setup_your_environment/)
> for instructions on enabling Deno in VS Code, Vim, Neovim, and other editors.

### 2. Run Your Scenarios

```bash
# Run all scenarios
probitas run

# Run scenarios with specific tag
probitas run -s tag:api

# Re-run only failed scenarios from previous run
probitas run --failed

# Run with JSON output
probitas run --reporter json

# List available scenarios
probitas list

# Initialize configuration file
probitas init
```

## Key Concepts

### Scenarios

A scenario is a sequence of steps that execute in order. Each step can:

- Return a value that's passed to the next step via `ctx.previous`
- Access all previous results via `ctx.results`
- Share state via `ctx.store`
- Use resources defined at the scenario level
- Have custom timeout and retry configuration

### Resources

Resources are lifecycle-managed objects with automatic cleanup (Disposable
pattern):

```typescript
import { client, expect, scenario } from "jsr:@probitas/probitas@^0";

export default scenario("Database Query Test", { tags: ["db"] })
  .resource("db", () =>
    client.sql.postgres.createPostgresClient({
      url: Deno.env.get("DATABASE_URL")!,
    }))
  .step("Insert test data", async (ctx) => {
    const result = await ctx.resources.db.query(
      "INSERT INTO users (name) VALUES ($1) RETURNING id",
      ["TestUser"],
    );
    expect(result).toBeOk().toHaveRowCount(1);
    return { userId: result.rows![0].id };
  })
  .step("Query inserted user", async (ctx) => {
    const { userId } = ctx.previous!;
    const result = await ctx.resources.db.query(
      "SELECT * FROM users WHERE id = $1",
      [userId],
    );
    expect(result)
      .toBeOk()
      .toHaveRowCount(1)
      .toHaveRowsMatching({ name: "TestUser" });
  })
  .build();
// Resource is automatically disposed (connection closed) after scenario
```

### Setup with Cleanup

For side effects that need cleanup without creating a reusable client:

```typescript
import { client, expect, scenario } from "jsr:@probitas/probitas@^0";

export default scenario("Redis Cache Test", { tags: ["redis", "cache"] })
  .resource("redis", () =>
    client.redis.createRedisClient({
      url: Deno.env.get("REDIS_URL")!,
    }))
  .setup(async (ctx) => {
    // Create test keys before scenario
    await ctx.resources.redis.set("test:counter", "0");
    // Return cleanup function
    return async () => {
      await ctx.resources.redis.del(["test:counter"]);
    };
  })
  .step("Increment counter", async (ctx) => {
    const result = await ctx.resources.redis.incr("test:counter");
    expect(result).toBeOk().toHaveValue(1);
  })
  .step("Verify counter value", async (ctx) => {
    const result = await ctx.resources.redis.get("test:counter");
    expect(result).toBeOk().toHaveValue("1");
  })
  .build();
```

### Tags

Organize scenarios with tags for filtering:

```typescript
import { client, expect, scenario } from "jsr:@probitas/probitas@^0";

export default scenario("Login Flow", { tags: ["auth", "critical", "e2e"] })
  .resource(
    "http",
    () => client.http.createHttpClient({ url: Deno.env.get("API_URL")! }),
  )
  .step("Login with valid credentials", async (ctx) => {
    const response = await ctx.resources.http.post("/auth/login", {
      body: { email: "test@example.com", password: "secret" },
    });
    expect(response).toBeOk().toHaveStatus(200);
    return response.json as { token: string };
  })
  .step("Access protected resource", async (ctx) => {
    const { token } = ctx.previous!;
    const response = await ctx.resources.http.get("/profile", {
      headers: { Authorization: `Bearer ${token}` },
    });
    expect(response).toBeOk().toHaveJsonProperty("email", "test@example.com");
  })
  .build();
```

Run with tag filters:

```bash
probitas run -s tag:auth
probitas run -s "tag:critical,tag:auth"  # AND logic
probitas run -s "!tag:slow"              # NOT logic
```

## Configuration

Create a `probitas.json` file in your project root:

```json
{
  "includes": ["probitas/**/*.probitas.ts"],
  "excludes": ["**/*.skip.probitas.ts"],
  "reporter": "list",
  "maxConcurrency": 4,
  "maxFailures": 0,
  "timeout": "30s",
  "selectors": ["!tag:wip"]
}
```

Configuration options:

- `includes` - Glob patterns for scenario files to include (default:
  `["**/*.probitas.ts"]`)
- `excludes` - Glob patterns for scenario files to exclude (default: `[]`)
- `reporter` - Output format: `list` or `json` (default: `list`)
- `maxConcurrency` - Maximum number of scenarios to run in parallel (default:
  number of CPU cores)
- `maxFailures` - Stop execution after N failures, 0 for unlimited (default:
  `0`)
- `timeout` - Default timeout for scenarios (e.g., `30s`, `5m`, `1h`)
- `selectors` - Default selectors to filter scenarios (e.g., `["tag:api"]`,
  `["!tag:slow"]`)

Generate a default configuration:

```bash
probitas init
```

## Available Clients

Probitas provides pre-configured clients for common services:

- **HTTP** - REST API testing (`client.http`)
  - **OIDC Authentication** - OpenID Connect authentication for HTTP clients
    (`client.http.oidc`)
- **GraphQL** - GraphQL query testing (`client.graphql`)
- **ConnectRPC** - RPC service testing (`client.connectrpc`)
- **gRPC** - gRPC service testing (`client.grpc`)
- **SQL Databases** - PostgreSQL, MySQL, SQLite, DuckDB (`client.sql.*`)
- **Redis** - Redis operations (`client.redis`)
- **MongoDB** - MongoDB operations (`client.mongodb`)
- **Deno KV** - Deno's key-value store (`client.denoKv`)
- **RabbitMQ** - Message queue operations (`client.rabbitmq`)
- **AWS SQS** - SQS queue operations (`client.sqs`)

Each client comes with specialized assertion utilities via the `expect()`
function.

### OIDC Authentication

The HTTP client supports OpenID Connect (OIDC) authentication for testing APIs
that require OAuth 2.0/OIDC authentication:

```typescript
import { client, expect, scenario } from "jsr:@probitas/probitas@^0";

export default scenario("OIDC Authentication Test", { tags: ["auth", "oidc"] })
  .resource(
    "http",
    async () =>
      await client.http.oidc.createOidcHttpClient({
        url: Deno.env.get("API_URL")!,
        oidc: {
          issuer: Deno.env.get("OIDC_ISSUER")!,
          clientId: "my-client-id",
        },
      }),
  )
  .step("Login with OIDC", async (ctx) => {
    const result = await ctx.resources.http.login({
      type: "authorization_code",
      username: "user@example.com",
      password: "password",
    });
    expect(result.ok).toBe(true);
  })
  .step("Access protected resource", async (ctx) => {
    // Authorization header is automatically added
    const response = await ctx.resources.http.get("/protected");
    expect(response).toBeOk().toHaveStatus(200);
  })
  .build();
```

The OIDC client supports:

- **Automatic token discovery** - Discovers OIDC endpoints from issuer URL
- **Manual configuration** - Or configure `authUrl` and `tokenUrl` directly
- **Authorization Code Grant** - For user authentication flows
- **Automatic token management** - Handles token storage and Authorization
  header injection

## Expectation API

Probitas provides client-specific expectation functions:

```typescript
import { client, expect, scenario } from "jsr:@probitas/probitas@^0";

export default scenario("E-Commerce Order Flow", { tags: ["e2e", "order"] })
  .resource(
    "http",
    () => client.http.createHttpClient({ url: Deno.env.get("API_URL")! }),
  )
  .resource("db", () =>
    client.sql.postgres.createPostgresClient({
      url: Deno.env.get("DATABASE_URL")!,
    }))
  .step("Create order via API", async (ctx) => {
    const response = await ctx.resources.http.post("/orders", {
      body: { items: [{ productId: "prod-1", quantity: 2 }] },
    });
    // HTTP-specific assertions
    expect(response)
      .toBeOk()
      .toHaveStatus(201)
      .toHaveHeadersProperty("content-type", /application\/json/);
    return response.json as { orderId: string };
  })
  .step("Verify order in database", async (ctx) => {
    const { orderId } = ctx.previous!;
    const result = await ctx.resources.db.query(
      "SELECT * FROM orders WHERE id = $1",
      [orderId],
    );
    // SQL-specific assertions
    expect(result)
      .toBeOk()
      .toHaveRowCount(1)
      .toHaveRowsMatching({ status: "pending" });
  })
  .step("Validate order total", async (ctx) => {
    const { orderId } = ctx.results[0] as { orderId: string };
    const response = await ctx.resources.http.get(`/orders/${orderId}`);
    const order = response.json as { total: number };
    // Generic value assertions (chainable)
    expect(order.total).toBeGreaterThan(0).toBeLessThan(10000);
  })
  .build();
```

## CLI Commands

```bash
# Run scenarios
probitas run [options]

# List scenarios without running
probitas list [options]

# Initialize configuration file
probitas init

# Cache scenario dependencies
probitas cache [options]

# Format scenario files
probitas fmt

# Lint scenario files
probitas lint

# Type check scenario files
probitas check

# Show version
probitas --version

# Show help
probitas --help
```

## Reporters

Choose output format based on your needs:

- `list` - Detailed human-readable output (default)
- `json` - Machine-readable JSON for CI/CD integration

```bash
probitas run --reporter json
```

## AI Integration

### Claude Code Plugin

Probitas provides a Claude Code plugin to enhance scenario development with AI
assistance:

```bash
# Add the plugin marketplace
/plugin marketplace add probitas-test/claude-plugins

# Install the Probitas plugin
/plugin install probitas@probitas-test
```

Or add to your project's `.claude/settings.json`:

```json
{
  "plugins": {
    "marketplaces": ["probitas-test/claude-plugins"],
    "installed": ["probitas@probitas-test"]
  },
  "enabledPlugins": {
    "probitas@probitas-test": true
  }
}
```

The plugin provides:

- **Scenario scaffolding** - Generate scenario templates with common patterns
- **Assertion suggestions** - Context-aware recommendations for expect calls
- **Client integration** - Auto-complete for client configurations
- **Error diagnostics** - AI-powered debugging for failing scenarios

See
[probitas-test/claude-plugins](https://github.com/probitas-test/claude-plugins)
for more details.

### LLM Documentation

For AI assistants and language models, comprehensive documentation is available
in machine-readable format:

- **Documentation website**:
  [probitas-test.github.io](https://probitas-test.github.io)
- **LLM-optimized**: [llms.txt](https://probitas-test.github.io/llms.txt) -
  Structured documentation following the
  [llms.txt standard](https://llmstxt.org/)
- **Markdown format**: All documentation provides `index.md` files for easy
  access and parsing

#### API Documentation

For detailed API reference, use `deno doc` to view type signatures and
documentation directly from the package:

```bash
# View expect API documentation
deno doc jsr:@probitas/probitas/expect

# View client API documentation
deno doc jsr:@probitas/probitas/client

# View main package documentation
deno doc jsr:@probitas/probitas
```

This provides the most accurate and up-to-date type information directly from
the source code.

The combination of narrative documentation (index.md/llms.txt) and API reference
(`deno doc`) enables AI assistants to provide accurate, comprehensive guidance
when working with Probitas.

## Contributing

Interested in contributing or maintaining Probitas? See
[CONTRIBUTING.md](CONTRIBUTING.md) for:

- Development environment setup
- Release process and workflow
- Architecture overview
- Related repositories

## Acknowledgments

Probitas is greatly inspired by [runn](https://github.com/k1LoW/runn), an
excellent scenario-based testing tool. Many core concepts in Probitas, such as:

- Accessing previous step results via `ctx.previous`
- Integration of database clients and other service helpers beyond traditional
  e2e testing scope
- Scenario-based workflow execution model

are influenced by runn's thoughtful design. We express our deep gratitude and
respect to the runn project and its maintainers for pioneering this approach to
scenario-based testing.

## License

See [LICENSE](LICENSE) file for details.

probitas

probitas Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

probitas

probitas Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README