llm-opstower

KO | EN

gitlite — search

#python #ai #llm #javascript #css3 #html5 #agent #automation #docker #vanilla-js #github-config #machine-learning

★ 71 Open GitHub ↗

DevOps AI Assistant CLI. Ask questions about your AWS services, cloudwatch metrics, and billing.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# 🗼 OpsTower.ai - DevOps AI Assistant

See beyond your dashboards, metrics, and logs. OpsTower.ai takes the grunt work out of DevOps, connecting to your systems (AWS, Docker, Kubernetes, etc) and debugging problems on its own.

🚧 As of Oct '23, OpsTower.ai can answer questions about your AWS resources and perform calculations on cloudwatch metrics from the command line. [Learn about our larger vision](#user-content-vision).

__🏆 OpsTower.ai is the current SOTA for accuracy in the [DevOps AI Assistant Open Leaderboard](https://github.com/opstower-ai/devops-ai-open-leaderboard) for AWS Services, AWS Cloudwatch Metrics, and AWS Billing.__

_📅 [Book a time on my calendar](https://calendly.com/derek-haynes) or email derek@opstower.ai to chat about what capabalities you'd like to see._

<a href="#user-content-vision">Vision</a>
&nbsp;&nbsp;•&nbsp;&nbsp;
<a href="#user-content-install">Install</a>
&nbsp;&nbsp;•&nbsp;&nbsp;
<a href="#user-content-usage">Usage</a>
&nbsp;&nbsp;•&nbsp;&nbsp;
<a href="#user-content-demo-mode">Demo Mode</a>

## Vision

OpsTower.ai is AI Assistant that takes the grunt work out of DevOps. OpsTower.ai connects to your systems (AWS, Docker, Kubernetes, etc) and debugs and researches problems on its own.

You've probably played with some fragile demos in this space. That's not OpsTower.ai. At the heart of OpsTower.ai is a robust evaluation framework that allows us to measure the agent's performance against a large dataset of questions across problem domains. We're building a platform that will allow us to iterate on the agent's capabilities and measure its performance over time. We'll transparently share our results.

See our [Capabilities Roadmap](#user-content-capabilities-roadmap) below for more details on current evaluation results and planned knowledge areas.

## Install

Installed as a plugin for [LLM](https://llm.datasette.io/), A CLI utility for interacting with LLMs.

Install `llm` if you haven't yet:

```bash
pip install llm
```

Install the OpsTower plugin:

```bash
llm install https://github.com/opstower-ai/llm-opstower/archive/refs/heads/main.zip
```

Set your API key. Use `demo` to try with a live AWS account or see [our docs](docs/aws_read_only_iam_user.md) for on creating a read-only IAM user.

```bash
llm keys set opstower --value demo
```

Make `opstower` your default model:

```bash
llm models default opstower
```

## Usage Examples

__AWS Services__

```bash
llm "list each ec2 instance id, name, state, last restart, image, and size"
```

__AWS Cloudwatch Metrics__

```bash
llm "What is the average CPU utilization of my EC2 instances?"
```
__AWS Billing__

```bash
llm "breakdown bill by aws service over the past 30 days"
```

_The default timeframe is the past hour. You can specify a different timeframe in the question (ex: "...over the past 10 minutes")._

## How it works

![how it works](https://www.opstower.ai/assets/images/agent_eval/agent_orch.png)

OpsTower.ai provides a OpenAI-compatible API. The `llm` CLI utility sends your credentials and question to our API. Server-side, OpsTower.ai generates AWS SDK code to answer your question and execute it in an isolated environment, summarizing the response.

Read how the agent is structured and our current [evaluation results](https://www.opstower.ai/2023-evaluating-ai-agents/) against an AWS question dataset.

## Authentication

You can query against your account by setting your API key in the following format:

```bash
llm keys set opstower --value AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY:AWS_REGION
```

## Demo Mode

You can execute commands against a demonstration AWS account that contains an EC2 instance, RDS instance, an Application Load Balancer, several Lambda functions, S3 buckets, and more.

Set the opstower key to `demo`:

```bash
llm keys set opstower --value demo
```

## Capabilities Roadmap

Just like how a Junior DevOps engineer isn't assigned with high-level tasks, OpsTower.ai's training is starting with a focus on information retrieval (ex: "what is our estimated AWS bill for this month"). After building up a base set of skills, OpsTower.ai will tackle reasoning you cannot infer from a ready-made dashboard (ex: "what resources appear to be not used and/or over-provisioned and how much could we reduce costs?").

| Knowledge area | Status | Accuracy % |
| -------- | -------- | -------- |
| AWS services  | Released  | [92%](https://github.com/opstower-ai/devops-ai-open-leaderboard#-current-leaderboard)  |
| AWS cloudwatch metrics | Released  | [89%](https://github.com/opstower-ai/devops-ai-open-leaderboard#-current-leaderboard)  |
| AWS billing  | Released  | [91%](https://github.com/opstower-ai/devops-ai-open-leaderboard#-current-leaderboard)  |
| AWS cloudwatch logs  |  🚧 | N/A |
| AWS deployment  |   | N/A  |
| AWS security  |   | N/A  |
| Advanced AWS reasoning  |   | N/A  |
| Docker  |   | N/A  |
| Kubernetes  |   | N/A  |
| Terraform  |   | N/A  |
| Advanced Kubernetes reasoning  |   | N/A  |
| Incident Investigation  |   | N/A  |

## Limitations

1. Only read-only operations are permitted. To be safe, you should only use an IAM user with read-only access.
2. OpsTower.ai does not support higher-level and/or abstact questions like "Has there been a sudden change in any critical ec2, rds, or s3 metrics?". Try to be specific.
3. Support for the `llm --continue` option is not yet available.

## Uninstall

```bash
llm uninstall llm-opstower
```

llm-opstower

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README