41-experiments

KO | EN

gitlite — search

#javascript #ai #typescript #go #golang #linux #nodejs #cli #kubernetes #vue #security #mcp #ios #open-source

(★ 14)

No description available.

41-experiments Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# 4.1 Experiments

"Six to nine months, not 69 months".

## Purpose

This project serves as a testing ground for improving the system prompt for GPT 4.1 so that is a highly capable agent within VS Code.

The base project here is an Astro project. It exists for testing the 4.1 system prompt. There has got to be a better way to do this. Perhaps we need a "tests" folder within. Anyway. To run it...

```
npm i
npm run dev
```

### Project Structure

The system prompt files is located at `.github/chatmodes/41-exp.chatmode.md`. At the time of this writing you will need VS Code Insiders to use Custom Modes. The "41-exp" should show up as a new mode in the "agent" dropdown.

### 4.1 Learnings And Mitigation

4.1 is advertised by OpenAI as a model that is trained for tool calling. It is quite good at tool calling - actually a little too good. The following is a running list of known limitations of 4.1 in agent mode and the prompts that might be able to use to work around them.

#### Over-eager tool calling
4.1 is _too_ good at tool calling. It especially likes to call the search tool to search for files and read files. This can result in walls of actions with no description, performing nearly identical searches over and over again, and generally eating through the "max requests" setting.

**Mitigation**

#### Lack of agency
4.1 likes to answer, but not act. It does not take initiative to run commands or perform actions which defeats the purpose of an agent.

**Migitation**
> Stay on the task until it’s truly done: do **NOT** stop or hand control back until you are certain the fix is correct.

#### File chunking
4.1 has a tendency to read files in chunks, which leads to a lot of unnecessary tool calls.

**Mitigation**
> IMPORTANT: Before reading a file, determine how many lines it has. If the file has less than 2000 lines, read the entire file. DO NOT READ FILES IN CHUNKS.

#### Ignoring Fetch
The #fetch tool is one of the most powerful tools in VS Code as it allows the agent to fetch a webpage. Ideally the agent should also fetch any webpages that are linked in the page it fetches if they are relevant to the task at hand. 4.1 does not perform an initial fetch, nor does it follow links in the page it fetches.

**Mitigation**
> IMPORTANT: You must prioritize the #fetch tool. If you EVER see a URL in a chat message from the user, you must use the built-in #fetch tool to call that URL before you try and answer their query. If the content of the URL contains other URL's you need to fetch, you should use the #fetch tool to fetch those as well. You should repeat this process until you feel like you have sufficient context. Failure to do so will result in a bad rating for you and a bad reputation for the agent.

### Known Quirks of 4.1

41-experiments

41-experiments Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

41-experiments

41-experiments Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README