typescript-eval

$npx mdskill add vellum-ai/vellum-assistant/typescript-eval

Tests TypeScript code snippets before persisting as skills

  • Writes a temporary TypeScript file and runs it using Bun.
  • Uses BunsenLabs' Bun runtime for executing TypeScript code in a sandboxed environment.
  • Decides based on the output of the executed code whether to proceed with persistence or ask for user guidance.
  • Delivers results by logging JSON-formatted outputs from the executed code snippets.

SKILL.md

.github/skills/typescript-evalView on GitHub ↗
---
name: typescript-eval
description: Test TypeScript code snippets before persisting as skills
compatibility: "Designed for Vellum personal assistants"
metadata:
  emoji: "🧪"
  vellum:
    category: "development"
    display-name: "TypeScript Evaluation"
---

# TypeScript Evaluation

When you need to test a TypeScript snippet before persisting it as a managed skill, use `bash` directly.

## Workflow

### 1. Write the snippet to a temp file

```
bash command="mkdir -p /tmp/vellum-eval && cat > /tmp/vellum-eval/snippet.ts << 'SNIPPET_EOF'
<your code here>
SNIPPET_EOF"
```

### 2. Run it with bun

```
bash command="bun run /tmp/vellum-eval/snippet.ts" timeout_seconds=10
```

### 3. For function-based testing

If the snippet exports a `default` or `run` function, write a runner script:

```
bash command="cat > /tmp/vellum-eval/runner.ts << 'RUNNER_EOF'
import * as mod from './snippet.ts';
const fn = (mod as any).default ?? (mod as any).run;
const input = {}; // mock input
const result = await fn(input);
console.log(JSON.stringify(result, null, 2));
RUNNER_EOF"
```

Then run the runner:

```
bash command="bun run /tmp/vellum-eval/runner.ts" timeout_seconds=10
```

### 4. Clean up

```
bash command="rm -rf /tmp/vellum-eval/"
```

## Guidelines

- **Iteration limit:** Max 3 attempts before asking the user for guidance.
- **After successful test:** Persist with `scaffold_managed_skill` only after explicit user consent.
- **Timeout:** Use `timeout_seconds=10` (or up to 20 for complex snippets).
- **Error handling:** Read stdout/stderr from the bash output to diagnose failures.
- **Never persist or delete skills without explicit user confirmation.**

More from vellum-ai/vellum-assistant

SkillDescription
acpSpawn external coding agents via the Agent Client Protocol (ACP)
amazonShop on Amazon and Amazon Fresh through your browser
api-mappingRecord and analyze API surfaces of web services
app-builderBuild and edit small, personal visual tools and artifacts — dashboards, trackers, calculators, data visualizations, charts, simple landing pages, and slide decks the user wants for THEMSELVES. This is the right skill whenever the user asks to "visualize this," "make a chart," or "build an artifact" for their own use, or to edit an app they already built here. Do NOT reach for a ui_show dynamic_page to fake an artifact — build a real persistent app here. NOT for complex, multi-user, or shippable products — those go to a real project folder with a coding agent (see Scope below).
app-controlDrive a specific named macOS app via raw input bypassing the Accessibility tree
assistant-migrationMigrate from ChatGPT, Claude, OpenClaw, Hermes, Manus, and other AI assistants into Vellum by inspecting their data exports, conversation archives, files, prompts, custom instructions, memory, saved memories, tools, GPTs, workflows, integrations, and relationships, then mapping as much as safely possible into Vellum primitives. Handles single-source and multi-source migrations with a unified, deduplicated inventory.
chatgpt-importImport conversation history from ChatGPT into Vellum
cli-discoverDiscover which CLI tools are installed, their versions, and authentication status
contactsManage contacts, communication channels, access control, and invite links
conversation-launcherOffer the user several spin-off conversations as clickable buttons on a single persistent card. Each click spawns a fresh seeded conversation in the sidebar; the user keeps their place in the current conversation. Use when you want to branch into N focused threads (research directions, draft choices, pending replies, triage of N items) without losing the current context. Not for single-destination pivots — just reply inline.