name: promptfoo-evaluation
description: Configures and runs LLM evaluation using Promptfoo framework. Use when setting up prompt testing, creating evaluation configs (promptfooconfig.yaml), writing Python custom assertions, implementing llm-rubric for LLM-as-judge, or managing few-shot examples in prompts. Triggers on keywords like "promptfoo", "eval", "LLM evaluation", "prompt testing", or "model comparison".
Promptfoo Evaluation
Overview
This skill provides guidance for configuring and running LLM evaluations using Promptfoo, an open-source CLI tool for testing and comparing LLM outputs.
Quick Start
# Initialize a new evaluation project
npx promptfoo@latest init
# Run evaluation
npx promptfoo@latest eval
# View results in browser
npx promptfoo@latest view
Configuration Structure
A typical Promptfoo project structure:
project/
├── promptfooconfig.yaml # Main configuration
├── prompts/
│ ├── system.md
## Code Examples
### README.md
Skill Demonstrations
This directory contains automated demonstrations of each skill using VHS (Video Home System) by Charmbracelet.
VHS allows you to write terminal recordings as code, making demos reproducible, version-controllable, and typo-free.
📁 Demo Structure
demos/
├── skill-creator/
│ ├── init-skill.tape # Initialize a new skill
│ ├── validate-skill.tape # Validate skill quality
│ └── package-skill.tape # Package for distribution
├── github-ops/
│ └── create-pr.tape # Create pull requests
├── markdown-tools/
│ └── convert-docs.tape # Convert documents
└── generate_all_demos.sh # Generate all GIFs
🎬 Generating Demos
Prerequisites
Install VHS:
macOS:
brew install vhs
Linux (with Go):
go install github.com/charmbracelet/vhs@latest
Or download from:
https://github.com/charmbracelet/vhs/releases
Generate All Demos
### generate_all_demos.sh
#!/bin/bash
set -e
echo "================================================"
echo "Generating Claude Code Skills Demo GIFs"
echo "================================================"
echo ""
Colors for output
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color
Check if VHS is installed
if ! command -v vhs &> /dev/null; then
echo -e "${RED}Error: VHS is not installed!${NC}"
echo ""
echo "Install VHS to generate demo GIFs:"
echo ""
echo "macOS:"
echo " brew install vhs"
echo ""
echo "Linux (with Go):"
echo " go install github.com/charmbracelet/vhs@latest"
echo ""
echo "Or download from: https://github.com/charmbracelet/vhs/releases"
echo ""
exit 1
fi
echo -e "${GREEN}✓ VHS found${NC}"
echo ""
Find all .tape files
TAPE_FILES=$(find demos -name "*.tape" -type f | sort)
TOTAL=$(echo "$TAPE_FILES" | wc -l | tr -d ' ')
CURRENT=0
echo "Found $TOTAL demo tape files"
echo ""
Generate each demo
for tape_fi
### index.html
Claude Code Skills Marketplace - Demo Gallery
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif;
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
color: #333;
line-height: 1.6;
}
.container {
max-width: 1400px;
margin: 0 auto;
padding: 20px;
}
header {
text-align: center;
padding: 60px 20px;
color: white;
}
header h1 {
font-size: 3rem;
margin-bottom: 10px;
text-shadow: 2px 2px 4px rgba(0,0,0,0.3);
}
header p
