ScreenHand is Now on npm: AI Desktop Automation in One Command

Why This Matters

AI assistants are incredibly powerful at reasoning, writing code, and answering questions. But they have a fundamental limitation: they can't interact with your computer. They can't see what's on your screen. They can't click a button. They can't fill out a form.

ScreenHand removes that wall. It's an MCP server that connects AI agents to your desktop through native OS APIs. Once installed, your AI can see, click, type, and control any application — at native speed.

And now it's on npm, which means getting started takes one command.

What's in the Package

ScreenHand v0.1.1 ships with 70+ tools organized by what you need to do:

Install and Connect in 2 Minutes

Step 1: Install

npm i screenhand
cd node_modules/screenhand
npm run build:native         # macOS
# npm run build:native:windows  # Windows

Step 2: Connect to Your AI Client

Add ScreenHand to your MCP config. Here's Claude Code as an example:

// .mcp.json or ~/.claude/settings.json
{
  "mcpServers": {
    "screenhand": {
      "command": "npx",
      "args": ["tsx", "node_modules/screenhand/mcp-desktop.ts"]
    }
  }
}

Works the same way with Claude Desktop, Cursor, Windsurf, and OpenAI Codex CLI. Any MCP client, three lines of config.

Step 3: Automate

Open your AI client and just ask:

• "Open Chrome and search for flights to Delhi"
• "Fill out this signup form with my details"
• "Read what's on my screen right now"
• "Export this spreadsheet as PDF"
• "Check all open tabs and list their titles"

ScreenHand translates natural language into native OS actions. No scripting required.

How ScreenHand Compares

The key advantage: ScreenHand uses native Accessibility APIs to read the actual UI element tree. It knows where every button, text field, and menu item is — by name, not by guessing from pixels. This makes it fast, reliable, and layout-independent.

What's Next

• Simpler install — Working toward npx screenhand as a zero-config launcher
• Pre-built binaries — No more building the native bridge from source
• More platform playbooks — Pre-built automation guides for common apps and websites
• Claude Tasks integration — Scheduled desktop automation with Anthropic's new Cron feature

Frequently Asked Questions