About

Imported from Mahi-887/IT_Companies_Agent (antigravity-awesome-skills-main/plugins/antigravity-awesome-skills/skills/pipecat-friday-agent/SKILL.md). Install upstream with npx skills add Mahi-887/IT_Companies_Agent --skill pipecat-friday-agent. Copyright stays with the author.

Pipecat Friday Agent

Name: pipecat-friday-agent
Author: Mahi-887

Overview

This skill provides a blueprint for building F.R.I.D.A.Y. (Replacement Integrated Digital Assistant Youth), a local voice assistant inspired by the tactical AI from the Iron Man films. It uses the Pipecat framework to orchestrate a low-latency pipeline:

STT: OpenAI Whisper (whisper-1) or gpt-4o-transcribe
LLM: Google Gemini 2.5 Flash (via a compatibility shim)
TTS: OpenAI TTS (nova voice)
Transport: Local Audio (Hardware Mic/Speakers)

When to Use This Skill

Use when you want to build a real-time, conversational voice agent.
Use when working with the Pipecat framework for pipeline-based AI.
Use when you need to integrate multiple providers (Google and OpenAI) into a single voice loop.
Use when building Iron Man-themed or tactical-themed voice applications.

How It Works

Step 1: Install Dependencies

You will need the Pipecat framework and its service providers installed:

pip install pipecat-ai[openai,google,silero] python-dotenv

Step 2: Configure Environment

Create a .env file with your API keys:

OPENAI_API_KEY=your_openai_key
GOOGLE_API_KEY=your_google_key

Step 3: Run the Agent

Execute the provided Python script to start the interface:

python scripts/friday_agent.py

Core Concepts

Pipeline Architecture

The agent follows a linear pipeline: Mic -> VAD -> STT -> LLM -> TTS -> Speaker. This allows for granular control over each stage, unlike end-to-end speech-to-speech models.

Google Compatibility Shim

Since Google's Gemini API has a different message format than OpenAI's standard (which Pipecat aggregators expect), the script includes a GoogleSafeContext and GoogleSafeMessage class to bridge the gap.

Best Practices

✅ Use Silero VAD: It is robust for local hardware and prevents background noise from triggering the LLM.
✅ Concise Prompts: Tactical agents should give short, data-dense responses to minimize latency.
✅ Sample Rate Match: OpenAI TTS outputs at 24kHz; ensure your audio_out_sample_rate matches to avoid high-pitched or slowed audio.
❌ No Polite Fillers: Avoid "Hello, how can I help you today?" Instead, use "Systems nominal. Ready for commands."

Troubleshooting

Problem: Audio is choppy or delayed.
- Solution: Check your OUTPUT_DEVICE index. Run a script like test_audio_output.py to find the correct hardware index for your OS.
Problem: "Validation error" for message format.
- Solution: Ensure the GoogleSafeContext shim is correctly translating OpenAI-style dicts to Gemini-style schema.

Related Skills

@voice-agents - General principles of voice AI.
@agent-tool-builder - Add tools (Search, Lights, etc.) to your Friday agent.
@llm-architect - Optimizing the LLM layer.

Use it

Copy one of these into your project. Installing also returns the manifest and these snippets.

yaml

targets:
  - https://api.opensmartroute.ai/api/v1/registry/mahi-887-it-companies-agent-pipecat-friday-agent/manifest   # or paste the manifest below

Manifest

An Open Capability Manifest: the router reads it to know what this does, what it costs and when to pick it.

mahi-887-it-companies-agent-pipecat-friday-agent.ocm.jsonjson

{
  "ocm": "1",
  "id": "mahi-887-it-companies-agent-pipecat-friday-agent",
  "kind": "skill",
  "name": "pipecat-friday-agent",
  "description": "Build a low-latency, Iron Man-inspired tactical voice assistant (F.R.I.D.A.Y.) using Pipecat, Gemini, and OpenAI.",
  "publisher": "Mahi-887",
  "version": "1.0.0",
  "capabilities": {
    "domains": [
      "general"
    ],
    "tags": [
      "skill-md",
      "pipecat",
      "voice",
      "gemini",
      "openai",
      "python",
      "github"
    ],
    "languages": [
      "en"
    ]
  },
  "quality_prior": 0.6,
  "examples": [
    "Build a low-latency, Iron Man-inspired tactical voice assistant (F.R.I.D.A.Y.) using Pipecat, Gemini, and OpenAI."
  ],
  "primary": false,
  "metadata": {
    "source": {
      "provider": "github",
      "repository": "https://github.com/Mahi-887/IT_Companies_Agent",
      "path": "antigravity-awesome-skills-main/plugins/antigravity-awesome-skills/skills/pipecat-friday-agent/SKILL.md",
      "ref": "ce44d590e1023e65cd9b40cc622fc5526a40df5a",
      "url": "https://github.com/Mahi-887/IT_Companies_Agent/blob/ce44d590e1023e65cd9b40cc622fc5526a40df5a/antigravity-awesome-skills-main/plugins/antigravity-awesome-skills/skills/pipecat-friday-agent/SKILL.md",
      "key": "Mahi-887/IT_Companies_Agent/antigravity-awesome-skills-main/plugins/antigravity-awesome-skills/skills/pipecat-friday-agent/SKILL.md"
    }
  },
  "instructions": "# Pipecat Friday Agent\n\n## Overview\n\nThis skill provides a blueprint for building **F.R.I.D.A.Y.** (Replacement Integrated Digital Assistant Youth), a local voice assistant inspired by the tactical AI from the Iron Man films. It uses the **Pipecat** framework to orchestrate a low-latency pipeline:\n- **STT**: OpenAI Whisper (`whisper-1`) or `gpt-4o-transcribe`\n- **LLM**: Google Gemini 2.5 Flash (via a compatibility shim)\n- **TTS**: OpenAI TTS (`nova` voice)\n- **Transport**: Local Audio (Hardware Mic/Speakers)\n\n## When to Use This Skill\n\n- Use when you want to build a real-time, conversational v",
  "cost": {
    "context_tokens": 715
  }
}

Fetch it by URL: GET /api/v1/registry/mahi-887-it-companies-agent-pipecat-friday-agent/manifest?version=1.0.0

Reviews

Star ratings from people who tried it. One review per account; edit yours any time.

No reviews yet. Install it, try it, and be the first to rate it.