feat: optional audio transcription via CLI

This commit is contained in:
Peter Steinberger
2025-11-25 23:06:54 +01:00
parent f945e284e1
commit 7d0ae151e8
7 changed files with 215 additions and 14 deletions

View File

@@ -59,6 +59,7 @@ Notes on this configuration:
- Host local paths for Twilio using the media server/Tailscale Funnel.
- Send buffers directly for the Web provider.
- Inbound media is downloaded (≤5MB) and exposed to your templates as `{{MediaPath}}`, `{{MediaUrl}}`, and `{{MediaType}}`. You can mention this in your prompt if you want Claude to reason about the attachment. Outbound media from Claude (via `MEDIA:`) is resized/recompressed on the Web provider path; control the cap with `inbound.reply.mediaMaxMb` (default 5).
- Voice notes: set `inbound.transcribeAudio.command` to run a CLI that emits the transcript to stdout (e.g., OpenAI Whisper: `openai api audio.transcriptions.create -m whisper-1 -f {{MediaPath}} --response-format text`). If it succeeds, warelay replaces `Body` with the transcript before invoking Claude.
## Testing the setup
1. Start a relay (auto-selects Web when logged in, otherwise Twilio polling):

View File

@@ -57,6 +57,7 @@ This document defines how `warelay` should handle sending and replying with imag
- `{{MediaUrl}}` original URL (Twilio) or pseudo-URL (web).
- `{{MediaPath}}` local temp path written before running the command.
- Size guard: only download if ≤5MB; else skip and log.
- Audio/voice notes: if you set `inbound.transcribeAudio.command`, warelay will run that CLI (templated with `{{MediaPath}}`) and replace `Body` with the transcript before continuing the reply flow; verbose logs indicate when transcription runs.
## Errors & Messaging
- Local path with twilio + Funnel disabled → error: “Twilio media needs a public URL; start `warelay webhook --ingress tailscale` or pass an https:// URL.”