Building a Self-Hosted Blog-to-Audio Pipeline on a $150 GPU

Listen to this article

0:00

/625

1×

I wanted every post on my blog to have a "listen" option — an audio narration in my own voice, generated entirely on my homelab hardware. No cloud APIs, no monthly fees, no sending my content to third parties. Here's the journey from idea to a fully automated pipeline that takes a Ghost blog post and produces a polished audio narration, uploaded and embedded on the live site.

The Hardware

The whole thing runs on a Proxmox LXC container with GPU passthrough:

• AMD RX 580 8GB — a card that costs about $50 used
• Vulkan compute — not CUDA, not ROCm, just standard Vulkan shaders
• LLM inference and TTS inference both run on this single GPU

That's it. No A100, no cloud instance, no expensive hardware. The RX 580 was already sitting in my server doing nothing.

Phase 1: The LLM Foundation

The first thing I needed was a local LLM to prepare blog text for spoken delivery. I compiled llama.cpp with Vulkan support and loaded Qwen 2.5 7B Instruct, later upgrading to Qwen 3 8B. Running at about 33 tokens per second on the RX 580 — not fast, but fast enough for a batch process.

The LLM does two passes on every post:

1. Narration prep — smooths section headings into spoken transitions, removes visual references and links, converts lists to prose, adds the intro/outro
2. Copy editor — a research-backed cleanup pass that uses SearXNG to verify how ambiguous terms should be pronounced, converts numbers to spoken form with context awareness, and adds prosody-friendly punctuation

The key insight: the LLM shouldn't rewrite the post. It should make minimal changes to preserve my voice while adapting for spoken delivery. Section headings like "The Failure Pattern" become "Alright, let me talk about what I call the failure pattern." Everything else stays as I wrote it.

The Qwen 3 Surprise

Upgrading from 2.5 to 3 brought better reasoning — it's smarter about catching numbers, understanding context, and identifying terms that need research verification. But Qwen 3 is a "thinking" model trained on heavily formatted data. It desperately wants to add stage directions (*[Music fades in]*), markdown formatting (**Narrator:**), and theatrical structure.

The solution wasn't better prompting — it was accepting the model's nature and adding a brute-force sanitization step. A simple regex pass after the copy editor strips any remaining asterisks, brackets, stage directions, and speaker labels. Three lines of code fixed what increasingly elaborate prompts couldn't. Don't fight the model. Let it do what it's good at, then clean up what it can't help.

Phase 2: The TTS Dead Ends

Kokoro-82M (Local, CPU)

My first TTS attempt was Kokoro-82M via ONNX Runtime on CPU. It produced clear audio but was painfully monotone — no emotion, no emphasis, completely flat reading regardless of content. At about 5x slower than real-time, a 7-minute blog post took 35 minutes to synthesize. And it sounded robotic.

The Podcast Experiment

I tried making it more interesting by generating a two-host podcast discussion instead of a narration. Two AI voices discussing the post, with research context pulled from SearXNG to add depth. The dialogue scripts were fun to generate, but the voices were firmly in the uncanny valley. Two robots having a fake conversation isn't better than one robot reading clearly. The NPR approach — one voice, reading clearly — is what actually serves readers.

ElevenLabs (Cloud)

I tested ElevenLabs and the quality was outstanding — natural prosody, emotion, breathing. But the free tier gives you about one episode per month, and the paid plans add up fast for a regular blog. More importantly, it violated my goal of running everything locally.

Phase 3: OmniVoice — The Breakthrough

Then I found OmniVoice.cpp — a C++/GGML port of OmniVoice that explicitly supports Vulkan. Same approach as llama.cpp but for TTS. I built it with ./buildvulkan.sh and it ran on my RX 580 at near real-time (RTF ~1.0). A 612M parameter model doing GPU-accelerated speech synthesis through Vulkan compute shaders on a $50 card.

Voice Cloning

The game changer was voice cloning. I grabbed a 15-second clip of myself from an old podcast episode, ran it through Whisper to get an accurate transcript (critical — the model needs to know what sounds map to what words), encoded it as a compact .rvq reference file, and now OmniVoice generates speech that sounds like me.

The key discovery: without an accurate transcript of the reference audio, voice cloning produces garbled nonsense. The model needs the text-to-phoneme alignment to understand your voice. Whisper transcription + OmniVoice encoding = consistent voice identity.

Long-Form Synthesis

Initially I was synthesizing paragraph by paragraph and concatenating with ffmpeg. This produced noticeable artifacts — fade-ins at every segment boundary, inconsistent voice between segments, and unnatural silence gaps.

The fix was elegantly simple: feed the entire script to OmniVoice in a single call. It has a built-in long-form mode that chunks the text into ~15-second segments internally, maintains voice consistency through the reference, and cross-fades between chunks. One call, continuous output, consistent voice throughout.

The Source Code Fix

But even with long-form mode, every chunk boundary had an audible fade-in. Through waveform analysis I traced it to a hardcoded constant in pipeline-tts.cpp:

audio = cross_fade_chunks(chunk_audios, sr, 0.3);

That 0.3 means 300ms of cross-fade — split into 100ms fade-out, 100ms silence, 100ms fade-in. Audible on every chunk boundary. I changed it to 0.05 (50ms — imperceptible), rebuilt, and the problem vanished.

I spent hours on ffmpeg filter chains, attack boosters, and trim heuristics trying to fix this downstream. The actual fix was one character change in the source code. The lesson: when you're post-processing to fix an artifact, you're probably solving the wrong problem. Fix it at the source.

Phase 4: Prosody Engineering

The biggest remaining challenge: the output sounded like a run-on sentence. No pauses, no breathing room, no emphasis. Without SSML or explicit prosody tokens, I needed to figure out what text patterns OmniVoice actually responds to.

I ran a systematic experiment: six different punctuation approaches on the same paragraph, measured the actual pause durations in the waveform:

Commas are the most reliable pause generator. Forward slashes — despite internet claims — actually reduce pauses. Short sentences boost amplitude variation (dynamic range), which creates the perception of expressiveness even without pitch control.

The copy editor now knows: when in doubt, add a comma. Use em dashes for dramatic pauses. Vary sentence length wildly. This is prosody engineering through punctuation.

Phase 5: Ghost Integration

The final piece: automatically embedding the audio on the blog.

1. Uploads the MP3 to Ghost via the Admin API media endpoint
2. Fetches the post's Lexical editor content
3. Injects a native audio node at the top of the document tree (Ghost v6+ uses Lexical, which has a first-class audio node type)
4. Updates the post via the Admin API

It's idempotent — re-running replaces the existing audio card. I ran all 24 posts through the pipeline in a single overnight batch (about 6 hours) and every post on the site woke up with an audio player the next morning.

Phase 6: SEO and Discoverability

The audio isn't just a reader convenience — it's an SEO differentiator.

What I added:

• Speakable schema markup — tells Google/voice assistants that the post content is optimized for audio
• AudioObject JSON-LD — structured data declaring the audio file, its URL, format, and duration
• llms.txt — Ghost 6.49+ generates this natively; it's an AI-readable index of all posts with markdown links
• Citation guidance — the About page includes preferred attribution format for LLMs that reference the content
• Transparency footer — every post discloses AI assistance in drafting and audio generation

The disclosure footer is a deliberate signal. Google penalizes undisclosed AI content. Transparent AI-assisted content with clear human direction signals quality, not spam. The audio itself is additional proof of investment — content farms don't generate voice-cloned narrations for every post.

I also fixed 16 posts where the audio player text was leaking into auto-generated excerpts ("Listen to this article0:00/275×") — polluting the llms.txt and OpenGraph descriptions. Setting proper custom_excerpt values via the Admin API cleaned this up across the entire site in one script run.

The Final Pipeline

Ghost Post
  → Text Extraction (strip HTML, images, code blocks)
  → Intro/Outro Templates (token replacement: {{post_title}}, {{author_name}}, {{publish_date}})
  → LLM Narration Prep (Qwen 3 8B, Vulkan)
  → Research-Backed Copy Editor (SearXNG + LLM, low temperature)
  → Regex Sanitization (strip any remaining formatting artifacts)
  → Long-Form Voice-Cloned TTS (OmniVoice, Vulkan, single-shot)
  → Ghost Admin API Upload + Audio Card Injection

Total processing time: about 25 minutes per post. Runs entirely on a $50 GPU in an LXC container. No cloud dependencies. 24 posts processed overnight in a single batch.

What I Learned

Start with the simplest thing that works, then iterate. The podcast format was a fun detour but the NPR-style narration — just reading the post clearly in my own voice — is what actually serves readers.

The LLM's job is restraint, not creativity. The biggest quality improvement came from telling the LLM to change as little as possible. My writing already sounds like me. The LLM just needs to smooth the rough edges for audio without imposing its own voice.

Don't fight the model's nature — compensate for it. Qwen 3 will always want to add theatrical formatting. Rather than increasingly elaborate prompts, a simple regex cleanup after the fact is more reliable. Accept the tool's quirks and build around them.

Punctuation is your prosody control. Without SSML or explicit prosody tokens, commas and em dashes are the primary tools for controlling pacing in the generated audio. The copy editor is essentially a prosody engineer.

Voice cloning needs accurate alignment. A 15-second reference clip works, but only if the model knows exactly what words are being spoken in that clip. Whisper transcription of the reference audio was the difference between garbled output and recognizable voice cloning.

Fix problems at the source, not downstream. I spent hours on ffmpeg filter chains trying to fix artifacts that were caused by a single hardcoded 0.3 in C++ source code. Post-processing is a code smell. If you're compensating for an upstream problem, go fix the upstream problem.

Audio is an SEO signal. Having a voice-cloned narration on every post differentiates from AI-generated content farms in a way that text alone can't. It's additional investment that signals quality to both readers and search engines.

The Stack

What's Next

The word-level prosody control model exists (omnivoice-word-control) with tokens for pitch, energy, duration, and contour per word. It's currently Python-only and non-commercially licensed. When it gets ported to C++ or the license changes, per-word control would take this from "good enough" to genuinely expressive.

Until then, the levers are punctuation, sentence structure, and the occasional patch to an open-source C++ codebase. The tools handle the rest.