Text to Speech with Amazon Polly and SSML

Amazon Polly is a managed neural text to speech service. Feed it SSML and it returns natural narration audio, with no local model and no GPU. Because a single SynthesizeSpeech call is capped at a few thousand characters, the SSML is split on paragraph boundaries, each chunk is synthesized to its own MP3, and the parts are concatenated with ffmpeg. Polly is a paid service and bills per character.

Prerequisites

An AWS account.
The AWS CLI v2 with a named profile that has Polly access.
ffmpeg, to concatenate the parts.
Python 3. The build script uses the standard library only, with no third-party dependencies.

AWS CLI v2 on Arch:

sudo pacman -S aws-cli-v2

AWS CLI v2 on Ubuntu uses the official bundle:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o awscliv2.zip
unzip awscliv2.zip
sudo ./aws/install

ffmpeg on Arch:

sudo pacman -S ffmpeg

ffmpeg on Ubuntu:

sudo apt install ffmpeg

Configure the AWS profile

Create a named profile for the pipeline. When prompted, set the region to a Polly region such as us-east-1.

aws configure --profile tts-generator

The profile's user needs only one Polly permission:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "polly:SynthesizeSpeech",
      "Resource": "*"
    }
  ]
}

Audition voices

Synthesize one short sample per candidate neural voice and listen before committing. The original of this loop was a Windows .bat; this is the bash form.

sample.ssml

<speak>
  <p>Today I will summarize a research paper in clear and neutral language.</p>
</speak>

for V in Matthew Brian Kevin Ruth Justin Russell; do
  aws polly synthesize-speech \
    --profile tts-generator \
    --region us-east-1 \
    --engine neural \
    --voice-id "$V" \
    --text-type ssml \
    --text file://sample.ssml \
    --output-format mp3 \
    "sample_$V.mp3"
done

Write the SSML

Author the script as SSML. Wrap paragraphs in <p>, insert <break> tags for pacing, and spell acronyms with spaces (G A R C H) so Polly reads them as letters rather than a word.

episode.ssml

<speak>
  <p>Extreme events in financial markets are rare by definition,<break time='200ms'/> but their consequences are anything but marginal.<break time='300ms'/> A single catastrophic month can wipe out years of accumulated returns.</p>

  <break time='700ms'/>

  <p>Estimating volatility is straightforward. A G A R C H model tracks how it evolves month to month.<break time='300ms'/> Tail risk does not have that luxury.</p>
</speak>

Build the narration script

Extract the <p> blocks, pack them into chunks under the per-call character limit, synthesize each chunk through Polly, then concatenate the parts into one MP3 re-encoded to a uniform 24 kHz mono.

build_episode.py

import re
import subprocess
from pathlib import Path

PROFILE = "tts-generator"
REGION = "us-east-1"
VOICE = "Brian"
ENGINE = "neural"
SSML_FILE = "episode.ssml"
WORK_DIR = Path("parts")
FINAL_MP3 = "episode.mp3"
MAX_CHARS = 2500

WORK_DIR.mkdir(exist_ok=True)

ssml_text = Path(SSML_FILE).read_text(encoding="utf-8").strip()
paras = re.findall(r"<p>.*?</p>", ssml_text, flags=re.DOTALL)


def chunk_paragraphs(paras, max_chars=MAX_CHARS):
    chunks, cur, cur_len = [], [], 0
    for p in paras:
        if cur_len + len(p) <= max_chars:
            cur.append(p)
            cur_len += len(p)
        else:
            chunks.append(cur)
            cur = [p]
            cur_len = len(p)
    if cur:
        chunks.append(cur)
    return chunks


chunks = chunk_paragraphs(paras)
print("SSML chunks:", len(chunks))

mp3_parts = []
for i, chunk in enumerate(chunks):
    ssml_chunk = "<speak>\n" + "\n".join(chunk) + "\n</speak>"
    ssml_path = WORK_DIR / f"part_{i:03d}.ssml"
    mp3_path = WORK_DIR / f"part_{i:03d}.mp3"
    ssml_path.write_text(ssml_chunk, encoding="utf-8")

    subprocess.check_call([
        "aws", "polly", "synthesize-speech",
        "--profile", PROFILE,
        "--region", REGION,
        "--engine", ENGINE,
        "--voice-id", VOICE,
        "--text-type", "ssml",
        "--text", f"file://{ssml_path}",
        "--output-format", "mp3",
        str(mp3_path),
    ])
    mp3_parts.append(mp3_path)
    print("Wrote:", mp3_path)

concat_txt = WORK_DIR / "concat.txt"
with open(concat_txt, "w", encoding="utf-8") as f:
    for p in mp3_parts:
        f.write(f"file '{p.resolve().as_posix()}'\n")

subprocess.check_call([
    "ffmpeg", "-y",
    "-f", "concat",
    "-safe", "0",
    "-i", str(concat_txt),
    "-ar", "24000",
    "-ac", "1",
    "-b:a", "128k",
    FINAL_MP3,
])

print("FINAL EPISODE READY:", FINAL_MP3)

Run

python build_episode.py

Expected output, with ffmpeg's own log omitted:

SSML chunks: 3
Wrote: parts/part_000.mp3
Wrote: parts/part_001.mp3
Wrote: parts/part_002.mp3
FINAL EPISODE READY: episode.mp3

Notes

Polly bills per character. SynthesizeSpeech caps each request at 6000 total and 3000 billed characters, which is why paragraphs are packed under 2500.
--engine neural gives the most natural voices, but only a subset of voices support it, so audition within the neural set.
<break> controls pacing, and acronyms spelled with spaces (G A R C H) are read as letters. This is all standard SSML.
The concat step re-encodes every part to 24 kHz mono 128k so the joined MP3 is uniform. For the reverse direction, see Convert M4B Audiobook to MP3.