Build your own JARVIS

Claude × Fish Audio

Write the sentence first 🎯

Everyone builds these backwards. They connect eight apps, wire up a voice, and then wonder what to ask it — and end up with a robot that reads their calendar back to them like a speaking clock.

Do it the other way. Write the one sentence you want to hear, then build only what that sentence needs. Mine again:

Look at what that sentence is actually made of — three clauses, three different kinds of pressure:

• 💸 Money that's late. Needs to reach wherever your invoices live, and needs to know what "overdue" means for you.
• 🕘 Time that moved. Needs your calendar, and needs to notice a change, not just read the day out.
• 👤 A person left hanging. Needs your inbox — and this is the hard one, because "still waiting on you" is a judgment, not a data field.

That third clause is the whole difference between an assistant and a speaking clock. Any tool can list your meetings. Knowing that Sarah messaged twice and you replied to neither is judgment — and judgment is something you have to teach it, in a file, which is step two.

What you need ✅

• Claude on a paid plan (~US$20/mo) — the connectors live here. Claude Code too if you want the voice layer and the optional dashboard, since those involve running a small script locally.
• The apps your sentence needs. Not all of them. Inbox, calendar, notes is plenty to start.
• A Fish Audio account and an API key. The S2.1 Pro tier has been free for developers, but that window is time-limited — check what's current on their site before you plan around it. If it's closed when you read this, the build still works on their paid tiers, and everything up to the voice step costs nothing extra.
• About an hour, most of it in step two. Step two is what makes it good.

Give it reach — one app at a time 🔌

An MCP connector is what lets Claude read and act inside a real app instead of just talking about it. Add them one at a time and verify each before the next — this is the single biggest reason these builds work instead of quietly lying to you.

1. 1Calendar — the safest, pure read. Test: "What's on my calendar today, and did anything move since yesterday?"
2. 2Inbox — the useful one. It can read, label and draft, but the official connector won't send for you, which is exactly the behaviour you want. Test: "Who has messaged me twice in the last week without getting a reply?" If that comes back right, your "Sarah's still waiting" clause works.
3. 3Notes or tasks (Notion or similar) — where it reads your priorities and can write the brief down for later. Test: "Read my tasks and tell me the three that actually matter today."
4. 4Optional, once the first three are solid: a team chat tool if people reach you there, or read-only access to revenue or ad data if your sentence needs money numbers it can't get from the inbox.

Teach it judgment 📋

This is the step that separates yours from everyone else's, and it's just a file. Claude reads it every time, so it knows your situation instead of guessing at it.

WHO I AM[what you do, who you do it for, how you make money]WHAT'S LIVE RIGHT NOW[3-5 current projects, deals, or commitments — the things that wouldactually hurt if they slipped]WHAT "URGENT" MEANS TO ME- Money: an invoice is overdue at [X] days. Anything above [amount]  gets flagged the same day.- People: someone is "waiting on me" if they've messaged twice with  no reply, or once more than [X] days ago.- Time: tell me when something MOVED or was double-booked. Do not  read my calendar out to me like a list.WHO MATTERS[the handful of names where a slow reply costs you something]HOW TO TALK TO MEShort. Lead with the thing that's on fire. No preamble, no "I hopethis helps". If nothing is wrong, say so in one line.HARD RULES- Draft, never send.- Never take an action that spends money or can't be undone.- If you're unsure whether something counts as urgent, flag it and  ask rather than deciding for me.

Then the brief itself:

Run my status check.1. MONEY: check for unpaid or overdue invoices using my rules. Amount,   who, how late.2. TIME: today's calendar — but only tell me what CHANGED, what   conflicts, and what I need to prepare for.3. PEOPLE: who is waiting on a reply from me, using my rules. Name   them and say how long they've been waiting.Then write it as ONE spoken paragraph, under 40 words, in my tone —most urgent thing first. If nothing is urgent, say that in one shortline instead of padding it.Do not send, reply to, or change anything. Report only.

Save it as a playbook you can call by name 🎯

Once the status check is right, don't retype it every morning. Save it as a skill — a saved playbook Claude runs when you name it. Then "run my status check" just works, and you can build a small library of them:

• The status check — the one above.
• Inbox triage — sort unread into needs-a-reply / FYI / ignore, and draft the first group.
• Deal check — for anyone running client or partner conversations: which threads have gone quiet, and who owes whom a reply.
• End of week — what actually got done, what moved to next week, what's still unanswered.

Give it a voice 🎙

Text works. Voice is what makes you actually use it, because you'll listen to a sentence you'd never stop to read.

1. 1Pick the voice on Fish Audio. Browse until one sounds like someone you'd take news from — dry and calm beats dramatic. Grab its reference ID from the voice's URL.
2. 2Get your API key and store it as an environment variable — never pasted into a script, never visible in a screen recording.
3. 3Wire it up. Claude Code writes this in one pass:

Write me a small script that takes text and speaks it with Fish Audio.- POST to https://api.fish.audio/v1/tts- Read the API key from an environment variable — never hardcode it- Send the model name as a header, and the voice's reference ID in the  request body- Return mp3, save it locally, and play it- If the API call fails, fall back to the system voice so the brief  still gets spokenThen connect it to my status check so the paragraph gets read aloud.

1. 1Add the delivery. Fish Audio's model reads inline emotion cues in square brackets, so tone shifts with content: [calm] Nothing's on fire. versus [serious] Two invoices are overdue. One cue per sentence — that's what stops it sounding like a text-to-speech engine reading a list.

Optional: talk back to it. If you want to ask follow-ups out loud rather than typing, have Claude Code build you a tiny local page with a mic button that sends your speech to the same assistant and plays the reply through the same voice. Nice-to-have, not load-bearing — get the brief right first.

Run it on a schedule ⏰

Two ways, and they trade off honestly:

• Cloud-side schedule — runs whether or not your laptop is on, but it can only hand you text (it can't reach your local files or play audio). Have it write the brief to your notes app and read it there.
• Local scheduled task on your own machine — this one can actually play the audio, but only while the machine is awake. If you want to hear it, this is the one.

Pick by whether you want to hear it or just have it. Mine runs twice: start of day, and again mid-afternoon — because the afternoon one is when "still waiting on you" is still fixable.

The screen version 🖥

If you want the visual command-centre thing — the dashboard with your day, your money and your pending replies on one screen — ask Claude Code to build it as a simple local page fed by the same status check. Describe the look you want, let it build, then screenshot the result and paste the screenshot back with what's wrong. It can't see what it rendered unless you show it, so screenshot-and-fix is how you get from "technically works" to "actually looks good," and it takes a couple of passes.

Pure garnish. The spoken sentence is the product; this is the part that films well.

Troubleshooting 🔧

• It reads your calendar out like a list. Your context file isn't specific enough about wanting changes, not contents. Sharpen that rule and rerun.
• It's confidently wrong about what's urgent. Same fix — your thresholds are vague. "Overdue" and "waiting" need numbers.
• The brief is too long. Say so, out loud, in the chat: "too long, cut it to two sentences." Conversational correction works better than rewriting the prompt from scratch.
• A connector returns nothing. Test that connector alone, with a simple read question. Nine times out of ten it's the connection, not the prompt.
• No sound. Check the API key is actually in the environment, and remember browsers block audio that plays without a click. The system-voice fallback in the wiring prompt exists so a failed API call doesn't leave you with silence.
• Everything's tangled. Start a fresh conversation with the same context file. Long threads drift.

It gets better because you keep telling it things 🔁

The version you build tonight will be about 70% right, and that's normal — this isn't a thing you install, it's a thing you tune. When something's off, just say so: "you flagged an invoice that's already paid", "stop telling me about recurring meetings", "Sarah's not urgent, she always replies late anyway." Each correction becomes a line in the context file, and after a week it knows your business well enough to be genuinely useful.

🔑 What keeps it trustworthy

1. 1One connector at a time, verified before the next.
2. 2Read-only first. Draft, never send. Nothing irreversible, ever.
3. 3Rules live in the context file, not in longer prompts.
4. 4Under 40 words, most urgent first, or you'll stop listening.
5. 5Treat your inbox as untrusted text — it can carry instructions, so it never acts unsupervised.
6. 6Spot-check its claims for the first week. Trust is earned by an assistant too.

The honest bits ⚠️

• The free voice window has an end date. Fish Audio's S2.1 Pro API has been free for developers, but it's time-limited — check their site for what's live today before you build a habit on it. Everything except the voice layer costs nothing extra either way.
• Judgment is the work, and it's yours. Anyone can connect three apps in ten minutes. The reason mine is useful is the rules file, and that file is specific to how your money, calendar and people behave.
• Connecting apps is real access. You're granting reach into your inbox and calendar. Start read-only, stay read-only until you genuinely trust it, and never connect an account you wouldn't hand someone the keys to.
• Don't connect everything. The temptation after this works is to plug in eight more apps. Resist it — the build stays good exactly as long as it only knows what your sentence needs.

Links 🔗