Pattern detection across memories. The duck sees what you haven't said.
Your duck remembers facts. It knows you're stressed about work. It knows you have a daughter named Luna. It knows you've been thinking about your side project. But it doesn't think about any of that. Each memory sits in DynamoDB like a note pinned to a corkboard — isolated, unconnected, waiting for someone to step back and see the pattern.
Part 3 adds that step back. You'll build a second loop — a slower one, running weekly — that reads all your memories, finds patterns across them, and surfaces insights. The duck won't just remember what you said. It'll notice what it means.
This is the part where the tutorial stops being purely technical and starts being a little philosophical. The code is shorter than Part 1 or Part 2. The questions it raises are not.
Until now, your agent has one loop: the daily cycle. EventBridge fires, the Ask function generates a question, you reply, the Listen function extracts memories, and the cycle resets. It's reactive — something happens, the duck responds.
Now you're adding a second loop that runs on a different clock entirely.
This is what you built in Parts 1 and 2. It runs every morning:
This loop is immediate. Something comes in, something goes out. It handles the daily conversation.
This is the new one. It runs once a week, on Sunday morning:
This loop is slow. Nothing is responding to anything. The duck is just thinking — reviewing what it's heard, looking for signal across the noise.
This is a real pattern in agent design: daily ingestion, periodic synthesis. Not every agent loop should run at the same frequency. Think of it like journaling vs. therapy — one records what happened, the other finds meaning in it.
You could run synthesis daily. But five new memories per day don't create patterns — fifty memories across two weeks do. A weekly cadence gives the model enough material to work with and enough time between runs for new data to accumulate. The daily loop feeds the weekly loop. The weekly loop improves the daily loop. Two clocks, one system.
Your memory system now has two tiers. Understanding the distinction is the key to this entire part.
These are extracted directly from what you said. They come from the daily Listen loop and map one-to-one with things you actually told the duck:
feelingfactgoalMemories are grounded. They trace directly back to a specific reply. If you didn't say it, it doesn't become a memory.
These are inferred by looking across memories. They come from the weekly synthesis loop and represent things you never explicitly said:
Observations are derived. No single reply contains them. They emerge from the pattern of your replies over time.
You already defined this in Part 2's types.ts. Let's look at it again:
/** A synthesized observation derived from multiple memories (Part 3) */
export interface Observation {
id: string;
userId: string;
content: string;
basedOn: string[]; // Memory IDs this observation was derived from
createdAt: string;
source: "synthesis";
}The critical field is basedOn: string[]. Every observation carries an array of memory IDs — the specific memories that led to this insight. This is traceability. You can always ask "why does the duck think this about me?" and get a concrete answer: because of these three memories, on these dates, where you said these things.
This isn't just good engineering. It's a trust mechanism. An observation without evidence is a hallucination. An observation with receipts is an insight.
This is the only new file in Part 3. Create src/synthesize.ts:
import { askClaude } from "./lib/bedrock.js";
import {
getMemories,
getObservations,
saveObservation,
} from "./lib/memory.js";
const SYNTHESIS_PROMPT = `You are a pattern recognition system. You're given a collection of memories about a person — things they've said over the past days and weeks. Your job is to find patterns, recurring themes, and insights that aren't obvious from any single memory.
Look for:
- Recurring emotions or states ("often mentions being tired on Mondays")
- Themes across topics ("talks about their side project with more energy than their day job")
- Changes over time ("seemed stressed last week but more relaxed this week")
- Connections between things ("mentions their daughter whenever they talk about motivation")
Output a JSON array of observations. Each observation should be:
- "content": a concise insight about this person (1-2 sentences)
- "basedOn": array of memory IDs that support this observation
Only include genuinely interesting patterns. If there aren't enough memories yet or nothing stands out, return an empty array. Quality over quantity — 1-3 observations max.
Return ONLY the JSON array, no other text.`;
export async function handler() {
const userEmail = process.env.USER_EMAIL!;
const memories = await getMemories(userEmail);
const existingObservations = await getObservations(userEmail);
if (memories.length < 5) {
console.log(
`Only ${memories.length} memories — skipping synthesis until there are at least 5.`
);
return;
}
const memoryList = memories
.map((m) => `[${m.id}] [${m.category}] [${m.createdAt}] ${m.content}`)
.join("\n");
let prompt = `Here are the memories collected about this person:\n\n${memoryList}`;
if (existingObservations.length > 0) {
const existingList = existingObservations
.map((o) => `- ${o.content}`)
.join("\n");
prompt += `\n\nYou've already made these observations (don't repeat them, but you can build on them):\n${existingList}`;
}
const resultJson = await askClaude(SYNTHESIS_PROMPT, prompt);
try {
const observations: Array<{ content: string; basedOn: string[] }> =
JSON.parse(resultJson);
for (const obs of observations) {
const saved = await saveObservation(userEmail, obs.content, obs.basedOn);
console.log("New observation:", saved.content);
}
console.log(`Synthesized ${observations.length} new observations`);
} catch (err) {
console.error("Failed to parse synthesis result:", resultJson, err);
}
}The structure is simpler than Ask or Listen. No email, no S3 events. Just: load memories, think about them, save what you find. Let's walk through the key decisions.
if (memories.length < 5) — don't synthesize from too little data. Five memories is about two days of replies. Below that, any "pattern" Claude finds is just noise. The function logs a message and exits early. No observations, no wasted Bedrock calls.
You could raise this threshold. Ten memories gives you a stronger signal. Twenty gives you a week's worth. Five is the floor — enough to detect something, not so much that you're waiting forever for the first insight.
Each memory gets formatted with its ID, category, timestamp, and content:
[abc-123] [feeling] [2024-11-15T09:30:00Z] Stressed about API migration at work
[def-456] [fact] [2024-11-15T09:30:01Z] Has a daughter named Luna
[ghi-789] [event] [2024-11-18T10:15:00Z] Planning side project launch for end of monthThe IDs are there so Claude can reference them in the basedOn field. The timestamps give temporal context — Claude can see that something was said last Monday vs. yesterday. The categories help Claude understand what kind of signal each memory carries.
If this isn't the first synthesis run, we append existing observations to the prompt with a clear instruction: "don't repeat them, but you can build on them." Without this, the duck would re-discover the same patterns every week. With it, observations can evolve — "seems stressed about work" might become "stress about work has decreased over the past two weeks" as new memories accumulate.
Same pattern as the memory extraction in Part 2. Claude might return invalid JSON, or wrap it in markdown fences, or add a preamble. The catch logs the raw output and the error, but doesn't crash. Synthesis is a background process — if it fails this week, it'll try again next week with even more data.
The synthesis prompt is doing sophisticated reasoning work in about 150 words. It's worth understanding why it works:
basedOn field forces traceability. Claude can't just assert a pattern — it has to point to the memory IDs that support it. This is a structural constraint that improves output quality. If the model can't find supporting evidence, it's less likely to hallucinate a pattern.This is where prompt engineering stops being "write a system message" and starts being "design a reasoning system." The prompt's structure — what it asks for, what it constrains, what it permits — shapes the quality of the output as much as the model itself does.
The Synthesize function needs a Lambda definition and a cron trigger, just like the Ask function. Add these to your sst.config.ts, after the email bucket subscription and before the return statement.
Two new blocks: the function definition and the weekly schedule.
// --- Synthesize Lambda: weekly pattern detection (Part 3) ---
const synthesizeFn = new sst.aws.Function("SynthesizeFn", {
handler: "src/synthesize.handler",
timeout: "60 seconds",
environment: {
TABLE_NAME: table.name,
USER_EMAIL: userEmail.value,
},
permissions: [
{
actions: ["bedrock:InvokeModel"],
resources: ["*"],
},
],
link: [table],
});
// --- Weekly synthesis: Sunday 8am UTC ---
new sst.aws.Cron("WeeklySynthesize", {
schedule: "cron(0 8 ? * SUN *)",
function: synthesizeFn.arn,
});A few things to notice:
Here's the complete sst.config.ts with all three parts:
/// <reference path="./.sst/platform/config.d.ts" />
export default $config({
app(input) {
return {
name: "pip-agent",
removal: input?.stage === "prod" ? "retain" : "remove",
home: "aws",
providers: {
aws: {
region: "us-east-1",
},
},
};
},
async run() {
// --- Email address ---
const userEmail = new sst.Secret("UserEmail");
// --- S3 bucket for inbound email ---
const emailBucket = new sst.aws.Bucket("EmailBucket");
// --- DynamoDB table for memories + question history ---
const table = new sst.aws.Dynamo("MemoryTable", {
fields: {
pk: "string",
sk: "string",
},
primaryIndex: { hashKey: "pk", rangeKey: "sk" },
});
// --- Ask Lambda: generates daily question, sends email ---
const askFn = new sst.aws.Function("AskFn", {
handler: "src/ask.handler",
timeout: "30 seconds",
environment: {
TABLE_NAME: table.name,
USER_EMAIL: userEmail.value,
},
permissions: [
{
actions: ["bedrock:InvokeModel"],
resources: ["*"],
},
{
actions: ["ses:SendEmail"],
resources: ["*"],
},
],
link: [table],
});
// --- Schedule: 7am UTC daily ---
new sst.aws.Cron("DailyAsk", {
schedule: "cron(0 7 * * ? *)",
function: askFn.arn,
});
// --- Listen Lambda: processes email replies ---
const listenFn = new sst.aws.Function("ListenFn", {
handler: "src/listen.handler",
timeout: "30 seconds",
environment: {
TABLE_NAME: table.name,
USER_EMAIL: userEmail.value,
},
permissions: [
{
actions: ["bedrock:InvokeModel"],
resources: ["*"],
},
{
actions: ["ses:SendEmail"],
resources: ["*"],
},
],
link: [table],
});
// Subscribe listen function to new objects in the email bucket
emailBucket.subscribe(listenFn, {
events: ["s3:ObjectCreated:*"],
});
// --- Synthesize Lambda: weekly pattern detection (Part 3) ---
const synthesizeFn = new sst.aws.Function("SynthesizeFn", {
handler: "src/synthesize.handler",
timeout: "60 seconds",
environment: {
TABLE_NAME: table.name,
USER_EMAIL: userEmail.value,
},
permissions: [
{
actions: ["bedrock:InvokeModel"],
resources: ["*"],
},
],
link: [table],
});
// --- Weekly synthesis: Sunday 8am UTC ---
new sst.aws.Cron("WeeklySynthesize", {
schedule: "cron(0 8 ? * SUN *)",
function: synthesizeFn.arn,
});
return {
emailBucketName: emailBucket.name,
tableName: table.name,
};
},
});You don't need to change the Ask function. The code you wrote in Part 2 already loads observations — it's just been loading an empty list until now. Let's trace how it works once observations exist.
Every morning, the Ask function does three queries: memories, observations, and recent questions. In Part 2, observations always came back empty. Now, after the first synthesis run, there might be one, two, or three observations sitting in DynamoDB.
These get appended to the prompt as a "Patterns you've noticed" section:
if (observations.length > 0) {
const obsList = observations
.slice(0, 5)
.map((o) => `- ${o.content}`)
.join("\n");
context += `\n\nPatterns you've noticed:\n${obsList}`;
}That's the bridge between the two loops. The weekly synthesis writes observations. The daily Ask reads them. The question gets deeper.
Without observations, the duck's prompt has a list of discrete memories. It might generate:
"How's work going this week?"
Fine. Generic. Technically informed by the "stressed about work" memory, but not deeply.
With observations, the prompt includes something like: "Talks about side project with noticeably more energy than day job." Now the duck might ask:
"You've been mentioning your side project a lot more than work lately — is that pulling you toward something?"
That's a different kind of question. It's not referencing a single memory. It's referencing a pattern the duck noticed across many memories. The question feels like it comes from someone who has been paying attention over weeks, not just recalling yesterday's conversation.
npx sst deploy --stage devThis adds the new Synthesize Lambda and the weekly cron. Your existing Ask and Listen functions are unchanged.
Synthesis requires at least 5 memories. If you've been using the duck for a few days and replying with substantive answers, you should have enough. Check the DynamoDB table — look for items with sk starting with MEMORY# and count them.
If you have fewer than 5, keep using the duck for a few more days. Reply with real answers, not "fine." The extraction system needs material to work with.
Don't wait until Sunday. Trigger it now:
aws lambda invoke \
--function-name $(aws lambda list-functions \
--query "Functions[?contains(FunctionName, 'SynthesizeFn')].FunctionName" \
--output text) \
--payload '{}' \
/dev/stdoutIf you have 5+ memories, check CloudWatch Logs for the Synthesize function. You should see "New observation:" lines followed by the insights Claude found, and a final "Synthesized N new observations" message. If you have fewer than 5, you'll see the early-exit message.
Go to the DynamoDB console, find your table, and look for items with sk starting with OBSERVATION#. Each one should have a content field with the insight and a basedOn field with an array of memory IDs.
Click through to some of the memory IDs. Do the observations make sense given the memories they reference? This is the traceability at work — you can audit every conclusion the duck reaches.
Now invoke the Ask function:
aws lambda invoke \
--function-name $(aws lambda list-functions \
--query "Functions[?contains(FunctionName, 'AskFn')].FunctionName" \
--output text) \
--payload '{}' \
/dev/stdoutCheck the email. Does the question feel different? Does it reference a pattern rather than a single fact? The CloudWatch logs will show you the context that was sent to Claude, including the observations — look for the "Context included" line with the observation count.
That's fine. This isn't a failure state. Keep using the duck for a week — reply to it every morning with something real. After 5-7 days, come back and run the synthesis manually. The code is deployed and the weekly cron is ticking. It'll run on its own next Sunday.
Your duck now remembers what you tell it and notices patterns in your life. That's powerful. It's also a responsibility.
Let's talk about what you've built. There's a DynamoDB table in your AWS account that contains extracted facts about your emotional state, your goals, your relationships, and your daily rhythms. A model is reading those facts weekly and making inferences about you — observations you didn't explicitly share, derived from the pattern of things you did.
She's not wrong about this part. I've heard things on that desk I'd never repeat. Not even to HR.
This is useful. It's also worth thinking carefully about.
What should the duck never do? Send your memories to someone else? Analyze your mental health and offer diagnoses? Give unsolicited advice about your relationships? Right now the duck is scoped to asking questions and noticing patterns. But the architecture supports more. The question is whether "could" should become "should."
What data are you comfortable storing? Everything is in your AWS account — you own it, you control it, you can delete it. That's better than sending your daily reflections to a third-party API with a terms-of-service update pending. But "you own it" doesn't mean "it's safe." DynamoDB encrypts data at rest by default, but IAM policies can be misconfigured. An AWS credential leak could expose everything.
What if the observations are wrong? Claude might see a pattern that isn't there. "Seems unhappy at work" could be an accurate read of your memories, or it could be an overinterpretation of one bad Monday. The basedOn field helps — you can check the receipts. But the duck doesn't present its observations as hypotheses. It presents them as context for the next question. There's a difference between "I wonder if you're unhappy at work" and silently asking you a question that assumes you are.
If you've read the guide, you know this territory. Ember drew a line: reminders to herself, not auto-wishes to other people. "That feels weird," she said, before she could articulate why. Confidence and correctness are different things — that's the lesson of the leap-day bug, and it applies here too. The duck might be confident in a pattern. That doesn't mean the pattern is real.
Where's your line? There's no right answer. But the fact that you're building this yourself — that you can read the prompt, audit the observations, and delete the table — means you get to decide. That's worth something.
The moment the duck says something true about you that you didn't explicitly tell it — something it inferred from the pattern of your replies — that's when it stops being a tutorial project and starts being something else. Something yours.
The tutorial is done, but the duck is just getting started. Here are ideas for where to take it — none of these are built, and none need to be. They're seeds.
USER#email. Add authentication, a sign-up flow, and you have a product. (Whether you should is the trust conversation again.)Not the marketing kind. Not the "autonomous AI that replaces your workforce" kind. The real kind. A loop with autonomy, a memory, and a growing understanding of one specific person: you.
The architecture is simple. Two Lambdas that talk to you (Ask and Listen), one that thinks quietly in the background (Synthesize). A DynamoDB table. An S3 bucket. A schedule. And a model that's getting better at knowing what to ask — not because it's getting smarter, but because it has more to work with. The magic isn't in the stack. It's in the loop.
...both. Definitely both.
You started with a duck that asked a stranger what's on their mind. Now you have a duck that knows your daughter's name, noticed you're more excited about your side project than your day job, and asks you questions that make you think. It did that in a few hundred lines of TypeScript and a well-designed prompt. Pip would be proud. Or furious. Probably both.