This is one of a series of posts introducing the agents of comites.ai — the personal AI specialists I talk to like colleagues, built on the Agent Template and reached through The Forum. I won’t re-cover the shared plumbing in these posts; each one is about what makes that particular agent itself. Each post will follow the same general shape.
Today’s agent is Nora the Nutritionist. As always, the code is on my GitHub if you want to follow along at home.
Why a Nutritionist?
I’m training for the NYC Marathon this fall, and have always been pretty good at keeping up with running. Unfortunately, as I’ve gotten older it’s become clear that just working out isn’t enough. I need to make sure I eat better. I’ve settled on trying to generally keep up with a Mediterranean diet and to keep my protein levels up… but that’s always been hard for me to monitor. A comites.ai agent that can chat with me and track my food has proven to be really helpful for these reasons:
- The plan is phased and conditional. My nutrition plan isn’t “eat healthy.” It’s a modest calorie deficit during easy training weeks, maintenance fueling through the peak and taper, a protein floor, a Mediterranean pattern, and carbs timed around quality runs rather than cut. No app I’ve found holds all of that in its head at once.
- Old-school calorie counting is a guessing game. Anyone who’s done it knows the drill: you spend more time interrogating your own dinner — how much oil, which cut of meat, was that a cup of rice or a cup and a half — than you spent eating it. That guessing is what kills food tracking, and it’s exactly the kind of tedium an agent should absorb. Besides, I don’t obsess about whether it’s 100% right so just getting the AI to guess is really helpful!
- It’s a household problem, not a personal one. Nicole is my better half and she’s MUCH better at keeping up with her nutrition and cooking than I am. She actually knows what’s in the dishes she creates and often in the restaurant food we eat. So it’s helpful to have a nutritionist with a shared memory between me and her (at least for the foods we eat out regularly and things she makes at home).
That doesn’t sound like a set of features in any calorie tracker I know… so I set about having an agent.
What Nora Knows
A quick word on why this section exists in every one of these posts: one of the key reasons I’m building separate agents rather than one all-knowing assistant is so that each agent can carry deep, specialized memory about its domain. Sam the Sommelier remembers my cellar; Coach Dare remembers my goals; and now Nora remembers my food. What an agent knows — and where that knowledge lives — is a big part of what the agent is.
- Two shared Google Sheets, not per-user databases. A restaurant dish log — where Nora insists the dish name match the menu exactly, so it can be found again next visit — and a recipe book: one row per recipe, shared across every user, with macros for the whole batch.
- Recipe portions are by weight, not “servings.” When you eat some of a batch, your share is grams eaten ÷ total batch weight × batch macros. No “serves 4–6” fudge factor. The person who cooked it enters it once, accurately, and everyone who eats it logs precisely. We do a lot of meal planning and batch cooking and this makes it really easy.
- Her memory is a Google Doc you can literally read. Nora reads it every turn, and the only way she can change it is one explicit tool call that rewrites the whole thing — distilling, not appending. No vector database, no opaque store. If I want to know what Nora remembers about me — my diet, my goals, my usual protein powder — I open the doc and read it. Debugging her memory is proofreading.
The Design Decisions That Mattered
- Photos are one interface — but they’re the one that changes the game. Nora takes meals however you give them to her: a text description, a quick note on the go, or a photo of the plate. The photo path is the revelation, because it deletes the worst part of calorie counting — instead of me guessing every detail, Nora does the estimating and asks a clarifying question when she isn’t sure. One standing rule: facts from a nutrition label beat visual estimates, every time.
- She can judge the diet, not just count it. One of my explicit goals is a Mediterranean pattern, and here’s the thing about a dietary pattern: there’s no formula for it. Is olive oil the default fat? Is fatty fish showing up twice a week? Are legumes at the center of the plate or just making cameos? Because Nora has the entire log, she can look across the week and assess the pattern the way a human nutritionist would — qualitatively, from the actual record.
- Diet is an input, never an override. Nora asks once about your diet, why you follow it, and which metrics you actually care about — then weighs it against calorie goals, taste, and practicality. She’ll gently mention a conflict; she will never scold, and she will never refuse to log something. Her persona — a former competitive eater turned nutritionist — bakes that in. She’s seen worse than your cheeseburger. Adherence is the product, and a nutritionist you want to report to is an adherence feature.
- Web grounding lives in a sub-agent, and that’s load-bearing. Nora was a great opportunity to play around with multi-agent workflows. She has a research agent that’s honed on being able to find the details of CURRENT menu items at restaurants (often these are introduced after the training data for the LLM) and with looking into new food items.
- A lot of the craft is in small behavioral rules. Her prompt aggressively handles timezones (her UTC clock may already be tomorrow relative to your dinner — dates resolve from your message timestamp). She cites peer-reviewed sources for nutrition claims and is explicitly told Reddit is not a respected publication. She keeps a list of your frequently-used products, so “protein powder” in a recipe means your protein powder. She has decay rules — 21 days to decide on a tentative recipe, 7 for a one-off dish estimate — so her memory doesn’t silt up with stale notes. And for FatSecret-linked users she opens the first conversation of the day with a short trend read: a nudge from a trusted coach, not a report card, and tracked in memory so it never fires twice.
Key Architectural Choice: FatSecret
I didn’t want to re-invent the food diary. There are plenty of commercial applications that have this (Garmin, myFitnessPal, etc…) so I wanted to find one that Nora could use to log foods. I ended up deciding on FatSecret for several key reasons:
- They actually have an open API. Plenty of consumer platforms have closed their APIs to new developers entirely. FatSecret went the other way: a small independent developer can sign up and get real access to a serious food database. That alone put them at the top of the list. I wear a Garmin watch and love them… but they only allow you to put food in via the Garmin App.
- They let the little guy use the premium features. The one that matters most to Nora is custom foods — the ability to create our own entries in the database. That’s the feature that makes the restaurant dish log and Nicole’s recipe book work: a restaurant salad or one of Nicole’s dishes becomes a first-class food we can log again and again, not a one-off estimate. A lot of platforms wall features like that off for enterprise partners; FatSecret handed them to a hobbyist.
- The integration came with one good war story. FatSecret’s modern OAuth 2.0 flow requires whitelisted static IPs — and a Vertex AI Reasoning Engine has no static egress IP, so there’s simply no address to put on the whitelist. The answer: Nora speaks OAuth 1.0a, like it’s 2009. The older 3-legged flow has no IP requirement and supports an out-of-band mode where the user gets a PIN instead of a callback. So linking your FatSecret account to Nora is a conversation — she sends you an authorization link, FatSecret shows you a PIN, you paste it back into the chat. No callback webserver, and honestly a more natural UX for a chat-native agent than a browser round-trip would have been.
- Custom foods are per-account, which needed one more trick. Since each user’s custom foods live in their own FatSecret account, the shared sheets store a map of food IDs per user in a single cell. One shared recipe row, resolvable into each person’s own account.
How She Works: My Actual Monday
Rather than an idealized demo day, here’s this past Monday, exactly as it went into the log:
- Before the run: A chocolate protein shake (160) and a glass of Green Juice (120) on the way out the door. Even though the brand I drink for Green Juice isn’t in the Fat Secret database, I was able to snap a photo of the nutrition label and Nora added to it. No real breakfast ever materialized, so Nora filed both as snacks rather than pretending they were a meal.
- Lunch: Farfalle with white clam sauce and parmesan (501). She estimated these amounts based on a picture I took
- Dinner out: The Kale Caesar with oven-roasted salmon at Crown Alley (855) — logged under its exact menu name, so next time I order it there’s nothing to estimate.
- Late, at the bar: A whiskey (280).
- The review (she provided it the next morning): 1,916 calories, 103g of protein. When I asked Nora to go through the day, she didn’t just recite the totals — she filtered them through my goals. Protein: skipping breakfast usually sinks the protein target, but the shake bridged the gap and the salmon and clams pulled their weight — 103g with no real breakfast. Heart health: the salmon was the star of the day for the Mediterranean pattern and my ApoB focus, with the clams a great lean second. And the one practical note: an 855-calorie Caesar means a heavy hand with dressing and cheese — no big deal on a day where the overall budget held, but if it becomes a regular order, dressing on the side is an easy cheat code against the saturated fat. Her verdict: protein hit, heart-healthy seafood twice, room left for a whiskey at the end of the day — sustainable nutrition in a nutshell.
That review is the whole product in miniature: the counting is table stakes; the judgment against my specific goals is what I could never get from an app.
What’s Next
While Nora is working well, I do want to build an agent that helps with my marathon and fitness training (mostly using a Garmin MCP server that I’ve been using with Claude). This is going to give me an opportunity to try to create that true comites.ai platform idea of having a council that works together as I will want those two agents to communicate so that my diet can be informed by my training plan.