Perception grounding

You're Claude Code, working in my repository.

Task
Add perception grounding to my app: send a photo plus a dietary prompt to Muse Spark in one Responses call, get one point per food item on the normalized 0 to 1000 grid with recommended, a health score out of 10, a justification and nutrition, then draw green or red dots over the image with the score above each dot and a tooltip that stays inside the image, and save a self-contained HTML overlay file.

Read this first
Meta's docs page "Perception grounding" as Markdown: https://dev.meta.ai/docs/cookbook/perception-grounding.md
It's the source of truth for request and response shapes. The same page for people: https://dev.meta.ai/docs/cookbook/perception-grounding

Setup
- Base URL: https://api.meta.ai/v1
- Auth: send Authorization: Bearer $MODEL_API_KEY, reading MODEL_API_KEY from the environment. Never hard-code, print or commit the key.
- Model: muse-spark-1.3-contributor (Contributor mode: Meta may use what you send to train future models; drop -contributor for Standard mode)
- Python: the OpenAI SDK (pip install openai) with base_url="https://api.meta.ai/v1" and api_key=os.environ["MODEL_API_KEY"] works, or plain requests.

Facts to get right
- Grid: (0, 0) is the top-left and (1000, 1000) the bottom-right of the image you sent; convert with x_px = x * width / 1000 against that exact image.
- Use structured output (text.format json_schema, strict) so the points parse first time; ask for one point per item, no duplicates, on the item itself.
- The recipe's own mode has the model write the whole HTML in one call; give it the image size and a placeholder image src you replace, and only open the result sandboxed.
- The recipe reads MODEL_API_KEY, MODEL_NAME (muse-spark-1.3-contributor here) and MODEL_BASE_URL (https://api.meta.ai/v1).

Starter code (Python)
import base64, json, os
from openai import OpenAI

client = OpenAI(base_url="https://api.meta.ai/v1", api_key=os.environ["MODEL_API_KEY"])
PROMPT = "I am pescatarian with high cholesterol. Put green dots on recommended food and red dots on not recommended food. Don't duplicate dots and make sure the dots are localized properly."
n, s = {"type": "number"}, {"type": "string"}
props = {"name": s, "x": n, "y": n, "recommended": {"type": "boolean"}, "health_score": n, "justification": s,
         "calories": n, "carbs_g": n, "protein_g": n, "fat_g": n}
item = {"type": "object", "properties": props, "required": list(props), "additionalProperties": False}
SCHEMA = {"type": "object", "properties": {"items": {"type": "array", "items": item}}, "required": ["items"], "additionalProperties": False}

photo = "data:image/jpeg;base64," + base64.b64encode(open("food_in_fridge.jpeg", "rb").read()).decode()
r = client.responses.create(model="muse-spark-1.3-contributor", input=[{"role": "user", "content": [
    {"type": "input_text", "text": PROMPT + " Give each item one point (x, y) on a 0-1000 grid of this image."},
    {"type": "input_image", "image_url": photo}]}],
    text={"format": {"type": "json_schema", "name": "food_points", "strict": True, "schema": SCHEMA}})
for it in json.loads(r.output_text)["items"]:
    print(it["name"], it["x"], it["y"], "green" if it["recommended"] else "red", it["health_score"])

Done when
- Every point parses and falls on its item when drawn.
- Green and red follow the dietary prompt, with the score visible above each dot.
- The saved HTML opens offline with the photo inlined and tooltips that stay in frame.
- The key comes from MODEL_API_KEY and never appears in code, logs or commits.
- API errors (4xx, 429) surface their message instead of failing silently.