Few Shot Tune Check
Status: Live.
Your muse
Rebuild ofLlama Best Practices: Fine-Tuning Llama Models in the Real World (opens in a new tab)Reframe
Tell me whether my classification task still needs fine-tuning, using a few labeled examples. This rebuilds Meta's video "Llama Best Practices: Fine-Tuning Llama Models in the Real World" (https://dev.meta.ai/llama/resources/videos/fine-tuning-llama-models-in-the-real-world) on current Muse models. It demonstrates: Fine-tuning strategies for domain-specific problems. Plan: Muse Muse builds apps, not training runs, and the hosted Muse models aren't fine-tuned through the API. The rebuild checks a few-shot Spark 1.3 setup against the user's labeled examples and says when fine-tuning open weights like Glimmer would still be worth it.
Checks passed
The plan: Few Shot Tune Check
Test few shot prompts on your examples and learn if tuning is still needed.
Can build part of it
- Describe your classification task and paste 6 to 20 labeled examples
- App holds out a few examples and prompts Spark 1.3 with the rest
- Review few shot accuracy, errors by label, and tuning recommendation
Won't do: Rebuild fit: reframe
See the full plan
Who it's for
Developers and product builders with a small text classification task
How it works
- Describe your classification task and paste 6 to 20 labeled examples.
- App holds out a few examples and prompts Spark 1.3 with the rest.
- Review few shot accuracy, errors by label, and tuning recommendation.
Screens
- Task Setup. Enter task description, labels, and pasted examples for testing.
- Fit Report. Show few shot accuracy, error patterns, and when open weights tuning would help.
What goes in
- Text: Task description, label list, and 6 to 20 examples as text plus label lines. Plain text under 100k characters.
What comes out
- Text: Plain language summary with few shot accuracy, error patterns, and keep or tune advice.
- JSON: Structured scores with overall accuracy, per label results, and tuning recommendation.
Models
Model Used for Why Muse Spark 1.3 Run held out few shot predictions and explain if tuning is needed Reads the task and examples, predicts held out labels, and returns structured scores plus tuning guidance. Limits
- Rebuild fit: reframe. The original video shows Llama fine tuning training runs, but this app builds no training run and cannot fine tune hosted Muse models. It only tests few shot prompting and advises when tuning open weights locally could be worth it.
- Accuracy is an estimate from a few held out examples, not a full test set.
- Needs at least 6 clean labeled examples with 2 or more labels.
- Gives general guidance for local open weights work and does not train or host a tuned model.
From your inspo
Rebuild of Meta developer video Llama Best Practices Fine Tuning Llama Models in the Real World. Keeps the focus on choosing a tuning strategy for domain tasks, reframed as a few shot check with current hosted models.
Takes
- choosing when a domain classification task needs fine tuning
Leaves out
- video footage
- video script
- Meta and Llama names and logos
Cost per run: about $0.007 (est.). One Spark 1.3 call with about 3,000 input tokens at $1.25 per million plus about 600 output tokens at $4.25 per million totals about $0.007. Total about $0.007 per run.
Recorded demo Done.
npm run verifyexits 0 (4 tests passed, manifest valid).