Tune Need Checker
Status: Live.
Your muse
Rebuild ofLlama Best Practices: Fine-Tuning Llama Models in the Real World (opens in a new tab)Reframe
Tell me whether my classification task still needs fine-tuning, using a few labeled examples. This rebuilds Meta's video "Llama Best Practices: Fine-Tuning Llama Models in the Real World" (https://dev.meta.ai/llama/resources/videos/fine-tuning-llama-models-in-the-real-world) on current Muse models. It demonstrates: Fine-tuning strategies for domain-specific problems. Plan: Muse Muse builds apps, not training runs, and the hosted Muse models aren't fine-tuned through the API. The rebuild checks a few-shot Spark 1.3 setup against the user's labeled examples and says when fine-tuning open weights like Glimmer would still be worth it.
Checks passed
The plan: Tune Need Checker
Test few shot prompting on your labels and see if fine tuning is still needed.
- Describe the classification task and list the allowed labels
- Paste 6 to 20 labeled examples and mark some for testing
- Run the few shot check with Spark on the held out examples
- Review per example results, accuracy, and the fine tune recommendation
See the full plan
Who it's for
Developers and product teams deciding between prompting and fine tuning for a text classifier.
How it works
- Describe the classification task and list the allowed labels.
- Paste 6 to 20 labeled examples and mark some for testing.
- Run the few shot check with Spark on the held out examples.
- Review per example results, accuracy, and the fine tune recommendation.
Screens
- Setup Task. Enter task goal and label set for the classifier.
- Example Library. Add labeled examples and choose few shot and test split.
- Results Report. Show accuracy, error patterns, and fine tuning recommendation.
What goes in
- Text: Task description plus label definitions in plain text under 2000 characters.
- Text: Labeled examples as text and label pairs, 6 to 20 rows for few shot check and test split.
What comes out
- Text: Accuracy summary with per example predictions and clear prompting or fine tuning recommendation.
- JSON: Structured scores with predicted labels, error notes, and failure patterns for export.
Models
Model Used for Why Muse Spark 1.3 Classify held out examples with few shot prompting and recommend prompting or open weight fine tuning. Default reasoning model that reads task text and examples and returns structured predictions and advice in one low cost call. Limits
- Rebuild fit: reframe. The original video shows fine tuning training runs, but this app does not train models because Muse Muse builds web apps and hosted models cannot be fine tuned, so it tests few shot Spark and advises when open weight fine tuning is worth it.
- Does not fine tune any model and does not launch training jobs.
- Results are estimates from a small sample and may change with more data.
- Advice about Glimmer or other open weights is general guidance and needs separate hosting to use.
From your inspo
Rebuild of Meta video Llama Best Practices: Fine-Tuning Llama Models in the Real World. It keeps the domain classifier fine tuning decision and reframes it as a few shot Spark check with guidance on open weight fine tuning.
Takes
- Deciding if a domain classification task needs fine tuning
Leaves out
- Video footage
- Video script and narration
- Meta and Llama names and logos
Cost per run: about $0.01 (est.). One Spark run with about 4000 input tokens at 1.25 per million is 0.005 and about 1200 output tokens at 4.25 per million is 0.0051 for a total of about 0.01. Total is 0.01.
Recorded demo Done.
npm run verifyexits 0, all 7 steps green.