Navigation text · little learning value
Model input
“Python tutorials | Java tutorials | Sign in | Subscribe | Accept cookies | Privacy policy | Next page”
Filter AI training data
Teach a quality filter which documents belong in a coding training corpus.
Step 01
Define and Align
Review borderline documents and explain what makes them useful training material.
Output schema
- KeepYes / No
Whether the text teaches a concrete programming concept.
- true
- false
- Content typeChoose one
The kind of content in the record.
- instructional_example
- navigation
- keyword_spam
- other
- ReasoningText
Why the content meets or fails the learning-value policy.
Collecting feedback · 1/3
Decide whether a document should be kept in a coding training corpus. Prefer substantive explanations and working code over boilerplate or keyword spam.
Validation accuracy
—
Example 1 of 3
3 annotations saved
Sutro can now generalize the rules based on your feedback.
Step 02
Model Sweep
Compare models against the same quality labels and choose a tradeoff.
Compare models on expert agreement and cost. The interactive chart uses illustrative measurements.
Illustrative sweep · select Value or Best to use that model in your function.
Step 03
Deployment
Call Sutro Functions like any other software. Swap the model as soon as something more performant is released.
Selected modelWaiting for sweep results.
Real-time
import sutro as so
input_data = {
"text": "Use enumerate to get both the index and value: for i, item in enumerate(items): print(i, item)."
}
result = so.run_function(
"training-data-filtering",
input_data,
)curl --fail-with-body -X POST "${SUTRO_API_URL%/}/v1/functions/training-data-filtering/run" \
-H "Authorization: Key $SUTRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input": {
"text": "Use enumerate to get both the index and value: for i, item in enumerate(items): print(i, item)."
}
}'Batch
import sutro as so
job_id = so.batch_run_function(
name="training-data-filtering",
data="coding_documents.csv",
)
results = so.await_job_completion(job_id)curl --fail-with-body -X POST "${SUTRO_API_URL%/}/v1/batch-inference" \
-H "Authorization: Key $SUTRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "training-data-filtering",
"inputs": [
{
"text": "Use enumerate to get both the index and value: for i, item in enumerate(items): print(i, item)."
}
]
}'Label a training corpus
Use schema-based annotation to turn your quality policy into consistent training-data labels. Review representative records, explain the edge cases, and apply the learned policy when labeling or filtering the full corpus.