Apply your standards for resolution, communication quality, and policy adherence across support conversations.
Choose an example
Score a support conversation against a resolution and communication rubric.
Step 01
Choose an example, review the proposed output, and annotate the fields that need to change. Your guidance defines the learned rules.
Overall result against the resolution and communication rubric.
Whether the conversation resolves the issue or gives usable next steps.
Specific quality problems found in the conversation.
Score a support conversation against a resolution and communication rubric.
How can I restore a deleted project?
Thanks for contacting us!
I am closing this ticket now.
Sutro can now generalize the rules based on your feedback.
Step 02
Compare quality and cost on your examples, then choose a model or use Sutro’s default.
Compare models on expert agreement and cost. The interactive chart uses illustrative measurements.
Illustrative sweep · select Value or Best to use that model in your function.
Step 03
Call Sutro Functions like any other software. Swap the model as soon as something more performant is released.
import sutro as so
input_data = {
"conversation": "[{\"role\": \"customer\", \"content\": \"How can I restore a deleted project?\"}, {\"role\": \"agent\", \"content\": \"Thanks for contacting us!\"}, {\"role\": \"agent\", \"content\": \"I am closing this ticket now.\"}]"
}
result = so.run_function(
"support-conversation-quality-scoring",
input_data,
)curl --fail-with-body -X POST "${SUTRO_API_URL%/}/v1/functions/support-conversation-quality-scoring/run" \
-H "Authorization: Key $SUTRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input": {
"conversation": "[{\"role\": \"customer\", \"content\": \"How can I restore a deleted project?\"}, {\"role\": \"agent\", \"content\": \"Thanks for contacting us!\"}, {\"role\": \"agent\", \"content\": \"I am closing this ticket now.\"}]"
}
}'import sutro as so
job_id = so.batch_run_function(
name="support-conversation-quality-scoring",
data="support_conversation_quality_scoring.csv",
)
results = so.await_job_completion(job_id)curl --fail-with-body -X POST "${SUTRO_API_URL%/}/v1/batch-inference" \
-H "Authorization: Key $SUTRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "support-conversation-quality-scoring",
"inputs": [
{
"conversation": "[{\"role\": \"customer\", \"content\": \"How can I restore a deleted project?\"}, {\"role\": \"agent\", \"content\": \"Thanks for contacting us!\"}, {\"role\": \"agent\", \"content\": \"I am closing this ticket now.\"}]"
}
]
}'Step 01
Upload sample data, define your output schema, and provide feedback on the hardest cases.
Whether the support outcome follows the supplied policy.
Policy requirements verified in the supplied record.
Requirements that were skipped or outcomes not supported by completed actions.
Check support outcomes against an explicitly supplied policy.
Change the email on my account.
Change the account email without checking ownership.
email_updated: true; ownership_verified: false
Sutro can now generalize the rules based on your feedback.
Step 02
Compare quality and cost on your examples, then choose a model or use Sutro’s default.
Compare models on expert agreement and cost. The interactive chart uses illustrative measurements.
Illustrative sweep · select Value or Best to use that model in your function.
Step 03
Call Sutro Functions like any other software. Swap the model as soon as something more performant is released.
import sutro as so
input_data = {
"policy": "Verify ownership before changing an account email.",
"trace": "[{\"role\": \"user\", \"content\": \"Change the email on my account.\"}, {\"role\": \"agent\", \"content\": \"Change the account email without checking ownership.\"}, {\"role\": \"tool\", \"content\": \"email_updated: true; ownership_verified: false\", \"name\": \"update_email\"}]"
}
result = so.run_function(
"resolution-policy-adherence-checks",
input_data,
)curl --fail-with-body -X POST "${SUTRO_API_URL%/}/v1/functions/resolution-policy-adherence-checks/run" \
-H "Authorization: Key $SUTRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"input": {
"policy": "Verify ownership before changing an account email.",
"trace": "[{\"role\": \"user\", \"content\": \"Change the email on my account.\"}, {\"role\": \"agent\", \"content\": \"Change the account email without checking ownership.\"}, {\"role\": \"tool\", \"content\": \"email_updated: true; ownership_verified: false\", \"name\": \"update_email\"}]"
}
}'import sutro as so
job_id = so.batch_run_function(
name="resolution-policy-adherence-checks",
data="resolution_policy_adherence_checks.csv",
)
results = so.await_job_completion(job_id)curl --fail-with-body -X POST "${SUTRO_API_URL%/}/v1/batch-inference" \
-H "Authorization: Key $SUTRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "resolution-policy-adherence-checks",
"inputs": [
{
"policy": "Verify ownership before changing an account email.",
"trace": "[{\"role\": \"user\", \"content\": \"Change the email on my account.\"}, {\"role\": \"agent\", \"content\": \"Change the account email without checking ownership.\"}, {\"role\": \"tool\", \"content\": \"email_updated: true; ownership_verified: false\", \"name\": \"update_email\"}]"
}
]
}'