Use Cases / screenshot-ui-box
Jev Screenshot UI Box (beta)
Image inputs and Box questions are beta and dreamer-ready, not recommended for production. Product and QA agents still want a typed “where is the primary button?” signal on screenshots.
Try a related template free on this host — 5 anonymous real-API runs with no signup — then ship the same schema with a prepaid jv_live_ key. See is Jev free? and playground.
Why Jev
Pass the screenshot as an image asset and ask a box question for the UI region. You get a single bounding box your overlay can highlight — useful for demos and eval harnesses, not as a sole production click target.
How to define the decision
assets.screenshot = { type: "image", format: "png", data_base64: "…" }. questions.cta = { type: "box", instructions: "Find the primary CTA button in screenshot" }. Keep instructions concrete.
When to use Jev
- Agent UI grounding prototypes
- Screenshot annotation demos
- Comparing Jev box vs a dedicated UI detector
When not to use Jev
- Auto-clicking production UIs from box alone
- Accessibility compliance certification
- Multi-element layout parsing
Failure modes and human fallback
- Upstream or network errors: retry with backoff; do not invent a decision.
- Low confidence: route to a human queue or safer default — never auto-act on money/destructive tools.
- Schema drift: keep a golden fixture set in CI so label changes fail the build before production.
- Official: Image inputs and Box questions are beta and dreamer-ready, not recommended for production.
- Screenshots may contain PII — strip secrets before sending assets.
- This host does not ship a screenshot upload playground tool yet — use the API or homepage local preview.
API example
Same prepaid jv_live_ balance as tools. Output tokens are free on this host's metering model.
curl https://jevtypesafe.org/api/v1/decide \
-H "Authorization: Bearer jv_live_YOUR_KEY_HERE" \
-H "Content-Type: application/json" \
-d '{"state":"Customer: I've been waiting five days for my refund.","questions":{"action":{"type":"choice","instructions":"Decide the next step","criteria":{"allow":"safe","review":"unsure","block":"unsafe"}}}}'FAQ
- Can I use this to auto-click my app?
- Not for production. Image inputs and Box questions are beta and dreamer-ready, not recommended for production.
- Where is the API shape documented?
- See /docs/api/ for assets and box (beta), including the official disclaimer.
Related use cases
Ready to build?
Free anonymous tool runs, then prepaid packs and a jv_live_ key — independent host, not affiliated with TypeSafe AI.