A few technical details

A Decisions request takes evidence (the input you provide, which can contain text and images) and questions (what to evaluate, including each question’s type, instructions, and any allowed choices or score levels). You can mix question types in a single call, and each question is answered independently against the same evidence.

For the comparison, the alternative is asking a general model to answer in JSON through the Responses API. I used the same model on both sides, gpt-6-luna, so the only thing that changes is the endpoint. Any difference comes from the shape of the API, not from a different model.

OpenAI’s documentation says the API returns typed answers “about 10x faster than the Responses API,” and that you pay only for input tokens, with no output-token charges: $0.10 per 1M tokens. Jev’s documentation lists $0.042 per 1M tokens, which is 58% cheaper than OpenAI’s. I tested Jev on the ticket urgency demo, in TypeSafe’s web playground.

Demo 1 (Score): how urgent is this ticket?

The first demo reads a support ticket and scores urgency on four ordered levels: can wait (0), soon (1), urgent (2) and critical (3). A score is the probability-weighted average of those level numbers, so it can land between two levels. The demo then reports the closest level.

I used five real CALS OIT tickets with identifying details removed, and I labeled each one myself before running anything. All three paths I compared matched my labels on four of five.

In the example below the ticket being evaluated was “Laptop is locked by BitLocker. Recovery key requested. keyboard is not functional. I need help unlocking this laptop located in a lab on campus. I have a picture showing the screen message.”

All three paths matched my label on four of five tickets. Decisions and Jev missed the same one, the NetID conflict, which was also among the hardest for me to label.

Each cell shows the score, then the confidence, and bold marks a miss once the score is rounded to the nearest level.

Say an agent uses one of these three to classify incoming tickets. Three things matter: how fast each ticket is classified, what the whole queue costs, and whether the agent knows when to hand a ticket to a person.

On speed, Decisions took about a quarter of a second per ticket and regular Luna took one to two seconds. Jev’s speed could not be measured in the playground. On cost, Decisions came in about 52% under regular Luna and Jev about 77% under. Classifying 10,000 tickets would run about $0.32 with Decisions, $0.15 with Jev and $0.67 with regular Luna, so the dollars are small, but the gap is consistent.

The third is knowing when to flag a ticket for human review, which needs a signal of how sure the classifier is. Decisions and Jev reported a more grounded confidence: it came from the probabilities they returned, and it was lowest on the tickets that were hardest to label, including the one they both got wrong. Regular Luna reported its own confidence, high regardless, 0.91 on the ticket it got wrong, so it gives an agent no clear signal to flag a ticket for review.

Demo 2 (Choice): which major fits?

The second demo takes a prospective student’s description of their interests and picks from 21 options (20 majors plus an option for interests outside CALS).

I gave the models a description from a student who wants to work on climate policy for an NGO or in government, captained the debate team, writes for the school paper, and cares less about lab work than about getting people to act on what scientists already know. That is a straddle, so my label accepted either Environment & Sustainability or Communication. Both paths answered Environment & Sustainability, which matches.

A run of 7 different descriptions (prompts) resulted in the following outputs:

Both paths gave the same answer on six of the seven cases, and differed only on the straddle, where Decisions chose Global and Public Health and regular Luna chose Nutritional Sciences. The vague student is the one case where the measured confidence looks too high: Decisions put 0.87 on Environment & Sustainability from a single sentence, while regular Luna was less sure at 0.52.

On speed, Decisions was about 7.6x faster, taking the median of each case’s own ratio. On cost, it came in about 38% cheaper per case.

Jev is not in this comparison: its playground allows only eight options, and I could not reach a 21-option test without paying.

In a choice question, the option descriptions do most of the work, so writing strong and accurate descriptions governs the accuracy of the decision.

Demo 3 (Predicate & Choice): Tomato pathology triage

The third demo combines two decision types, and it is the first to use the Decisions API’s image input, which Jev does not offer. A predicate first checks whether the photo is usable, which makes it a quality gate: a photo that fails is stopped there and never reaches the choice over eight outcomes, so no further API call is spent on input too poor to classify. I ran the same two steps on both paths.

In the above examples, for usable photos, both paths answered correctly. For example, for Zippering, an option I described to the model as ‘One or more dry brown scars extending lengthwise down the tomato fruit, often from the stem toward the blossom end,’ both paths correctly identified the condition.

Decisions put 0.50 on Zippering and 0.31 on Late blight, with a confidence of 0.43, so the answer arrives with a visible runner-up. Regular Luna reported 0.78 for the same answer and offered no alternatives.

The totals for the two steps were 600 ms and $0.000406 for Decisions, against 4.04 s and $0.000650 for regular Luna: about 6.7x faster and 38% cheaper.

Demo 4 (Predicate): can I pass through here?

The fourth demo samples a phone’s live camera stream to evaluate if a path may not easily be accessible to a person in a wheelchair. It is sending a frame every two seconds. Each frame carries four independent predicates in a single request: is the path obstructed, does the obstruction look movable, is there a step without a ramp, and does the path look too narrow to pass.

I used predicates rather than a choice because the combination is the signal. Obstructed and movable is a box someone can shift. Obstructed but not movable is built into the building and is a true path blocker.

In the two frames taken from the video one could see:

The model returns four numbers, and the headline on screen comes from a short rule in my own code that checks them in order: a step with no ramp first, then a movable obstruction, then a built one, then a path that is too narrow, and otherwise a clear path. The order can be changed without touching the model or the prompt.

The staircase scored 0.95 on “too narrow,” which I would not call right for a normal staircase, but the headline ignores it because the step rule outranks it.

A one minute video is 30 frames and costs about a third of a cent ($0.003). At that rate, OpenAI’s published limits for the model, on its lowest paid tier, would let about 66 phones stream at once.

Final thoughts

Use Jev (text) or the Decisions API (image or text) when you have high-volume classification where the answers are known in advance, where an application has to act on the result, and where knowing how close a call was has value. Use a regular LLM for anything that needs a written explanation, an open-ended answer, or a category you can’t list ahead of time.

On speed, OpenAI’s documentation claims about 10x faster than the Responses API, and my own timings came in at about 7x across the demos.

The natural next step is to move from demos to pilots. For the ticket demo, that could mean running the classifier on incoming IT tickets to see how its urgency scores hold up against how the team actually prioritizes them. For the major explorer, it could mean asking the admissions office whether incoming applicants would find it useful. For the tomato demo, it could mean working with plant pathology faculty to test the classification on different crops. None of these is arranged yet; they are the conversations I would want to have.

On the cases I tried, the Decisions API was faster and cheaper than the same model answering in words, and it handed back probabilities I could do something with. Getting good results came down to how clearly I described the possible answers, and what to do with the probabilities is still up to whoever owns the decision.