Jev Troubleshooting: Common Errors and Proven Fixes

Every Jev integration hits the same handful of failures, and almost all of them have short fixes. This guide is the reference: a symptom-to-fix table for HTTP errors, a section on the JSON parsing traps that account for most “the API is broken” reports, and what to do when confidence values look off. Work through it top to bottom and you will resolve the vast majority of issues without filing anything.

TL;DR: 401 means the key; 404 means the model slug; 429 means slow down and back off; parse failures mean the message content was not JSON-loaded; weird confidence usually means a vague question. Check the table below before anything else, and confirm the exact model slug on the OpenRouter model page first — it is the most common culprit.

HTTP errors: symptom, cause, fix

A minimal retry wrapper covers 429 and 5xx behavior in one place:

import json, os, time, requests

def ask_jev_with_retry(prompt: str, max_retries: int = 3) -> dict:
    # Confirm the exact model slug on the OpenRouter model page
    for attempt in range(max_retries):
        resp = requests.post(
            "https://openrouter.ai/api/v1/chat/completions",
            headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
            json={
                "model": "typesafe/jev-1.13",
                "messages": [{"role": "user", "content": prompt}],
            },
            timeout=30,
        )
        if resp.status_code == 200:
            return json.loads(resp.json()["choices"][0]["message"]["content"])
        if resp.status_code in (429, 500, 502, 503):
            time.sleep(2 ** attempt)  # exponential backoff
            continue
        resp.raise_for_status()  # 401/404/400 will raise here with context
    raise RuntimeError("Jev call failed after retries")

The wrapper returns the fixture-shaped answer on success; the response shape shown is an example fixture and official field names should be confirmed in the official documentation. For the official TypeSafe AI channel’s own error codes and retry guidance, follow the official documentation at typesafe.ai.

Parse failures: the usual suspects

These account for most “the API is broken” reports that turn out to be client-side:

1. Treating content as a dict. The message content is a JSON string. resp.json()["choices"][0]["message"]["content"] gives you a string; json.loads it before accessing answer or confidence.
2. Assuming the envelope is the answer. answer lives inside the parsed content, not at the top level of the HTTP response. The HTTP envelope is OpenAI-compatible; the judgment payload is inside it — example fixture, confirm exact field names in the official documentation:

{
  "answer": "billing",
  "confidence": 0.94,
  "rationale": "The message disputes a charge amount rather than reporting a bug."
}

3. No defense against a malformed payload. Rare, but wrap the parse and retry once before paging anyone:

try:
    result = json.loads(resp.json()["choices"][0]["message"]["content"])
except (json.JSONDecodeError, KeyError):
    result = ask_jev_with_retry(prompt)  # one retry, then alert

The retry returns a fresh fixture-shaped answer; the shape shown is an example fixture and official field names should be confirmed in the official documentation.

Confidence anomalies

When confidence looks wrong, work down this list:

• All answers very high. The question probably has no real boundary — everything looks obviously yes or no because the criterion is trivial. Tighten the criterion.
• All answers low. The question is underspecified or the inputs are noisier than the question assumes; add boundaries and few-shot examples per the question-design guide.
• Confidence swinging for similar inputs. Normalize inputs before judging (strip HTML, truncate long text deterministically) and make sure the same scale and option list is used on every call.
• Confidence high but answers wrong on your labels. Re-check the option definitions for overlap; mutually exclusive definitions are covered in the state-and-questions guide.

If none of these move the needle, isolate one input and run it through the first-call tutorial’s minimal request — most residual issues turn out to be state accidentally included in the prompt. The product FAQ scoring case shows a full pipeline where these checks run automatically.

This guide applies to Jev 1.13.