Repository navigation
Add Cheaper Inference as a model gateway backend - #144
Open
aiapienthusiast wants to merge 1 commit into
Open
aiapienthusiast wants to merge 1 commit into
aiapienthusiast wants to merge 1 commit into
Conversation
Cheaper Inference is an OpenAI-compatible LLM gateway at its own endpoint (api.cheaperinference.com/v1), reached with one CHEAPER_INFERENCE_API_KEY. Follows the Novita backend (CodeGraphContext#102): a thin subclass of OpenAICompatChatModel, credentials resolved through the same env/.env lookup in config.py, a new branch in the registry's get_model(), a `models --check` probe, and a `cheaperinference` extra. Like OpenAI and Novita, this backend reads no per-call cost, so calls are counted as unpriced unless a rate card is supplied. Model ids on this gateway are bare (gpt-5.4-mini), not author/slug, so vendor() reads every spec on it as `cheaperinference`. It is not mapped to one lab in BACKEND_VENDOR, because the gateway serves several. Behaviour change: the `models --check` backend column grows from 12 to 17 characters so the 16-character name keeps the table aligned. The test that pins `bedrock unprobed` and both cookbook transcripts are updated to the new width. The deep-dive test count was refreshed with GRAPHARC_UPDATE_FIGURES=1.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds Cheaper Inference as a model gateway backend.
Cheaper Inference is an OpenAI-compatible LLM gateway.
One API key gives access to models from several labs.
It uses the same pattern as #102 (Novita).
Spec:
cheaperinference/gpt-5.4-mini. Key:CHEAPER_INFERENCE_API_KEY(env or.env).Changes:
grapharc/gateway/cheaperinference.py:CheaperInferenceChatModel, a thin subclass ofOpenAICompatChatModelwith a fixed endpoint, likenovita.py.config.py:CHEAPER_INFERENCE_KEYSandcheaper_inference_api_key().registry.py: the backend inBACKENDS, aget_model()branch, a bare-backend example.cli/main.py,cli/probe.py: the key line ingrapharc modelsand a--checkprobe.pyproject.toml,uv.lock: acheaperinferenceextra, also inall.02-models.md(examples, table, transcripts) and the--checktranscript in06-serving-and-ops.md.tests/test_gateway_cheaperinference.py(offline, plus onelive-marked test), and the backend lists in the existing tests.Two points to review
gpt-5.4-mini), notauthor/slug.So
vendor()reads every spec on it ascheaperinference.I did not map it to one lab in
BACKEND_VENDOR, because the gateway serves several.The module docstring states the effect on
different_providers.BACKEND_COLUMNwas 12.I changed it to 17, so the
--checktable stays aligned.This changes the width of every row.
I updated the
bedrock unprobedassertion intest_cli.pyand both cookbook transcripts to the new width.The transcript in
06-serving-and-ops.mdalso gets the Novita row that it did not have.If you prefer a shorter backend name and no width change, I can change it.
Testing:
uv run pytest: green (2,424 selected, 14 deselected).uv run ruff check .: clean.grapharc models --checkoutput matches the new transcripts.docs/deep-dive.mdwithGRAPHARC_UPDATE_FIGURES=1. I did not re-run the other checks that paragraph lists.livetest.