Repository navigation
Add opt-in keyless Keenable search for OfficeQA rollouts - #368
Open
Ilya Bogin (ilya-bogin-keenable) wants to merge 1 commit into
Open
Ilya Bogin (ilya-bogin-keenable) wants to merge 1 commit into
Ilya Bogin (ilya-bogin-keenable) wants to merge 1 commit into
Conversation
env.search_mode=keenable runs the OfficeQA controller search loop on Keenable's search API, beside the Parallel mode from microsoft#324. It is never selected automatically: the default and auto stay offline. Each generated query is one POST to the keyless endpoint (/v1/search/public) with an X-Keenable-Title: SkillOpt header; the task question is not sent. If the user sets KEENABLE_API_KEY, it is sent in an X-API-Key header to the keyed endpoint instead. No key, credential or account is bundled. It uses the core httpx dependency, so no extra is needed. Service errors fail the rollout without falling back, and one deadline (search_timeout_seconds) covers all queries of a round. The config reference gets a data-boundary section listing exactly what a request carries, plus the comparability caveat about the paper's Google-backed search setting. Closes microsoft#352
Author
|
@microsoft-github-policy-service agree company="Keenable" |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
env.search_mode=keenable, an opt-in web search mode for the OfficeQA controller search loop, in the same shape as #324.Closes #352
Conditions from #352
_normalize_search_modereturnskeenableonly when it is spelled out; the default,autoand unknown values stayoffline(tests:test_auto_and_other_modes_never_select_keenable,test_fresh_config_never_calls_keenable).KEENABLE_API_KEYis read only if the user sets it, and the docs say users who enable the mode use the service under Keenable's own terms.httpx;pyproject.tomland CI are unchanged.docs/reference/config.mdlists exactly what a request carries (the generated query andmax_results,X-Keenable-Title: SkillOpt, SkillOpt'sUser-Agent, andX-API-Keyonly whenKEENABLE_API_KEYis set) and includes the comparability caveat about the paper's Google-backed search setting.tests/test_officeqa_keenable_search.py(29 tests) runs the client againsthttpx.MockTransportand asserts the exact request; nothing reaches the live service.keenable_search.py, its branch inrollout.py, the config reference, CHANGELOG and the tests.Tested
python -m pytest -q tests/test_officeqa_keenable_search.py tests/test_officeqa_parallel_search.py: 61 passed on Python 3.12 and 3.10 (.[dev,parallel]).python -m pytest -q: 2062 passed, 11 skipped, 4 xfailed on both 3.12 and 3.10. The one failure,tests/test_backend_pi.py::test_default_model_from_env, comes from a localpibinary on myPATH; all 21 pi tests pass with that directory removed fromPATH.ruff checkon the changed files: clean.mkdocs build --strict: passes.rollout.process_one(..., search_mode="keenable")with a scripted model and no key set completed and returned evidence for both queries. No keyed run and no live LLM run.Disclosure: I work at Keenable, which runs this API.