Repository navigation
Conversation
gen-ai is adding a stored workflowStatus multipart part that tells the browser a dashboard workflow ended with nothing left open. It rides on the turn's last visible answer, next to the text. The SSE client put every non-text part it did not handle into unhandled_parts, and render_answer_text appends those to the text the evaluators score, so dashboard evals would grade a JSON blob as part of the answer and log an unknown-part warning on every decline turn. Register the type and skip it in _handle_multipart: it carries nothing an evaluator scores. jira: GDAI-2733 risk: low
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1926 +/- ##
=======================================
Coverage 86.15% 86.15%
=======================================
Files 344 344
Lines 25874 25876 +2
=======================================
+ Hits 22291 22293 +2
Misses 3583 3583 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
4 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
gen-ai is adding a stored
workflowStatusmultipart part (GDAI-2733) that tells the browser a dashboard workflow ended with nothing left open, e.g. after the user declines a proposed chart. It rides on the turn's last visible answer, next to the text:{"type": "multipart", "parts": [ {"type": "text", "text": "Got it, leaving the dashboard as it is."}, {"type": "workflowStatus", "skills": ["dashboard_editor"], "status": "ended"} ]}Without this change the SSE client puts the part into
unhandled_parts, andrender_answer_textappends it to the text evaluators score:Got it, leaving the dashboard as it is.\n\n[workflowStatus]\n{"skills": ["dashboard_editor"], "status": "ended"}plus an "unknown multipart part type" warningGot it, leaving the dashboard as it is._handle_multipartnow skips the part, and_KNOWN_PART_TYPESlists it.The gen-ai side is gooddata/gdc-nas#29907. This one is safe to merge first: until gen-ai sends the part, the new branch never runs.
tests/tavern-e2ein gdc-nas locks gooddata-eval in itsuv.lock, so that lock needs to move to a release with this change before or together with the gen-ai change.Test plan
test_workflow_status_part_stays_out_of_the_answer: a status part followed by a text part scores as the text alone. Fails when the branch is removed, whencontinuebecomesreturn, and when the part goes tosearch_results.test_known_part_types_matches_the_documented_gen_ai_unionincludesworkflowStatus.make -C packages/gooddata-eval format lint type-check: clean.test_only_the_first_questionnaire_gets_the_structured_answers) fails the same way on master and is unrelated.Risk
low: an eval-only parser change that drops one part type the server does not send yet.jira: GDAI-2733
risk: low
Summary by CodeRabbit