Skip to content

Conversation

@AlexanderOnischenko
Copy link
Owner

Motivation

  • Make the runner's expectation checks robust to casing differences for contains and equals comparisons and to minor formatting differences in list/set elements.
  • Keep error messages showing the original (un-normalized) strings while performing comparisons using normalized values.

Description

  • Added normalization helpers _normalize_text and _normalize_strings in examples/demo_qa/runner.py and updated _match_expected to use them for comparisons.
  • For contains checks, compare norm(expected) in norm(actual) by using _normalize_text on both sides.
  • For equals checks, perform norm(expected) == norm(actual) and added support for list/tuple/set comparisons by normalizing elements and comparing lists or sets as appropriate while still reporting original strings in ExpectedCheck.detail.
  • Added unit tests in examples/demo_qa/tests/test_demo_qa_runner.py covering case-insensitive contains, case-insensitive equals, and normalized list comparisons (test_match_expected_contains_pass_and_fail, test_match_expected_equals_is_case_insensitive, test_match_expected_list_comparison_normalizes_elements).

Testing

  • No automated test suite was executed as part of this change (no pytest run in the rollout).
  • New unit tests were added but not run: test_match_expected_equals_is_case_insensitive and test_match_expected_list_comparison_normalizes_elements in examples/demo_qa/tests/test_demo_qa_runner.py.

Codex Task

@AlexanderOnischenko AlexanderOnischenko merged commit a13283c into dev Jan 22, 2026
@AlexanderOnischenko AlexanderOnischenko deleted the codex/update-case-insensitive-checks-in-runner.py branch January 22, 2026 17:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants