fix(openai): Handles Bedrock-style context overflow errors for OpenAI-compatible endpoints #1529
+110
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds logic to recognize and handle context window overflow errors that are returned in a Bedrock-style format by OpenAI-compatible endpoints that wrap Bedrock models. Handles OpenAI rate limit errors consistently Ensures rate limit errors from OpenAI are consistently treated as throttling exceptions across different code paths. This prevents potential misinterpretation of rate limits as context window overflows. Fixes Bedrock-style overflow detection when rate limited.
dinindunz
requested a deployment
to
manual-approval
GitHub Actions
Waiting
mkmeral
reviewed
Jan 21, 2026
Updates the model to recognize a wider range of context overflow error messages. This change expands error handling to include messages from various providers, ensuring more robust detection of context window overflows when using OpenAI-compatible endpoints.
1. test_stream_alternative_context_overflow_messages (3 parametrized tests) - Tests that stream() method properly converts APIError with alternative overflow messages to ContextWindowOverflowException - Parametrized to test all 3 error message patterns 2. test_structured_output_alternative_context_overflow_messages (3 parametrized tests) - Tests that structured_output() method properly converts APIError with alternative overflow messages - Parametrized to test all 3 error message patterns 3. test_stream_api_error_passthrough (1 test) - Tests that APIError without overflow messages passes through unchanged - Ensures we don't accidentally catch unrelated API errors Test Coverage The new tests verify: - ✅ All 3 alternative overflow messages are detected - ✅ Both stream() and structured_output() methods handle the errors - ✅ Errors are properly converted to ContextWindowOverflowException - ✅ Original exception is preserved as __cause__ - ✅ Non-overflow APIError exceptions pass through unchanged
dinindunz
requested a deployment
to
manual-approval
GitHub Actions
Waiting
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes context overflow detection for OpenAI-compatible endpoints that wrap Bedrock models (e.g., Databricks Model Serving).
Problem:
The OpenAI provider only caught BadRequestError with code "context_length_exceeded". When using Databricks endpoints serving Bedrock models, context overflow errors come as APIError with Bedrock-style messages like "Input is too long for requested model". These were not converted to ContextWindowOverflowException, preventing SummarizingConversationManager and other conversation managers from triggering.
Solution:
Impact:
Agents using OpenAI-compatible endpoints that wrap Bedrock models can now properly handle context overflow with conversation managers.
Related Issues
Closes #1528
Documentation PR
N/A
Type of Change
Bug fix
Testing
How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli
hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.