-
Notifications
You must be signed in to change notification settings - Fork 572
fix(litellm): fix gen_ai.request.messages to be as expected
#5255
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
fix(litellm): fix gen_ai.request.messages to be as expected
#5255
Conversation
…inal messages and handle data URLs correctly
Semver Impact of This PR🟢 Patch (bug fixes) 📋 Changelog PreviewThis is how your changes will appear in the changelog. New Features ✨
Bug Fixes 🐛Litellm
Other
Documentation 📚
Internal Changes 🔧Release
Other
🤖 This preview updates automatically when you update the PR. |
sentry_sdk/integrations/litellm.py
Outdated
| return { | ||
| "type": "blob", | ||
| "modality": "image", | ||
| "mime_type": mime_type, | ||
| "content": content, | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
litellm accepts input which follows OpenAI format, is there an opportunity to deduplicate with the openai-python input handling here?
…AI messages Add transform_content_part() and transform_message_content() functions to standardize content part handling across all AI integrations. These functions transform various SDK-specific formats (OpenAI, Anthropic, Google, LangChain) into a unified format: - blob: base64-encoded binary data - uri: URL references (including file URIs) - file: file ID references Also adds get_modality_from_mime_type() helper to infer content modality (image/audio/video/document) from MIME types.
Replace local _convert_message_parts implementation with the shared transform_message_content function, removing ~50 lines of duplicated code.
| "modality": get_modality_from_mime_type(mime_type), | ||
| "mime_type": mime_type, | ||
| "content": content, | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Non-base64 data URIs incorrectly treated as blobs
Medium Severity
The transform_content_part function treats all data URIs starting with data: as base64-encoded blobs, but the test test_convert_message_parts_data_url_without_base64 expects data URIs without the ;base64 marker to be returned as type: "uri" instead of type: "blob". The comment says "Check if it's a data URI (base64 encoded)" but the code doesn't actually check for the ;base64 marker before treating content as base64. For data:image/png,rawdata, the code returns type="blob" but the test asserts type="uri".
Issues
Closes https://linear.app/getsentry/issue/TET-1635/redact-images-litellm