fix(integrations): google-genai: reworked gen_ai.request.messages extraction from parameters
#5275
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
… if mime_type and file_uri are present (Cursor comment)
…i-report-image-inputs
Semver Impact of This PR🟢 Patch (bug fixes) 📋 Changelog PreviewThis is how your changes will appear in the changelog. New Features ✨
Bug Fixes 🐛
Documentation 📚
Internal Changes 🔧Release
Other
🤖 This preview updates automatically when you update the PR. |
…i-report-image-inputs
| if isinstance(function_response, dict): | ||
| tool_call_id = function_response.get("id") | ||
| tool_name = function_response.get("name") | ||
| response_dict = function_response.get("response") or {} | ||
| # Prefer "output" key if present, otherwise use entire response | ||
| output = response_dict.get("output", response_dict) | ||
| else: | ||
| # FunctionResponse object | ||
| tool_call_id = getattr(function_response, "id", None) | ||
| tool_name = getattr(function_response, "name", None) | ||
| response_obj = getattr(function_response, "response", None) or {} |
There was a problem hiding this comment.
I've seen this .get() vs getattr pattern a lot in our AI integrations. Feels like introducing a helper function that would try both at once would potentially deduplicate a lot of code.
Not something that needs to be done in this PR, mostly thinking out loud.
![sentry[bot]](https://avatars.githubusercontent.com/u/1396951?s=60&v=4){kind=small}
…AI messages Add transform_content_part() and transform_message_content() functions to standardize content part handling across all AI integrations. These functions transform various SDK-specific formats (OpenAI, Anthropic, Google, LangChain) into a unified format: - blob: base64-encoded binary data - uri: URL references (including file URIs) - file: file ID references Also adds get_modality_from_mime_type() helper to infer content modality (image/audio/video/document) from MIME types.
…rmats Replace inline_data and file_data dict handling with the shared transform_content_part function. Keep Google SDK object handling and PIL.Image support local since those are Google-specific.
| "type": "blob", | ||
| "mime_type": mime_type, | ||
| "file_uri": file_uri, | ||
| } |
There was a problem hiding this comment.
Missing modality field in blob outputs from Part handling
Medium Severity
Blob outputs for Part objects (file_data, inline_data), File objects, and PIL images are missing the modality field. However, dict parts handled via transform_content_part do include modality. The get_modality_from_mime_type function was imported but never used in these code paths, suggesting the intent was to include modality but it was forgotten. This creates inconsistent output format where some blobs have modality and others don't.
Additional Locations (2)
| "type": "blob", | ||
| "mime_type": mime_type, | ||
| "content": BLOB_DATA_SUBSTITUTE, | ||
| } |
There was a problem hiding this comment.
Part object inline_data with non-bytes data silently dropped
Low Severity
For Part objects with inline_data, only bytes data is handled. If data is not bytes (e.g., a base64 string), the condition at line 340 fails and the function falls through to return None, silently dropping the data. However, dict inline_data with string data is preserved via transform_content_part. This asymmetric behavior could lead to silent data loss when processing Part objects with non-bytes inline_data.
Description
Previously we only extracted only text parts were extracted. Now the full range of possibilities are covered.
Issues
Closes https://linear.app/getsentry/issue/TET-1638/redact-images-google-genai