Browser use viewport 1080p #74
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix remaining TS items + update Python template for Anthropic CUA to utilize computer controls instead of Playwright. Still do to: optimize click location issues.
…8 viewport and Claude Sonnet 4.5 Updates both TypeScript and Python Anthropic Computer Use templates: - Set viewport to 1024x768@60Hz (Anthropic recommended size) - Update model to claude-sonnet-4-5-20250929 - Fix coordinate alignment between browser viewport and computer tool dimensions Changes: - pkg/templates/typescript/anthropic-computer-use/ - tools/computer.ts: display_width_px=1024, display_height_px=768 - session.ts: viewport 1024x768@60Hz - index.ts: model updated to claude-sonnet-4-5-20250929 - pkg/templates/python/anthropic-computer-use/ - tools/computer.py: width=1024, height=768 - session.py: viewport 1024x768@60Hz - main.py: model updated to claude-sonnet-4-5-20250929 Test replays (magnitasks.com Kanban drag test - moved 5 items to Done): - TypeScript: https://proxy.iad-awesome-blackwell.onkernel.com:8443/browser/replays?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE4MDAwMTgyNTYsInNlc3Npb24iOnsiaWQiOiJmZDA3NGRxZjY5bnNlcjk4aDliNGtrb3giLCJjZHBQb3J0Ijo5MjIyLCJjZHBXc1BhdGgiOiIiLCJpbnN0YW5jZU5hbWUiOiJicm93c2VyLXN0ZWFsdGgtcHJvZHVjdGlvbi01LWFsbG93ZWQtaGFtbWVyaGVhZC00MjcxIiwiZnFkbiI6InF1aWV0LXRyZWUtM3kybnd6c2EucHJvZC1pYWQtdWtwLWJyb3dzZXJzLTAub25rZXJuZWwuYXBwIiwibWV0cm8iOiJodHRwczovL2FwaS5wcm9kLWlhZC11a3AtYnJvd3NlcnMtMC5vbmtlcm5lbC5ydW4vdjEiLCJ1c2VySWQiOiJ3ODdoNHd1dTRoazNmeHFyZW5iNzFrMnAiLCJvcmdJZCI6ImlxMnRmMjUzbWlsOWptOWhmZjI3bDhyMiIsInN0ZWFsdGgiOnRydWUsImhlYWRsZXNzIjpmYWxzZSwicmVwbGF5UHJlZml4IjoiczM6Ly9rZXJuZWwtYXBpLXByb2Qvc2Vzc2lvbnJlcGxheXMvaXEydGYyNTNtaWw5am05aGZmMjdsOHIyL2ZkMDc0ZHFmNjluc2VyOThoOWI0a2tveCIsImtlcm5lbEh0dHBTZXJ2ZXJQb3J0Ijo0NDQsInRpbWVvdXRTZWNvbmRzIjozMDAsImNyZWF0ZWRBdCI6IjIwMjYtMDEtMTVUMTM6MDQ6MTYuNzc2OTEwOTc5WiIsImltYWdlIjoib25rZXJuZWwva2VybmVsLWN1LXYyNTo5NmYzOGU0Iiwic3RlYWx0aFByb3h5SWRlbnRpZmllciI6Ijg3NTY1X25YREZGQDIxNi4yNDcuMTAyLjE1MDo2MTIzMiIsImxpdmVTbHVnIjoia3c5b0lBc1VzRkxlIiwicHJpdmF0ZUlQIjoiMTcyLjE2LjIuMjAxIiwidmlld3BvcnRXaWR0aCI6MTAyNCwidmlld3BvcnRIZWlnaHQiOjc2OCwidmlld3BvcnRSZWZyZXNoUmF0ZSI6NjB9fQ.GHE2BXg6qrtNMoqO6NvuJ9fbHTW15igfmXl7W-ls3Qg&replay_id=wipxrn813lmajv7ukdkuykoa - Python: https://proxy.iad-awesome-blackwell.onkernel.com:8443/browser/replays?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE4MDAwMTc4OTUsInNlc3Npb24iOnsiaWQiOiJseTVxOXQxa3F6YXR3NzE1N3lpYzl2M3IiLCJjZHBQb3J0Ijo5MjIyLCJjZHBXc1BhdGgiOiIiLCJpbnN0YW5jZU5hbWUiOiJicm93c2VyLXN0ZWFsdGgtcHJvZHVjdGlvbi01LXJlYWwtd2F0Y2htZW4tNTUxNCIsImZxZG4iOiJ0d2lsaWdodC1ib25vYm8tZGFvZTd5ZngucHJvZC1pYWQtdWtwLWJyb3dzZXJzLTAub25rZXJuZWwuYXBwIiwibWV0cm8iOiJodHRwczovL2FwaS5wcm9kLWlhZC11a3AtYnJvd3NlcnMtMC5vbmtlcm5lbC5ydW4vdjEiLCJ1c2VySWQiOiJ3ODdoNHd1dTRoazNmeHFyZW5iNzFrMnAiLCJvcmdJZCI6ImlxMnRmMjUzbWlsOWptOWhmZjI3bDhyMiIsInN0ZWFsdGgiOnRydWUsImhlYWRsZXNzIjpmYWxzZSwicmVwbGF5UHJlZml4IjoiczM6Ly9rZXJuZWwtYXBpLXByb2Qvc2Vzc2lvbnJlcGxheXMvaXEydGYyNTNtaWw5am05aGZmMjdsOHIyL2x5NXE5dDFrcXphdHc3MTU3eWljOXYzciIsImtlcm5lbEh0dHBTZXJ2ZXJQb3J0Ijo0NDQsInRpbWVvdXRTZWNvbmRzIjozMDAsImNyZWF0ZWRBdCI6IjIwMjYtMDEtMTVUMTI6NTg6MTUuMzk0MjQyNTc3WiIsImltYWdlIjoib25rZXJuZWwva2VybmVsLWN1LXYyNTo5NmYzOGU0Iiwic3RlYWx0aFByb3h5SWRlbnRpZmllciI6Ijg3NTY1X25YREZGQDE0MC4yMzMuMjQ5LjE3NDo2MTIzNCIsImxpdmVTbHVnIjoiak5DdGdpdHRreGtrIiwicHJpdmF0ZUlQIjoiMTcyLjE2LjcuMTMzIiwidmlld3BvcnRXaWR0aCI6MTAyNCwidmlld3BvcnRIZWlnaHQiOjc2OCwidmlld3BvcnRSZWZyZXNoUmF0ZSI6NjB9fQ._AhzTu1HwawrWwDgo66K3FZkEh4dpiOEVPmBTO4A21A&replay_id=pa0ha28zodehf1e1jyv1qibn Resolves KERNEL-725
…8 viewport and Claude Sonnet 4.5 Updates both TypeScript and Python Anthropic Computer Use templates: - Set viewport to 1024x768@60Hz (Anthropic recommended size) - Update model to claude-sonnet-4-5-20250929 - Fix coordinate alignment between browser viewport and computer tool dimensions Changes: - pkg/templates/typescript/anthropic-computer-use/ - tools/computer.ts: display_width_px=1024, display_height_px=768 - session.ts: viewport 1024x768@60Hz - index.ts: model updated to claude-sonnet-4-5-20250929 - pkg/templates/python/anthropic-computer-use/ - tools/computer.py: width=1024, height=768 - session.py: viewport 1024x768@60Hz - main.py: model updated to claude-sonnet-4-5-20250929 Test replays (magnitasks.com Kanban drag test - moved 5 items to Done): - TypeScript: https://proxy.iad-awesome-blackwell.onkernel.com:8443/browser/replays?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE4MDAwMTgyNTYsInNlc3Npb24iOnsiaWQiOiJmZDA3NGRxZjY5bnNlcjk4aDliNGtrb3giLCJjZHBQb3J0Ijo5MjIyLCJjZHBXc1BhdGgiOiIiLCJpbnN0YW5jZU5hbWUiOiJicm93c2VyLXN0ZWFsdGgtcHJvZHVjdGlvbi01LWFsbG93ZWQtaGFtbWVyaGVhZC00MjcxIiwiZnFkbiI6InF1aWV0LXRyZWUtM3kybnd6c2EucHJvZC1pYWQtdWtwLWJyb3dzZXJzLTAub25rZXJuZWwuYXBwIiwibWV0cm8iOiJodHRwczovL2FwaS5wcm9kLWlhZC11a3AtYnJvd3NlcnMtMC5vbmtlcm5lbC5ydW4vdjEiLCJ1c2VySWQiOiJ3ODdoNHd1dTRoazNmeHFyZW5iNzFrMnAiLCJvcmdJZCI6ImlxMnRmMjUzbWlsOWptOWhmZjI3bDhyMiIsInN0ZWFsdGgiOnRydWUsImhlYWRsZXNzIjpmYWxzZSwicmVwbGF5UHJlZml4IjoiczM6Ly9rZXJuZWwtYXBpLXByb2Qvc2Vzc2lvbnJlcGxheXMvaXEydGYyNTNtaWw5am05aGZmMjdsOHIyL2ZkMDc0ZHFmNjluc2VyOThoOWI0a2tveCIsImtlcm5lbEh0dHBTZXJ2ZXJQb3J0Ijo0NDQsInRpbWVvdXRTZWNvbmRzIjozMDAsImNyZWF0ZWRBdCI6IjIwMjYtMDEtMTVUMTM6MDQ6MTYuNzc2OTEwOTc5WiIsImltYWdlIjoib25rZXJuZWwva2VybmVsLWN1LXYyNTo5NmYzOGU0Iiwic3RlYWx0aFByb3h5SWRlbnRpZmllciI6Ijg3NTY1X25YREZGQDIxNi4yNDcuMTAyLjE1MDo2MTIzMiIsImxpdmVTbHVnIjoia3c5b0lBc1VzRkxlIiwicHJpdmF0ZUlQIjoiMTcyLjE2LjIuMjAxIiwidmlld3BvcnRXaWR0aCI6MTAyNCwidmlld3BvcnRIZWlnaHQiOjc2OCwidmlld3BvcnRSZWZyZXNoUmF0ZSI6NjB9fQ.GHE2BXg6qrtNMoqO6NvuJ9fbHTW15igfmXl7W-ls3Qg&replay_id=wipxrn813lmajv7ukdkuykoa - Python: https://proxy.iad-awesome-blackwell.onkernel.com:8443/browser/replays?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE4MDAwMTc4OTUsInNlc3Npb24iOnsiaWQiOiJseTVxOXQxa3F6YXR3NzE1N3lpYzl2M3IiLCJjZHBQb3J0Ijo5MjIyLCJjZHBXc1BhdGgiOiIiLCJpbnN0YW5jZU5hbWUiOiJicm93c2VyLXN0ZWFsdGgtcHJvZHVjdGlvbi01LXJlYWwtd2F0Y2htZW4tNTUxNCIsImZxZG4iOiJ0d2lsaWdodC1ib25vYm8tZGFvZTd5ZngucHJvZC1pYWQtdWtwLWJyb3dzZXJzLTAub25rZXJuZWwuYXBwIiwibWV0cm8iOiJodHRwczovL2FwaS5wcm9kLWlhZC11a3AtYnJvd3NlcnMtMC5vbmtlcm5lbC5ydW4vdjEiLCJ1c2VySWQiOiJ3ODdoNHd1dTRoazNmeHFyZW5iNzFrMnAiLCJvcmdJZCI6ImlxMnRmMjUzbWlsOWptOWhmZjI3bDhyMiIsInN0ZWFsdGgiOnRydWUsImhlYWRsZXNzIjpmYWxzZSwicmVwbGF5UHJlZml4IjoiczM6Ly9rZXJuZWwtYXBpLXByb2Qvc2Vzc2lvbnJlcGxheXMvaXEydGYyNTNtaWw5am05aGZmMjdsOHIyL2x5NXE5dDFrcXphdHc3MTU3eWljOXYzciIsImtlcm5lbEh0dHBTZXJ2ZXJQb3J0Ijo0NDQsInRpbWVvdXRTZWNvbmRzIjozMDAsImNyZWF0ZWRBdCI6IjIwMjYtMDEtMTVUMTI6NTg6MTUuMzk0MjQyNTc3WiIsImltYWdlIjoib25rZXJuZWwva2VybmVsLWN1LXYyNTo5NmYzOGU0Iiwic3RlYWx0aFByb3h5SWRlbnRpZmllciI6Ijg3NTY1X25YREZGQDE0MC4yMzMuMjQ5LjE3NDo2MTIzNCIsImxpdmVTbHVnIjoiak5DdGdpdHRreGtrIiwicHJpdmF0ZUlQIjoiMTcyLjE2LjcuMTMzIiwidmlld3BvcnRXaWR0aCI6MTAyNCwidmlld3BvcnRIZWlnaHQiOjc2OCwidmlld3BvcnRSZWZyZXNoUmF0ZSI6NjB9fQ._AhzTu1HwawrWwDgo66K3FZkEh4dpiOEVPmBTO4A21A&replay_id=pa0ha28zodehf1e1jyv1qibn Resolves KERNEL-725
…nto kernel-computer-use
Updated invokecommand example for the anthropic templates
…sistency TypeScript template: - Add xdotool-format key mappings for consistency with Python template - Rename methods from convertToOnKernelKey to convertToKernelKey - Fix scroll fallback to use lastMousePosition instead of [0, 0] - Fix scroll amount using ?? operator to handle zero correctly - Remove unused KeyboardUtils import - Fix error message: "OnKernel" → "Kernel" - Reset all state fields (liveViewUrl, replayViewUrl) on session stop - Handle replay recording failures gracefully with try/catch Python template: - Wrap cleanup in try/finally to ensure browser deletion on errors - Handle replay recording failures gracefully with try/except - Preserve unexpected Anthropic content block types in loop
…otool behavior Anthropic's reference implementation uses xdotool where each scroll_amount unit equals one scroll wheel click (~120 pixels). Previously: - TypeScript used the value directly - Python used a 10x multiplier Both now use 120x to match Anthropic's expected behavior for AI agents.
Wrap replay stopping logic in try/finally to ensure browser session is always deleted even if stopReplay() fails. This prevents resource leaks on the Kernel platform when replay recording is enabled and stopping fails. Matches the existing Python implementation behavior.
Change browser window and viewport size from 1024x786 to 1920x1080 for better compatibility with modern web applications.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
| return {"final_result": result.final_result()} | ||
| return {"errors": result.errors()} | ||
| finally: | ||
| client.browsers.delete_by_id(kernel_browser.session_id) |
There was a problem hiding this comment.
Accidentally committed QA test directory files
Medium Severity
The qa-20260116-124334/py-browser-use/ directory and its files (.env.example, .gitignore, README.md, main.py, pyproject.toml, session.py) appear to be QA test artifacts that were accidentally committed. According to .cursor/commands/qa.md, QA directories should be removed after testing is complete. These files duplicate template code and should not be in the repository.
Additional Locations (2)
| await new Promise(resolve => setTimeout(resolve, 500)); | ||
| return await this.screenshot(); | ||
| } | ||
|
|
There was a problem hiding this comment.
TypeScript ignores key modifier parameter for click actions
Medium Severity
The ActionParams interface defines a key?: string parameter (line 42 of types/computer.ts) that the Python implementation uses to hold modifier keys during click actions (for keyboard shortcuts like Ctrl+click). However, the TypeScript ComputerTool.call method destructures params but never extracts or uses the key parameter. When a click action is requested with a modifier key, the TypeScript version silently ignores it, while the Python version properly holds the key down during the click. This creates inconsistent behavior between the two language implementations.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based on bug pointed out by customer updating the default viewport sizing. Also we'll need to make changes in the docs as well for BU.
Note
Modernizes computer-use templates and improves testability.
KernelBrowserSessionmanagers and tool rewrites for mouse/keyboard/scroll and screenshots;cursor_positionnow returns an errorrecord_replay) with returnedreplay_url; updates READMEs with setup/usage and limitations@onkernel/sdk/kernelto 0.24.0; removes Playwright depsWritten by Cursor Bugbot for commit 637b49a. This will update automatically on new commits. Configure here.