Commit Graph

33 Commits

Author SHA1 Message Date
4369611610 fix: add debug logging and visual indicators for OCR results 2025-05-19 20:19:36 +02:00
93a01b792b fix: update OCR result coordinates to center position 2025-05-19 20:11:56 +02:00
3d5f71ec84 fix 2025-05-19 17:19:24 +02:00
20f05ca991 fix: emphasize priority in search_pc function description 2025-05-19 17:09:21 +02:00
859e1c2f0b fix: missing bracket 2025-05-19 17:07:03 +02:00
d9a9eba4c7 updated win func 2025-05-19 17:05:38 +02:00
b89051a37f fix 2025-05-19 17:02:48 +02:00
72a876410c more context to gpt 2025-05-19 16:51:46 +02:00
46a5bce956 refactor: Update function descriptions for clarity and consistency 2025-05-19 16:41:02 +02:00
e639e1edd3 refactor: Rename press_windows_key to windows_key for consistency 2025-05-19 16:33:59 +02:00
9bd15d45c5 feat: Add functionality to press Windows key and update function registry 2025-05-19 16:32:09 +02:00
105ab4a04b feat: wip: give OCR+positions 2025-05-19 16:10:02 +02:00
5be7f9aadb feat: Add OCR functionality to process method; integrate Tesseract for text extraction from screenshots 2025-05-19 15:59:46 +02:00
20764d5d19 fix: Simplify click position extraction for screenshot crosshair in tool execution 2025-05-19 13:43:04 +02:00
158529a2bd fix: Parse tool call arguments as JSON for improved handling in process method 2025-05-19 13:41:25 +02:00
b583094e20 fix: Enhance screenshot functionality; add crosshair drawing and save screenshot to file 2025-05-19 13:39:26 +02:00
d7c4f9b0cb fix: Update image handling in process method; ensure only the last two messages retain images and improve debugging output 2025-05-19 13:27:16 +02:00
035252c146 fix: Enhance logging for tool calls in process method; handle potential errors in next steps assignment 2025-05-19 13:21:15 +02:00
892f41f78a fix: Shorten image data in message copies for better debugging; update logging to reflect changes 2025-05-19 13:17:51 +02:00
0af7dc7699 fix: bug 2025-05-19 13:14:38 +02:00
2bcddedca5 fix: Adjust message handling in process method; ensure correct image assignment and add next steps output 2025-05-19 13:13:28 +02:00
b881f04acc fix: Update process method return type and handle image attribute correctly; improve error handling 2025-05-19 13:10:46 +02:00
670066100f feat: Implement logging functionality; add logger configuration and retrieval 2025-05-19 13:05:36 +02:00
52c455b20c fix: Remove unused PyQt5 and tkinter overlay code; simplify click indicator function 2025-05-19 12:58:34 +02:00
a4e078bc19 tempfix: remove mouse overlay 2025-05-19 12:51:59 +02:00
1925a77d85 Add screenshot re-execution logic in AIProcessor; append outputs from tool calls 2025-05-19 09:34:21 +02:00
e573ecb553 Add confirmation function and re-execution logic in AIProcessor; clean up web server request handling 2025-05-19 09:30:58 +02:00
f7feb12946 Add screenshot functionality and new commands for wait and reprompt 2025-05-19 09:15:08 +02:00
66330bfc73 Implement click indicator with red circle display; update server run parameters 2025-05-19 09:00:39 +02:00
41f7d0e210 Refactor mouse button handling to use string literals instead of ButtonType constants; add debug print for screenshot action in web server 2025-05-19 08:53:46 +02:00
84d65cb505 Add Pillow dependency and implement screenshot functionality in web server 2025-05-19 01:18:54 +02:00
7e612c1af7 Add initial implementation of AI agent with mouse and keyboard control features 2025-05-19 00:48:14 +02:00
ed34ebca6a Initial commit 2025-05-18 17:32:16 +00:00