Screen tools

Capture the screen for a pill

In a pill’s editor, choose Add an action, then Screenshot, and set Capture to Screen under the cursor, Active window, All screens or A region I drag. For a region, drag a rectangle; Escape or a right-click cancels.

A screenshot isn’t saved or sent anywhere by itself. What happens next depends on the actions after it, such as Read text, Ask AI or Note.

Copy text from a picture

In a pill’s editor, add Screenshot, Read text and Copy to clipboard, in that order, to copy text from part of the screen. Set Picture on Read text to Latest screenshot, Attached image or Copied image.

Reading happens on your PC and the picture isn’t sent to an AI provider. It uses text recognition for your language, which you add in Windows language settings. If Read text finds nothing, check that your language is added there.

Find and highlight something

The Find on screen pill takes a screenshot, asks you what you’re looking for, and draws a box around it. Type a description such as “the settings button”.

By default it uses Jev, a model from TypeSafe that reads pictures. Add a TypeSafe key in Models first. The screenshot is sent to TypeSafe. To keep the picture on your PC, set Find by to Text on the Find step: it then looks for words on screen and sends nothing.

If nothing is found, the pill stops. Set If not found to Carry on to continue with the next action.

Render element draws the box. Set its Colour, Thickness, Corners, Stays for and Animation under its options. The box is an estimate, so check it before you follow it with a Click.

Click a found place

Add a Click action after Find on screen and choose the found spot in “Where”. “Which part” picks the centre, an edge or a corner, and Button picks Left click, Right click or Double click. Move right and Move down shift the click by a few pixels.

“Where” needs a screenshot taken earlier in the same pill. Use “Run up to here” in the pill editor to see where it would click without clicking.

Pick a color

In a pill’s editor, choose Add an action, then Pick a color. When it runs, the screen freezes and a magnifier follows the pointer. Click a pixel to get its Hex, RGB and HSL values, or press Escape to cancel. Follow it with Copy to clipboard and choose the value you want.