Pete Hahnloser@beehaw.org to Technology@beehaw.orgEnglish · 23 days agoAnthropic publicly releases AI tool that can take over the user’s mouse cursorarstechnica.comexternal-linkmessage-square6fedilinkarrow-up140arrow-down10file-text
arrow-up140arrow-down1external-linkAnthropic publicly releases AI tool that can take over the user’s mouse cursorarstechnica.comPete Hahnloser@beehaw.org to Technology@beehaw.orgEnglish · 23 days agomessage-square6fedilinkfile-text
minus-squareKissaki@beehaw.orglinkfedilinkEnglisharrow-up6·22 days agoIIRC Windows has an accessibility feature where the cursor jumps to the primary default action in opening dialogs. Doing it screenshot based seems inefficient if y du could iterate through windows and controls.
minus-squareflashgnash@lemm.eelinkfedilinkarrow-up3·22 days agoMakes it work universally, even if the gui isn’t made with a standard toolkit Also it’s ai they don’t care about efficiency
minus-squareToribor@corndog.sociallinkfedilinkEnglisharrow-up2·21 days agoYeah this is one of those things where accessibility settings can probably get you 90% there but screenshots and machine learning can probably close the gap somewhat reliably (even if it’s much less efficient).
IIRC Windows has an accessibility feature where the cursor jumps to the primary default action in opening dialogs.
Doing it screenshot based seems inefficient if y du could iterate through windows and controls.
Makes it work universally, even if the gui isn’t made with a standard toolkit
Also it’s ai they don’t care about efficiency
Yeah this is one of those things where accessibility settings can probably get you 90% there but screenshots and machine learning can probably close the gap somewhat reliably (even if it’s much less efficient).