control UI: click does not work

[FhemPiUser]

Hi all,

I want to automate a click of a button on a webpage running in full screen mode (browser app: fully). Therefore, I have defined a control ui action with a script content "click(x,y)", but it does not work. I also "click(pattern)". the coordinates as well as the pattern where identified by the overlay function. Did I miss anything?

The accessibility function is activated, the display is on an unlocked. The device is a android 5.0 device. I trigger the action using a http request trigger and I can see in the log that the action is indeed executed.

It also doesn't work with another browser app (firefox).

The button is a pagebutton of fhem tablet ui (Javascript).

Any hint?

[FhemPiUser]

is this maybe the reason?

[should be fixed in Android 5.1] Action Control UI: Some functions like click() don't work on some list items on Android 5 due to an internal bug of Android. The bug has been reported to Google and was recently marked as fixed in a future version (I hope they don't mean Android 6). Currently there's no known workaround except to use action Execute Root Command: input tap x y to simulate a touch on the coordinates on screen.

[Martin]

It's probably caused by a limitation of the Android accessibility feature and the browser in use. Clicking buttons often does not work since the button can not be detected as a button component (is not marked with a green border around the button). You could try to use action Execute Root Command: input tap <x> <y> (on a rooted device) or try to use keyboard simulation with function sendKey to focus the button and then to issue a sendKey("ENTER") to press the button (not sure if this works on the button in question since it might not react to the enter key at all when it only handles mouse clicks).

Regards,
Martin