Attempting to Interact with the UI

[sitruk]

Hello, I have a VPN app that I would like to perform some mild automation with. I am able to launch it and have it automatically connect. However, disconnecting is another story. It appears the best case scenario would be to launch the VPN app and control the UI to tap the disconnect button. I have tried a few options including root command to click a location on the screen. If I use "Control UI: click(720, 1093) the flow completes without error, but nothing happens.

Can anyone shed any insight on my issue and maybe guide me in the right direction?

[Desmanto]

I assumed you have tried clickById();

Code: Select all

clickByld("com.privateinternetaccess.android:id/fragment_connect_slider");

If yes, then probably the problem is the timing. Try to add sleep(1000) or longer at the control UI script.
Remember, the Control UI must come after the launch app/shortcut action, so it will execute after it, not at the same time.

Code: Select all

sleep(1000);
clickByld("com.privateinternetaccess.android:id/fragment_connect_slider");

If it works, you can try lower value to see if it still works, such 500 or 200.

[sitruk]

I assumed you have tried clickById();

Code: Select all

clickByld("com.privateinternetaccess.android:id/fragment_connect_slider");

If yes, then probably the problem is the timing. Try to add sleep(1000) or longer at the control UI script.
Remember, the Control UI must come after the launch app/shortcut action, so it will execute after it, not at the same time.

Code: Select all

sleep(1000);
clickByld("com.privateinternetaccess.android:id/fragment_connect_slider");

If it works, you can try lower value to see if it still works, such 500 or 200.

Yes, I did in fact try the clickByld option. I also added a sleep step but it doesn't seem to help. I added the sleep step into the script at 1000 as well as 5000 and there is no difference.

[Desmanto]

Maybe the app has the secured flag on. It prohibits any accessibility access. But root command should have bypassed it. Try to use control UI with other element to make sure the Control UI works at the app. Try to click that hamburger menu at top left. It seems the app doesn't support automation/intent plugin so no way to stop it using intent.

Since you are on nougat, you can try to use gesture and try to swipe the toggle to left.

[sitruk]

Maybe the app has the secured flag on. It prohibits any accessibility access. But root command should have bypassed it. Try to use control UI with other element to make sure the Control UI works at the app. Try to click that hamburger menu at top left. It seems the app doesn't support automation/intent plugin so no way to stop it using intent.

Since you are on nougat, you can try to use gesture and try to swipe the toggle to left.

Thank you for your guidance. You were right, it appears this app is not accepting any kind of UI Control. However, I tried the gesture option as you suggested and just using a simple tap gesture did the trick. Thank you again!

[digitalstone]

On a sidenote:
Isn't the code behind the "Control UI" element getting discontinued by Google?
I also want to dive into the Control UI, but if it goes "away" then it might not be worth to build anything with it.
I haven't read any news on that lately (or i might be looking with my butt).

-- EDIT --
Sorry to "hijack" this topic, but it seemed a proper subject + it reminded me.

[Desmanto]

@sitruk : That VPN app is way too secured But I just realize the can gesture can be used that way too. (I am still using LP 5.1, waiting for the next phone with Oreo).

@digitalstone : The status now is undefined, someone have linked the group discussion as "Won't Fix", but can't find it anywhere now. So far, google haven't made any drastic move, but keep pushing dev behind the stage. It seems all dev must comply to certain set of accessibility term of usage if they want to keep their app. Considering no big news yet lately, it seems there is no further restriction from google yet (but I don't know what happened behind the scene).

CUI is one of my top 3 most used actions, it accounts for 64 usages, or about 7% of total actions; after script (206) and notification on screen (92). Well, not all are in use, but I can say at least 50 above are inside working flows. Removing this accessibility feature will really create a deep impact on someone who has "laziness disability" like me.

[sitruk]

@sitruk : That VPN app is way too secured But I just realize the can gesture can be used that way too. (I am still using LP 5.1, waiting for the next phone with Oreo).

@digitalstone : The status now is undefined, someone have linked the group discussion as "Won't Fix", but can't find it anywhere now. So far, google haven't made any drastic move, but keep pushing dev behind the stage. It seems all dev must comply to certain set of accessibility term of usage if they want to keep their app. Considering no big news yet lately, it seems there is no further restriction from google yet (but I don't know what happened behind the scene).

CUI is one of my top 3 most used actions, it accounts for 64 usages, or about 7% of total actions; after script (206) and notification on screen (92). Well, not all are in use, but I can say at least 50 above are inside working flows. Removing this accessibility feature will really create a deep impact on someone who has "laziness disability" like me.

HAH! Yea it appears to be that way.

I am just now starting to delve into CUI and I am dying to find more reasons to use it now. Does the screen need to be on and unlocked in order for some or all CUI functions?

[Desmanto]

HAH! Yea it appears to be that way.

I am just now starting to delve into CUI and I am dying to find more reasons to use it now. Does the screen need to be on and unlocked in order for some or all CUI functions?

You will find a lot of interesting things it can do. Yes, the screen must be on and unlocked, so you need to coupled it with turn screen on and set keyguard off (or dismiss keyguard). Locked screen will cause all CUI function fail without any error shown.

[sitruk]

HAH! Yea it appears to be that way.

I am just now starting to delve into CUI and I am dying to find more reasons to use it now. Does the screen need to be on and unlocked in order for some or all CUI functions?

You will find a lot of interesting things it can do. Yes, the screen must be on and unlocked, so you need to coupled it with turn screen on and set keyguard off (or dismiss keyguard). Locked screen will cause all CUI function fail without any error shown.

Ok I kind of discovered that through trial and error but I just wanted to hear it for sure. Thank you