Flow Execution Policy & Automagic Emergency Stop

[Desmanto]

This is a test flow to help us to understand how Flow Execution Policy and Automagic Emergency Stop works. It doesn't serve any other purpose.
For you who just wanna test the flow, you can jump directly to the "Flow Execution Policy Test flow" below.

1. Flow Options, FEP AES.png (152.23 KiB) Viewed 35359 times

Flow Execution Policy (FEP)
Automagic has 4 ways to determine what happen when a trigger is received and the current flow is running.
1. Parallel : multiple trigger will run on its own flow execution
2. Skip : multiple trigger will miss if previous triggered one still running
3. Wait (Queue) : Multiple trigger will queue and executed one by one after the running one finish.
4. Stop : stop current running execution and execute new trigger.


King and minister Analogy
If i can make an analogy for it, using king (flow), his minister (trigger) and minister's program (execution).
1. Parallel : The king allow the ministers to execute their program at the same time (parallel execution). Since He has separated all of them based on their department (independent parallel branching), so no interference to each other. Minister of telecommunication won't interfere much to the program from minister of health care.
2. Skip : The king will only allow one program at one time. No other minister can interfere with the running program until it is finished. So all other minister will be eliminated if they disobey (triggers ignored)
3. Wait (Queue) : Wait is more civilized. There is only one program at one time. If other minister (new trigger) try to propose new program, they will be put on queue (trigger wait), until the current program has been finished. So every minister proposal will be accepted (no trigger ignored), but executed based on the queue.
4. Stop : Every new minister program (new trigger) will stop the previous minister program (stop current execution). So it is still only one program at one time. There will be a lot of unfinished program out there if the current program keep replaced by the new one.
I hope the analogy make sense. Now we move to the example of usage.

Usage of each FEP
1. Parallel
is used when you have no specific usage. This is the default when you create new flow. The triggers usually not so frequent, may be ignored if appear in quick succession, and the flow usually finished as soon as it get triggered (very fast). Usually we just deal with local variable, no GloVar/external output involve; since it may be chaos if the trigger execute parallelly and change GloVar in those successive parallel execution. (Example : 1st trigger change GloVar to "hello", 2nd trigger change it "World", while 1st execution still using it as reference). It is also use when you have multiple trigger separate using parallel expression. Each branch execute its own logic, not interfering other branch. See here for more info Multiple Choice - Series vs Parallel

I have a bunch of one trick pony flow before. Each consist of one trigger with 3-5 actions. Those all has similiar function, example to do something with network or send intent; but not necessary related one to each other. So I just combined them into single flow with 5-10 triggers and separate them using parallel expression to check for multiple trigger.

2. Multiple Trigger.png (168.61 KiB) Viewed 35359 times

This is the example of 7 flow combined into single flow with 7 triggers. Some of the triggers are shortcuts, notification, UI event or media storage event. Each event/action are not related to each other, so I should let them execute parallelly. If i use another FEP, then it will behave differently. Skip FEP will ignored some of the trigger, causing certain event not triggered properly. Wait FEP will delay the flow and make the event not instantly reacted to. Stop FEP will cause some consecutive trigger not executed until finish.

But not all parallel expression checking should use Parallel FEP. It may use another if they have related action in each branch. By default, just use this one first if you are unsure about other FEP.

2. Skip
is used when you need only single execution of certain flow and don't care about the new trigger (or must ignore the new trigger) until this execution is finished. Any new trigger happened during the execution will be ignored (missed) anda can't interfere with current execution in any way. Usually the flow takes more time to finish, since there are timeout or delay in the flow. (Example are take screenshot, downloading file, pinging host etc). When you use GloVar or external file from the beginning until the end of the flow, you surely don't want those value changed by the parallel execution of the same flow. So you must wait until this execution finish first before running another execution.

It is also used when you have action in the flow which can trigger the flow again. You maybe have flow with trigger Wifi State off, then during certain part of the flow, you turn it on and turn it off again. This will trigger the flow again. It also happen to bluetooth, GPS, GloVar, or any trigger that can be triggered again by its own action. The best example for this is clipboard. You may have flow with Clipboard changed trigger, to process the clipboard (example change certain youtube shortlink to full link before sharing it). Then at the end of the flow, you use action Copy Text to Clipboard to set the processed text back to clipboard. This action will trigger the Clipboard changed again, and execute the flow again and again and again until you hit automagic emergency stop. The flow will always runaway anytime you copy something and get disabled in the end. This is the example of my Multiclipboard flow.

3. Multi Clipboard.png (162.11 KiB) Viewed 35359 times

You can't use parallel FEP in this case, as it is a certain recipe for runaway/infinite loop. Wait FEP also won't work, since the the clipboard trigger still get queue and executed after current one finish. Stop FEP also still result in runaway, as it will stop current execution and re-execute with the new clipboard copied. So the only way you can protect this kind of loop is to use Skip FEP. You must ignore the new trigger caused by the action Copy Text to Clipboard. But if you just leave that action at the end of the flow, since the execution time per element is quite fast (only about 3-5 ms), it will still trigger the flow and loop again. You must put action sleep, example 1 second, after the Copy Text to Clipboard. So it will wait at least 1 second before starting to accept new trigger again. At my phone, i can put the sleep as low as 500 ms to protect it.

Extra note : For you who came from tasker, you maybe know something called Cooldown timer, to protect the profile from looping infinitely. Automagic Skip FEP + sleep (time) at the end of the flow works exactly just like the cooldown timer. This Skip FEP can be used to protect your flow from multiple execution. If you have a flow to backup files and it takes several minutes. You have time trigger and manual trigger via shortcut. Sometimes you press the shortcut and think you haven't tap it. So you tap it again. Or maybe the time trigger also happen around the time you execute it manually. If the FEP is parallel, you might have a corrupted backup in the end, since both parallel execution will try to write to the same file. Using skip will ensure only first backup works, the rest of the trigger won't do anything to the executing flow.

3. Wait
is almost the same as skip, except you can't ignore the trigger. You use it when you need only single execution and MUST react to every trigger happened. No trigger can be ignored, any new trigger must be registered and put into queue; but they may not interfere with current execution. This is the only FEP that can have multiple trigger stacked up and executed in order. If you open the flow, the Trigger element will show number of triggers queuing up since current execution. You can remove the queue by stopping the flow from 3 dot menu or stop flows action. This FEP behaves almost the same as Skip FEP, where the flow takes more time to finish (usually longer), deal with GloVar and external output. The difference are wait FEP appreciates every trigger and maintain sequence (order). The key point is the sequence of the triggers.

I don't use this Wait FEP so far. I have a flow concept since the first week I use Automagic, but I haven't finish it until now LOL I know it can be done, it just takes a long time to finish the whole script. The flow purpose is to Scan Tax document automagically and output it to csv, ready to be imported to the program. It deals with GloVar, take several seconds to connect to the server, parse xml and output to csv; hence it supposed to use this Wait FEP. But can't say much until I finish it. So I will just take another example : reading notification from messaging app.

You create a flow which will read out the sender name, app name and the message from the notification, the flow active when you are driving. Speech output will need several seconds to complete single trigger, since you want to read out something like : "Alan, Whatsapp. where are you?" A lot of the time, people just text us several short text consecutively. This will trigger the flow several times, while the flow haven't finish the speech yet. If you use Parallel FEP, depends on the phone, probably there will be chaos since each trigger will try to compete for the speech output resource. Skip FEP will miss several notification if previous one hasn't finished yet when the new notification arrived. We will miss some important messages. Stop FEP also won't work either. If the message arrived in bulk, probably only the last one will be read out. By using Wait FEP, we will ensure every message will be read out, even though it will take more time as they will be executed sequentially.

[Desmanto]

Continued.... (as I can upload 3 image per post).

4. Stop
is used when you want only single execution, but the current execution should be stopped and restart from beginning if there is a new trigger coming. This is mostly used for delay timer, to do something after certain time has passed if there is no new trigger coming. The flow usually has sleep in the middle of the flow and do something after sleep timer ended.

I have shared several flows which utilize this Stop FEP
Navigation Apps Auto turn on/off GPS = viewtopic.php?f=3&t=6899
Windows Recognizer = viewtopic.php?f=3&t=6898
Screen Saver = viewtopic.php?f=3&t=6963
I will use screensaver as example here.

4. Screensaver.png (243.51 KiB) Viewed 35357 times

The flow when trigger by the widget, will hide the screensaver, wait for 10 seconds and show the screensaver again. If I use Parallel FEP, it will cause chaos, as any tap will be executed on its own logic, and will spam several 10 seconds timer and show the screen saver multiple of time after the timer ended. We probably click the screensaver several times and that will spam another new waves of 10 seconds timer. We should wait until every parallel execution finish first and tap the screensaver to hide it, quickly disable the flow and stop it also.

Using Skip FEP, only the first trigger will work. No matter what we do after the first trigger, the flow still wait until 10 seconds and always shows the screensaver. We tap the screen saver, the cycle repeat again, always at 10 seconds interval. Using Wait FEP in this flow will be the real nightmare. Since every trigger will be registered and put into queue. If you have tap 10 times, the flow execute the first one and queue 9 of the rest. After 10 seconds, screensaver shows up. The flow immediately execute the next one, leaving 8 queue. When you tap it, you expect the tap will hide the screensaver. But in fact, that tap registered and queue behind the other 8. You have to wait for those 8 to finished first (8x10 seconds), before you current tap can be executed and hide the screensaver. Imagine if you rage tapping the screen 100 times. You will be locked for 100x10s = 1000s. Not a cool way to test the flow.

So this flow can only use Stop FEP as the logic, not other. By using Stop, it will make sure the timer always get reseted when there is a new trigger (tap/scroll) from user.

I have summarized the typical characteristic of each FEP. It doesn't have to be like the one in the table, as it is only general guide. Usage can be vary depends on the purpose.

5. FEP comparison.png (18.31 KiB) Viewed 35357 times

Flow Execution Policy Test flow
After wall of text, now we need some experiment to really understand the difference among each FEP. Below is the flow.
Flow Execution Policy Test

6. FEP test.png (142.52 KiB) Viewed 35357 times

Make sure you change the Sound notification Agile to match the notification sound you have at your phone. Since not every phone has "Agile" notification sound.

Quick Description
The flow using proximity sensor near as the trigger, since it is easy to trigger it quickly. The first action is to produce the sound notification so you know the flow is executing. Make sure you have turn on the notification volume so you can hear it. There are 7 sleeps with 500ms each to simulate a running flow. Imagine that those sleeps are the real action. At the end, we have vibrate action to mark the end of the flow. We are going to test each Execution policy and see how it behave when we use it. You can change it at 3 dot menu > Options > Flow Execution Policy. Make sure it is Parallel, save it. When you finish with one, come here again and change to the next FEP, Skip, Wait, and Stop. Enable the flow and let's start the show.

1. Parallel : Try to wave over your proximity furiously for several times. You should hear MANY notification sound everytime you cover the proximity. It means the flow executed parallelly. Everytime you cover, it will trigger the flow. After the sleep timer finish, you will also receive MANY vibration. You can even make the whole elements red (so many parallel execution), by waving regularly at the proximity. This is the default FEP, useful when you need immediate reaction from trigger and have no problem with parallel execution.
2. Skip : Try to wave furiously again. You should hear only SINGLE notification sound, even if you wave as furious as in parallel. Only the first wave will trigger the flow, all subsequent wave will be ignored until the flow finish till the last vibrate. You can only trigger it again after the last vibrate finished executing. This FEP is used to protect your flow from being executed twice by the trigger there. It also protect that flow from triggering itself as in clipboard processing flow.
3. Wait : Try to wave gently for several times. You should still hear only SINGLE notification. But the second wave will put a small number at the top right corner of the trigger element. Everytime you wave again, this number increase, but the flow keep executing without stopping. After several waves, stop your waving hand. You can see the flow is still executing and the number decrease everytime the flow finish a cycle (until the the last vibrate). The flow will keep on executing, giving sound and vibrate, until the number disappear (all queued trigger has been executed). You can use 3 dot menu > Stop or Action Stop Flows to stop the flow and delete all queued trigger (number disappear). This FEP is used when all triggers matter, but must be processed in sequence.
4. Stop : Try to wave gently again for several times. You should hear MANY nofitication everytime you wave. But this is still single execution only. You can see that if you wave fast enough, the flow never has the chance to be executed until finish. Let say the flow now execute until the 4rd sleep (bottom right) and you wave again. The flow immediately stopped and restart again from the first notification sound. The vibrate action will never be executed as long as there are new triggers keep coming before the flow executed until that step. This FEP is used mostly in delayed timer, where you create a kind of timer waiting for the trigger. If there is no more trigger happens, the last action then get the chance to be executed.

I hope after you test the flow, you gain a deep insight how each FEP work and when they can be useful. In most of the time, if there is no specific need, I just leave the FEP as default (parallel). Currently I have 72 flows (10+ of them are test flows); 55 parallel, 9 skip, 2 wait, 6 stop. Maybe I can say that only 20% using skip, 10% stop, 5% wait, the rest are parallel.


Automagic Emergency Stop (AES)
Since I have explained it so far until here, I think I would just continue with the AES too, as it is located below FEP.

Automagic has its own built-in way to handle runaway flow. Runaway flow is unstoppable flow which run in an infinite loop or get triggered too often that it executes so many times within a short period of time. The example of runaway flow has been referenced above : the clipboard processing flow. If the Copy text to Clipboard is not protected using Skip FEP + Sleep action, then it will trigger its own flow and runaway until AES stop it.

AES will stop any flow if the elements of execution has reach certain value within a minute. Global Default is 60 elements. Each condition or action will be counted as one execution. Only those executed will be counted. If there is branching, and the flow run thru one of them; only that branch elements will be counted. Parallel execution will be counted all as well, if there are 3 parallel, those counted as 3 per step. Any loop will be counted everytime they are executed again. Trigger element is not counted as 1 execution. But if there is a flow with single trigger and no any other element connected to it, that trigger will be counted as 1 everytime it gets triggered.

At any moment of execution, if the elements executed has reach AES value within a minute, it will be stopped immediately with error "Emergency Stop". The flow will be disabled to make sure it doesn't get trigger anymore or executed from other widget/flow. However it still can be abused though, by using Set Flow State Enabled, the flow which got disabled still can be triggered and run again (and runaway again). So make sure you design your flow properly.

Default global AES value can be changed at Automagic Settings > General > Automagic emergency stop. Or at each flow, you can set specific value too, 3 dot menu > Options > Automagic emergency stop > Specific and change to the value needed. Most of the time, default 60 is enough. But you may need to increase if you are testing a flow with a long loop within several seconds. Or decrease if you are testing loop with dangerous quick successive triggers.

I have a flow to auto renumber video. It used to have 5-7 elements per execution. Every tap of rename will trigger the rename and can be executed as fast as every seconds. If I renumber 20 videos continously, I will reach minimum 20x5 = 100 elements within 20 seconds. This definitely will be stopped by default AES, which is only 60. Taking the maximum 7 elements per element, i have to increase the AES to at least 7x60 = 420, or maybe just 500 in case I rename 60 videos per minute.

But later, i optimize the flow, combined so the script/expression elements. Now I have only 2-4 elements execution! This means I only need max 4x60 = 240, or in most of cases, 200 is sufficient. So you will see why I can't help commenting other's flow when there are a lot of duplicate elements or script/expression connected consecutively. Those can be optimized to make the elements execution lower. We should try to set the AES as low as needed (default is enough), as we never know that one day the flow just runaway without our consent (some error or logical flaw can cause it). If the AES is too high, the flow might be not stopped and continue running on background, killing our battery and reducing overall perfomance. Or maybe worse, if the flow involve hardware, such as turning on flashlight or other, it may reduce the life expectancy of our hardware.


Testing AES
Now it is the test time. Using the same Flow Execution Policy Test above, go to 3 dot menu > Options > change the FEP to skip (so it is easier to check) and AES to 10. We have 9 elements per run, so we expect after executing the flow the second time, it should stop after executing the first sound notification. Try it out.

You can try to increase the AES value, enable and trigger the flow again and see how it behaves to the new value.

I am sorry for a very long wall of text. I don't have much time to make it shorter. Hope this help and have a nice day.

[tphg]

Just curious a little bit on the flow, how can you add in the notes with dark-yellow color in your flow?

[Desmanto]

It is new feature in automagic 1.34.0. When you add new element, you should see it already : Action, Condition, Note, Template. Or just long press on any blank space in the flow, there will be option to Add note. In case you don't see it, please check your playstore and update automagic to the latest version.

[tphg]

It is new feature in automagic 1.34.0. When you add new element, you should see it already : Action, Condition, Note, Template. Or just long press on any blank space in the flow, there will be option to Add note. In case you don't see it, please check your playstore and update automagic to the latest version.

Found it, thank you @Desmanto. Now I could add note and resize it, but still couldn't find how to make the dotted line connecting to the flow element.

[Desmanto]

You can't add the note to the element. But you can add the element to the note. The connection can only made from the element to note, not the reverse. After made, you can tap the connector again to reposititon it as usual

[tphg]

You can't add the note to the element. But you can add the element to the note. The connection can only made from the element to note, not the reverse. After made, you can tap the connector again to reposititon it as usual

Oh, I see. Thank you.
I think I must read the Automagic changelog to learn such new interesting features.

[Econdoc]

The wording in AM for FEP is not entirely clear for STOP. I quote: "Stop the currently executing instance and execute afterwards." Your diagram shows that Call 1 is killed and Call 2 is executed. So far, so good. It is unclear from your diagram whether Call 1 is executed after Call 2 has completed execution. If so, is there a way to Stop Call 1 and prevent it from running again?

[Desmanto]

Execute afterward should mean the second trigger. For stop, when Call 1 is stopped by Call 2, Call 1 is dead already, no more execution from Call 1. But if there is Call 3, Call 2 will be killed and stop immediately, now Call 3 is in charge. There is no way you can prevent Call 3 in this method. Probably you are asking for something like FEP - STOP then SKIP.

If what you want is to stop the Call 1 and prevent it from triggering the flow again (so no Call 3 possible), you have to separate it to 2 flows.

The basic concepts of FEP - stop is actually similar like this.
Flow 1 : Trigger 1 - Stop Flow 2 - do stuff
Flow 2 : Trigger 2 - Stop Flow 1 - do stuff

Everytime there is Call 2, Flow 2 triggered, stop the Flow 1; and vice versa. But if you want only flow 2 can stop flow 1, not the vice versa. You would remove the stop Flow 2 from Flow 1. And you want when Call 2 is in charger, nothing can triggered the Flow 1; in that case you should disable it and enable it back after the flow finish.

Flow 1 : Trigger 1 - do stuff
Flow 2 : Trigger 2 - Stop Flow 1 - Disable Flow 1 - do stuff - Enable Flow 1

This seems to be too long, but easier to understand and as the base of concept below.


You can actually do it in single flow only, by using FEP Parallel instead. The concept is similar to above, except you don't disable/enable flow. Instead, when triggered by Call 2, you use script to set a glovar to certain value, example : global_call1 = 0. and set it back to 1 after Call 2 finish the flow. (similar to disable and enable it back). In branch of Call 1 trigger 1, use addition expression to check if global_call1 == 0, then don't execute the main flow. The trigger 1 still got triggered, but never continue to the Call 1 branch when the global_call1 == 0.

[Econdoc]

Thanks for the complete explanation. I would suggest to Martin that the wording on the STOP option be changed, something like:
"Stop the currently running instance and execute the newer instance." The use of "afterwards" is misleading.