Automagic Forum

Have a condition that can compare two images, if they are the same, then it's true.

What exactly you're looking for? Plain compare files or some kind of «visual match»? The last involves different formats, sizes, colorspaces, EXIF data and so on.

I can give you a use case scenario and that might better explain the feature that I'm requesting.

I take a screenshot and save the image to a variable I then crop this image and save it.
I take another screenshot, crop and compare it with the first and if they are the same then the condition is true.

Seems there no «fuzz» logic of any kind needed? Just compare pixel data?

It looks simple, but I failed to find the way to compare objects, stored in Init Variable Image File. Of course, comparing different objects is always returns false (even the underlying structures are same)
.
Though, we can FR for a something like object_hash(), but implementation of this can be more complex than expected in common case (deep object traversal, serialization…, but I'm not a Java dev). Instead of it, as an immediate solution, we can ask for an another variable in Init Variable Image, something like image_hash.

@robhoc : I have tried getPixelColor() as proof of concept. The screenshot, cropping to 121x28 and then loop check it based on the saved portion of the image takes around 3 seconds. bigger image probably will take much more time. It can match at around 50-100% percent with the similar image. Somehow, I see that the image pixel is different although it seems exactly the same. Probably there is some variation on the alpha channel. Changing to the get-argb method, get varied result, but takes longer, up to 7 seconds. So better use getPixelColor() only.

Using this with automagic forum, chrome with desktop mode, "Logout [Desmanto]" button, it works 100% everytime it detect it at the same position. But if I scroll the browser a bit, or zoom in out, the detection drops dramatically, usually in the range of 0-5 % only. So this kind of detection only works best when the cropped area is always the same (button in a fixed UI element).

@beelze : this thread is actually a continuation from here : viewtopic.php?f=5&t=7683
Hash method doesn't work, probably because metadata. And we need to torelate some variation, let say confidence matching level 95% above. So there is slight margin for error before counting it as different.

Another possible solution, if there is someone who know about openCV and have the binary for android, probably we can use that too to do the comparison.

Desmanto wrote:I have tried getPixelColor()

As expected, it is horribly slow. I even not thinked about it.

As I said before, we should not blindly compare pixels, because of (simplified!):
a) we have not a fullscreen images but croppings, so we can't rely on the same offset/size
b) as you mentioned, there can be small scrollings
c) we can't rely on «stable» font/UI elements rendering (even if someone successfully run 100500 equality tests, the rendering algorithms are complex and not fully «pixel-predictable»)

Image comparing is a COMPLEX task, as I said before. and it's obviously not an AM task (exactly like a media processing/encoding, for instance)

We should not reinvent the wheel: famous imagemagick project contain all about image processing. Please consider compare command. Search github for the imagemagick android ports, I've seen a couple.

A little test:
1904×1904 .jpg image (car photo) has been sharpened a little and cropped a little. After 15 minutes of tests and reading manuals, I was finally successful: (0 absolute error count, number of different pixels, -fuzz affected)

... but it took almost 20 seconds on my Core i5

Tasker has the facility to compare images but I'm trying to move away from Tasker as I'm running out of space on my 8Gb phones, with all the plugins necessary.

I can use the autoinput plugin for now.

Thanks for the input.

Desmanto wrote:Hash method doesn't work, probably because metadata. .

Probably because it hashes not an Object, but str(Object). Which, I guess is a something like "blah.blah.Bitmap @somewhere".

@beelze : I also expect it will be slow, since we have to take screenshot first. It simply prove my concept and the method is similar to what I found by googling. I would expect a better finished algorithm, probably from openCV, as it is mostly computer vision problem. I can't find any working android binary for the imagemagick. Found this https://github.com/ayaromenok/Android_I ... 7/releases but can't execute it from shell. I prefer the binary for the shell command, so we can use execute root command to run it. Or probably better if Automagic adopts it into action (as requested)

The point robchoc trying to make is actually related to autoinput. Autoinput has the action to Compare the cropped screenshot : http://i.imgur.com/X8sdRXc.jpg
While we can do it already using the loop to the getPixelColor() now (probably autoinput also use the same algorithm), it will be easier to use if Automagic has the action built-in. Something probably called : Compare image. Then we can adjust for tolerance, or we can compared with several method. Probably ignore resolution (to compare resized image), ignore certain color, compare all argb channel, compare only portion of the image, compared shifted image and etc.

I also don't want Automagic to reinvent back the wheel, just use the open source solution available out there and pick one as the best solution.

Since this solution is mostly to replace UI detection in Control UI, it should be working fine. As the UI always has the fixed element, it is just not detectable by current Control UI or UI event; hence we have to detect it using image based comparison. And the cropped portion also will make the script run faster than comparing whole screen.

Let's define what we discussing exactly.
If we're talking about «compare 2 images, common case» for similarity (i.e. calculate some abstract difference) which can be used in different contexts (comparing 2 photos in series, compare original and processed image, compare screenshots) – it worths discussing. In contrast, any partial solution (like screenshot-only comparison) should be implemented outside AM.

In any case, we need at least «compensate the motion» before comparing, eliminate shift and size difference. In other words, find overlapping rectangle. Otherwise even for the 1 pixel offset, we will get high absolute error count. The funny part is recursion – in order to find this «overlapping», we need a method to (fuzzy) comparing images' parts!
No doubts, trying to implement it is surely reinventing the wheel.

OpenCV looks a proper tool, but it's a very heavy dependency for the AM. Of course, we do not need a whole OpenCV, but someone should fork needed parts and maintain it over years. Here's a relatively fresh imagemagick port, but it looks like one-time homework.

The good news is that we can use a ready and well-tested code from geeqie (i'm used «find duplicates» in geeqie many times and it's good):
1. I've compared original car photo with sharpened/cropped version, got 96 similarity
2. 2 screenshots, which I took carelessly (significant shift and size difference), got 96 similarity
3. 2 screenshots, which I took carefully, got 99 similarity

The C source code is not complex and can be easily ported to Java or whatever needed.