Zori
New Member
Posts: 30
|
Post by Zori on May 16, 2017 7:15:40 GMT
Hello guys how are you? I wanna develop a pretty simple script who does the following: 1.Search for an image within the screen and keep its location in X 2.Run a loop of like 30 minutes: 2.1 Search for 1 of 10 images 2.1.1 Click anywhere(Except of X) 2.1.2 Get out of the loop 2.2 Click X
Now i can write the basic stuff but i didn't understand some things: -Where do I need to put the images of the image recognition? -How do I trigger the image recognition? -How do I save a location from an image recognition? -How do I use the saved location in order to stimulate a click?
And some questions about it: -Is image recognition each perform of the loop of 10 items will take bunch of resources and time? -If so, is there a way to reduce it?
Thanks in advance <3 <3 <3
|
|
|
Post by AnkuLua on May 16, 2017 8:02:48 GMT
-Where do I need to put the images of the image recognition? Put the images to the folder "image". Refer to the example of calculator script. ankulua.boards.net/thread/4/first-example-usage-ankulua-application-How do I trigger the image recognition? -How do I save a location from an image recognition? -How do I use the saved location in order to stimulate a click? Use exists or wait to do the things. match = wait("target.png", 5) -- wait the target.png for maximally 5 seconds. If found, save the result to match click(match) -- click the saved result Before concerning the speed, make it work first.
|
|
Zori
New Member
Posts: 30
|
Post by Zori on May 16, 2017 11:25:36 GMT
-Where do I need to put the images of the image recognition? Put the images to the folder "image". Refer to the example of calculator script. ankulua.boards.net/thread/4/first-example-usage-ankulua-application-How do I trigger the image recognition? -How do I save a location from an image recognition? -How do I use the saved location in order to stimulate a click? Use exists or wait to do the things. match = wait("target.png", 5) -- wait the target.png for maximally 5 seconds. If found, save the result to match click(match) -- click the saved result Before concerning the speed, make it work first. Managed to make it work, now I have some holes in my code: -How do use image recognition only to get boolean flag (if the screen contains it, return true otherwise return false) -How do I stimulate a click by coordinates? -Can i manipulate Match object with and/or? (Wanna check if one object from 10 objects is visible, if it is so return true or enter the loop) So the scanning works (I used exists, don't know what is its return value), but it takes quite some time, is there a way to reduce it?
|
|
|
Post by Mercobots on May 16, 2017 12:14:14 GMT
|
|
Zori
New Member
Posts: 30
|
Post by Zori on May 16, 2017 13:26:56 GMT
Thank you for the quick reply! Now I got my code working but the image recognition doesn't work all the time, maybe its related to the way i cropped or to the success rate?
|
|
|
Post by Mercobots on May 16, 2017 13:55:30 GMT
you r welcome importantbe sure you set proper set CompareDimension() and ScriptDimension() according your resolution There are some very important settings. Some for the script to work properly. And Some are for development and debugging. Settings for script functionSettings:setCompareDimension(boolean compareByWidth, int pixels):This function is used to set the image matching (comparing) dimension. compareByWidth: If the application that we want to automate is scaled in width, set this to true. It's the normal case for most Apps. pixels: This value is the number of screen pixels when the image matching is performed. The value is width or height according to the value of compareByWidth. Return value: none This is the secret that AnkuLua can work on different devices with the same script. Take the example of our first example. We capture the images from device with 960 on width. So we have the following code in our script. Settings:setCompareDimension(true, 960) If the device width to run the script is different, the captured screen will be first scaled to 960. Then the image matching is performed. Becuase all captured screen is scaled to our setting, the script works independent of running device. There is one thing to note. The number of pixels is App usable screen size, not real screen size. Device with navigation bar is one case that App usable screen size is different from real size. In the following picture, App usable screen size is inside the yellow rectangle. do some tests EDITEDtestMyImg = function(img) local similiraty = 0.9 local timer = Timer() local _try = 1 --
while true do toast("try Nº" .. _try) local pattern = Pattern(img):similar(similiraty) timer:set() if exists(pattern) then local m = getLastMatch() m:highlight(1) print("---------------") print("[FOUND]") print("- time elapsed: " .. timer:set()-1) print("- used-similiraty: " .. similiraty) print("- real-similiraty: " .. m:getScore()) print("---------------") break else print("---------------") print("[NOT FOUND]") print("- time elapsed: " .. timer:set()) print("- used-similiraty: " .. similiraty)
print("---------------") end similiraty = similiraty - 0.05 _try = _try + 1 end end
testMyImg("img.png")
only change this linetestMyImg("img.png")
|
|
Zori
New Member
Posts: 30
|
Post by Zori on May 20, 2017 11:26:39 GMT
Mercobots and AnKuLua I will explain the idea behind my code and what I did in order to develop it. It's mainly a bot for a game named King Of Thieves, who will found you decent gems. So here's the idea, the whole script will run on this screen, which will have 3 basic elements: Red - totem Blue - gems Green - skip button Each player has a totem in his dungeon, and this totem can have a ritual with 3 gems running. The main purpose of the bot is to find a player's dungeon with specific gem in it, for an example. This example contains 3 gems which are called golden gems, and there are 10 of those, you can find them here in the 2 right columns. Those gems are very rare, and in order to find them you need to skip a lot of dungeons, using the skip button. So now after I explained the idea behind my bot, this is what I have done: I opened remoteAKL on 960 resolution and I opened screenshot (some weren't from my phone) of the golden gems in the totem, then I used Photoshop to crop them. After I have cropped the gems I inserted the .png files to the script's folder and started to code using your amazing help. This is the code I ended up with:
--============ Settings =============
setImmersiveMode(true)
Settings:setCompareDimension(true, 960)
Settings:setScriptDimension(true, 960)
--============ Program ==============
function existsMultiMax(target, region)
local oldROI = Settings:getROI()
local maxScore = 0
local maxIndex = 0
local match
if (region ~= nil) then Settings:setROI(region) end
for i, t in ipairs(target) do
if (i == 1) then usePreviousSnap(false) else usePreviousSnap(true) end
if (exists(t, 0.5)) then -- check once
local score = getLastMatch():getScore()
if (score > maxScore and score > 0.99) then
maxScore = score
maxIndex = i
match = getLastMatch()
end
end
end
resumeROI(oldROI)
usePreviousSnap(false)
if (maxScore == 0) then
return -1
end
return maxIndex, match
end
function resumeROI(oldROI)
if (oldROI) then
Settings:setROI(oldROI)
else
Settings:setROI()
end
end
function findGolden()
myImages ={
"smallYellow.png",
"smallRed.png",
"smallGreen.png",
"smallBlue.png",
"smallPurple.png",
"bigYellow.png",
"bigRed.png",
"bigGreen.png",
"bigBlue.png",
"bigPurple.png",
}
i, match = existsMultiMax(myImages)
if match then
setHighlightStyle(0x8fffff0000, true)
match:highlight(2)
toast("found -> " .. myImages[i])
loc = Location(100,100)
click(loc)
print("Found a golden!") --Letting you know we found a golden
flag = true
else
click(skip)
end
end
skip = wait("Skip.png", 5)
flag = false
do
while flag == false do
findGolden()
end
end
Now the main idea works like this: 1.Check if there is a skip button. 1.1 If so, search in the whole image for golden gems using the code Mercobots gave me. 1.1.1 If you find one, popup a message saying you found one, mark the gem and get out of the loop. 1.1.2 If you didn't find one, click the skip button and go back to 1.1. And I checked it on some screenshots I had and it worked. But in real time it didn't work and failed detecting the gem itself/detecting the right gem, so I tried lowering the score, and it found other gems as the golden gems. What have I done incorrectly? How do I make the bot found at least 1 good gem without failing detecting it or detecting incorrectly? It's something univalent, how can I make the image recognition find the gem without failing? Thanks for you time and patient <3 <3
|
|
|
Post by Mercobots on May 21, 2017 0:06:03 GMT
hello I really hope this help you , without playing the game it's hard to script - The script is not tested
- don't forget to change the path of commonLib.lua to yours - You will need a new image , the chest.png, it should be like this one - If needed adjust the region calculation at getChestRegion()-- Constants vars --
-- Path ROOT = scriptPath() DIR_LIB = ROOT .. "lib/"
-- Engine (Main Vars)
TIME_OUT = 3 SIMILARITY = 0.85 SCRIPT_DIMENSION = 1280 COMPARE_DIMENSION = 1280
-- BASIC SETTINGS Settings:setScriptDimension(true, SCRIPT_DIMENSION) Settings:setCompareDimension(true, COMPARE_DIMENSION) -- Settings:set("MinSimilarity", SIMILARITY) Settings:set("AutoWaitTimeout", TIME_OUT) -- setHighlightStyle(0x8fffff0000, true)
-- load commonLib dofile(DIR_LIB .. "commonLib.lua")
-- IntelliJ IDE DEBUG -- this var is just to trick the ide local _true = true
-- function to create a small region to find the goldens local getChestRegion = function() if exists("chest.png") then local m = getLastMatch()
-- get sizes and axis local r = { x = m:getX(), y = m:getY(), w = m:getW(), h = m:getH(), } -- Region Calculation r.y = r.y - (r.h * 2) return Region(r.x, r.y, r.w, r.h) else return nil end end
-- find golden and return current condition local findGolden = function() local goldens = { "smallYellow.png", "smallRed.png", "smallGreen.png", "smallBlue.png", "smallPurple.png", "bigYellow.png", "bigRed.png", "bigGreen.png", "bigBlue.png", "bigPurple.png", }
local i, match = existsMultiMax(goldens, getChestRegion())
if match then match:highlight(2) toast("found -> " .. goldens[i]) -- loc = Location(100, 100) click(loc) -- print("Found a golden!") --Letting you know we found a golden
-- get the golden name local golden_name = string.gsub(goldens[i], "(%.%w+)$", "") return true, golden_name
elseif existsClick("skip.png") then return false, "_no_golden" else return false, "_error" end end
-- Main While -- with this small loop you can keep track of your script and manipulate the actions while _true do print("main loop") print("-----------------------") local golden_found, msg = findGolden()
if golden_found then print("golden found -> " .. msg) -- msg = name of the golden found -- do stuff elseif msg == "_no_golden" then print("golden Not found -> " .. msg) -- goto next dungeon -- continue the while elseif msg == "_error_" then print("golden Not found -> " .. msg)
-- out of dungeon view -- do stuff to openanother dungeon end print("-----------------------") end
|
|
Zori
New Member
Posts: 30
|
Post by Zori on May 21, 2017 1:58:53 GMT
hello I really hope this help you , without playing the game it's hard to script - The script is not tested
- don't forget to change the path of commonLib.lua to yours - You will need a new image , the chest.png, it should be like this one - If needed adjust the region calculation at getChestRegion()CODE Wow man, thank you so much for the help. The basic format is working but the bot can't track the gems and find the specific one. What are the steps of finding an item 100% correctly? because in minimumSimilarity of 0.99 it cant find the chest/gems, and its the same chest in every dungeon. Have I done something wrong?
|
|
|
Post by Mercobots on May 21, 2017 2:33:11 GMT
hello
Try minimumSimilarity 0.7 or 0.8 and add this line
local getChestRegion = function() .... .... -> Region(r.x, r.y, r.w, r.h):highlight(2) return Region(r.x, r.y, r.w, r.h) .... .... end
be sure the goldens are inside the region
EDITED btw the getChestRegion() will return a region of chest.png if was found, otherwise will return nil
once the existsMultiMax(goldens, getChestRegion()) is fired,he will try find the goldens by the given region if chest found, if not will take whole screen.
|
|
Zori
New Member
Posts: 30
|
Post by Zori on May 21, 2017 9:07:07 GMT
hello Try minimumSimilarity 0.7 or 0.8 and add this line local getChestRegion = function() .... .... -> Region(r.x, r.y, r.w, r.h):highlight(2) return Region(r.x, r.y, r.w, r.h) .... .... end be sure the goldens are inside the region EDITED btw the getChestRegion() will return a region of chest.png if was found, otherwise will return nilonce the existsMultiMax(goldens, getChestRegion()) is fired,he will try find the goldens by the given region if chest found, if not will take whole screen. Using this code: --============ Settings =============
setImmersiveMode(true)
Settings:setCompareDimension(true, 960)
Settings:setScriptDimension(true, 960)
--
Settings:set("MinSimilarity", 0.7)
Settings:set("AutoWaitTimeout", 1)
--
setHighlightStyle(0x8fffff0000, true)
-- load commonLib
dofile("/sdcard/AnkuLua/firstBot/commonLib.v1.1.lua")
-- IntelliJ IDE DEBUG
-- this var is just to trick the ide
local _true = true
-- function to create a small region to find the goldens
local getChestRegion = function()
if exists("chest.png") then
local m = getLastMatch()
-- get sizes and axis
local r = {
x = m:getX(),
y = m:getY(),
w = m:getW(),
h = m:getH(),
}
Region(r.x, r.y, r.w, r.h):highlight(2)
-- Region Calculation
r.y = r.y - (r.h *3)
r.h = r.h*2
return Region(r.x, r.y, r.w, r.h)
else
return nil
end
end
-- find golden and return current condition
local findGolden = function()
local goldens = {
"smallYellow.png",
"smallRed.png",
"smallGreen.png",
"smallBlue.png",
"smallPurple.png",
"bigYellow.png",
"bigRed.png",
"bigGreen.png",
"bigBlue.png",
"bigPurple.png",
}
local i, match = existsMultiMax(goldens, getChestRegion())
if match then
match:highlight(2)
toast("found -> " .. goldens[i])
--
loc = Location(100, 100)
click(loc)
--
print("Found a golden!") --Letting you know we found a golden
-- get the golden name
local golden_name = string.gsub(goldens[i], "(%.%w+)$", "")
return true, golden_name
elseif exists("Skip.png") then
print("no golden")
return false, "_no_golden"
else
print("no golden")
return false, "_error"
end
end
-- Main While
-- with this small loop you can keep track of your script and manipulate the actions
flag = false
while flag == false do
print("main loop")
print("-----------------------")
local golden_found, msg = findGolden()
if golden_found then
print("golden found -> " .. msg)
flag = true
-- msg = name of the golden found
-- do stuff
elseif msg == "_no_golden" then
-- getChestRegion():highlight(2)
click("Skip.png")
print("golden Not found -> " .. msg)
-- goto next dungeon
-- continue the while
elseif msg == "_error_" then
-- getChestRegion():highlight(2)
click("Skip.png")
print("golden Not found -> " .. msg)
-- out of dungeon view
-- do stuff to openanother dungeon
end
print("-----------------------")
end I get the following result. I swiped between screenshots and it didn't recognize the goldens. Any idea why is it failing?
|
|
|
Post by Mercobots on May 21, 2017 9:31:37 GMT
The red square (region) should cover the golden s
|
|
Zori
New Member
Posts: 30
|
Post by Zori on May 21, 2017 9:42:49 GMT
The red square (region) should cover the golden s It is if i put the calculation like this: r.y = r.y - (r.h *3)
r.h = r.h*2
Region(r.x, r.y, r.w, r.h):highlight(2) But it still fail to recognize it
|
|
|
Post by Mercobots on May 21, 2017 9:54:37 GMT
|
|
Zori
New Member
Posts: 30
|
Post by Zori on May 21, 2017 11:21:54 GMT
Ok, where do i find the images after i record? because i cant find that luar/... directory
|
|