Basic Objects and Methods Introduction

AnkuLua use Sikuli object structure and function. But not all the objects and functions are supported.
However, they are compatible for supported objects and methods/functions/API. Methods, functions and API are the same in AnkuLua. These terms will be used interchangeably. 
We will introduce what are supported here.
For detailed description, you can get from Sikulix web page. Links are provided for each object and functions.

Let's start from Objects.

---

Objects

Image String: An image string refers to the image file for matching.

Location:
A location is the coordinate of the screen.
The supported methods are getX, getY, grow and offset.

--sample code
loc = Location(100,200)
print(loc:getX()) -- 100
print(loc:getY()) -- 200
loc2 = loc:offset(11, 22)
print(loc2) -- L(111, 222)

AnkuLua only methods:
    typeOf(): return "Location" string.

Pattern:
A pattern is used, to associate an image file with additional attributes used in find operations and when acting on a match object.
The supported methods are mask, similar, targetOffset, and getTargetOffset.

-- modified from example1.lua of calculator plus
-- ========== Settings ================
Settings:setCompareDimension(true, 960)
Settings:setScriptDimension(true, 960)

-- ==========  main program ===========
click(Pattern("6.png"))
plus = Pattern("+.png"):similar(0.8) -- set the similarity on "+.png"
click(plus)

nine = Pattern("6.png"):targetOffset(0, -67) -- 9 is above 6 and the distance is 67 pixels in 960x540 screen
click(nine)
print(nine:getTargetOffset()) -- L(0, -67)
click("-.png")
click("7.png")
click("=.png")

AnkuLua only methods:

    typeOf(): return "Pattern" string.

    getFileName(): return the file name string.

    color(): force color image matching.
        Note: if want to use previous snapshot, the previous snapshot should be in color.

    gray(): force the image as gray for matching

    isColor(): return if the pattern will use color image matching.

pat = Pattern("target.png"):color()
existsClick(pat)
print(pat:isColor() and "color" or "gray")


Region:
A Region is a rectangular area on a Screen and is defined by its upper left corner (x, y) being the pixel with this offset relative to the upper left corner of the screen (usually (0, 0) ) and its dimension (w, h) as its width and height in pixels.
Create the region with Region(x, y, w, h)
The supported object methods are getCenter(), getTarget(), getLastMatch(), getX(), getY(), getW() and  getH().
And following methods accept region as parameters. click(), find(), exists(), wait(), findAll(), waitVanish(), dragDrop() and  type().
We will go to the details of click, find, exists, wait, findAll, waitVanish, dragDrop and type later.

AnkuLua only methods:

    typeOf(): return "Region" string.


Match:
An object of class Match represents the result of a successful find operation. It has the rectangle dimension of the image, that was used to search. It knows the point of its upper left corner on an existing monitor, where it was found.
Since class Match extends class Region, all methods of class Region can be used with a match object.
AnkuLua Match also support following methods: getScore().

AnkuLua only methods:

    typeOf(): return "Match" string.

---

Methods (Sikuli compatible)

For every matching method, there are basic object arguments. The following acronyms are used for each objects.
P: Pattern
S: String
M: Match
R: Region
L: Location

click(PSMRL): Perform a tap on the click point.

-- modified from example1.lua of calculator plus
-- ========== Settings ================
Settings:setCompareDimension(true, 960)
Settings:setScriptDimension(true, 960)

-- ==========  main program ===========
click(Pattern("6.png"))
plus = Pattern("+.png"):similar(0.8) -- set the similarity on "+.png"
click(plus)

nine = Pattern("6.png"):targetOffset(0, -67) -- 9 is above 6 and the distance is 67 pixels in 960x540 screen
match = find(nine)
click(match)
print(match:getScore())  -- the similarity value of "6.png"
print(match:getTarget()) -- the coordinates of "9.png"
print(match:getCenter()) -- the center of "6.png"
print("match upper left coordinate = (" .. match:getX() .. "," .. match:getY() .. ")")
-- upperleft corner coordinate

click("-.png")
click("7.png")
click(Location(885, 515)) -- "="

Some games are found with problem for the original click method.
Therefore, an alternative click method is introduced since version 3.2.0.
If the original click method does not work for your games (Apps), use following function to enable the new method.
setAlternativeClick(enable)

find(PS): Find a particular GUI element, which is seen as the given image.

findAll(PS): Repeatedly find ALL instances of a pattern, until no match can be found anymore.
Note: findAll(PS) returns table with matches, but region:findAll(PS) returns Java List object.
Therefore, to get table with matches from region:findAll(PS), use one of the followings

findAll() throws exception if the target image not found.
If you don't want the exception, use findAllNoFindException() instead.


test1 = findAll("target.png"))

for i, m in ipairs(test1) do
   click(m)
end

print (table.getn(test1)) -- print the number of matches

reg = Region(10, 10, 300, 500)
test2 = listToTable(reg:findAll("target.png"))

for i, m in ipairs(test2) do
   click(m)
end

test3 = regionFindAll(reg, "target.png")

for i, m in ipairs(test3) do
   click(m)
end


wait([PS][, seconds]): Keeps searching the given pattern in the region until the image appears ( would have been found with Region.find()) or the specified amount of time has elapsed.

exists(PS[, seconds]): Check whether the give pattern is visible on the screen.

waitVanish(PS[, seconds]): This method keeps searching the given pattern in the region until the image vanishes (can not be found with Region.find() any longer) or the specified amount of time has elapsed.

dragDrop(PSMRL, PSMRL): Perform a drag-and-drop operation from a starting click point to the target click point indicated by the two PSMRLs respectively.
The behavior of dragDrop could be modified by following methods
setDragDropTiming(downMs, upMs)
downMs: Press the drag position for downMs milliseconds before moving.
upMs: Hold on drop position for upMs milliseconds before releasing the touch.
setDragDropStepCount(stepCount)
stepCount: The number of steps moving from drag to drop positions.
setDragDropStepInterval(intervalMs)
intervalMs: The interval in millisecond between steps.

In summary, the behavior of dragDrop is

press the drag position for downMs milliseconds
move to drop position in stepCount steps
wait intervalMs between steps
keep holding on drop position for upMs millisecods


type([PSMRL, ]text): Type the text at the current focused input field or at a click point specified by PSMRL.

All above methods could be called directly or from Region objects.
If called directly, the entire screen is searched for the specified PS.
Otherwise called within Region, only the specified region is searched.

---

The following methods could only be called from Region and Match objects.

highlight([text, ][seconds]): Highlight the region for some period of time.
If no text parameter, the highlight is without text. The highlight style could be set with method setHighlightStyle (color, fill). Here the color is in the format of ARGB integer. fill is the boolean about the fill of the region or not.

If the text parameter given, there is text inside the highlight.
And the lightlisht style could be set with setHighlightTextStyle (bgColor, textColor, textSize)。

bgColor (default 0xf8666666) is the background color. textColor (default 0xf8ffffff) is the text color. textSize (default 16)is the text siez.


If there is no second, this method behaves like a switch. The first call turn on the highlight. And the second turn it off.


highlightOff(): turn off the highlight

Don't know the status of highlight with highlight() switch. You can use this to make sure it's off.


highlightUpdate(text): update the highlight text


Example:
Settings:setScriptDimension(true, 960)
right = Region(400, 0, 560, 640)
left = Region(0,0, 400, 640)
-- ==========  main program ===========
left:highlight("Hello!") -- Text highlight with "Hello!". The background color is gray. And the text color is white.
right:highlight()
click("6.png")
click("+.png")
click("9.png")
setHighlightStyle(0x8fffff00, true)
getLastMatch():highlight(3)
right:highlight()
left:highlightOff()

The result of above sample code is 


For those methods generating exception, some images are saved in /sdcard/AnkuLua/captured_images for debugging.
findException.png: the resized captured screenshot
regionException.png
: the region image in the resized screenshot if region searching used.

---

Other Methods (not in Sikuli)

AnkuLua provides following methods for convenience. Those are not provided in Sikuli.

waitClick(PSMRL [, seconds]): Keeps searching the given pattern in the screen until the image appears or the specified amount of time has elapsed. If the pattern found, click on the target. If not, exception is generated.
A Match object that contains the best match will be returned.


existsClick(PSMRL [, seconds]): Keeps searching the given pattern in the screen until the image appears or the specified amount of time has elapsed. If the pattern found, click on the target. No exception is generated even the pattern not found.
The return value is boolean, true if pattern found.

doubleClick(PSMRL)
The method name says everything. It could be called directly or from Region object.

findAllNoFindException(PS)
It's the same as findAll() except that no exception will be thrown.

regionFindAllNoFindException(region, PS)
It's the same as regionFindAll() except that no exception will be thrown.

keyevent(keycode)
Send special key, like home, back.
Here is the corresponding key definitions from google.

keyevent(4) -- back
keyevent(3) -- home
wait(2)
keyevent(187) -- app_switch
swipe(PSMRL, PSMRL [, swiping seconds]) 
Perform a swipe operation from a starting click point to the target click point indicated by the two PSMRLs respectively.
The behavior of swipe is similar to dragDrop. But the timing is not controllerable.
Some devices have problem on dragDrop. If you just want to swipe the screen, use swipe for better compatibility.
If parameter "swiping seconds" provide, the script can control the speed of swipe.

touchDown(PSMRL [,seconds]), touchMove(PSMRL [,seconds]), touchUp(PSMRL [,seconds])
These three methods provides the most basic touch capabilities.

manualTouch(actionTable)
Because calling low level command each time, the speeds of touchDown(), touchMove() and touchUp() are low.
In normal cases, we know the whole moving path in advance. We can use manualTouch() to improve the speed a lot. Because only one low level command is called.

The only parameter actionTable is a table consisting of all actions. Every action is a table with action and target. Here action is one of "touchDown", "touchMove", "touchUp" and "wait". For "wait", the target is seconds. For others, it's Location.
It's clearer to see an example.

-- ========== Settings ================
Settings:setCompareDimension(true, 1280)
Settings:setScriptDimension(true, 1280)
-- ==========  main program ===========
p1 = Location(100, 300)
p2 = Location(400, 300)
p3 = Location(400, 600)
p31 = Location(401, 601)

setManualTouchParameter(5, 1) -- see below for explanation

actionList = { {action = "touchDown", target = p1},
   {action = "wait", target = 0.2},
   {action = "touchMove", target = p2},
   {action = "touchMove", target = p3},
   {action = "touchUp", target = p31} }

manualTouch(actionList)


setManualTouchParameter(int maxDistance, int insertWaitTimeMs)

If moving too far each time, the game's response may not be correct. We can use manualTouch() to set the behavior of touchMove in manualTouch().
maxDistance: The maximum moving distance. If the touchMove distance is larger than this value, multiple moving points with maxDistance distance will be inserted automatically. The default value is 5.
insertWaitTimeMs: When needing to insert moving points, one insertWaitTime milliseconds are also inserted before moving. The default value is 1.
With tuning of parameters, we can reach balance of speed and correctness.

---

General Methods (not related with searching and clicking)

save(fileName)
saveColor(fileName)

save() and saveColor() must be called from Region or Match objects. There is no return value.
It will save the image to fileName under imagePath.
save() saves the image in gray-scale.
saveColor() save the image in color format.
Here is one example about calculator plus.
match = waitClick("6.png",2)
match:save("save6.png") -- save the region to save6.png
print(getLastMatch())
click("+.png")
click("9.png")
click("-.png")
click("7.png")
click("save6.png") -- We can now click the saved image
click("=.png")

scriptPath()
scriptPath() method return the path of the script.
Users can then use the path in dofile or require.

local dir = scriptPath()
print (dir)
package.path = package.path .. ";" .. dir .. '?.lua'
print(package.path)
require(somelib)
dofile(dir .. "script.lua")
getLanguage()
Return the language setting in Android system
It could be used to support multi-language

function fileExists(name)
   local f=io.open(name,"r")
   if f~=nil then io.close(f) return true else return false end
end

function loadStrings(path)
   local language = getLanguage()
   local file = path.."strings."..getLanguage()..".lua";
   if (fileExists(file)) then
       dofile(file)
   else
       if (fileExists(path.."strings.lua")) then
           dofile(path.."strings.lua")
       end
   end
end


getCountry()
Return the country setting in Android system


getVersion()
Return the string of AnkuLua version


getAndroidVersion()
Return the number of Android API level.


getDeviceID(), getIMEI(), getSIMSerial(), getIMSI(), getWiFiMAC(), getMacAddr()
All these functions return the corresponding IDs.
They could be used in scripts for sale.

Note: Because WiFiMAC always returns 02:00:00:00:00:00 above Android version 6.0. You can use getMacAddr() instead. But the returned values for these two methods are different.

getUserID()
This function returns the user account email in Pro2.
It returns nil in trial version.

setClipboard(String string)
Modify the clipboard's string.
Under root or daemon methods, with Android 7.0 or above, you can input UTF string with setClipboard() and keyevent(279).
setClipboard("Olá Mundo\nEngligh")
keyevent(279)
Under accessibility mode, type() supports UTF.

typeOf()
To judge the type of an variable, lua provides the method "type()". However, Sikuli also use "type()" for inputting text.
Therefore, to judge variable's type, please use typeOf() instead.

You can get back lua's original type().
inputText = type
type = typeOf

[Next]: User Interface Methods
[Previous]: Settings
[Home]: AnkuLua Introduction