Sikuli GUI Automation Testing Tool – Beginner’s Guide Part #2

The most practical tutorials on Sikuli GUI automation testing tool:

In part-1 of this “introduction to Sikuli tutorial series”, we have discussed about Sikuli, how it works, and how to create a simple Sikuli project.

In this 2nd part, you are going to learn some advanced concepts like – how to create Sikuli maven project and how Sikuli can be used with Selenium WebDriver to automate web pages.

This part is essential because there is no built-in method to open a website/web application in Sikuli. So whenever you’re executing a Sikuli script, you need to make sure that the website is already opened in a browser. It is an overhead task.

To overcome this, Sikuli can be used together with Selenium WebDriver. Selenium contains a method to open a website in a browser. After opening the website, we are able to run Sikuli scripts on that website.

Sikuli GUI Automation tool 2

What is covered in this Sikuli Tutorial#2:

  • Installing Eclipse Maven Plugin
  • Installing Apache Maven
  • Install Sikuli Script Jar in Maven Repository
  • Creating Sikuli Maven Project
  • Example program: Open a file in Windows Explorer
  • Executing Sikuli Maven Project from Command line
  • Selenium vs Sikuli
  • Integrating Sikuli With Selenium WebDriver

Installing Eclipse Maven Plugin

Step #1:
Open Eclipse, Go to Help   -> Install a new Software. Click on “Add” button and add the following URL.

sikuili10.
Click on “OK” button.

Step #2:
Check All the checkboxes listed, click “Next” and install the maven plugin.

(Click on image to enlarge)

sikuili11

Installing Apache Maven

Step #1:
Download latest version of maven from here.

Step #2:
Extract the downloaded zip file and put it under somewhere in your machine.
Copy the bin folder path of Maven, and append the path in the environment variable.
(It requires JAVA_HOME variable in the environment variable. Please set JAVA_HOME variable in your environment)

Step #3:
Check whether maven installed correctly, Open command prompt and type “mvn -version”. It should return something like this,

(Click on image to enlarge)

sikuli12

It indicates Maven successfully installed on your machine.

Install Sikuli Script Jar in Maven Repository

As I mentioned in part -1, we’ve already got sikuli-script.jar, next we need to install sikuli-script.jar in the maven repository.
By using the following command we can install sikuli-script.jar in the maven repository.

Mvn install: install-file -Dfile=D:\Jars\Sikuli-r930\win32\Sikuli-IDE\sikuli-script.jar -DgroupId=com.test.sikuli -DartifactId=sikuli -Dversion-1.0.1 -Dpackaging=jar

(Click on image to enlarge)

sikuli13

Creating Sikuli Maven Project

Step #1:
Open Eclipse and create new Maven Project.

Step #2:
Add the following dependencies in your POM file.

<dependency>
<groupId>com.test.sikuli</groupId>
<artifactId>sikuli</artifactId>
<version>1.0.1</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
</dependency>

(Click on image to enlarge)

sikuli14


Step #3:
Create a package inside src/test/java and Create a class inside the package. Now you can start writing the Sikuli script inside this class.

Sikuli Example Program: Open a file in Windows Explorer

Step #1:
Create a Sikuli Maven Project, as explained above.

Step #2:
Take the screenshot of required elements and put it inside the Maven project.

sikuli15

– file.png

Step #3:
Create a class with name “Test1”, and Paste the following code inside the sikuli class.

package com.test;

import org.junit.Test;
import org.sikuli.script.FindFailed;
import org.sikuli.script.Screen;

public class Test1 {

@Test
public  void openFileTest() throws FindFailed, InterruptedException {
// TODO Auto-generated method stub
Screen s=new Screen();
s.find("file.png");
s.doubleClick("file.png");
System.out.println("File icon clicked");

}
} 

Executing Sikuli Maven Project from Command line

Step #1:
Open Command Prompt and cd to the project directory.

Step #2:
Execute the above project from a command prompt using the following command.

mvn clean test -Dtest=Test1

sikuli16

Selenium Vs Sikuli

sikuli17

Integrating Sikuli With Selenium WebDriver

Step #1:
Create a new Java Project in eclipse by clicking New -> Java project.

Step #2:

  1. Right-click on the Project Go to Build Path   -> Configure Build Path.
  2. Switch to Libraries Tab.
  3. Click on “Add External Jars” and Add Selenium library jars as well as Sikuli-scritp.jar

sikuli18

Step #3:
Create a package inside src/ folder and create a class under that package.

Step #4:
Take All required screenshot of web elements and save inside the project.

Step #5:
Copy the following code inside that class.

package com.test;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.sikuli.script.FindFailed;
import org.sikuli.script.Screen;

public class OnlinePainting {

public static void main(String[] args) throws FindFailed {
// TODO Auto-generated method stub

WebDriver driver=new FirefoxDriver();
WebDriverWait wait=new WebDriverWait(driver,20);
driver.manage().window().maximize();
driver.get("http://www.thecolor.com/Coloring/a-puppy-with-a-kitten.aspx");
Screen screen=new Screen();
screen.wait("1398665726055.png", 20);
screen.click("1398666382715.png");
screen.click("1398666248846.png");
screen.click("1398666729252.png");
screen.click("1398666188894.png");
screen.click("1398665763634.png");
screen.click("1398666592027.png");
screen.click("1398666610951.png");
screen.click("1398666308624.png");
screen.click("1398666326406.png");
screen.click("1398666570749.png");
screen.click("1398666703708.png");
screen.click("1398666382715.png");
screen.click("1398666857321.png");
screen.waitVanish("1398665763634.png");

}
} 

Step #6:
Right-click on the project, Select RunAs -> Java Application.

Before Execution:

sikuli19
After Execution:

sikuli20

Conclusion

  • Sikuli scripts can be easily integrated with Selenium WebDriver to automate flash websites.
  • Sikuli can automate windows as well as all other applications.
  • As it uses a Visual match, we can automate almost anything, we see on the screen.
  • It provides extensive support to Flash objects. i.e. we can automate adobe flash player components. (Audio player, video player)
  • Sikuli scripts can be created as maven project and can be run from command prompt.
  • Hence, Sikuli is most friendly, automation tool to automate challenging flash/windows applications.

With this, we are concluding our Sikuli tutorial series. Feel free to post your queries in comments.

 

 




Recommended reading

25 comments ↓

#1 Charles

Thanks for posting 2nd part of Sikuli GUI automation tool and explaining how to use it with Selenium.

#2 Katie Surman

we at our team using this tool for more than a year now. it’s good except few limitations.

#3 ayesha

Awesome man , hope you would become father of tetsing

#4 Henk Noppe

Our testers are using this at a daily basis. It really is a greate test tool! The only problem we run up is to let Jenkins kick start the Sikuli test suite. Any one knows a solution for this?

#5 Micky

Thanks for sharing your experiences. Can we automate captcha with Sikuli..?

#6 Anitha

@micky – yes .. we can automate captcha. But you should not reload the page. Otherwise captcha image will be changed.

#7 Surbhi

Hi STH team
Thanks for sharing such nice posts. These are really helpful in my testing career as I leaned lot of things from this blog. plz share selenium articles as well.

#8 Vinh

You can start a website by using app.open() to open the browser you want to test and type in the URL. It’s important to remember that Sikuli supports typing in keys. So you can do a lot of stuff with shortcut keys. As for using sikuli and Jenkins. Try this site http://technicaltesting.wordpress.com/2012/05/07/setting-up-a-jenkins-slave-for-sikuli-based-tests/.

#9 Hal

We use Sikuli extensively to automate our mobile websites via iOS Simulator and GenyMotion Android emulators. It is absolutely a fantastic tool, an automated and speedy black box tester.

It is an unusually powerful tool when scripted in it’s native Jython – but just the API with Java alone makes it so clunky and complicated just to click something from what I see here.

For instance, the ‘find’ operation is inherent and implied with ‘click’ in Jython, so you just click something. You don’t import several things, explicitly find the image, then explicitly click the image, etc.

#10 Sudheendran

How to identify an correct image when the images are identical. Example: We have the search icon more than once in a web page.

Please help me.

#11 Tanya

How to handle the IE10 Download popup using Sikuli?

#12 suraj

Well explained really helpful article to learn and install sikuli for testing purpose.

#13 AkuLua App

If intereseted, now you can run sikuli directly on non-rooted Android devices.
https://play.google.com/store/apps/details?id=com.appautomatic.ankulua.trial

#14 bairi

unable to use images when i exported the jar file but in eclipse it is working fine.

#15 Kvsc

Thanks for such a useful information.
Our testers are using this at a daily basis. It really is a useful testing tool.
Regards,
Kvsc

#16 AVS

Can i get a tutorial to automate testing for a installation wizard.(i mean a wizard which shows offers to users)

#17 ajit

what is dependcy for sikuli to be used in pom.xml file so that it can be used by all the member in the team?

#18 Krishna

Hi,
Thank you very much for tutorial. I have one question, can we use Siluli for Kibana kind of web sites? Where we need to deal with more graphs and its validations. Please provide some insight.

#19 vijaya

Can you share the images that you screenshot in above scripts?

#20 geetha

Can we use sikuli without triggering mouse event, because, each time I need to wait for the action to complete before I take the control of the mouse. If I have 10 test cases, and each test case, if I use sikuli, then I need to wait until the actions are completed. Anyother way to handle this.
Thanks
Geetha

#21 Deepak Mahapatra

Hi sir ,Thanks for the tutorial . Can I click on wordpad and enter some text in it .If so Please provide me the codes please.

#22 Sher Hassan

Hi Vijay ,

Please write a complete series of tutorials about using Sikuli with Katalon Studio. I’ve one question “How can we select a checkbox out of multiple checkboxes” using Sikuli.

Regards,

Sher Hassan

#23 Subbrao

Please mail (subbrao.spi@gmail.com) me Sikuli Doc either in pdf or word doc

#24 inderpreet singh

we can open the browser through sikuli.Follow the below steps:

Step1. select the browser image and use doubleClick function.
Step2: select the address bar and type the url on it and enter using type()

#25 Azhar

How to get dynamic text from image using sikuli and java?

Leave a Comment