Sikuli GUI Automation Testing Tool – Beginner’s Guide Part #2

The most practical tutorials on Sikuli GUI automation testing tool:

In part-1 of this “introduction to Sikuli tutorial series”, we have discussed Sikuli, how it works, and how to create a simple Sikuli project.

In this 2nd part, you are going to learn some advanced concepts like – how to create Sikuli Maven project and how Sikuli can be used with Selenium WebDriver to automate web pages.

This part is essential because there is no built-in method to open a website/web application in Sikuli. So whenever you’re executing a Sikuli script, you need to make sure that the website is already opened in a browser. It is an overhead task.

Sikuli GUI Automation Testing Tool

To overcome this, Sikuli can be used together with Selenium WebDriver. Selenium contains a method to open a website in a browser. After opening the website, we are able to run Sikuli scripts on that website.

Create A Sikuli Maven Project

Installing Eclipse Maven Plugin

Step #1) Open Eclipse, Go to Help   -> Install a new Software. Click on the “Add” button and add the following URL.


Click on the “OK” button.

Step #2) Check All the checkboxes listed, click “Next” and install the Maven plugin.


Installing Apache Maven

Step #1) Download the latest version of the Maven from here.

Step #2) Extract the downloaded zip file and put it under somewhere in your machine.
Copy the bin folder path of Maven, and append the path in the environment variable.
(It requires a JAVA_HOME variable in the environment variable. Please set the JAVA_HOME variable in your environment)

Step #3) Check whether maven installed correctly, Open command prompt and type “mvn -version”. It should return something like this,


It indicates Maven successfully installed on your machine.

Install Sikuli Script Jar In Maven Repository

As I mentioned in part -1, we’ve already got sikuli-script.jar, next we need to install sikuli-script.jar in the maven repository.
By using the following command we can install sikuli-script.jar in the maven repository.

Mvn install: install-file -Dfile=D:\Jars\Sikuli-r930\win32\Sikuli-IDE\sikuli-script.jar -DgroupId=com.test.sikuli -DartifactId=sikuli -Dversion-1.0.1 -Dpackaging=jar


Creating The Sikuli Maven Project

Step #1) Open Eclipse and create a new Maven Project.

Step #2) Add the following dependencies in your POM file.



Step #3) Create a package inside src/test/java and Create a class inside the package. Now you can start writing the Sikuli script inside this class.

Sikuli Example Program: Open A File In Windows Explorer

Step #1) Create a Sikuli Maven Project, as explained above.

Step #2) Take the screenshot of the required elements and put it inside the Maven project.


– file.png

Step #3) Create a class with name “Test1”, and Paste the following code inside the sikuli class.

package com.test;

import org.junit.Test;
import org.sikuli.script.FindFailed;
import org.sikuli.script.Screen;

public class Test1 {

public&nbsp; void openFileTest() throws FindFailed, InterruptedException {
// TODO Auto-generated method stub
Screen s=new Screen();
System.out.println("File icon clicked");


Executing The Sikuli Maven Project From Command Line

Step #1) Open Command Prompt and cd to the project directory.

Step #2) Execute the above project from a command prompt using the following command.

mvn clean test -Dtest=Test1


Selenium Vs Sikuli


Integrating Sikuli With Selenium WebDriver

Step #1) Create a new Java Project in eclipse by clicking New -> Java project.

Step #2) 

  1. Right-click on the Project Go to Build Path   -> Configure Build Path.
  2. Switch to Libraries Tab.
  3. Click on “Add External Jars” and Add Selenium library jars as well as Sikuli-scritp.jar


Step #3) Create a package inside src/ folder and create a class under that package.

Step #4) Take All required screenshot of web elements and save inside the project.

Step #5) Copy the following code inside that class.

package com.test;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.sikuli.script.FindFailed;
import org.sikuli.script.Screen;

public class OnlinePainting {

public static void main(String[] args) throws FindFailed {
// TODO Auto-generated method stub

WebDriver driver=new FirefoxDriver();
WebDriverWait wait=new WebDriverWait(driver,20);
Screen screen=new Screen();
screen.wait("1398665726055.png", 20);"1398666382715.png");"1398666248846.png");"1398666729252.png");"1398666188894.png");"1398665763634.png");"1398666592027.png");"1398666610951.png");"1398666308624.png");"1398666326406.png");"1398666570749.png");"1398666703708.png");"1398666382715.png");"1398666857321.png");


Step #6) Right-click on the project, Select RunAs -> Java Application.

Before Execution:

After Execution:



  • Sikuli scripts can be easily integrated with Selenium WebDriver to automate flash websites.
  • Sikuli can automate windows as well as all other applications.
  • As it uses a Visual match, we can automate almost anything, we see on the screen.
  • It provides extensive support to Flash objects. i.e. we can automate adobe flash player components. (Audio player, video player)
  • Sikuli scripts can be created as a Maven Project and can be run from command prompt.
  • Hence, Sikuli is most friendly, automation tool to automate challenging flash/windows applications.

Feel free to post your queries in comments.

=> Check ALL Sikuli Tutorials Here.

Recommended Reading

31 thoughts on “Sikuli GUI Automation Testing Tool – Beginner’s Guide Part #2”

  1. Our testers are using this at a daily basis. It really is a greate test tool! The only problem we run up is to let Jenkins kick start the Sikuli test suite. Any one knows a solution for this?

  2. Hi STH team
    Thanks for sharing such nice posts. These are really helpful in my testing career as I leaned lot of things from this blog. plz share selenium articles as well.

  3. We use Sikuli extensively to automate our mobile websites via iOS Simulator and GenyMotion Android emulators. It is absolutely a fantastic tool, an automated and speedy black box tester.

    It is an unusually powerful tool when scripted in it’s native Jython – but just the API with Java alone makes it so clunky and complicated just to click something from what I see here.

    For instance, the ‘find’ operation is inherent and implied with ‘click’ in Jython, so you just click something. You don’t import several things, explicitly find the image, then explicitly click the image, etc.

  4. How to identify an correct image when the images are identical. Example: We have the search icon more than once in a web page.

    Please help me.

  5. Hi,
    Thank you very much for tutorial. I have one question, can we use Siluli for Kibana kind of web sites? Where we need to deal with more graphs and its validations. Please provide some insight.

  6. Can we use sikuli without triggering mouse event, because, each time I need to wait for the action to complete before I take the control of the mouse. If I have 10 test cases, and each test case, if I use sikuli, then I need to wait until the actions are completed. Anyother way to handle this.

  7. Hi sir ,Thanks for the tutorial . Can I click on wordpad and enter some text in it .If so Please provide me the codes please.

  8. Hi Vijay ,

    Please write a complete series of tutorials about using Sikuli with Katalon Studio. I’ve one question “How can we select a checkbox out of multiple checkboxes” using Sikuli.


    Sher Hassan

  9. we can open the browser through sikuli.Follow the below steps:

    Step1. select the browser image and use doubleClick function.
    Step2: select the address bar and type the url on it and enter using type()

  10. Starting ChromeDriver 2.37.543627 (63642262d9fb93fb4ab52398be4286d844092a5e) on port 20028
    Only local connections are allowed.
    Mar 21, 2018 11:28:20 AM org.openqa.selenium.remote.ProtocolHandshake createSession
    INFO: Detected dialect: OSS
    *** classpath dump
    0: /C:/Program%20Files/Java/jre1.8.0_161/lib/resources.jar
    1: /C:/Program%20Files/Java/jre1.8.0_161/lib/rt.jar
    2: /C:/Program%20Files/Java/jre1.8.0_161/lib/jsse.jar
    3: /C:/Program%20Files/Java/jre1.8.0_161/lib/jce.jar
    4: /C:/Program%20Files/Java/jre1.8.0_161/lib/charsets.jar
    5: /C:/Program%20Files/Java/jre1.8.0_161/lib/jfr.jar
    6: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/access-bridge-64.jar
    7: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/cldrdata.jar
    8: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/dnsns.jar
    9: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/jaccess.jar
    10: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/jfxrt.jar
    11: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/localedata.jar
    12: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/nashorn.jar
    13: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/sunec.jar
    14: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/sunjce_provider.jar
    15: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/sunmscapi.jar
    16: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/sunpkcs11.jar
    17: /C:/Program%20Files/Java/jre1.8.0_161/lib/ext/zipfs.jar
    18: /G:/selen/chii/bin/
    19: /G:/selen/selenium-java-3.11.0/client-combined-3.11.0.jar
    20: /G:/selen/selenium-java-3.11.0/client-combined-3.11.0-sources.jar
    21: /G:/selen/selenium-java-3.11.0/libs/byte-buddy-1.7.9.jar
    22: /G:/selen/selenium-java-3.11.0/libs/commons-codec-1.10.jar
    23: /G:/selen/selenium-java-3.11.0/libs/commons-exec-1.3.jar
    24: /G:/selen/selenium-java-3.11.0/libs/commons-logging-1.2.jar
    25: /G:/selen/selenium-java-3.11.0/libs/gson-2.8.2.jar
    26: /G:/selen/selenium-java-3.11.0/libs/guava-23.6-jre.jar
    27: /G:/selen/selenium-java-3.11.0/libs/httpclient-4.5.3.jar
    28: /G:/selen/selenium-java-3.11.0/libs/httpcore-4.4.6.jar
    29: /G:/selen/selenium-java-3.11.0/libs/okhttp-3.9.1.jar
    30: /G:/selen/selenium-java-3.11.0/libs/okio-1.13.0.jar
    31: /G:/selen/sikulixsetup-1.1.0.jar
    *** classpath dump end
    [error] RunTimeINIT: *** terminating: libs to export not found on above classpath: /sikulixlibs/windows/libs64

    how can i resolve this error,plz send me the link of sikuli jar file which is suitable for my project versions

  11. Hi,I am using Sikuli in selenium with C#. I am facing an error as below:
    Sikuli4Net.sikuli_UTIL.SikuliActionException: ‘Result: FAILorg.sikuli.script.Pattern cannot be cast to java.lang.String’.
    Below is my code

    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Threading.Tasks;
    using OpenQA.Selenium;
    using OpenQA.Selenium.Support;
    using NUnit.Framework;
    using OpenQA.Selenium.Chrome;
    using Sikuli4Net.sikuli_JSON;
    using Sikuli4Net.sikuli_REST;
    using Sikuli4Net.sikuli_UTIL;

    namespace Chart1
    class Program
    static void Main(string[] args)
    APILauncher launcher = new APILauncher();
    IWebDriver driver = new ChromeDriver();
    Screen screen1 = new Screen();
    Pattern pattern1 = new Pattern(“C:/>TestBase/GoogleBase.PNG”);
    screen1.Click(pattern1, true); //FACING ERROR HERE

    Please help in getting this resolved.


Leave a Comment