Tuesday, September 11, 2012

Automate Applet window by Selenium 2 and Sikuli


Handling Web Applet is a big headache while creating automation script with Selenium. Other windows can be handled by their Instance or Class but Applet window does not show any different Instance or Class for its content. Suppose, if we want to click a upload button in an Applet window, we won't get any separate Instance or Class for this Upload button. If we try any window information tool (e.g. AutoIt window info) and point to the Upload button, it will show the Instance/Class of the whole Applet window not the information of the button. So far, I have found solutions by pointing the position of the button in the script. But it will be a huge problem if the resolution of the window changes. Moreover, position based clicking sometime shows strange result.

To solve this problem, I have combined 2 widely used library - Selenium 2 and Sikuli, where the Sikuli library will handle the Applet window. Hope you already have a little idea on Eclips, Selenium 2 and Sikuli. This script is written in Java. Before you start, make sure you have Java installed.

1. At first Create a new project in Eclips. Name of my project is "testApplet"



2. Download Selenium 2, Download and install Sikuli

3. Right click on project name in "Package Explorer" , hover on New and then add a new folder. Name the folder "lib" (You can use any name for the folder, this folder will contain the jar files of Selenium 2 and Sikuli). Now go to the workspace of Eclips and search for the directory with project name (e.g. testApplet), open this directory. Search for the "lib" directory, copy "selenium-server-standalone-2.x.jar" and "sikuli-script.jar" to this directory. You will find "sikuli-script.jar" in this location - "C:\Program Files\Sikuli X". Now Right click on Project name, Build Path>Configure Build Path>Add External JARs and select the "jar" files in the "lib" directory.Library adding process can be simply done by Adding External Jars in projects Build Path without copying the library to workspace directory, but I always prefer above process to make the project independent.



4. Now Refresh the Project Explorer, you will see two jar files will be added into the "lib" folder, and there will be 2 "Referenced Libraries"

5. Add a new java file. Right click on source, New>File, Name the file "Main.java" or anything you want.

6. Add another folder in Project Explorer named "img". Go to http://www.java.com/en/download/testjava.jsp and capture the screenshot of the link "Sun Microsystems.Inc". Save the image on "img" directory of the workspace. Refresh the Explorer window and you will get a view like this:




7. Now we will try to click on "Sun Microsystems.Inc" link in the Applet window. Paste the following code on "Main.java" -

import org.openqa.selenium.firefox.*;
import org.openqa.selenium.WebDriver;
import org.sikuli.script.Screen;

public class Main
{
  public static void main(String[] args)
  {
    String sUrl = "http://www.java.com/en/download/testjava.jsp";

    // Instantiate the Internet Explorer browser.
    WebDriver mydriver = new FirefoxDriver();
          
    // Open the main webpage.
    mydriver.get(sUrl);
    Screen s = new Screen();
    //Click on the content of Applet;
    try{s.click("img/appWindow.png",0);
    }
    catch(Exception e){
        e.printStackTrace();
    }   
  }
}
8. Run the Project. Check the mouse movement over "Sun Microsystems.Inc" and the magic click will navigate you to the http://java.com/en/ web site. If you having problem with getting the Applet window in http://www.java.com/en/download/testjava.jsp, you need to update your Java or need to click on "test the currently installed version of Java" link by Selenium command (hope it is the most easy job for you :)).

9. That's it, you have clicked a link in the Applet window. Now you can automate anything inside the Applet window. Enjoy automated testing!

6 comments:

  1. hi i compiled and tried to run your example but facing some issues.
    Can you help me resolve this problem.

    I can open the site in firefox using your code but that does not click the image.

    ReplyDelete
    Replies
    1. Hello Nipun, thank you very much for your patience. Can you tell me what kind of issues are you facing?

      Delete
    2. Have you added the Sikuli library correctly? Can you please your image location again?

      Delete
  2. How to resolve this error? can u please help?

    -------------------------------------------------------------
    Exception in thread "main" java.lang.UnsatisfiedLinkError: D:\Users\cakula\AppData\Local\Temp\tmplib\Win32Util.dll: Can't load IA 32-bit .dll on a AMD 64-bit platform
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(Unknown Source)
    at java.lang.ClassLoader.loadLibrary(Unknown Source)
    at java.lang.Runtime.load0(Unknown Source)
    at java.lang.System.load(Unknown Source)
    at com.wapmx.nativeutils.jniloader.NativeLoader.loadLibrary(NativeLoader.java:44)
    at org.sikuli.script.Win32Util.(Win32Util.java:19)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at org.sikuli.script.Env.getOSUtil(Env.java:91)
    at org.sikuli.script.ScreenHighlighter.init(ScreenHighlighter.java:180)
    at org.sikuli.script.ScreenHighlighter.(ScreenHighlighter.java:293)
    at org.sikuli.script.Screen.initBounds(Screen.java:105)
    at org.sikuli.script.Screen.(Screen.java:117)
    at Main.main(Main.java:16)

    ReplyDelete
    Replies
    1. Hello Chiranjivi,
      Can you please check your check your Sikuli and Java? I think your machine is 64bit and you have installed 32bit Sikuli/Java.
      Thanks!

      Delete