A Simple Approach for Desktop Application UI Automation With PowerShell

Introduction to Microsoft UIAutomation:

UIAutomation is an open source project PowerShell extension for GUI automation tests. It supports Win32, Windows Form, WPF, and so on.

In this article, we’ll focus on how PowerShell uses the module to implement UI automation.  To explain this we will use Windows Form application as an example. 

Getting started with MS UIAutomation:

Desktop Application UI Automation With PowerShell

Since the project provides the only library for the PowerShell scripts, it does not require installation, we can just download the module and import it to the script.

Also read => 35+ Best GUI Testing Tools with Complete Details

To know more about PowerShell and to understand how to get started with it, check out:

Import UIAutomation:

1) Download and unzip the latest package from its official website: UI Automation PowerShell Extensions to local path, for example, C:\UIAutomation

2) Import the module into the PowerShell script: (Note: When you run this command, make sure you are not running PowerShell as an Administrator)

Import-Module C:\UIAutomation\UIAutomation.dll

Now we can use the commands defined in the module to start UI automation testing.

UI Automation with examples:

Let’s begin with a simple Windows form application, which covers some typical controls we may see in GUI automation testing.

Also read => GUI Testing on Smart Devices – Testing Guidelines

UI Automation

In this example, we are told to automate the steps to submit a form with some basic personal information:

Step #1) Input the name “Anna Smith”
Step #2) Select the gender “Female”
Step #3) Choose “Yes” as the answer of if graduated.
Step #4) Click “Submit”

The script to do it is like this:

Start-Process "Test Form"

#Step1: input the name “Anna Smith”
Get-UIAWindow -Name 'Test Form' | Get-UIATabItem -Name 'BasicInfo'| Get-UIAEdit -AutomationId 'textBox1' -Name 'Name'|Set-UIATextBoxText "Anna Smith"

#Step2: select the gender “Female”
Get-UIAWindow -Name 'Test Form' | Get-UIATabItem -Name 'BasicInfo'| Get-UIAComboBox -Name 'Gender' | Invoke-UIAListItemSelectItem -ItemName "Female"

#Step3: choose “Yes” as the answer of if graduated.
Get-UIAWindow -Name 'Test Form' | Get-UIATabItem -Name 'BasicInfo'| Get-UIARadioButton -AutomationId 'Graduated' -Name 'Yes'|Invoke-UIAControlClick

#Step4: click “Submit”
Get-UIAWindow -Name 'Test Form' | Get-UIAButton -Name 'Submit'|

You probably can tell from the above script, how it is manipulating the controls in the application. Let us understand further.

Take the first step as an example:

Get-UIAWindow -Name 'Test Form' | Get-UIATabItem -Name 'BasicInfo'| Get-UIAEdit -AutomationId 'textBox1' -Name 'Name'|Set-UIATextBoxText "Anna Smith"

Here is a breakdown on what is happening in this line:

1) The script first finds the top window and then finds its child control through the pipeline. It does this until it finds the target control- the edit box (or textbox) with the name “Name”.

2) On finding the control, it claims for the type of the control so that you know what command to use, e.g. Get-UIAWindow – is a window and Get-UIAEdit – is an edit/text box.

3) It also requires one or more properties of the control. In this example, we use only Name as the property to identify the control. Note: AutomationId and Class are also properties that are commonly used when finding a control.

4) After the target control is found, we use another command to invoke the action on the control. In this example, it is Set-UIATextBoxText to set the text for a text box.

This is the main way for UI Automation to realize UI automation.

You can find the commands for different types of controls and the command to invoke actions in the help documents of its official site: UI Automation PowerShell Extensions

The rest of the steps are similar in vein. So, let us now move on to other important topics.

Find the parent-child relationship and the properties of the control:

It is simple to write such pipelines, but the significant part is how to find the properties and the child controls containing the target control. In my experience, these are the two I have tried:

#1) Use UIAutomationSpy in the package:

There is a tool in the package we downloaded that can capture the controls on the desktop. When the UI is ready on the desktop, open UIAutomationSpy.exe:

(Note: Click on any image for enlarged view)

UIAutomationSpy


  • Click “Start” to start the capture.
  • Move the cursor to the control you what to capture.
  • When you find the control highlighted and script displayed on its window interface, it means the capture is successful.
  • Click “Stop” to stop the capture.
  • Click “Hierarchy” tab on the left and you’ll see the parent-child relationship of the controllers from top to the target

Click Hierarchy tab

  • Click “Script” tab on the right, you can see a full pipeline for the control:

Click script tab

Please note that we don’t have to include every control and every property from the pipeline to search for the target control. Actually, what we use is quite simple (see the example above):

Get-UIAWindow -Name 'Test Form' | Get-UIATabItem -Name 'BasicInfo'| Get-UIAEdit -AutomationId 'textBox1' -Name 'Name'

Unfortunately, no official silver bullet exists on how many controls should be included. It requires trying and experience.

What’s not so great about UIAutomationSpy:

  • It does not have a friendly interface
  • You’ll have to capture one control after another and get the details you want.

#2) Use external tools to inspect the controls for windows desktop

A suggested tool is inspect.exe provided by Microsoft. When you open the tool, you’ll get every detail of the controls on the window:

Windows inspect

What’s not so great about inspect.exe

  • There is no way to export the details.
  • The UI Automation module provides some commands such as Get-UIAControlChildren to get all the child controls. inspect.exe can transverse all the controls under the window, but the performance is not assessed for complex windows.

Advanced implementation: simplify the approach to writing the script

From the above introduction, you may see that pipelines are the main way to write the script using the module, although sometimes the pipelines are not so long. There is an approach to make it simpler and easier, and the main idea is to extract the data of the controls into an XML file, and join the pipelines when needed from the XML file.

Let’s continue with the above example:

First, generate an XML file including the properties and the parent-child relationship of the controls. We add nodes for every control required: the node name is unique so that we can find the control immediately by name, and the command and its properties are set as the properties of the nodes.

Here is the XML file for the Test Form we test:

<TestForm method="Get-UIAWindow" Name="Test Form" Seconds="5">
 <Basicinfo method="Get-UIATabItem" Name="BasicInfo">
 <Name method="Get-UIAEdit" Name="Name" AutomationId="textBox1" />
 <Gender method="Get-UIAComboBox" Name="Gender" >
 <Male method="Get-UIAListItem" Name="Male"/>
 <Female method="Get-UIAListItem" Name="Female"/>
 </Gender>
 <Graduated method="Get-UIARadioButton" Name="Yes" AutomationId="Graduated" />
 <NotGraduated method="Get-UIARadioButton" Name="No" AutomationId="Graduated" />
 </Basicinfo>
 <Submit method="Get-UIAButton" Name="Submit" />
</TestForm>

Some functions should be defined to join the pipelines by searching the xml file and execute them:

function getPipeline($nodeName)
{
$object_xml = (Invoke-WebRequest(controls.xml'))
$control = $object_xml.SelectSingleNode("//$nodeName ")
$pipeline = ""
do 
{
$string = ""
$string = $control.method
foreach($a in $control.Attributes)
{
 if(!$a.Name.ToLower().Equals("method"))
 {
 $string = $string + " -" + $a.Name + " '" +$a.Value +"'"
 }
}
$ pipeline = $string + " |" +$ pipeline
$control= $control.ParentNode
}
while (!$control.Name.Equals("#"))
return $pipeline.Remove($pipeline.length-1,1)
}

function setText($textbox,$value)
{
$pipeline = getPipeline $textbox
$iex = $pipeline + "| Set-UiaEditText -text $value"
Invoke-Expression $iex
}

function selectItem($combobox,$item)
{
#get the pipeline and invoke the expression
}

function toggleRadioButton($radioButton)
{
#get the pipeline and invoke the expression
}

function click($control)
{
#get the pipeline and invoke the expression
}

So far, these are almost one-time tasks unless the controls themselves change.

Now we can automate the steps in the following scripts:

#import the functions defined previously
. \MyLibrary.ps1
setText "Name" "Anna Smith"
selectItem "Gender" "Female"
toggleRadioButton "Graduated"
click "Submit"

In this way, we can get rid of bunches of pipelines in the scripts, and the proficiency will be improved greatly.

Recommended read => Introduction to Sikuli GUI Automation Tool

Conclusion:

Microsoft UIAutomation is a great library for those who want to have UI automation tests on Windows desktop with PowerShell.

The above introduction is just a segment of the tool. Its simplified approach is inspired by the idea of extracting data from testing scripts, which is a good way to improve our proficiency in writing automation tests. Explore more about it here.

Please share your comments below and we would love to hear experiences from UIAutomation users.

Thanks for reading!




Recommended reading

6 comments ↓

#1 Sudhir

good tutorial

#2 Nagireddy S

Nice Article… Need more articles on Automation

#3 rahul saxena

thanku for this to share a imporatnt information

#4 Virgil Glavan

can you help me please with this
Import-Module C:\UIAutomation\UIAutomation.dll

Import-Module : Could not load file or assembly ‘file:///C:\UIAutomation\UIAutomation.dll’ or one of its dependencies. Operation is not supported. (Exception from HRESULT: 0x80131515)
At line:1 char:1
+ Import-Module C:\UIAutomation\UIAutomation.dll
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Import-Module], FileLoadException
+ FullyQualifiedErrorId : System.IO.FileLoadException,Microsoft.PowerShell.Commands.ImportModuleCommand

#5 Anil Prakshale

This is very good article

#6 Krishna

How to read the content from text box?

Leave a Comment