User Interface Automation Framework

The Microsoft UI Automation APIs enable navigation of user interfaces programmatically.

Some of the areas of application are UI Automation testing,  manufacturing Quality checks, Accessibility,

The UIAFramework  is a light weight framework that can be used to achieve UI Automation.either by Scripting languages or C#. It has two components ControlDBTool and UIADriver. The UIAFramework is based on .net CLR 3.5 and works with Windows XP and above. The process involves two steps.

1. Identify controls participating in UI automation in an user interface and generate a control database using ControlDBTool.
2. Using the UIADriver component, manipulate the control by invoking a method  or property from the supported control patterns of the control. For e.g., Invoke method of Button control.

ControlDBTool

ControlDBTool is a GUI based tool that aids identification of UI elements that participates in UI automation. The users can basically select UI elements from an application and save them into a file based repository. ControlDBTool also supports record and play feature where code can be generated from user actions.

UIADriver

UIADriver is an easy to use managed class library that provides an interface to simulate user interactions on various UI elements such as Buttons, Grids, combo boxes etc. This is exposed to script based and unmanaged clients via COM.
UIADriver supports both UIA (user interface automation) and MSAA (Microsoft Accessibility Api) technologies. These basically provide the foundation for UI automation and are part of the OS. UIA is the successor of MSAA and can be used for both WPF and Win32/Winform based UI controls.  However, it exhibits anomalies with certain winform based controls. In such cases, MSAA can be used to fill the gap.
The code generator of the ControlDBTool discussed earlier uses the UIADriver library to perform UI operations.

Example

In this example, calucalator.exe is launched and  multiplication calculations are performed as discussed below.

1. Launch calucalator 

2. Launch ControlDBtool and set toolbar only mode

.

3. Click on record button and perform following actions on the calculator using mouse

4. click digits 8 7 8 4 x 1 6 x 1 0 4 0 as shown in the below

5. stop recording the controldb tool 

6. save the recording

7. recorded actions are listed as below. if there is any unmapped event, it needs to be mapped to the action.

8. set  controldb tool to regular mode

9. save the recording

10. click on generate code and save the selection

11. select all the actions

12. select C# code from the dropdown

13. Save the code to calc.cs

14. edit calc.cs file and add highlighted code in main() and save.

	   
public static void Main()
{
	string brs = null;
	System.Diagnostics.Process.Start("calc.exe");
	System.Threading.Thread.Sleep(2000);

	UIAAutomationElement.UIADriver.SetAutomationElement (objsample.button_Clear_clearButton_1294_432);15. open a command window from the saved location

16. compile the code to generate exe using following command

17. execute runcalc.exe. the playback looks as in the video below

Source and Binaries can be found here.