How to do GUI testing and Demo playback?

[thomas10001]

Is there any class or functions to do Demo playbacks or GUI testing?

 

What I need to do is to interact with another software's (a compiled exe) userinterface!

 

From my program I need to be able to move and perform clicks in the other software and to perform keyboard input, ie to copy data from a text field in the other software's userinterface in to my software for processing.

 

I saw that sun java has a robot class is there any similar things in .NET?

 

Thomas

[thomas10001]

I found how to move the mouse and do keyboard input. But not how to emulate a mouse click. Anyone who knows?

 

Is there any way to perform a mouse click using the keyboard? That would solve my problem.

 

 

// to move mouse

System.Windows.Forms.Cursor.Position = new System.Drawing.Point(95,105);

// to do keyboard input

System.Windows.Forms.SendKeys.Send("^a"); // Ctrl-a

[DiverDan]

There used to be a freeware program designed for this...I think it was called Ghost Surf.

[thomas10001]

I found a system function sendinput which is to simulate user interaction as keyboard input, mouse and hardware input. After lot of search on internet I found a bit of code which I added definitions from winuser.h and made a class Robot. It works for mouseinput as move and click etc.

 

But I tried to add the struct for keybdinput. But I can't get it to work. Anyone got a clue what's wrong?

 

Even the System.Windows.Forms.SendKeys.Send(string) doesn't work for me

 

Here is the Robot class

 

(The formatting got a bit unorganized when I pasted it)

------------------------------------------------------------------------

using System;

using System.Runtime.InteropServices;

 

namespace HoldemAdvice

{

/// <summary>

/// Summary description for Robot.

/// </summary>

public class Robot

{

public static int MOUSEEVENTF_MOVE = 0x0001; /* mouse move */

public static int MOUSEEVENTF_LEFTDOWN = 0x0002; /* left button down */

public static int MOUSEEVENTF_LEFTUP = 0x0004; /* left button up */

public static int MOUSEEVENTF_RIGHTDOWN = 0x0008; /* right button down */

public static int MOUSEEVENTF_RIGHTUP = 0x0010; /* right button up */

public static int MOUSEEVENTF_MIDDLEDOWN = 0x0020; /* middle button down */

public static int MOUSEEVENTF_MIDDLEUP = 0x0040; /* middle button up */

public static int MOUSEEVENTF_XDOWN = 0x0080; /* x button down */

public static int MOUSEEVENTF_XUP = 0x0100; /* x button down */

public static int MOUSEEVENTF_WHEEL = 0x0800; /* wheel button rolled */

public static int MOUSEEVENTF_VIRTUALDESK = 0x4000; /* map to entire virtual desktop */

public static int MOUSEEVENTF_ABSOLUTE = 0x8000; /* absolute move */

 

public static int KEYEVENTF_EXTENDEDKEY = 0x0001;

public static int KEYEVENTF_KEYUP = 0x0002;

public static int KEYEVENTF_UNICODE = 0x0004;

public static int KEYEVENTF_SCANCODE = 0x0008;

 

public static int INPUT_MOUSE = 0;

public static int INPUT_KEYBOARD = 1;

public static int INPUT_HARDWARE = 2;

 

public Robot()

{

}

 

[DllImport("User32.dll", SetLastError=true)]

public static extern int SendInput(int nInputs, ref INPUT pInputs, int cbSize);

 

[DllImport("User32.dll", SetLastError=true)]

public static extern int SendInput(int nInputs, ref KEYBOARD pInputs, int cbSize);

 

 

public struct INPUT

{

public int type;

public MOUSEINPUT mi;

//public KEYBDINPUT ki; // doesn't work when this is declared here

}

 

// making another struct to use when keyboard inputs

public struct KEYBOARD

{

public int type;

public KEYBDINPUT ki;

}

 

 

public struct MOUSEINPUT

{

public int dx;

public int dy;

public int mouseData;

public int dwFlags;

public int time;

public int dwExtraInfo;

}

 

 

public struct KEYBDINPUT

{

public int wVK;

public int wScan;

public int dwFlags;

public int time;

public int dwExtraInfo;

}

 

 

public int sendinput(ref INPUT input)

{

// Call the API

int resSendInput;

resSendInput = SendInput(1, ref input, Marshal.SizeOf(input));

if (resSendInput == 0 || Marshal.GetLastWin32Error() != 0)

System.Diagnostics.Debug.WriteLine(Marshal.GetLastWin32Error());

return resSendInput;

}

 

public int sendinput(ref KEYBOARD input)

{

// Call the API

int resSendInput;

resSendInput = SendInput(1, ref input, Marshal.SizeOf(input));

if (resSendInput == 0 || Marshal.GetLastWin32Error() != 0)

System.Diagnostics.Debug.WriteLine(Marshal.GetLastWin32Error());

 

return resSendInput;

}

 

}

}

 

 

----------------------------------------------------------------------

 

Here is my calls to Robot

 

Robot robot = new Robot();

 

// move the mouse to (50,445) (where I have a textfield for testing)

Robot.INPUT input = new Robot.INPUT();

input.type = Robot.INPUT_MOUSE;

input.mi.dx = 50-System.Windows.Forms.Cursor.Position.X;// relative movements!

input.mi.dy = 445-System.Windows.Forms.Cursor.Position.Y;

input.mi.dwFlags = Robot.MOUSEEVENTF_MOVE;

robot.sendinput(ref input);

 

// make a left mouse down

input.mi.dwFlags = Robot.MOUSEEVENTF_LEFTDOWN;

robot.sendinput(ref input);

 

// make a left mouse up

input.mi.dwFlags = Robot.MOUSEEVENTF_LEFTUP;

robot.sendinput(ref input);

// now my text field got the focus

 

// try to enter a (65) in the text field. But no result.

Robot.KEYBOARD input2 = new Robot.KEYBOARD();

input2.type = Robot.INPUT_KEYBOARD;

//input2.ki.wVK = 65;// have tried both wVK and wScan but no result. What

input2.ki.wScan = 65;// can be wrong?

input2.ki.dwFlags = Robot.KEYEVENTF_SCANCODE;

robot.sendinput(ref input2);

[Denaes]

When I did this application, I did it Via SendKeys.

 

I posted the entire code somewere on this site.

 

I would open a file, search for the process, make sure it's on top, and use tab and space to do things. tab to get between controls. Send text into textboxes and space to click a button.

 

It got the program done - which was automating backups in the middle of the night.

 

Works better than moving a mouse because if someone moves the window you want to manipulate - the mouse will miss it's mark because it's based on coordinates. The only way I can see around missing cordinates is going into window handles and basing it off of the windows coordinates... seems like way too much trouble

 

I guess, just in principle it's nice to know how to move the mouse and click it.

 

--edit--

 

Link to post that gets the job done. http://www.xtremedotnettalk.com/showthread.php?t=76773

 

I'm thinking of altering it to read from file... maybe even open up a "macrorecorder" and record what you do. Can a .Net program view SendKeys if another program is active (ie your app is in the background and you're using another app)

Edited December 7, 2004 by Denaes