Wonder if it is possible to look for part of a screen and click on it with Macro Express Pro 6

[hufi]

Hi Forum

I am trying to automate a process using Teams. And I wonder if it is possible to look on the screen to the teams icon and press on we find it or press on the ... to acess to the rest of the possibles choices and press on the team icon.

 

In fact, the teams icons can ve displayed on your left side of the screen but it can depends on the configuration of your team software

 

More generally, can we provide to Macro Express Pro 6 a picture so that the automate looks for this picture and if he finds it, he makes an action and if not, he makes another action

 

thanks for your support and help

 

regards

 

Hufi

[acantor]

Rather than looking for images and clicking on them, I think it will be easier to automate tasks in Teams by sending sequences of keystrokes. 

 

Many key sequences in Teams are standard; they work in almost all well-behaved applications. Other key sequences in Teams are truly bizarre, and hard to figure out. (For example, the only way to perform one task via keyboard is to press a key combination Microsoft stopped supporting around the time of Windows ME.)

 

Perhaps Microsoft has documented ways to interact with Teams via keyboard / no mouse. You could start there. My recollection is that the set of keys for performing most tasks in Team include:

 

F6

Ctrl + 1, Ctrl + 2, etc.

The four arrow keys

Tab and Shift + Tab

spacebar

Enter

 

Once you figure out the key sequence, you can do things like this in Macro Express. (Note that this is for illustration purposes only. The sequence won't do anything useful in Teams.)

 

Text Type (Simulate Keystrokes): <CONTROL>2
Text Type (Simulate Keystrokes): <TAB><TAB><TAB><TAB>
Text Type (Simulate Keystrokes): <ARROW DOWN><ARROW DOWN><ARROW DOWN>
Text Type (Simulate Keystrokes): <ENTER>

 

 

 

[hufi]

Thanks a lot Acantor for your answer.....In fact it is the right way to proceed.....

 

Anyway do you know if it is possible to look for a picture and click on it....In some case it should be the only solution

 

thanks 

 

Hufi

[rberq]

In the far past I tinkered with software that was supposed to look for pictures within a screen, and tried to call them from Macro Express.  I didn't have much luck, but I wasn't too serious about it, either.  There may be fancier products available now, that you could use. 

 

IF you can look for specific colors, and IF they are distinct from the general colors on the screen, and IF there is a limited number of screen areas where you have to look, then the macro command Get Pixel Color is useful.  For example, the code below scans vertically down the page looking for a data-entry field.  It works because I know approximately where the field will be, and field's color is different from everything around it.  But the method is too slow for scanning an entire screen, and likely will find many false positives outside of a narrow search area.

 

// Find the type-in area by color
Variable Set Integer %xc% to 300
Variable Set Integer %yc% to 100
Repeat Start (Repeat 150 times)
  Get Pixel Color at (%xc%, %yc%) Relative to Screen into %color%
  If Variable %color% Equals "4343115"
    Repeat Exit
  Else
    Variable Modify Integer: %yc% = %yc% + 8
  End If
End Repeat
If Variable %color% Equals "4343115"
Else
  Macro Return
End If
Variable Modify Integer: %yc% = %yc% + 15
Mouse Move: %xc%, %yc% Relative to Screen // Move mouse down a little from the top edge of the entry field
Mouse Left Double Click

 

[acantor]

Rberq's example may be the most elegant way, using Macro Express, to "hunt" for a pixel colour. But because pixel checking is handled mathematically, the approach can be challenging to debug if the script fails.

 

If you pixel check the position of the mouse cursor, rather than its x, y coordinates, it's easier to spot the problem when the script doesn't work.

 

This pixel hunting method runs noticeably slower than Rberq's method.

 

Mouse Move: 800, 240 Relative to Current Window
Repeat Start (Repeat 20 times)
  Get Pixel Color from Beneath the Mouse into %PixelColour%
  Delay: 50 milliseconds
  If Variable %PixelColour% Equals "14120960"
    Mouse Left Click
    Repeat Exit
  End If
  Mouse Move: 0, 50 Relative to Last Position
End Repeat

 

I've written a few scripts that calculate the starting coordinates, or the x or the y coordinate, based on the size of the window. For example, if I know the target I'm aiming for lies along a horizontal line half way across the window, I might do this:

 

// Half the width of a window is the x position
Variable Set Integer %Width%: Set to the Current Window's Width
Variable Modify Integer: %x% = %Width% / 2
 
// Start hunt at the top of the window
Variable Set Integer %y% to 0
 
Repeat Start (Repeat 20 times)
  Mouse Move: %x%, %y% Relative to Current Window
  Get Pixel Color from Beneath the Mouse into %PixelColour%
  Delay: 50 milliseconds
  If Variable %PixelColour% Equals "14120960"
    Mouse Left Click
    Repeat Exit
  End If
  Variable Modify Integer: %y% = %y% + 20
End Repeat

 

 

[hufi]

thanks both of you for your answer, I will try it....

[rberq]

11 hours ago, acantor said:

If you pixel check the position of the mouse cursor, rather than its x, y coordinates, it's easier to spot the problem when the script doesn't work.

 

Very true.  But as you say, it runs slower using the mouse.  You can have the best of both worlds by checking x,y coordinates, AND moving the mouse to those coordinates as a visual aid during debugging.  After you have found any script problems, disable the mouse movement command and just leave it in place in case you need it again later.