Issues navigating in browser

[UAMike]

I have been incorporating the Ctrl+F function in a Chrome browser to allow my macros to interact with various parts of a page (or series of pages).

A given macro will TextType Ctrl+F to search for a specific phrase that I know will only appear once on a given page. In Chrome, the text found gets highlighted in orange, then when you hit esc, it will turn blue. I use the blue highlighted text to tab to a specific text box, to click buttons, to gather information that then gets stored into variables, etc.

This has worked perfectly until very recently. Now, after a string is found and highlighted orange, it doesn't always highlight blue when I hit esc. Sometimes it will, sometimes it won't. As a temporary fix, I have the macro perform the same search twice... and on the second attempt the text will undoubtedly highlight orange, then blue when I hit esc. I thought perhaps there was a window focus issue or a timing issue, but that hasn't led me anywhere. I can't even reproduce this behavior if I manually search for something in my browser. 

Any ideas? What other steps can I take to debug this issue? Since I rely on the Ctrl+F function so much, it really slows down my workflow to have the macro repeat the search a second time.

[acantor]

It sounds like a timing issue. What happens if you add extra long pauses between each step, say, 1000 milliseconds?

It shouldn't make a difference, but maybe try F3 instead of Ctrl+f.

 

If you can't find a solution, consider rewriting your script for Firefox. I use the same technique regularly with Firefox, and I can almost always make MEP scripts work.

[acantor]

P.S. Make sure your script is sending "Ctrl+f" and not "Ctrl+F"

[UAMike]

On 3/16/2019 at 8:33 PM, acantor said:

It sounds like a timing issue. What happens if you add extra long pauses between each step, say, 1000 milliseconds?

It shouldn't make a difference, but maybe try F3 instead of Ctrl+f.

 

If you can't find a solution, consider rewriting your script for Firefox. I use the same technique regularly with Firefox, and I can almost always make MEP scripts work.

 

I've tried slowing down the macro and still get the same results, only slower lol. Unfortunately I don't have the option to use Firefox (this is my work machine and they only want us using Chrome). It's always frustrating when something used to work just fine and all of the sudden doesn't 😤

 

I will try using Ctrl+F instead of Ctrl+f. Will also try F3 and report back.

[rberq]

Are you sure the web page is completely (or sufficiently) loaded before the search?  You could try Wait for Web Page before searching for text. 

That might explain why you can't reproduce the problem manually -- it takes you a lot longer to type in the search than it takes the macro, so by the time you finish typing the browser is ready for you. 

[acantor]

Try recreating the macro, step by step, from scratch.

[UAMike]

On 3/20/2019 at 8:10 AM, rberq said:

Are you sure the web page is completely (or sufficiently) loaded before the search?  You could try Wait for Web Page before searching for text. 

That might explain why you can't reproduce the problem manually -- it takes you a lot longer to type in the search than it takes the macro, so by the time you finish typing the browser is ready for you. 

 

I believe the Wait for Web Page only works in IE, whereas I am using Chrome. In any case, it's not always the first thing that I search for on a page that the macro gets tripped up on... sometimes it's the 2nd, 3rd, etc, by which time the page has definitely loaded completely. 

[UAMike]

7 hours ago, acantor said:

Try recreating the macro, step by step, from scratch.

 

I will try. I also tried using F3 to search and that may have led to some marginal improvement in the search behavior.

[acantor]

I've written macros for web apps that search for text via Ctrl + f and F3. I don't think the difference is significant.

 

Firefox supports search for link text only, which I find helpful. Not sure if Chrome has the same feature.

[terrypin]

I was eventually able to reproduce this behaviour (in Chrome). I think it only arises if I first use the vertical scroll bar to locate the first of multiple hits.

 

 

[rberq]

Speaking of the vertical scroll bar, I have several macros where I would like to use it.  But it's hard to find.  Pixel color seems to be totally unreliable, to distinguish where the "grab-able" part is, as opposed to the "groove" that the bar slides in.  Does anyone know why?  (Windows 7) 

[acantor]

Maybe you don't need to find the scroll bar. Instead, aim for the scrollable information, and then press navigation keys, e.g., up, down, Page Up, Page Down, Home, End, etc.

[terrypin]

For macros working on the scroll bar I usually first drag it to either top or bottom, and then further move it if necessary from that known position. In practice I've found that the 'guess' of its initial, unknown position works reliably.

 

Typical example:

 

Mouse Speed: 5 milliseconds
Delay: 0.1 seconds
Mouse Move: 1423, 218 Relative to Screen // Roughly middle of scroll bar.
Delay: 0.1 seconds
Mouse Left Button Down
  Delay: 0.1 seconds
  Repeat Start (Repeat 20 times)
    Mouse Move: 0, 4 Relative to Last Position
    Delay: 1 milliseconds
  End Repeat
  Delay: 0.1 seconds
Mouse Left Button Up

 

[rberq]

8 minutes ago, terrypin said:

For macros working on the scroll bar I usually first drag it to either top or bottom, and then further move it if necessary from that known position. In practice I've found that the 'guess' of its initial, unknown position works reliably. 

 

Right.  But with a short but unpredictable scroll bar length, AND unknown top-to-bottom position, how to you move the mouse to the middle of it?  Best workaround I have found is to move the mouse to the very top or bottom of the scroll area, click it a bunch of times, and by then the grab-able part IS under the mouse.  It would be more elegant and emotionally satisfying🤗 to be able to do a quick pixel-color scan to find it. 

[rberq]

2 hours ago, acantor said:

Maybe you don't need to find the scroll bar. Instead, aim for the scrollable information, and then press navigation keys, e.g., up, down, Page Up, Page Down, Home, End, etc.

Yep, I have used that technique too.  I'm just puzzled by the nature of the scroll bar area, why pixel color can't be reliably obtained. 

 

[terrypin]

2 hours ago, rberq said:

Right.  But with a short but unpredictable scroll bar length, AND unknown top-to-bottom position, how to you move the mouse to the middle of it?  Best workaround I have found is to move the mouse to the very top or bottom of the scroll area, click it a bunch of times, and by then the grab-able part IS under the mouse.  It would be more elegant and emotionally satisfying🤗 to be able to do a quick pixel-color scan to find it. 

 

Agreed. The apps in which I run my few ‘scrolling macros’  have relatively long  scroll bar lengths and their tops are either fixed or easily calculated.

 

I’m curious too. Maybe UAMike could show us an example?

[acantor]

The colours of some scrollbars may not be (easily) discoverable because its colours change when the mouse pointer is hovering over it. This isn't an issue in every application, but I've seen it.

 

My workaround is to snap a screenshot of the object am trying to detect, paste it into a graphics program – even MS-Paint will do – and then read off the pixel colour there.

[acantor]

Not sure this code will help, but I use routines like this to search for pixel colours along a user-specified horizontal line starting from the right side of the window. So it might help for vertical scroll bars, which are on the right edge of windows.

 

My script counts the number of times the target pixel colour has been detected. If it's found %HitTarget% times, the macro declares success. Note the script is NOT looking for consecutive pixels. It's looking for the total number of hits along a horizontal line. Consecutive would be better!

 

I can find scroll bars in Firefox fairly reliably using this routine.

 

(However, the window must be more than %PixelsFromTop% pixels high! In actual use, I would calculate this value rather than set it.)

 

Even if you try this macro in Firefox, your value of %PixelColourTarget% may be different than mine. The correct value seems to be dependent on the version of Firefox, the version of Windows, and your display theme settings.

 

Variable Set Integer %PixelColourTarget% to 16777215 // We are searching for this pixel colour
Variable Set Integer %PixelsFromTop% to 200 // ... along a horizontal line this distance from the top of the window 
 
Variable Set Integer %WinWidth%: Set to the Current Window's Width // Maximum number of pixels to check along the horizontal line
Variable Set Integer %HitTarget% to 5 // How many times must the pixel be found?
 
Variable Set Integer %HitCount% to 0 // Keep track of how many times the pixel is found
Variable Set Integer %x% to %WinWidth% // Start searching at right side of the window
 
Repeat Start (Repeat %WinWidth% times)
  Get Pixel Color at (%x%, %PixelsFromTop%) Relative to Current Window into %PixelColour%
  If Variable %PixelColour% Equals "%PixelColourTarget%"
    Variable Modify Integer %HitCount%: Increment
    If Variable %HitCount% Equals "%HitTarget%"
      Mouse Move: %x%, %PixelsFromTop% Relative to Current Window
      Text Box Display: Pixel Found!
      Macro Stop
    End If
  End If
  Variable Modify Integer %x%: Decrement
End Repeat
Mouse Move: %x%, %PixelsFromTop% Relative to Current Window
Text Box Display: Pixel not Found!

<VARIABLE SET INTEGER Option="\x00" Destination="%PixelColourTarget%" Value="16777215" _COMMENT="We are searching for this pixel colour"/>
<VARIABLE SET INTEGER Option="\x00" Destination="%PixelsFromTop%" Value="200" _COMMENT="... along a horizontal line this distance from the top of the window "/>
<COMMENT/>
<VARIABLE SET INTEGER Option="\x0A" Destination="%WinWidth%" _COMMENT="Maximum number of pixels to check along the horizontal line"/>
<VARIABLE SET INTEGER Option="\x00" Destination="%HitTarget%" Value="5" _COMMENT="How many times must the pixel be found?"/>
<COMMENT/>
<VARIABLE SET INTEGER Option="\x00" Destination="%HitCount%" Value="0" _COMMENT="Keep track of how many times the pixel is found"/>
<VARIABLE SET INTEGER Option="\x00" Destination="%x%" Value="%WinWidth%" _COMMENT="Start searching at right side of the window"/>
<COMMENT/>
<REPEAT START Start="1" Step="1" Count="%WinWidth%" Save="FALSE"/>
<GET PIXEL COLOR Option="\x01" Rel_To_Screen="FALSE" X="%x%" Y="%PixelsFromTop%" Destination="%PixelColour%"/>
<IF VARIABLE Variable="%PixelColour%" Condition="\x00" Value="%PixelColourTarget%" IgnoreCase="FALSE"/>
<VARIABLE MODIFY INTEGER Option="\x07" Destination="%HitCount%"/>
<IF VARIABLE Variable="%HitCount%" Condition="\x00" Value="%HitTarget%" IgnoreCase="FALSE"/>
<MOUSE MOVE Option="\x02" X="%x%" Y="%PixelsFromTop%" _PROMPT="0x000A"/>
<TEXT BOX DISPLAY Title="Pixel Found!" Content="{\\rtf1\\ansi\\ansicpg1252\\deff0\\deflang1033{\\fonttbl{\\f0\\fnil\\fcharset0 Tahoma;}{\\f1\\fnil Tahoma;}}\r\n\\viewkind4\\uc1\\pard\\f0\\fs24 Colour = %PixelColourTarget% was discovered %HitTarget% times. The current one is %x% pixels along a horizontal line %PixelsFromTop% pixels from the top of the window.\\f1 \r\n\\par }\r\n" Left="Center" Top="Center" Width="278" Height="200" Monitor="0" OnTop="TRUE" Keep_Focus="TRUE" Mode="\x00" Delay="0"/>
<MACRO STOP/>
<END IF/>
<END IF/>
<VARIABLE MODIFY INTEGER Option="\x08" Destination="%x%"/>
<END REPEAT/>
<MOUSE MOVE Option="\x02" X="%x%" Y="%PixelsFromTop%" _PROMPT="0x000A"/>
<TEXT BOX DISPLAY Title="Pixel not Found!" Content="{\\rtf1\\ansi\\ansicpg1252\\deff0\\deflang1033{\\fonttbl{\\f0\\fnil\\fcharset0 Tahoma;}{\\f1\\fnil Tahoma;}}\r\n\\viewkind4\\uc1\\pard\\f0\\fs24 Colour = %PixelColourTarget% was not found along a horizontal line %PixelsFromTop% pixels from the top of the window.\\f1 \r\n\\par }\r\n" Left="Center" Top="Center" Width="278" Height="200" Monitor="0" OnTop="TRUE" Keep_Focus="TRUE" Mode="\x00" Delay="0"/>

 

[rberq]

15 minutes ago, acantor said:

Not sure this code will help, but I use routines like this to search for pixel colours along a user-specified horizontal line starting from the right side of the window. So it might help for vertical scroll bars, which are on the right edge of windows. 

 

Yes, I was using similar code to scan along a vertical line (the scroll bar area), though being south of the border I searched for colors rather than colours.😉

I just tried your trick of taking a screen shot and pasting it into Paint to look at the scroll bar.  The slider is consistently a gray shade, but the slot the slider moves in is a combination of the same gray with a lot of white pixels mixed in, for shading -- same problem as when working with the original screen rather than a snapshot.  I just did a little more testing, and found if I scan half a dozen pixels vertically, without finding any color variation, then I have reliably found the slider rather than the slot.  Perhaps I will try that technique next time I want to use the scroll bar. 

 

Oops!  Seems I have hijacked this thread.  I hope the original poster got his answers.