Wait For Web Page

[Nicolas]

I'm really really really fed up with this STUPID "Wait for Web Page" command that from time to time never stops ... I have to reboot to have it work again (sometimes)

 

it may happen with a Wait for Web Page "" (all instances) or with the new option in the "Web Site" command as well, I'm using Internet Explorer 6 under Windows XP (it never occurred on another computer with W2000) and I really don't know what I can do, except maybe develop my own "Web for Web Page" macro (looking for the pixels in the status bar) or buy another software as this feature is vital in all my macros

 

this evening, I already rebooted 12 times and it still does not want to work as it should ...

 

does anybody has any idea ? or a work-around ?

[paul]

IMHO this is one of ME's weakest areas. I'm sure Insight recognizes this because many new releases include so-called improvements to this feature.

 

Even waiting for pixel colours to appear in specified location(s) works only until the owners of the web page in question decide to replace what previously worked perfectly with some newer version (which of course often doesn't work as well).

[Noggin the Nog]

Something that I do may or may not help. After requesting the web page I allow a certain minimum loading time, say 20 secs. At that point I click the mouse on the Address Bar (near the right hand end), followed by Copy to Clipboard and examine the text using Set Text Variable from Clipboard. If the text indicates it is the required page, I allow a further time to complete page loading and then continue with the macro. The mouse click, copy and interpretation is in a timed repeat loop, say every 5 secs. Once the complete loading time has expired, the macro jumps out of the repeat loop with an Exit or Break command.

 

The first time the new page name appears in the Address Bar, clicking with the mouse will highlight the new page name. On subsequent clicks, the entry will be deselected and nothing returned. The procedure does not always work, but mostly!

[joe]

Hello Nicolas!

 

There was a project that we had worked on a while ago. It required accessing and gathering data from approximately 20,000 web pages per month. It was run 24/7, which meant a new page every 30 seconds. There are four things that we did to make the waiting for web pages more reliable:

1. We made sure that IE would check for a new page each time (Tools | Internet Settings | General | Settings)
   
2. We would place the Wait for Web Page command immediately after the Web Site command. No pauses or delays of any kind between them. This prevented the Wait for Web Page command from missing the "okay" signal from the O/S (now the two commands can be command into one).
   
3. We waited for the IE "progress bar" to finish by trapping for a change in pixel colors within a Repeat Loop.
   
4. We placed a Delay of 1 second following the loop.
   

We had actually set up two computers. One simply gathered the data using the above steps and saving it to a file, while the other processed the data. By taking these steps, we were able to get the macro to run reliably for a long period of time. I think the longest it ran was 10 days before memory leaks from Microsoft O/S froze the machine.

[Nicolas]

Thanks for your answers, I changed my settings in Tools | Internet Settings | General | Settings, we'll see it it can help !

 

I don't think it is a question of delays because when it happens for one page, then "Wait for Web Page" does not stop anymore, for any page, until I reboot ... I don't know it if is a problem in Macro Express, Internet Explorer or a conflict between ME and something else ...

 

This problem would not be so annoying if another enhancement I asked for (last year I think) was available : the possibility to ask Macro Express to continue after the timeout delay rather than raising an error. In this case, I would be able to process the Web page, even if I had to wait for this delay everytime ... But maybe there is a way to do it ?

[Nicolas]

I've lost another night, having a macro that did not continue after the web page was loaded ... so I decided to develop my own (small) program to do it !

 

of course it will not be so easy to use as a "Wait for Web Page", as I'll have to launch another program, waiting for its return, but I'll tell you if it works

 

I simply have to replace "Wait for Web Page" commands in something like 700-800 macros ...

[Noggin the Nog]

Inspired by some comments by floyd in another thread and joe's here, I found a very good way to improve my own web page loading without a "Wait for..." line which I could not use in my application. I first tried using the 'get pixel color' as suggested by joe. It worked quite well but I had pages where the progress indicator was almost instantaneous. What I now do (and I'm sure others have done before me) is as follows:

 

1. Request the page

2. Go into a 2 sec loop which gets control from the address bar (edit) %C1% and converts to string variable %T1%, and gets control from the status bar (msctls_statusbar32) %C2%, and converts to string variable %T2%.

3. If the address bar control string is the required page and the status bar string is "Done", then kick out of repeat loop and save the page.

 

So far works really well. It would not work on sites where Done is overwritten by something else like a banner. Previously I had to allow a large time to cope with variations in download times. This method is very time-efficient.

[Nicolas]

2. (...) gets control from the status bar (msctls_statusbar32) %C2%, and converts to string variable %T2%.

3.  If (...) the status bar string is "Done", then kick out of repeat loop and save the page.

... ok it's in french, but in my case I had to wait for a status bar string "Terminé" so

 

it was really a good idea ! It works fine and to process 50 Web pages (in fact several pages for each of them) :

 

using Wait for Web Page : 30-31 minutes (1)

using Macro Run / my program : 37-38 minutes (2)

using your trick : 25 mn (3)

 

 

(1) because I always add a 2s delay before and after this command or it does not work really well

(2) because calling an external program is slower and this program has to connect to the Windows Shell eveytime

(3) no need for delays, no external call, simply a loop every second

 

Unfortunatly I won't use it on all my macros because i'm too lazy to open 600-700 macros and replace between 1 and 4-5 "Delay / Wait for Web Page / Delay" commands with a "Macro Run" command but I'll use this trick for the most critical ones !!

 

once again, thanks

[marcoval]

I too think it's a useless command and needed it to be rock solid. So I did some tweaking an it works fantastic.

 

What I did was changed the "throbber" in the upper right hand corner to be blue when it isn't busy and red when it is busy. Then ME looks for the pixel color change every .5 seconds. It has worked flawlessly ever since.

 

Instructions for changing that animation can be found at...

 

http://www.virtualplastic.net/html/logo_thr.html

 

Look for the heading "Throbber" (What a horrible name)

[kpunches]

The Throbber suggestion worked like a charm!!

 

I changed the throbber so that it was blue when IE was inactive and red when loading. Had to fudge with the graphic and SmBrandBitmap Reg entry quite a bit. Couldn't ever get the bitstrip to take up all of the space available for the graphic. Wound up using 22 X 22 X 264 for the BitStrip. I really don't care about the look of it...just want to know EXACTLY when IE stops loading the page so my macro can continue unabated.

 

Then I wrote the Macro (aptly dubbed "wait for web page"..cheeky eh?) below:

 

 

You'll notice that I have it set up to check the pixel color every .5 seconds and that I have it repeat for up to 30 possible cycles. This gives my pages up to 15 seconds to load. I have a T1 so if the page takes longer than that I just skip that step in the macro and move on to the next.

 

Now I run this macro instead of using the "timed delay" command whenever I want to load a new page.

 

Works like a charm. Thanks for all the great advice. Thanks to this I have cut 3 minutes off of a 9 minute routine. Huge timesaver and it's fullproof.

 

Cheers!

 

~Keith

[Neel_M]

I would like to thank Noggin for the method of using the text in the status bar to determine the condition of the web page.

 

This has solved all my problems with a macro that I have been trying to perfect, and this new method has reduced a job that took appx 16 hours to complete down to appx 10 hours. This is a major improvement in time and throughput in what we were trying to accomplish.

[paul]

Unfortunatly I won't use it on all my macros because i'm too lazy to open 600-700 macros and replace between 1 and 4-5 "Delay / Wait for Web Page / Delay" commands with a "Macro Run" command but I'll use this trick for the most critical ones !!

Here the trick is to examine that part of your macro that contains the new code using the Direct Editor, then write a new macro that inserts this code into your 600-700 macros. Joe and I use this technique to distribute our PGM Library.

[roostermt]

Newbie here.

 

How do I get a pixel color number. Not the coordinates but the actual number.

 

I'd like to use it as a wait for page.

 

Rooster

[paul]

How do I get a pixel color number. Not the coordinates but the actual number.

 

I'd like to use it as a wait for page.

Use the Mouse Locator tool.