Problem with variables

[sdspieg]

Wondered if somebody here could help us out. We are writing a macro to help us fill out a form that allows you to select a whole set of items (they're newspaper articles and the search results are shown 20 at a time with a checkbox in front of each article). The macro then opens the full-text of the checked articles as one large html-file and saves them 20 at a time. The problem is that we have a few thousands results and would like to save them all. We figured out how to check all checkboxes, how to open the large html-file and to save those (well actually, we're still struggling to find a way use the date/time stamp to create unique filenames, - we haven't quite figured this out yet, but I do think we'll be able to manage that one.

 

What we can't manage (yet) is how to get to the next page for the next set of search results. There is no 'next page' that we could have the macro click on , but there IS a logic in the consecutive pages' url's: http://*****.*****.*****/searchresults/articles.jsp?pg=2, http://*****.*****.*****/searchresults/articles.jsp?pg=3, etc. Is there a way to repeat the macro in such a way that it stores that URL as a variable and then just incrementally adds a number to the page number (i.e. first ...2, then...3, then...4 and so on in a loop until it no longer finds the higher number)?

 

Any help greatly appreciated!

 

-Stephan

[Cory]

Yes, of course. Use an integer variable and increment each time. For instance in Yahoo Groups they all have a 'messageid' portion in the URL. Instead of clicking or following I simply started at 1 and incremented it each loop. Something like this:

<REP3:08:000002:000001:0001:1:01:T1><NMVAR:08:01:0:0000001:0:0000000><HTTP2:1:T:http://*****.*****.*****/searchresults/articles.jsp?pg=%N1%><ENDREP>

Another thing to bear in mind is not to get stuck in the mindset that you have to click on links in the web page. I often grab the web page I need with WGET (downloads the HTML w/o IE to file) then use the data in the HTML to launch IE or another pass at WGET. IOW think of navigating with a series of launches instead of trying to click on links. Oh, "View Source" works well for this also.

[sdspieg]

Yes, of course. Use an integer variable and increment each time. For instance in Yahoo Groups they all have a 'messageid' portion in the URL. Instead of clicking or following I simply started at 1 and incremented it each loop. Something like this:

<REP3:08:000002:000001:0001:1:01:T1><NMVAR:08:01:0:0000001:0:0000000><HTTP2:1:T:http://*****.*****.*****/searchresults/articles.jsp?pg=%N1%><ENDREP>

Another thing to bear in mind is not to get stuck in the mindset that you have to click on links in the web page. I often grab the web page I need with WGET (downloads the HTML w/o IE to file) then use the data in the HTML to launch IE or another pass at WGET. IOW think of navigating with a series of launches instead of trying to click on links. Oh, "View Source" works well for this also.

 

Thanks so much - what a great forum!!!