rrathje Posted August 5, 2014 Report Share Posted August 5, 2014 I am scrapping a web page to find call data for our company. The page size changes based on data. I've tried tabbing, but they use Java script on their page. Is there a way to do a lookup such as "Find "Yesterday's Calls"" that would then take and highlight a word somewhere on the page and then to either grab the coordinates of that word so I can move the mouse there or automatically move the mouse there. Keep in mind the coordinates change on every page. Thanks Quote Link to comment Share on other sites More sharing options...
rberq Posted August 5, 2014 Report Share Posted August 5, 2014 With some web pages, I have had ME do a Ctrl-a to highlight all text on the page, then copy to clipboard and transfer to a ME text variable. Then you can scan the whole text string looking for "Yesterday's Calls" or whatever other words delimit the beginning and end of the data you want to extract. Whether it works or not depends a lot on how the page is presented. Sometimes the page is in sections, and you have to click on the right section before doing the highlight/copy-to-clipboard. You can test in a few seconds to see if it will work, by manually doing the Ctrl-a and Ctrl-c, then paste into Notepad and see what you have. Quote Link to comment Share on other sites More sharing options...
rrathje Posted August 5, 2014 Author Report Share Posted August 5, 2014 Thanks, I've tried that prior, there is a lot of "junk" on the page and if I can get to this certain coordinate, it is all nice and neat to download in a csv. Quote Link to comment Share on other sites More sharing options...
joe Posted August 5, 2014 Report Share Posted August 5, 2014 The best way to read and write to web pages is accessing IE's DOM, which is easily done using MXPro's External Script command and VBScript. In order to accomplish what you want programmatically (reading webpage fields, creating CSV files, and writing to webpage fields) HTML is a good thing to know. Here is an article on how to access a webpage in general from MXPro and get data off from it. This article covers only anchor elements but any type of element on a webpage is accessed in the same manner. Quote Link to comment Share on other sites More sharing options...
Cory Posted August 5, 2014 Report Share Posted August 5, 2014 I get the raw HTML and use RegEx to extract my data. Using the DOM is often easier but too often I find that it's simpler to use RegEx. I used to manipulate the string of the raw HTML in MEP but these days I do it in VB.NET. EG if the data is in an HTML table the sequence of rows is usually uniform. So if I create a match using the row and cell tags I can return a match collection that can easily be converted to a CSV, DB, or whatever. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.