Can ME Scan Words & Report Information

[sebby]

Good Day Everybody,

 

Maybe someone may have some insight on the function I would like Macro Express do. I would like to have the program scan a specific document and/or webpage on request then check/scan for a list of words and report those that were found. Then last thing would be to have that information paste into an excel spreadsheet for further analysis. I'm really looking forward for some ideas on whether or not this is actually possible then if so the process or functions that could be needed making it work.

 

Thanks in advance for the help!

[Cory]

Yes, I routinely write macros to do this sort of thing. Then contents of the file or web page can be loaded into a variable and that variable can be 'scanned' in one of many ways. It depends on just exactly how you want to report it.

[sebby]

Hi Cory,

 

Thanks for your kind response. I thought using a variable would be the best thing considering all other options available. My experience using variable is pretty much limited to multiple choice menu, thus learning or finding a way to manipulate it for such a purpose got my curiosity going. Now I am limited to the extent of the information that can be provided, nevertheless I will give you an example which will be almost identical to the real thing. The site has the information displayed using an ID then there is a box of text underneath, therefore it would look like this:

 

ID 458595584511

 

Text...

 

The way I thought would be best is having a macro that would copy the entire page then having it paste into a word document or perhaps excel, then when some of the words are found in each of the description it would pick up the ID and report it back. Now once I have the ID from the finding, it will be moved into an url or possibly into a search field from the site it was found and pull up that specific page up for my review. I still have no idea if the variable can only scan a word at the time or string of words, I imagine it could be possible to build a database of possibilities and having it checked back and forth. Those are all ideas that I had without much mean to put it together. I am always looking to learn something new and this I must admit will be a nice change of pace.

 

Hopefully the explanation given makes sense,

 

Thanks again your help

[Cory]

First I would like to suggest that you use the "Reply" button in the message you want to respond to instead of the "Add Reply" button at the bottom. Some of us view the forum in Outline mode and the Add Reply destroys the continuity of the threads. It's no biggie, it just make sit easier for us to follow and therefore more likely to respond ;-)

 

The first consideration is how best to get the text into the variable. I choose to use an external VBScript to download the web page to a file then load that file into MEP. This is more advanced but it is faster and far more reliable than using a web browser which is prone to all kinds of timing and reliability issues. Search the posts here and you can find that I posted it up. It looks intimidating but you can ignore all that. All you need to know is the place to insert your URL and resulting file name which you can easily substitute. I am confident you will find it easy to learn.

 

However if you want to keep things conceptually simple to start with and don't have a large number of pages to process you could use your web browser and do a CTRL+A to select all the text and execute the clipboard Copy command. Then use Variable Set String to the contents of the clipboard.

 

Hint: I suggest you not use the 4 predefined variables. Instead create your own named variables. So much nicer to have meaningful names and not be limited to 99 elements.

 

Another alternative to the coping from the browser is to "View Source" and copy that to your variable. This is the raw HTML and this is often much easier to parse.

 

I'm still not clear about your scenario so let me make up my own. Let's say an eBay window routinely pops up and I want to grab the item number and check the description for certain keywords. Let's say the label "Item Number: " occurs before it and "Item Description: " the description. Once the text of the page is in a variable I would use the Variable Set Integer to the position of text "Item Number: " then add 13. This would be the space after. I would then use Variable Modify String to Delete Part of Text form character 1 rto that position. I would then get the position of the next space, subtract 1 and use Variable Modify String to copy 1 to that position to another variable. Now let me back up a bit. Since you are likely to be doing several this is probable in a loop with an integer counter, let's call it %Counter% So I would have a variable named %Item Number% with, say, 100 elements. So on the first iteration I could copy that bit of text to %Item Number[%Counter%]% (AKA %Item Number[1]%).

 

Now let's say I had a big lists of keywords I wanted it to search for hits on. You could use the Text File Process command to go thru every one inside this larger loop. I would use the logic of "If Variable Contains" to see if any of the words or phrases were a match. Now again I don't know if you want to keep track of which hits it made so let's keep it simple and just report if there was any hits without keeping track of which. I would then have another array boolean (true/false) array variable As this inner loop runs I would have a statement to the effect of "if description contains keyword set %Hit% to TRUE else FALSE.

 

Finally within this loop I would output the item number and hit result to a text file using the Variable Modify String >Append to Text file with the CR LF option. And I would simply construct a line of output in a temp string that I would set to "%Item Number[%Counter%]%,%Hit[%Counter%]%". This creates a CSV file I can now later use in another macro or load into Excel.

 

I think that about covers it. I don't' have tome to consider this all carefully so if I missed something just ask. I'm also available to write such things for you or even teach you via remote and/or phone but of course I have to charge for that.

[terrypin]

Cory:

 

Have you scheduled that reply on a 1 minute repeat?! I see 3 identical copies so far.

 

Mind you, it probably warrants it, as it's a highly educational post which I've duly filed, thanks.

 

--

Terry, East Grinstead, UK

[paul]

Hint: I suggest you not use the 4 predefined variables. Instead create your own named variables. So much nicer to have meaningful names and not be limited to 99 elements.

Of course I agree with you completely about meaningful names. But you're not limited to 99 elements with the predefined variables - simply adjust the number of elements as desired, exactly like any other array variable.

[Cory]

Have you scheduled that reply on a 1 minute repeat?!

[spoken like HAL 9000] You have to understand one thing Terry, I'm not a real person. I'm actually a very sophisticated macro created by Joe. Every once in awhile I experience a glitch and repeat myself. I am putting myself to the fullest possible use, which is all I think that any conscious entity can ever hope to do. ;-)

 

[clears throat]Yeah the forum was slow and giving fits yesterday and every time I tried to post the browser would time out waiting for the confirm page. I must have tried 3 times. Joe couldn't get in either. All I could do is sit there and wonder what the heck the forum web icon is. And what is that anyway? A cartoon orange wearing a cowboy hat and smiling? Sorry, OT.

[Cory]

But you're not limited to 99 elements with the predefined variables - simply adjust the number of elements as desired, exactly like any other array variable.

Of course. I didn't mention that scenario because I feel if one is skilled enough to modify the number of elements then it would seem more logical to create a named variable instead of expanding the elements of the old guard. IE an unlikely hypothetical. That and I have an irrational hatred for the old 4. Do you have any idea how many hours I have spent and lines of code have been written to work around that limitation? :-) It's one reason I don't understand why any seasoned ME3 user would not want to migrate to MEP. In fact that feature alone would be worth it to me.

[paul]

It's one reason I don't understand why any seasoned ME3 user would not want to migrate to MEP. In fact that feature alone would be worth it to me.

Well, MEP is still full of bugs, and is extremely slow at many operations ME3 ran in the blink of an eye. You know, I still have no clue why AutoIt can process large text files very quickly, while I have never had the patience to wait for MEP to complete the same operation (after waiting for up to 30 minutes where AutoIt ran in less than 2 seconds). This "feature" is so bad that I question the technical competence of those responsible for such atrocious code.

[Cory]

I agree with you on the performance issue. I guess I'm just optimistic they can solve them soon. I've been dabbling in VB.NET and find things even quicker then you AutoIT anecdote. I can suck in a huge amount of data into arrays and do all sorts of complex procedures with very inefficient looping and for all practical purposes it's instantaneous. It reminds me of a stream tractor, there just doesn't seem to be anything one can do to slow it down. And if ISS wants to appeal to those they consider "Pro" users they need to get this fixed or, like me, "Pros" with needs will look elsewhere for solutions. EG I have a fairly simple macro that compares a Db extract (TSV sucked into arrays) to a client contact list. It doesn't do anything exotic and has no UI operations (all variable manipulation) and the client list is only about 1400 and the contact list about 2.5 times that. Yet the macro takes 12 minutes to run on the user's machine. That's glacial in the computer world. But my self-criticism is that this should be either a feature of the database or a properly compiled app, not a scripted macro. IE I'm using the wrong tool for the job. But there are times when user macros benefit from such operations but so far they have been limited enough that the slowdown isn't' noticeable.

[paul]

I agree with you on the performance issue. I guess I'm just optimistic they can solve them soon.

My fear is that these performance problems are caused by flawed basic design, and therefore cannot be solved without a fundamental rewrite. For example, in its early days (and even now) MEP had a number of very silly display problems, which I always thought were caused by not using standard graphics techniques (e.g. prompting for a variable used to cause the running macro to freeze, requiring MEP to be unloaded and reloaded). My current problem (reported some time ago but never further referenced by Insight) is the command categories in the script editor. They go funny on me after a while, so that the category names disappear, and the detailed commands misbehave when I double-click on one (an entirely different command appears). Running the mouse up and down the invisible categories without pressing any buttons usually fixes this, but it's irritating.

[paul]

I've been dabbling in VB.NET and find things even quicker then you AutoIT anecdote.

What version of VB.Net are you using? Did you ever learn VB6? Do you find the language relatively easy to use?

I've not really taken the plunge into the world of .Net. Perhaps I should!

[Cory]

The current version. I never learned VB6 and the only BASIC I ever did was in high school in the early 80's. Pascal in college was the most advanced language I ever learned. I did a lot of research on what language to learn and it's hard to find objective advice. It's like asking what pickup is best at a redneck bar. A Ford guy will tell you it's a Ford, the Chevy guy a Chevy, and of course the Dodge guy insists it's a Dodge. One easy way to clear the waters is that all the old arguments of one being faster that the other is out. Since all of them crom C#, C++, and VB are now based on the .NET framework they all perform essentially the same. IOW it really is semantics and personal preference and what one is familiar with. Now another thing is that VB6 is fundamentally different than VB.NET. VB.NET is the first true OOP version and that was a requirement for me. BTW also a lot of the bashing other language programmers do is out of date and based on their experiences with VB6. My other requirement was that it was Db friendly which it is.

 

But I think what really made up my mind was that it is syntactically very similar to other tools. VBScript which is used in automation and websites, Powershell, VBA which it the language for automation in every MS app. I reckon if I learn the proper VB that it would be easy to back into everything else.

 

I now find the language and the new IDE of Visual Studio very easy to use indeed. There is an old saying that people who learned procedural based languages are fundamentally scarred and can never properly adapt their mental programming paradigm to OOP. At first I felt this way but now I see the methods, properties and such all as just little subroutine calls which they really are. The property of width does not exist until it's called for. And in the end it's all compiled into machine code which is procedural. Once I made that leap I find it very easy to understand. One thing I complained about languages like this before was that one needed to have a 'bible' at hand all the time to find all the possible methods and exact syntax for everything. One thing I like about MEP is that I get nice dialogs and instead of having to research some obscure option and figure out the exact usage and syntax MEP just gives me a check box. IE the old way was like writing macros in the MEP direct editor. But the new IDE sees you typing and guesses ahead and when it gets close you can arrow down the list and tab or just tab if it's on top to the nect period, parameter or whatever. Also if one doesn't know all the methods of an object the drop down shows them all and one can just select one and even get help and a description. There are several clever things like this that make it way easier. I just finished the section with variables, arrays, loops, logic, and such and it was all very familiar. It's like being an American auto mechanic and moving to Germany. All the tools and things one does are basically the same, you just need to learn all the new words and metrics. The real skills are tings like knowing how to troubleshoot a charging system and that's all the same. All you need to do is learn the German terminology.