Jump to content
Macro Express Forums

Web Form Filling Strategy & MEP "Control" Questions


Recommended Posts

Greetings All,

 

I'm spenidng more & more time writing MEP scripts to complete web forms. Probably 90% of this is selecting radio buttons on a web page; "How would you rate the quality of the of the service you received?" - with 5 radio buttons, poor to excellent, etc. Typical feedback sort of stuff. Now I have a few questions:

 

1) Currently I say something like:

 

// Enter Order #...

Mouse Move: 506, 377 Relative to Screen

Delay: 80 milliseconds

Mouse Left Click

Delay: 80 milliseconds

 

Blah blah blah

 

...

 

And I repeat this over & over, moving from one question / statement on a form to the next. Typiocal deviations are hitting the "Next" (page of form) button or entering occasional text here & there. Find the next mouse coord, click the radio button. Repeat ad nauseum, ad infinitum. Always the same (per site). Of course manually harvesting the coordinates with the Mouse Locator is painstaking / tedious. I appreciate the Mouse Locator Tool... but it would be slick if there were a tool that would just register mouse click coords (and nothing else - no time delays, etc.). The good news is that every feedback / survey (per site) is filled out exactly the same way - every time. No deviation necessary.

 

Has anyone ever had success using a "coordinate file" for this sort of thing? How did you do it? I'm thinking if I put all the click / comment coordinates in a simple text file, and just process the text file. Take action based on the coord's purpose. Maybe a CSV file and invent a little syntax - so I can differentiate between a coord for a click on a radio button and a coordinate for a text insertion (comment). I'm thinking a "coord file" would >>greatly<< compact the overall code. But I digress...

 

2) Another thing I need to understand: What, exactly is "control"? What is a 'control event'? What is a 'Window control'? I see it referenced in PGM's "Macro Express Explained" - but never clearly (simply?) defined. I see references to it in commands throughtout Macro Express Pro. I'm told that "control" is the bread and butter of Macro Express, yet I can't find any sample macros that DO show examples (that even seem to be remotely relavant) of control. Would this be a more efficient / expeditious way to go rather than the repetitive mouse coord movements / clicks?

 

3) My final question: How can one determine how many lines there are in a text file without "pre-processing" it in advance and incrementing a counter? Is there a command to determine this? I occasionally need to extract a random line from a text file. The text file may be smaller or larger than it was the last time the macro was run. Right now, I have to manually track the length of the text file and update the variable that holds the "ceiling" of the random number generator code.

For Example:

 

(Let's assume there are currently 37 lines in the text file...But this has to constantly be verified before each run!)

 

// =-=-=-=-=-=-=-=-=-=-=-=-=-=-=

This will extract a random quote from an ASCII text file.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Variable Set Integer Number_of_Comments_In_File to 37

Variable Set Integer Which_Comment_To_Use to a random value between 1 and %Number_of_Comments_In_File%

Text File Begin Process: C:\somepath\somewhere\Comments.txt

Text File End Process

// =-=-=-=-=-=-=-=-=-=-=-=-=-=-=

End of random quote extraction.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

 

Anyway, that's probably enough (too much?! Sorry!!) for now!

 

I sincerely appreciate any hints, tips, etc. I have thick skin (for CONSTRUCTIVE criticism! ;-) ). Flames and rants to /devnull

 

Thanks for all the time and consideration here,

 

--Pete

Link to comment
Share on other sites

I am experienced.

...................................

As Jimmy would say. I have written tons of macros that enter or harvest tens of thousands of records.

 

1- I have used coordinate tables with good success. Especially for distributed macros where everyone’s environment is different. I have also created training macros where the user and define the different positions. EG “Position your mouse over the ‘OK’’ button and hit Enter to continue” and do that for all the interaction points. Also you can have it do a review ‘walk thru’ of all the positions for the user. The difference is I tend to save these in the registry instead of a local file. But in networked installations an INI file might be preferable.

2- Essentially controls are most every object you can interact with in proper programs. Evert text box, button and so on. There is a samples.mex with your MEP install that will demo controls using calculator. When I create macros they are always my first attempt. You can interact with them with much less hassle, timing issues, and you can even do things like click buttons or send text to items that aren’t even visible. Do yourself a favor and learn all about them. Having said that the main window area in web browsers are one beg control so for entering forms and such are useless. Bain of my existence.

3- There is no simple command to retrieve the number of lines in a file that I am aware of. What I do is perform a Text File Process that only contains a single command which is an integer increment. However in really large files I have opted for a quicker method. I would read the entire file into a var then get the number of characters. Sometimes file size can be used instead. Then I replace all the CR-LF (Carriage Return Line Feeds are the end of line delimiters) then get the var size again and divide by two. This is the number of lines. I suspect in some cases it’s faster. If you don’t know what the non-printing ASCII characters like CR and LF are let me know and I’ll explain more.

 

BTW do us all and yourself a favor and break these up into separate topics next time. Much easier especially if it breads a lot of responses.

 

I’ve written several long winded dissertations here on what I think are the best methods for talking web page automation. I encourage you to search for those because to be frank I’m to unmotivated to write it all up again. But I will suggest to avoid mouse moves unless your web page is Java or some other script. You see Java tends to only hear most of your text type commands. In this case you should consider the dreaded mouse moves and clicks. Also you can ‘download’ and process HTML files as text with a VBScript I posted which avoids the perils of browser interaction in many cases. Also if you have any programming ability there is a way to interact with web form elements programmatically much like the controls you mentioned. I’ve not done it yet but seen it demo’d and I swear that my next project will use this method instead. But of course a lot of this is dependent on exactly what you are doing. Also when entering web form stuff enter the data, copy it back and check it, then submit, then go back and verify. And build in a recovery or logging mechanism for the eventual misfires. Sounds like a lot but it’s worth it.

 

Feel free to give me a call too if you like and we can chat. Just send me a PM and I’ll hook up with you. I have spent so much time on this area I feel compelled to share my learnings with everyone.:)

Link to comment
Share on other sites

Item 3: We all have wondered at some time or other if there's a better way to count lines. Realistically, it does not take very long to do small files. My old PC will count a 33,000 line file in 42secs, about 720 lines per sec. It would be nice if you could process a text file without copying each line into a variable. I would think that would speed things up no end. Request for a feature?

 

Just for the hell of it I found a free text editor from which I could get the line count. I wrote a macro that opens the text file with the editor, activates the line count, copies it using a Control, closes the text editor and processes the Control into a Text or Integer variable, all in less than 3 seconds for the above file. So I guess there are alternatives, fast but clumsy.

Edited by JohnS
Link to comment
Share on other sites

Item 3: We all have wondered at some time or other if there's a better way to count lines. Realistically, it does not take very long to do small files. My old PC will count a 33,000 line file in 42secs, about 720 lines per sec. It would be nice if you could process a text file without copying each line into a variable. I would think that would speed things up no end. Request for a feature?

 

Just for the hell of it I found a free text editor from which I could get the line count. I wrote a macro that opens the text file with the editor, activates the line count, copies it using a Control, closes the text editor and processes the Control into a Text or Integer variable, all in less than 3 seconds for the above file. So I guess there are alternatives, fast but clumsy.

Here's some code in MEP and AutoIt to achieve this (runs in 3 seconds on an 11MB file containing 83,260 lines):

Variable Set String %t[1]% to "d:\temp\test2.txt"
External Script: AutoIt
Variable Modify String %t[2]%: Convert to Integer (%n[1]%)
Text Box Display:

The AutoIt script is very simple:

$File1 = FileRead($cmdline[1])
$File2 = StringReplace($File1, @CRLF, "")
ConsoleWrite(@extended)

<VARIABLE SET STRING Option="\x00" Destination="%t[1]%" Value="d:\\temp\\test2.txt"/>
<EXTERNAL SCRIPT Language="AutoIt" Dest="%t[2]%" Script="$File1 = FileRead($cmdline[1])\r\n$File2 = StringReplace($File1, @CRLF, \"\")\r\nConsoleWrite(@extended)\r\n\r\n" Parameters="%t[1]%"/>
<VARIABLE MODIFY STRING Option="\x04" Destination="%t[2]%" Variable="%n[1]%"/>
<TEXT BOX DISPLAY Content="{\\rtf1\\ansi\\ansicpg1252\\deff0\\deflang3081{\\fonttbl{\\f0\\fnil\\fcharset0 Tahoma;}{\\f1\\fnil Tahoma;}}\r\n\\viewkind4\\uc1\\pard\\f0\\fs16 Number of lines is %n[1]%\\f1 \r\n\\par }\r\n" Left="Center" Top="Center" Width="278" Height="200" Monitor="2" OnTop="FALSE" Keep_Focus="TRUE" Mode="\x00" Delay="0"/>

Link to comment
Share on other sites

Thanks Cory & John (And now I just see Paul's note here but haven't digested it yet!)

 

Last night I spent a couple hours just reading everything I could that looked remotely relevant with the word "control" in it, within the Macro Express documentation (Help files)- I printed it out and went through it all. I haven't truly absorbed everything, but I'm thinking "Mouse Click on Control" will be my friend (perhaps)! I plan to do a little test macro later this morning.

 

Cory, most everything you said made a good degree of sense. I shall search for your past dissertations in this forum and try to benefit from those, rather than make you reinvent the wheel just for me.

 

I think one thing that threw me a curve was that one particular page I was trying to interact with was apparently not "control friendly". Once I read this could be the case (I thought it was my test macro), I realized the problem. Arrrgggghh. Ok,I've since made a little progress. Pretty slick so far. I did just a very simple little test last night- go to a page, click a certain button using control rather than a mouse movement (per the sample code I showed yesterday). Definitely worth me continuing my research in it. This is all part of the ME learning curve, I guess. Sure, it might take 6 hours to develop one type / facet of a macro, but if you can use it again (and again) - that time investment suddenly is much less. Kind of like making specialty IC chips (Pentium processors or whatever...) the first one may cost millions (R&D, etc.). The rest are essentially "free", but you have to average the cost to recover the investment. I digress.

 

1) Coordinate files: Good! Glad others have used these with good success. This confirms that my logic may have some merit! Always a comfort! :rolleyes:

 

2) Control questions in general. Thanks for all the info. As I mentioned, I'm now making forward progress.

 

3) Regarding determining the number of lines in a text file: This would seem like a basic / simple thing to add in a future version. Easy for me to say, I don't have to code it! It seems agreed that this would be a nice feature to have. I hadn't thought of counting CR/LF's... There are some possibilities there. Still, that just seems a little more convoluted than it should be. I use EditPad Pro - the instant I open the text file it shows me the line count on a status bar in the bottom of the pane. BTW, EditPad Pro seems to behave very well with Macro Express. I believe EditPad Lite is FREE! And it will also show you the line count at the bottom of its pane. If anyone's looking for an above average, simple but extremely capable text editor, I highly recommend it. I use EditPad Pro to create, edit, massage, etc; all my text files. It understands regular expressions, etc. I think it blows away UltraEdit, etc. Since I also do some HTML and other things, I like the available color schemes / plug ins, etc. Disclaimer: I have NO pecuniary connection with JGSoft, et al. :-)

 

 

Thank-you Cory, John and Paul, for your contributions here, and for your patience with my noob questions. In the past, I've crafted some rather complex macros (interacting between SharePoint & a proprietary ticket system, disaster recovery and so on) - but I have always found 'intelligent

' form completion - specifically repetitive button clicking - to be a little tedious. And of course, as fate would have it, that's where I seem to be spending more of my time (repetitive form / feedback / survey ) completions. Nice to know there's a more precise way to do it!

 

Yes, next time, I will segregate this into different topics! Sorry 'bout that! :huh:

 

Thanks again! Very helpful info. I appreciate your willingness to help in the future. I think I'm "loaded for bear" now!! I'm headed over to Panera's for breakfast. I'm taking the laptop and plan to script a 'test' macro to really focus on what I need to do. After your notes and after reading the documentation and a promising little code experiment, I feel very optimistic now! I'll give you a shout if I get hung up on anything.

 

Have a great day, all!

 

--Pete

Link to comment
Share on other sites

Adirondacker - I have several text editors that show the number of lines however there was no way of extracting the information directly. If it's not a usable Control you are out of luck. I had to try a few new editors until I found one that worked. It was small, fast and required no installation.

 

Paul - That's a neat solution. I'm sure I tried AutoIt years back but migrated to ME. With my method I tried a 9MB file with 226,000 lines. That took 7 seconds. Mine's almost as big as yours! Total including character count of 9703175 took a minute.

Link to comment
Share on other sites

Has anyone ever had success using a "coordinate file" for this sort of thing? How did you do it? I'm thinking if I put all the click / comment coordinates in a simple text file, and just process the text file.

Yes, this can work well. You can write a simple macro to get current mouse position and append the x-y coordinates to a file. So you "learn" each screen once by positioning the mouse manually and firing off the macro with a hot-key combination. Or, have a repeat loop in the macro so it waits for you to click the mouse and gathers the coordinates after each click.

 

Be very careful not to move the mouse while the macro is getting mouse position. The command takes a "significant" amount of time, and if you move the mouse at all you may wind up recording the wrong coordinates.

 

If you want to get fancy, collect the pixel color at each point where you collect mouse coordinates. Then your playback macro can verify that it is really in the right place before it clicks, and warn you if things get out of sync.

Link to comment
Share on other sites

Paul - That's a neat solution. I'm sure I tried AutoIt years back but migrated to ME. With my method I tried a 9MB file with 226,000 lines. That took 7 seconds. Mine's almost as big as yours! Total including character count of 9703175 took a minute.

The way I see it, MEP and AUtoIt can be used together in harness. MEP is good at providing the interface and the fancy bits, while AutoIt is often better at doing the "heavy lifting". And MEP's seamless harmonization with AutoIt makes the whole thing quite straight-forward.

Link to comment
Share on other sites

Hi John!

 

Could you drop me a note at my email address? For reasons that escape me I can not send you a PM - I'm constantly told I can't send one for (exactly 24 hours) from now...

 

Thanks much!

 

--Pete

 

pfferris AT gmail DOT com

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...