Web forms

[dhunt]

I have a cloud application with a web form.
The location of web browser on my screen may vary.
The web form is scrollable.
Currently to enter data on this form I scroll down until I get to a section that says "Screenings"
I click on screenings and below it a text box appears (a search box).
I then type "Mammogram" in the search box  and then in a drop box below appears the mammogram screening item.
I click the item and click the save button.

With macro express I can save these steps, but it only works if the window is in the exact same place and the form is scrolled down in the exact same place.
I'd like to have some type of 'smart' macro which recognizes the form, finds the clickable Screening item and performs all the steps automatically.
I'd appreciate a nice nudge in the right direction please.  Thank you.

[Cory]

MEP can't recognize controls in a Web browser. The best solution I've found was to tab. All pages will have the same control set as zero and start there. DO a fresh load of the page and count the tabs it takes to get to where you need to go. Also, try to do everything with the keyboard.

[dhunt]

Excellent.

Playing around with the form looks like tab and other keyboard controls may just work.
Thank you.

[acantor]

You might find this article about "smart macros" of interest....

 

https://www.cantoraccess.com/publications/macros_2007_csun_smartmacros.shtml

[kunkel321]

Yep.  I use macros on webforms a lot.  I find that navigating with the keyboard, is the way to go.   

Some tips:  Tab to the droplist, then press "m" to go right to "mammogram."  Or, if mammogram is the 4th thing in the list that starts with a m, then press "mmmm."    Tab to a checkbox or radio button and use <Space> to check/uncheck or select the item.  Shift+Tab will jump backwards, though I never use that in macros.  

Here's an example of some text that might get sent to the webform:

<CTRLD>a<CTRLU><DELETE>Please find the attached consent-for-evaluation form. Upon receipt of %txtName%'s consent-for-eval form we will begin the evaluation process. Remember that this consent form is only for the evaluation.  If %txtName% does qualify for an Individualized Education Program (IEP), then you'll need to sign an additional permission form for us to start %txtHisHer% services.  Please call if you have any questions. Stephen Kunkel, School Psych @ %TxtSchool%
<TAB><TAB><CTRLD>a<CTRLU><DELETE>No other options were applicable.  A file review and input from parents and staff suggest that an evaluation is appropriate at this time.
<TAB><TAB><CTRLD>a<CTRLU><DELETE>N.A.
<TAB><TAB><CTRLD>a<CTRLU><DELETE>This is based on the recent referral for special education testing.

I always click in the first edit field, then activate the macro.  As you might guess, Ctrl+A is "select all."  That allows me to delete/clear any old text that is in the text boxes.   

[rberq]

Another keyboard-navigation trick, to get a known starting position on the web form, is ALT-d to select the URL, then <END> to get to the end of it, then start tabbing from there. 

[acantor]

8 hours ago, rberq said:

Another keyboard-navigation trick, to get a known starting position on the web form, is ALT-d to select the URL, then <END> to get to the end of it, then start tabbing from there. 

This is a very good technique, but it may only work reliably when you open a web page for the first time. If you use the same technique to navigate a second time to the same field, tabbing into the page from the Address Bar may begin at the last focused field.

 

A method which I find more forgiving is to identify a spot within the window, usually near the upper left corner of the content. Click at that spot. If you find the appropriate area to click, the tab order will be reset, and pressing Tab will always take you to the first field. 

[kunkel321]

I like the Alt+d tip.  Another good one is Ctrl+0, which sets the zoom to 100%.  I'm not sure how you'd use that in a macro though.  

Regarding the top post, I do wish there was a way to make the macro "identify" the form.  For example with my above-pasted special education form text, there are a couple different types of forms.  "Referral for testing" which will be one of multiple different scenarios, or "results of testing"  which will be one of multiple different scenarios.  I'd like the macro to "see" which form I have open, and only offer text templates relevant to that form.  With the web database I use, the window title never changes and the URL never changes.   

[acantor]

 

Quote

 

With the web database I use, the window title never changes and the URL never changes. 

 

 

Ah... you are working with a poorly designed application! That's too bad because that makes macro scripting challenging, when it might be relatively straightforward.

 

The developers probably gave little thought that they may have legal obligations to ensure the application is accessible. A title bar that does not reflect the content of a page is a strike against WCAG conformance at even the most fundamental level:

 

https://www.w3.org/TR/UNDERSTANDING-WCAG20/navigation-mechanisms-title.html

 

Approach the developers and ask if there are plans to make the database meet WCAG 2.0 or 2.1, which is the international standard for accessible web content. Adding a meaningful <title> to each page is not that onerous, and could potentially help a lot of other users.

 

Without meaningful titles (or unique URLs), macro automation will likely be a major undertaking. I developed macros for a web app that lacked meaningful titles, and it took days to come up with scripts that worked somewhat reliably, followed by months of tweaking to get them to work very reliably. My scripts rely heavily on tracking pixel colours, and perform hundreds or thousands of steps to determine which page is displayed. To give a sense of the complexity of scripting for inaccessible applications, here is an excerpt from a script that determines which page is displayed:

 

Variable Set Integer %x% to 0
Variable Set Integer %y% to 0
Variable Set Integer %TabsTotal% to 0
Variable Set Integer %WinWidth%: Set to the Current Window's Width
Variable Set Integer %WinHeight%: Set to the Current Window's Height
 
:Start
Repeat Start (Repeat %WinWidth% times)
  Get Pixel Color at (%x%, %y%) Relative to Current Window into %PixelColour%
  If Variable %PixelColour% Equals "14599344"
    Repeat Exit
  Else
    Variable Modify Integer %x%: Increment
    Variable Modify Integer %y%: Increment
  End If
End Repeat
 
:FoundBlueZone
Repeat Start (Repeat %WinWidth% times)
  Get Pixel Color at (%x%, %y%) Relative to Current Window into %PixelColour%
  If Variable %PixelColour% Does not Equal "14599344"
    Variable Modify Integer %x%: Decrement
    Repeat Exit
  Else
    Variable Modify Integer %x%: Increment
  End If
End Repeat
 
:FoundCaseMasterTop
Repeat Start (Repeat %WinHeight% times)
  Get Pixel Color at (%x%, %y%) Relative to Current Window into %PixelColour%
  If Variable %PixelColour% Equals "3167105"
    Repeat Exit
  Else
    Variable Modify Integer %y%: Increment
  End If
End Repeat
 
Variable Modify Integer: %y% = %y% - 20 // This hack should not be necessary, but makes the macro work at a wide range of zoom levels!

Mouse Move: %x%, %y% Relative to Current Window
 
:FoundCaseMasterBottom
Variable Modify Integer: %WinWidth% = %WinWidth% - 300 // Shorten trajectory of pixel search - otherwise routine may find the separator near the right edge of the window. 

Repeat Start (Repeat %WinWidth% times)
  Get Pixel Color at (%x%, %y%) Relative to Current Window into %PixelColour%

  If Variable %PixelColour% Is Greater Than or Equal To "12763327" // White separator lines between tabs
  AND
  If Variable %PixelColour% Is Less Than or Equal To "14599344" // White separator lines between tabs
    Variable Modify Integer %TabsTotal%: Increment
    Variable Modify Integer: %x% = %x% + 5 // The separator is several pixels wide at certain zoom levels, so skip ahead a little
  End If
  Variable Modify Integer %x%: Increment
End Repeat
 
Macro Stop
 
:FoundTab
 
Switch( %TabsTotal% )
 
Case: 0 // Case List
  Variable Set String %Screen% to "CaseList"
End Case
Case: 7 // Demographics
  Variable Set String %Screen% to "Demographics"
End Case
Case: 13 // Case Master
  Variable Set String %Screen% to "CaseMaster"
End Case
 
Default Case
  Text Box Display: Not handled yet: %TabsTotal%
End Case
 
End Switch

 

[kunkel321]

Good to know!  I didn't even realize WCAG was a thing..  I'll suggest this to the developers.  As I look closer, I see that there actually is a difference in the URL when I go to different forms.  Unfortunately the difference appears to be a random number such as "document?q=3BOJIA9BUU."  I compared a bunch of them and there doesn't seem to be a pattern.  

 

I like your pixel-sampling work-around.  How long does it take to scan a normal sized window though??

 

Interestingly, the other topic on this forum "Issues navigating in browser," overlaps a bit with this one.  For example using Ctrl+f to see if certain words are on a page.  I don't think it's possible for MEp to "see" whether or not anything was found though.   Another interesting thing:  In Chrome, if you press F12, you can see the actual code of the webpage and search it.  Again though, I'm not sure that you can utilize this with a Macro (also you might not want to mess about in the code of webpages).  

[acantor]

Quote

I like your pixel-sampling work-around.  How long does it take to scan a normal sized window though??

 

It's impractical to scan an entire window pixel by pixel using Macro Express. Checking a 1680 x 1050 screen means testing nearly two million pixels. Macro Express takes too long: minutes, not seconds. Other macro scripting packages search for colours more quickly, but this is not Macro Express' forte.

 

However, the task becomes do-able in Macro Express by restricting the search to, say, five or ten thousand pixels. The code sample above checks pixels along lines rather than pixels within areas.

 

In the "Start" section, the macro checks pixels along a diagonal beginning from the upper left corner of the window.

 

When the colour is found, it checks right horizontally ("FoundBlueZone" section).

 

Then it checks down vertically ("FoundCaseMasterTop" section).

 

Then it checks right horizontally ("FoundCaseMasterBottom").

 

By restricting the pixel search to lines instead of areas, this macro runs in a fraction of a second. But the script was not easy to code.

 

Quote

For example using Ctrl+f to see if certain words are on a page.  I don't think it's possible for MEp to "see" whether or not anything was found though.

 

It's possible if you can predict (or calculate) where selected text — which in my browser is a shade of blue — will appear. In one application I automated, there was a fixed-width frame on the left side of the window containing a list of links, one per line. The script searched for link text using Ctrl + f. Then it checked every pixel along a single vertical line within the frame for that shade of blue. If the macro found the colour, it clicked on the link. If the macro did not locate the colour after searching the height of the window, it stopped.

[rberq]

Another variant on pixel checking:  I had an application with several forms, each with the same screen title, and each form contained a dozen or more spots for entering data.  The problem was for the macro to know which form it was dealing with.  The entry slots were in different locations, and/or were different widths, on the several forms.  The data-entry space was a light color and the non-entry space was a gray background.  So the macro read a table of pixel locations, with an indicator for each location whether it should be the light or dark color.  If all locations matched what was expected, the form identity was known.  If a location did not match, then the next table (for the next form) was loaded and testing continued until a form was identified.  It was pretty fast since only a few dozen pixels had to be tested for each form.  When a form changed due to an application update, it was simple to update the corresponding table, with no changes to the macro itself.  In fact, I had a separate macro to build the tables, where I manually moved the mouse to an area to be tested, then triggered the macro with a hot key to record mouse location and pixel color. 

 

Incidentally, it is MUCH faster to check pixel colors at specific locations, than to move the mouse and get the color at mouse location. 

[kunkel321]

Some good thoughts here on pixel testing.  It occurs to me that, with rbert's example of sampling a few specific points, you might have a problem if the zoom factor of the page were ever changed, then the points might be at different locations.  This might be a good use for "Ctrl+0" which would set the page to 100% zoom.  

 

In my last post (above), I mentioned the other thread, "Issues navigating in browser."  It occurs to me that, using the method discussed there, you might be able to identify a form.  If a particular web form has a unique phrase such as "Referral," then you could do Ctrl+f, type "referral," then Esc.  If the word is found it would (i.e. should) be left highlighted.  You could then copy to clipboard, and check if the clipboard = "referral."  This would be really cludgy and I would hate it.  Still though, should work in theory.  

[rberq]

It is not really kludgy, and in fact can work pretty fast.  I have a number of macros that use CTRL-a to highlight a whole web page, look for specific text, then do something.  Below is a snippet of code that checks an email and either deletes it or saves it in a mail folder depending on text.  

 

The key to making the macro very fast is to set the Macro Express clipboard delay (preferences) to zero and use a different technique to assure the clipboard command has completed – see text of the clipboard macro following this code snippet.  My fear, when I created the clipboard macro, was that it would return a partially-filled clipboard to the calling macro.  In other words, the test for "something" in the clipboard would be satisfied before "everything" was in the clipboard.  That did not happen, however, and the macro has worked flawlessly for years.  

 

    // ....    Amazon not in window title, so see if "Amazon.com order" is in the text somewhere.
    Mouse Move Screen 1300, 700 (somewhere within the body of the email text)
    Mouse Left Button Click             (click provides focus for CTRL-a to work properly)  
    Delay 200 Milliseconds
    Text Type: <CTRLD>a<CTRLU>
    Delay 200 Milliseconds
    Macro Run: 0_Generic_Copy_To_Clipboard
    Variable Set String %T1% from Clipboard
    Mouse Left Button Click  (clears the highlighting)
    Delay 200 Milliseconds
    If Variable %T1% contains "amazon.com order" 

Macro 0_Generic_Copy_To_Clipboard
// Set clipboard to nulls -- check to make sure it happens
Clipboard Empty
Repeat Start (Repeat 10000 times)
  If Clipboard Text Equals ""
    Repeat Exit
  Else
  End If
Repeat End
// Copy to clipboard (CTRL-c)
Clipboard Copy
// Loop until clipboard is non-null, that is, copy to clipboard has completed.  Since the value
// to be copied may in fact BE null, we limit the loop to a nominal 1 second, then quit, leaving it null.
// (Due to overhead, the maximum loop time will be more like 2 seconds than 1.)
Repeat Start (Repeat 100 times)
  Delay 10 Milliseconds
  If Clipboard Text Equals ""
  Else
    Repeat Exit
  End If
Repeat End
// Return to caller
Macro Return