Scope bug?

[terrypin]

I've been trying to establish the causes of various macro scope failures. Macros that should not run (because their scope specs are not met) do run. And if the activation hotkey for these macros happen to be also used in some of my applications, the macro trys to run instead of the application's own command, creating havoc. Today I isolated the cause to what seems a major bug in ME Pro.

 

Here's an example that should be easily reproduced. This one-line script just displays a message when activated. It's scoped for Notepad. ("The window only needs to be running. The title is a partial match.")

 

<TEXT BOX DISPLAY Content="{\\rtf1\\ansi\\ansicpg1252\\deff0\\deflang2057{\\fonttbl{\\f0\\fnil\\fcharset0 Tahoma;}{\\f1\\fnil Tahoma;}}\r\n\\viewkind4\\uc1\\pard\\f0\\fs16 Notepad is running\\f1 \r\n\\par }\r\n" Left="Center" Top="Center" Width="278" Height="200" Monitor="0" OnTop="FALSE" Keep_Focus="TRUE" Mode="\x00" Delay="0"/

 

With Notepad not running, it still gets activated. I tried many hotkeys, including: Ctl+3, Ctl+Sh+6, Ctl+Sh+7, Alt+1, Alt+Ctl+7, Win+Ctl+Y.

 

This could explain unresolved wild behaviour in the past. The immediate implication is that I cannot trust any of my macros with scope "The window only needs to be running. The title is a partial match.".

 

--

Terry, East Grinstead, UK

[paul]

I confirm your finding. If the Scope is set to program instead of window, it all works correctly.

 

I then changed your sophisticated macro to repeat with all windows, check the variable set each time to see if it contains the case-insensitive characters notepad, and append to a 2nd variable if it does. After all windows had been evaluated, the displayed result was blank. I did this just to make sure that MEP wasn't finding some spurious window name containing notepad.

Variable Set to ASCII Char 13 to %tCr%
Variable Set to ASCII Char 10 to %tLf%
Repeat with Windows: All Windows: Store in variable %t[1]%
 If Variable %t[1]% Contains "notepad"
Variable Modify String %t[2]%: Append Text (%t[1]%%tCr%%tLf%)
 End If
End Repeat
Text Box Display:

Actually, I wonder whether this is a scope bug, or a window bug because I don't think I find windows activation as reliable in MEP as it was in ME3.

[Cory]

Confirmed.

..........................................................................

I created a simple macro as you described and set the scope to "XCVBBVCX" just in case there happened to be something with the word "notepad" in it and it still ran.

[chris]

Could I ask something here? When using the window scope with the title of Notepad, could you verify that there are no hidden windows with notepad in the title? During the scope testing, Macro Express does not differentiate between visible and invisible windows and on the newer versions of Windows, it seems to keep hidden windows around that contain the titles of programs you have recently run.

 

So, when a macro is firing that doesn't look like it's supposed to, could you please open Macro Express and bring up the window list (by editing scope > Add Window button) and look in the Hidden tab? I'll bet that you find a window title that will cause the scope to pass.

 

@paul: During your repeat loop, did you have the "Ignore Case" option checked in the "If Variable" command? I only ask because it's not obvious from the code you posted.

 

Also, in what way are you finding the window title activation to be unreliable?

[terrypin]

Could I ask something here? When using the window scope with the title of Notepad, could you verify that there are no hidden windows with notepad in the title? During the scope testing, Macro Express does not differentiate between visible and invisible windows and on the newer versions of Windows, it seems to keep hidden windows around that contain the titles of programs you have recently run.

 

So, when a macro is firing that doesn't look like it's supposed to, could you please open Macro Express and bring up the window list (by editing scope > Add Window button) and look in the Hidden tab? I'll bet that you find a window title that will cause the scope to pass.

 

Chris: How much money was the bet? There are no fewer than 227 hidden windows (a subject in its own right, which we discussed at length a while back), but none contains the string 'Untitled - Notepad'.

 

But you've raised a point that now confuses me. Can you clarify your comment about ME Pro not differentiating please. If I use a command like Wait for Window Title, select the Visible tab, choose one of those listed, and select the Partial option, I expect that the command or activation will require a visible window containing that string. Are you saying that's no longer true? Will the command (or activation) occur even if there is one of my many hidden windows contining that string? If so, what's the point of categorising them as Visble and Hidden? And it would make the use of such commands highly impractical.

 

@paul: During your repeat loop, did you have the "Ignore Case" option checked in the "If Variable" command? I only ask because it's not obvious from the code you posted.

 

Also, in what way are you finding the window title activation to be unreliable?

 

--

Terry, East Grinstead, UK

[terrypin]

I confirm your finding. If the Scope is set to program instead of window, it all works correctly.

 

I then changed your sophisticated macro to repeat with all windows, check the variable set each time to see if it contains the case-insensitive characters notepad, and append to a 2nd variable if it does. After all windows had been evaluated, the displayed result was blank. I did this just to make sure that MEP wasn't finding some spurious window name containing notepad.

Variable Set to ASCII Char 13 to %tCr%
Variable Set to ASCII Char 10 to %tLf%
Repeat with Windows: All Windows: Store in variable %t[1]%
 If Variable %t[1]% Contains "notepad"
Variable Modify String %t[2]%: Append Text (%t[1]%%tCr%%tLf%)
 End If
End Repeat
Text Box Display:

Actually, I wonder whether this is a scope bug, or a window bug because I don't think I find windows activation as reliable in MEP as it was in ME3.

 

Shouldn't the test strictly have been

If Variable %t[1]% Contains "Untitled - Notepad" ?

 

--

Terry, East Grinstead, UK

[Yehnfikm8Gq]

I have to echo terrypin's comment about Hidden windows. We all work on the assumption that normally, things only happen with Visible windows. If you set scope for a Hidden window using the Hidden list that's a different matter.

[chris]

@terrypin:

The visible/hidden tabs are for your convenience only. When Mike originally wrote Macro Express, he included invisible windows in a lot of the functions. Unfortunately, Mike passed away several years ago, so I don't have the option of asking him why he did that. So, we maintained that in Macro 3 and Pro (it's never good to just change functionality without providing a backwards compatibility path). We added the tabs because people were confused why there were so many windows listed when they couldn't see them. But the underlying functionality has remained the same.

 

So, in answer to your question:

I expect that the command or activation will require a visible window containing that string. Are you saying that's no longer true?

I'm saying that that was never true. It's been that way since before I joined the company back in 1999.

 

As for the scope, after typing my previous post, I went through the code, just so I could see if there were any problems with picking up non-existent windows. When we search for a running window, we ask Windows itself for a full list of the windows. The OS takes a snapshot and returns that to us, which we then store, temporarily. The program then runs through every window title in that snapshot and checks to see if it matches. If it does, then the macro is queued up for launch. The snapshot of running windows is then discarded. That way, we don't run into situations where we're looking at old data.

 

The "program" option of the scope does the exact same thing, except it looks at the running processes instead of the window titles. The fact that changing the scope from Untitled - Notepad to Notepad.exe indicates that the code is sound as it runs through the same code, except for how the snapshot is acquired. So, we have a situation where Windows is telling us that some window title contains "Untitled - Notepad" somewhere in its title. This has become much more frequent with Windows Vista since the hint windows and taskbar preview windows get populated with the title of the window your mouse is hovering over. And, these windows don't get immediately destroyed when the program closes. Windows 7 is even worse about this. That's why I asked you to look at the hidden window list. Also, if you have the Macro Express editor up and the name of the macro contains the window title you're looking for, that can also cause it (because the titlebar of the editor contains the title you're looking for). This is a rather common occurrence.

 

So, since I last posted, I've made a change to the program. On the window scope, there is a new option to include hidden windows (which is enabled by default for backwards compatibility). When you get the upgrade (which should hopefully be out soon), you'll need to go back into the scopes and turn that option on.

 

I have to echo terrypin's comment about Hidden windows. We all work on the assumption that normally, things only happen with Visible windows. If you set scope for a Hidden window using the Hidden list that's a different matter.

I understand that viewpoint and don't think that I discount that. When we were developing Macro Express 3, we came across this situation. Specifically, we found that when we used the "Window Activate" command, we were activating hidden windows more times than not. We initially changed the code to only work on visible windows, but we found that there were many "big" customers who relied on the hidden window being activated (I think they knew for sure that that was what was happening), so removing that broke some things. So, we rewrote the code to give visible windows the priority by sorting them to be first in that snapshot I mentioned above. That way, it improved the reliability, but kept the Macro Express 2 customers happy. For backwards compatibility reasons, we maintained that in MEPro.

 

I hope that explains the way things work. If there are any more questions or comments, please let me know.

[terrypin]

So, since I last posted, I've made a change to the program. On the window scope, there is a new option to include hidden windows (which is enabled by default for backwards compatibility). When you get the upgrade (which should hopefully be out soon), you'll need to go back into the scopes and turn that option on.

 

Thanks Chris. That was a revelation. And may on its own have been behind some occasional inexplicable results. But I'm pleased to hear that you've fixed it and look forward to the next release.

 

Of course, this still leaves my reported bug unresolved. Can you reproduce it? And after your program patch?

 

--

Terry, East Grinstead, UK

[Yehnfikm8Gq]

Looks like we must thank terrypin for his excessive use of hotkeys for this revelation! Thanks to Chris for digging out the info. I have no idea if I can attribute any problems to this. It makes you want to stay away from short partials until the update.

 

Any explanations should go into the Help too.

[terrypin]

It makes you want to stay away from short partials until the update.

 

And Exact Match too in some cases it seems!

 

I just changed the scope of my one-line macro to "The window only needs to be running. The title is an exact match." With Notepad closed, the macro still gets activated.

 

Once again, I checked all Hidden windows and there is no 'Untitled - Notepad'.

 

I could waste the rest of the night testing this. But I already feel sure this is a bug and await Chris's confirmation he has reproduced it, if not isolated its cause.

 

BTW, at the risk of muddying the water further, my tests over the last couple of hours have been frustrated by having to use Tools > Restore Keyboard & Mouse Hooks several times. Does anyone (ISS or user) have any idea why this might be necessary so frequently? That aside, and accepting that these 'hooks' are not the sharpest in the tin, would it make sense for a future version to do this automatically every minute or so? I mean a feature that actually works. The present option 'Auto restore the keyboard and mouse hooks' clearly does not.

 

--

Terry, East Grinstead, UK

[chris]

Terry, could you export and post a copy of the macro that's giving you problems? I want to take a quick look inside it because we're not able to reproduce the issue.

 

As for needing to restore the hook frequently, it would have to do with whatever software is running on your computer. The only time this is needed is if there is another program which is hooking the keyboard and not forwarding the necessary messages back down the hook chain. Restoring the hooks pulls Macro Express out of the chain and then puts it back at the top so that it gets first priority. If you need to keep doing that then there is a program that is doing the same thing automatically.

 

The auto-restore hooks feature should work when you switch between windows. When a window becomes focused, Macro Express will pull the hooks out and put them back on top. But, if you have another program interfering with the hooks, then I can see why that isn't working very well.

[Yehnfikm8Gq]

...my tests over the last couple of hours have been frustrated by having to use Tools > Restore Keyboard & Mouse Hooks several times. Does anyone (ISS or user) have any idea why this might be necessary so frequently?

 

I've haven't even thought about hooks since ME was just ME. You seem to have so much trouble with "common" stuff. The obvious culprit outside of ME would be interactions with other software but I'm sure you've looked into that. My PC used to run with less than 30 processes. Now it's in the 50s. Every prog wants 2 or 3. I have a few older progs that now conflict - could be direct or indirect due to updates in system files.

 

After recent posts about hooks I trolled around. I had never realized that Opus uses hooks. I don't use any non-Windows shortcuts for it but it made me more aware of unseen conflicts.

[chris]

John, you'll find that a lot of programs use hooks, either locally or at the system level. When we upgraded our menu control components that we use in ME3 and MEPro, we found that the components installed their own hooks that caused problems activating our macros within our own program. Adobe Photoshop uses hooks in such a way that they completely bypass our ability to capture mouse movements within that program.

 

On Windows XP there was a program that you could use to find all the hooks throughout the system and what program created them. I wish Vista and 7 still had that because there are now so many programs that do so.

[paul]

@paul: During your repeat loop, did you have the "Ignore Case" option checked in the "If Variable" command? I only ask because it's not obvious from the code you posted.

 

Also, in what way are you finding the window title activation to be unreliable?

You're right in saying my pseudo-code doesn't show whether I used the Case Insensitive setting. But I think my description above the code does show this information, doesn't it? (Yes, I used Case Insensitive).

[paul]

Shouldn't the test strictly have been

If Variable %t[1]% Contains "Untitled - Notepad" ?

Huh? The Contains sub-command doesn't need the entire string. It's sufficient simply to look for the word "notepad". Your "Untitled - Notepad" string certainly does contain the characters "notepad", which is all I was interested in!

[paul]

@paul:Also, in what way are you finding the window title activation to be unreliable?

From time to time MEP misses activating a macro based on a window title. It's impossible to reproduce because the behaviour is not consistent.

 

For example, I have a macro that runs only when Roboform issues a request for its master password. In my normal environment, I have control over when Roboform issues this request because I deliberately log out of Roboform in order to force this very request to occur (this is the only way I have discovered to insert macro code specific to a particular Roboform passcard). In this circumstance, MEP seems 100% reliable. But sometimes Roboform issues a request for its master password simply because I haven't used Roboform for some time, and it logs itself out. In this circumstance, MEP is not 100% reliable.

[terrypin]

Terry, could you export and post a copy of the macro that's giving you problems? I want to take a quick look inside it because we're not able to reproduce the issue.

 

Chris: I wrote 4 one-line macros, the only difference being the scoping.

Here is a table showing the results for each macro when tested in the 3

states of Notepad.

 

Of the 12 possible combinations, 4 gave what seem to me to be incorrect

results, supporting my belief that there's a basic bug here somewhere.

 

 

							 RESULTS: Y=Runs, N=Doesn't run, (X)=ERROR

						 ........ STATUS of NOTEPAD .........
HOTKEY  MACRO NAME		   ON TOP   ONLY RUNNING	NOT RUNNING 
------  -----------------	------   ------------	-----------
Alt+1   Exact OnlyRunning	  Y		 Y			   Y (X)

Alt+2   Partial OnTop		  Y		 N (X)		   N

Alt+3   Partial OnlyRunning	Y		 Y			   Y (X)

Alt+4   Exact OnTop			Y		 N (X)		   N

 

I'm surprised you couldn't reproduce the problem as Paul and Cory did, but I've attached an MEX file as requested, containing all four.

 

ScopeBugTests.mex

 

FWIW, I'm using XP Pro.

 

Maybe hear from you tomorrow? (24 February 2010, 22:54 UK time here).

 

--

Terry, East Grinstead, UK

[Jeff Cleveland]

I entered a formal bug report [iSS8273] Macro Express Pro - Problem with Scope a couple of days ago. I have two macros that use Ctrl+1. The scope is shown in the attachment. About the time I installed 4.1.5.1 Ctrl+1 embedded in Eudora email program and Adobe Pagemaker quit working correctly. I have used Ctrl+1 often in these two utilities. By disabling one of the two macros the problem disappears.

 

With the discussion above, looking at the scope for my macro, I do not see that hidden windows would come into play as my macro will only work if the window is on top (can hidden windows be on top?).

 

Jeff

[Cory]

When using the window scope with the title of Notepad, could you verify that there are no hidden windows with notepad in the title?

If you read my post carefully you will see I was anticipaing your question. So as a simple way to avoid bumping into any hidden windows I set teh text to "XCVBBVCX". It's highly unlikely there are any windows with this string in it. So it's not a hidden window.

[paul]

I then changed your sophisticated macro to repeat with all windows, check the variable set each time to see if it contains the case-insensitive characters notepad, and append to a 2nd variable if it does. After all windows had been evaluated, the displayed result was blank. I did this just to make sure that MEP wasn't finding some spurious window name containing notepad.

Could I ask something here? When using the window scope with the title of Notepad, could you verify that there are no hidden windows with notepad in the title?

Did my earlier post, part of which I repeat above, also not address this very issue?

[paul]

I then changed your sophisticated macro to repeat with all windows, check the variable set each time to see if it contains the case-insensitive characters notepad, and append to a 2nd variable if it does. After all windows had been evaluated, the displayed result was blank. I did this just to make sure that MEP wasn't finding some spurious window name containing notepad.

Could I ask something here? When using the window scope with the title of Notepad, could you verify that there are no hidden windows with notepad in the title?

Did my earlier post, part of which I repeated above, also not address this very issue?

[terrypin]

Maybe hear from you tomorrow? (24 February 2010, 22:54 UK time here).

--

Terry, East Grinstead, UK

 

Chris: Any progress please?

 

I subsequently also sent a formal bug report but so far have had no ticket assignment.

 

--

Terry, East Grinstead, UK

[terrypin]

Chris: Any progress please?

 

I subsequently also sent a formal bug report but so far have had no ticket assignment.

 

--

Terry, East Grinstead, UK

 

Anyone else reported this bug? With any ticket assignment?

 

After several hours work pulling together the analysis above in post #18 I had hoped for some response, even if only 'We're on the case."

 

I followed it up later with a formal bug report a week ago but have still had no ticket.

 

--

Terry, East Grinstead, UK

[kevin]

Your bug report was merged with an existing bug report [iSS8273]. An issue with scope was found and fixed. We are working on a new version of Macro Express Pro that contains the fix.