Challenge: Macros to drag without the need to click or hold down a mouse button

[acantor]

In my work with people with disabilities, I often assist individuals for whom clicking is difficult. For many of them, dragging is even more challenging (and sometimes impossible) especially if they have missing fingers and/ or limited finger strength and coordination.

 

There's a feature built into Windows called "ClickLock." Once enabled, a user uses the mouse to slide the pointer to the start location, and briefly holds down the left mouse button. This causes the button to lock in the down position. The button stays locked until the user clicks the button.

 

ClickLock is impractical for many of my clients because they are must hold down the mouse button to lock, and click to unlock. They need a solution that doesn't require clicking or holding down a mouse button.

 

I've used Macro Express to build scripts that click and drag without the need to actually click or drag. I've experimented with several methods. Some methods work nicely. Others were rejected by my clients as too complicated. I'm not sure what's the best approach! So I'm curious what other Macro Express fans might dream up.

 

If you accept this challenge...

 

Assume users have a mouse and can easily slide the pointer around the screen; but can't click and can't drag.

 

Test your macro (or macros) to ensure you can do four tasks. Without physically clicking or dragging a mouse button...

 

1.    Grab a non-maximized window by its title bar, and move it elsewhere on the screen.

2.    Select a block of text within a document.

3.    Drag a selected block within the same document.

4.    Drag a selected block from one window, and move (or copy) it to a different window.

 

Choose whatever macro activation method makes sense to you. But the activation should not involve mouse buttons.

 

I'll post a Macro Express solution that several of my clients actually find useful.

[rberq]

I have a macro that zooms Firefox screens, to make the text bigger.  The macro runs all the time Firefox is active.  It checks the mouse position once every second.  If I move the mouse to the extreme left margin of the screen, the macro types the Firefox key sequence to zoom in.  The longer the mouse stays at the extreme left, the more times the zoom-in happens.  If I move to the extreme right, it types the sequence to zoom to actual size.  If the mouse is at extreme right, but has not moved since the previous observation, then zoom to actual size is not done, since it was already done previously. 

 

To take one of your scenarios – moving a block of text within a document – I would try something similar.  (Design below is based on Microsoft Word.) 

1) Mouse monitor macro running all the time, but takes no part in what follows except activating another macro.

2) Mouse monitor activates the block-move macro based on user moving mouse to extreme left in top half of screen.  (Bottom half of screen, and top/bottom halves at extreme right reserved for the other three scenarios.)

3) Block-move macro does its own monitoring of mouse position, probably once every second. 

4) User moves mouse to first character of text block to be moved, and holds position.  Macro detects zero mouse movement for two seconds, presses left mouse button, flashes text box on screen briefly to inform user.  (Text box may not be needed if user watches for cursor to change as a result of mouse button being pressed.)

5) Block-move macro continues to monitor mouse position every second. 

6) User moves mouse to end of text block to be moved, and holds position.  Macro detects zero mouse movement for two seconds, releases left mouse button, flashes text box on screen briefly to inform user.  At this point the text block has been highlighted.     

7) User moves mouse to middle of highlighted text block, holds position as before.  Macro presses left button, user drags block to target position.  After two seconds of no movement, macro releases left mouse position.  Text block should now be in its new position.     

 

Well, that’s the general idea.  Some refinements would no doubt be needed.  For example, it is often difficult for me to hold the mouse totally still, so a few pixel’s of tolerance might be built in.  I think some variations would work for the other three scenarios.  Macros would be very much tailored to the specific scenario. 

 

You didn’t mention whether the user could use the keyboard in any way, so I assumed he could not.  All the above would be more reliable, and easier to code, if the user had some other way (keystrokes) to signal pressing/releasing the mouse buttons.   If no keyboard allowed, then perhaps no mouse movement for two seconds, followed by very rapid movement in one direction or another to signal which button function the macro will activate.  That would require high-CPU monitoring of mouse position, not just once per second.  I have written a couple macros that do that sort of mouse tracking, but then it’s a (not too difficult) matter of the user learning to move at the correct speed. 

[acantor]

Assume a user is capable of operating a keyboard to type (at least a little) and press hotkeys.

 

Individuals who have difficulties handling a mouse often have difficulties typing. Many of my clients use Dragon Professional, the speech recognition application. (In fact, much of my work involves teaching people to use Dragon, and customizing it.) Driving a computer entirely by voice is possible, but challenging. The ability to use a keyboard to perform at least some tasks is really helpful.

[rberq]

1 hour ago, acantor said:

Assume a user is capable of operating a keyboard to type (at least a little) and press hotkeys.

 

OK, then revise my design by eliminating 99.9 percent of it.  If the user can press hotkeys, and move the mouse, then the mouse button presses and releases and clicks can be done by simple macros.  If the user can NOT press hotkeys, can Dragon activate the keys that then activate the macros? 

 

I don't see where the problem is -- it's awkward for the user, but doable with minimal macros.  What am I missing?

[acantor]

Hi rberq,

 

You're not missing anything. One could easily create a hotkey macro to lock the mouse, and a second hotkey macro to unlock it. Maybe that's the best solution. That's the way I've done it in the past. Here's part of a keyboard layout I developed for a one-toe typist. Note the drag lock/unlock buttons:

 

 

But I'm not convinced this is the best approach. Approaches vary in convenience, reliability, and ease-of-use. I once introduced dragging macros to someone who flat-out refused to use them! I've also provided dragging macros to users who couldn't remember how to make them work. The reason, I'm guessing, is that the approach I implemented was too unfamiliar.

 

One of my early attempts relied on the "Variable Set Handle" instruction. It was supposed to make dragging from window to window more reliable. It seemed like a good idea at the time, but it wasn't!

[acantor]

The following solution works well for two people. The drag is initiated by a single key press. The drag stops automatically after %DragTime% seconds. I arrive at the value through trial-and-error experimentation with the client. For one, it was four seconds. For the other, it was five seconds.

 

To use the macro, here are the steps.

 

1. Move the mouse pointer to the start position of the drag.

2. Press a hotkey to activate the macro (e.g., F12).

3. The macro locks the left mouse button. The user has %DragTime% seconds to move the mouse pointer to the end position.

4. The macro releases the left mouse button.

 

There are two versions: "basic," and with bells-and-whistles.

 

The bells-and-whistles version has three enhancements that I think are useful:

 

1. Wiggle the mouse cursor to signal the macro is active.

2. Wiggle the mouse cursor to signal the macro is done.

3. Press the Ctrl key to exit early. So instead of waiting four (or five or whatever) seconds, touching the Ctrl key

causes the timing loop to exit immediately.

 

Basic version:

 

// Minimal effort, no-click mouse drag macro.
 
// HOW TO TWEAK THE MACRO
// Decide how many seconds a user needs to drag. (%DragTime% = 3 seconds? 4 seconds? 10 seconds?)
 
// HOW TO USE THE MACRO
// 1. Hover the mouse pointer over the starting position of drag. Don't click.
// 2. Activate the macro.
// 3. Move mouse pointer to its final position within %DragTime% seconds.
// 4. Voilà!
 
Variable Set Integer %DragTime% to 4 // Number of seconds to drag
 
Mouse Left Button Down // Start dragging
 
  Delay: %DragTime% seconds
   
Mouse Left Button Up // Stop dragging

<COMMENT Value="Minimal effort, no-click mouse drag macro." _BACK="0080FFFF"/>
<COMMENT/>
<COMMENT Value="HOW TO TWEAK THE MACRO" _BACK="0080FFFF"/>
<COMMENT Value="Decide how many seconds a user needs to drag. (%DragTime% = 3 seconds? 4 seconds? 10 seconds?)" _BACK="0080FFFF"/>
<COMMENT/>
<COMMENT Value="HOW TO USE THE MACRO" _BACK="0080FFFF"/>
<COMMENT Value="1. Hover the mouse pointer over the starting position of drag. Don't click." _BACK="0080FFFF"/>
<COMMENT Value="2. Activate the macro." _BACK="0080FFFF"/>
<COMMENT Value="3. Move mouse pointer to its final position within %DragTime% seconds." _BACK="0080FFFF"/>
<COMMENT Value="4. Voilà!" _BACK="0080FFFF"/>
<COMMENT _BACK="00FFFFFF"/>
<VARIABLE SET INTEGER Option="\x00" Destination="%DragTime%" Value="4" _COMMENT="Number of seconds to drag"/>
<COMMENT/>
<MOUSE LEFT BUTTON DOWN _COMMENT="Start dragging"/>
<COMMENT/>
<DELAY Flags="\x01" Time="%DragTime%"/>
<COMMENT/>
<MOUSE LEFT BUTTON UP _COMMENT="Stop dragging"/>

 

Bells-and-whistles version:

 

// Minimal effort, no-click mouse drag macro.
 
// HOW TO TWEAK THE MACRO
// Decide how many seconds a user needs to drag. (%DragTime% = 3 seconds? 4 seconds? 10 seconds?)
 
// HOW TO USE THE MACRO
// 1. Hover the mouse pointer over the starting position of drag. Don't click.
// 2. Activate the macro.
// 3. Move mouse pointer to its final position within %DragTime% seconds, or press Ctrl to quit sooner.
// 4. Voilà!
 
Variable Set Integer %DragTime% to 4 // Number of seconds to drag
Variable Modify Integer: %DragCycle% = %DragTime% * 10 // Test for Ctrl more often than %DragTime% seconds
 
Repeat Start (Repeat 8 times) // Wiggle the mouse pointer to show the macro is active.
  Mouse Move: 2, 0 Relative to Last Position
  Delay: 20 milliseconds
  Mouse Move: 0, 2 Relative to Last Position
  Delay: 20 milliseconds
  Mouse Move: -2, 0 Relative to Last Position
  Delay: 20 milliseconds
  Mouse Move: 0, -2 Relative to Last Position
  Delay: 20 milliseconds
End Repeat
 
Mouse Left Button Down // Start dragging
 
  Repeat Start (Repeat %DragCycle% times) // E.g., if drag time = 4 second, loop and test for Ctrl 40 times
    Get Key State "Control" into %CtrlKeyState% // Check if Ctrl is pressed
    If Variable %CtrlKeyState% Equals "TRUE"
      Repeat Exit
    End If
    Delay: 100 milliseconds
  End Repeat
   
Mouse Left Button Up // Stop dragging
 
Repeat Start (Repeat 8 times) // Wiggle the mouse pointer to show the macro's work is done.
  Mouse Move: 2, 0 Relative to Last Position
  Delay: 20 milliseconds
  Mouse Move: 0, 2 Relative to Last Position
  Delay: 20 milliseconds
  Mouse Move: -2, 0 Relative to Last Position
  Delay: 20 milliseconds
  Mouse Move: 0, -2 Relative to Last Position
  Delay: 20 milliseconds
End Repeat

<COMMENT Value="Minimal effort, no-click mouse drag macro." _BACK="0080FFFF"/>
<COMMENT/>
<COMMENT Value="HOW TO TWEAK THE MACRO" _BACK="0080FFFF"/>
<COMMENT Value="Decide how many seconds a user needs to drag. (%DragTime% = 3 seconds? 4 seconds? 10 seconds?)" _BACK="0080FFFF"/>
<COMMENT/>
<COMMENT Value="HOW TO USE THE MACRO" _BACK="0080FFFF"/>
<COMMENT Value="1. Hover the mouse pointer over the starting position of drag. Don't click." _BACK="0080FFFF"/>
<COMMENT Value="2. Activate the macro." _BACK="0080FFFF"/>
<COMMENT Value="3. Move mouse pointer to its final position within %DragTime% seconds, or press Ctrl to quit sooner." _BACK="0080FFFF"/>
<COMMENT Value="4. Voilà!" _BACK="0080FFFF"/>
<COMMENT _BACK="00FFFFFF"/>
<VARIABLE SET INTEGER Option="\x00" Destination="%DragTime%" Value="4" _COMMENT="Number of seconds to drag"/>
<VARIABLE MODIFY INTEGER Option="\x02" Destination="%DragCycle%" Value1="%DragTime%" Value2="10" _COMMENT="Test for Ctrl more often than %DragTime% seconds"/>
<COMMENT/>
<REPEAT START Start="1" Step="1" Count="8" Save="FALSE" _COMMENT="Wiggle the mouse pointer to show the macro is active."/>
<MOUSE MOVE Option="\x03" X="2" Y="0" _PROMPT="0x000A"/>
<DELAY Flags="\x02" Time="20"/>
<MOUSE MOVE Option="\x03" X="0" Y="2" _PROMPT="0x000A"/>
<DELAY Flags="\x02" Time="20"/>
<MOUSE MOVE Option="\x03" X="-2" Y="0" _PROMPT="0x000A"/>
<DELAY Flags="\x02" Time="20"/>
<MOUSE MOVE Option="\x03" X="0" Y="-2" _PROMPT="0x000A"/>
<DELAY Flags="\x02" Time="20"/>
<END REPEAT/>
<COMMENT/>
<MOUSE LEFT BUTTON DOWN _COMMENT="Start dragging"/>
<COMMENT/>
<REPEAT START Start="1" Step="1" Count="%DragCycle%" Save="FALSE" _COMMENT="E.g., if drag time = 4 second, loop and test for Ctrl 40 times"/>
<GET KEY STATE Key="Control" Dest="%CtrlKeyState%" _COMMENT="Check if Ctrl is pressed"/>
<IF VARIABLE Variable="%CtrlKeyState%" Condition="\x00" Value="TRUE" IgnoreCase="TRUE"/>
<REPEAT EXIT/>
<END IF/>
<DELAY Flags="\x02" Time="100"/>
<END REPEAT/>
<COMMENT/>
<MOUSE LEFT BUTTON UP _COMMENT="Stop dragging"/>
<COMMENT/>
<REPEAT START Start="1" Step="1" Count="8" Save="FALSE" _COMMENT="Wiggle the mouse pointer to show the macro's work is done."/>
<MOUSE MOVE Option="\x03" X="2" Y="0" _PROMPT="0x000A"/>
<DELAY Flags="\x02" Time="20"/>
<MOUSE MOVE Option="\x03" X="0" Y="2" _PROMPT="0x000A"/>
<DELAY Flags="\x02" Time="20"/>
<MOUSE MOVE Option="\x03" X="-2" Y="0" _PROMPT="0x000A"/>
<DELAY Flags="\x02" Time="20"/>
<MOUSE MOVE Option="\x03" X="0" Y="-2" _PROMPT="0x000A"/>
<DELAY Flags="\x02" Time="20"/>
<END REPEAT/>

 

These are the best solutions I've come up with to date, but I'll continue to tinker!

 

I wish there was a way to use the same hotkey to activate the macro and to quit early... but I don't think we can do that, at least not easily, if we're using a Function key to activate a macro.