lemming

You could try the ASCII File Process command instead of Split String, and see if you can get better performance. It supports tab-delimited text.

But yeah, 36 minutes is waaaay to long :-(

I did a Split String on CRLF on a TSV file with approx. 61k lines and it took 36 minutes so beware that I don't think this command was intended for any heavy lifting.

 

This is intolerable for my purposes so I'm going to have to find another solution and I think I'm just trying to push MEP to far at this point so it's no condemnation of MEP. I'm just using the wrong tool for the job.

OK, I've looked at your code. I believed you've triggered an obscure bug in MEP.

Part of the problem is that you have defined many of your variables twice, e.g. PIXEL and %PIXEL%

Delete all the duplicate variables that have %% in them. E.g. delete %PIXEL%

However, this doesn't solve the problem right away. You will now need to "redefine" all your variables within the code.

Basically, you have to delete any mention of a variable and re-pick it from the Variables box. Do NOT type in the variable name manually. E.g. for line 9, dbl-click to bring up the Variable Set Integer window. In the Destination Variable box, delete the word %PIXEL% completely. Click on the Variables box, select PIXEL and click OK. The word %PIXEL% will appear in the Destination Variable box again.

Yeah, I know it looks just the same, but that's how to fix this. You will need to repeat this process on every line which contains those variables. I encountered a similar problem some months ago when I accidentally defined a varible twice. Took me quite a while to find this solution.

-Lemming

Attachment...

On a side note, the environment variable %RANDOM% expands to a random decimal number between 0 and 32767. It's been supported since DOS days!

In MEP, you can read this value via Variable Set String -> Set from Environment Variable

Perhaps WP is intercepting all Alt and Ctrl key presses, thus preventing your macros from working.

As an experiment, try changing your hotkey to ; (the semicolon key alone), without any other key combo. If that works, then you'll need to find a hotkey combo without Alt and Ctrl.

Try XLS2CSV:

www.brothersoft.com/xls2csv-69870.html

Does exactly as its name says. Small, fast and free. You don't even need to have Excel installed. Once data is in CSV format, it is a lot easier to read with ME3/MEP.

I've been using XLS2CSV for a project where I need to extract data from dozens of xls files every day.

I don't use Crystal Reports. Does it display a "wait for" window or dialog box? If so, you can just stick an If Window Not Running command in a loop with appropriate delays. Something like:

Start Repeat 120 times
 If Window Not Running (Insert your window title here)
   Break
 Delay 500 ms
End Repeat

This example will run for a maximum of one minute (120 x .5 sec), and will exit the loop earlier if the specified window is not running. I'm assuming here that the "wait for" window will disappear.

Sure, I've converted a few ME3 scripts to AutoHotkey over the years. Both AutoIT and AutoHotkey are more powerful than Macro Express, although their learning curves are steeper. They don't come with nice GUIs either, unless you count SmartGUI Creator for AutoHotkey, or the Window Spy utility for AutoIT/AutoHotkey.

I thinking what you're really asking is: "Is there a program or script to automatically convert MEP scripts to AutoIT scripts?"

Unfortunately, the answer is no (so far). As Alan mentioned, you will need to go through your MEP scripts slowly, and translate the logic to AutoIT/AutoHotkey code.

-Lemming

Hey there,

I've put a lot of work into creating MEP files and was wondering if it's possible to convert them into AutoIT scripts or somehow make an executable file since our company won't let everyone install MEP on their machines.

Thanks in advance for any help.

Bummed,

Mike

Can you see the console window at all? I'm guessing the external program encountered some problem and was prompting for input.

If the console window stays on your screen long enough, you can just manually capture the text within, as a fail-safe.

To do so, right-click anywhere on the title bar or within the window. Then navigate to "Select All" by sending arrow keys and Enter (The exact sequence varies with Windows versions). Once all the text is selected (wait about 60 ms), send Enter again and the text will be copied to the clipboard.

I have macros on an SBS 2003 Server which stopped functioning a week or two ago, possibly after a system update. The macros contain an External Script or a Program Launch command which capture the output to a variable. The macro freezes at this command and will go no further. Command line Executable will still appear in the process list. Uncheck the console capture and the macro executes normally. The same macros work fine on other machines including a Windows Server 2003 Standard box.

 

ISS provided very in depth assistance but it appears the problem is with the server, not MEP. Has anyone experienced this or does anyone have any idea how I can fix it?

Yes, it can be slightly confusing if you've never used this before, or if you were formerly an ME3 user.

I would second Kevin's suggestion to avoid using the T variable.

So, if you set %InputData% as the "Variable array to receive result", the command will automatically create an array of numbered variables and fill them with each respective line from the file, i.e. Line 1 will go into %InputData[1]%, Line 2 into %InputData[2]%, Line 3 into %InputData[3]%, and so on. You do not have create those numbered variables yourself.

However, you do need to define %InputData% as an array. The first time you create a variable you should get a warning prompt saying "Variable currently not defined? Create now?" or similar. Click Yes, and in the variable properties box, make sure the "Create as an array" box is checked. In the Elements box, set a sufficiently large number. This needs to be slightly larger than the number of records you will process.

-Lemming

Thanks for your reply Paul. That part I understood but the area which I'm having issues with is choosing how the variables are defined (I hope I'm using the right terminology). In other words I have no clue how to fill this section out as seen in the image -->

Thus far I've set the following as....

 

Begin processing on record: 1

Variable array to receive results: T1,T2,T3,T4 etc

 

However, what do I set for the properties for the above variables and do I need more variables?

If you're not averse to using an external program, you can use Sed to extract the digits with just two Sed commands:

/status/!d

s/status \([0-9]\{7,8\}\) .*$/\1/

The first line removes any lines that do not contain the work "status". The second line extracts any 7 or 8 digits after the word "status".

Sed is part of GNU utilities for Win32 (unxutils.sourceforge.net), which includes other classic Unix utilities like Grep and Awk. These 3 support regular expressions, and I got them to work with Macro Express Pro.

QUICK START

1. Download Sed numbers.mxe and Sed numbers.zip from here

2. Unzip Sed numbers.zip and put the 3 files (sed.exe, Sed_Script.txt, NeVeR.txt) into a folder which MEP can read. Take note of the folder path. Also note that Windows 7/Vista may block access to certain folders.

3. Import Sed numbers.mex into Macro Express Pro

4. Change the WorkingFolder variable in the script to the full path of the folder which contains the 3 files from Step #2

5. Run the script, and it will process NeVeR.txt and report the results

The Sed_Script.txt file has comments which explain the Sed commands. I also learned a lot from this site:

Famous Sed One-Liners Explained

Sed numbers.mxe

Sed numbers.zip

Hi Terry, I've been testing GNU utilities for Win32 (unxutils.sourceforge.net), which includes classic Unix utilities like Grep, Sed, and Awk. These 3 support regular expressions, and I got them to work with Macro Express Pro.

For your text problem, I wrote an MEP script which calls Sed.

The MEP script is only 6 lines long - all it does is set a working folder, launch Sed, and report the results.

The Sed script which is doing the actual heavy lifting is only 3 lines long! (excluding comments)

All thanks to regex ;-)

QUICK START

1. Download Run Sed.mex and Run Sed.zip from here

2. Unzip Run Sed.zip and put the 3 files (sed.exe, Sed_Script.txt, ExampleForCory.txt) into a folder which MEP can read. Take note of the folder path. Also note that Windows 7/Vista will block access to certain folders.

3. Import Run Sed.mex into Macro Express Pro

4. Change the WorkingFolder variable in the script to the full path of the folder which contains the 3 files from Step #2

5. Run the script, and it will process ExampleForCory.txt and report the results

The Sed_Script.txt file has comments which explain the Sed commands. I also learned a lot from this site:

Famous Sed One-Liners Explained

-Lemming

Run Sed.mex

Run Sed.zip

Hi Paul, that's an interesting use for "set to the length of variable". I'll have to remember that one.

It's good to have alternative approaches to the problem!

An alternative (better?) method might be:

set string t10 to your gappy string
repeat until n1 <> n1 -- endless loop
 set integer n1 to the length of variable t10
 variable modify string: replace "  " in t10 with " " --replace all instances
 set integer n2 to the length of variable t10
 if n2 = n1 repeat exit --because nothing changed, we're done
end repeat

This short script demonstates how to remove extra spaces from data, for example:

Before:
14770512        7       0       9       1  
this   is   too  gappy?
testing    testing   1       0       5       0

After:
14770512 7 0 9 1 
this is too gappy?
testing testing 1 0 5 0

The script makes use of the "Set Variable to Position of Text" command, which is essentially a search command. The script repeatedly searches for 2 consecutive spaces and replaces them with 1 space.

Compress spaces demo.mex

Since you've already tried AutoIt, might I suggest AutoHotkey? It's got lightning-fast built-in commands called ImageSearch and PixelSearch, which do exactly what their names suggest.

They're so fast that I can even use them for scripting games.

Just for fun, I wrote a macro to write all the pixels on my 1280x1024 screen to a file. I then rewrote the macro in AutoIt.

 

MEP takes 3.5 minutes to run, while AutoIt takes 2.25 minutes; so AutoIt is not that much quicker in this case.

Just caught this off Slashdot:

Linux: Open Source OCR That Makes Searchable PDFs

on Thursday July 22, @03:21PM

An anonymous reader writes "In my job all of our multifunction copiers scan to PDF but many of our users want and expect those PDFs to be text searchable. I looked around for software that would create text searchable pdfs but most are very expensive and I couldn't find any that were open source (free). I did find some open source packages like CuneiForm and Exactimage that could in theory do the job, but they were hard to install and difficult to set up and use over a network. Then I stumbled upon WatchOCR. This is a Live CD distro that can easily create a server on your network that provides an OCR service using watched folders. Now all my scanners scan to a watched folder, WatchOCR picks up those files and OCRs them, and then spits them out into another folder. It uses CuneiForm and ExactImage but it is all configured and ready to deploy. It can even be remotely managed via the Web interface. Hope this proves helpful to someone else who has this same situation."

http://linux.slashdot.org/story/10/07/22/1852234/Open-Source-OCR-That-Makes-Searchable-PDFs

I haven't tried it myself, but it sure looks promising. It's a Live CD though, so you would need a dedicated PC/notebook, or at least a VPC/VMware session to keep it running. The setup process looks straightforward. Desc from their main website at http://www.watchocr.com:

"WatchOCR is free OCR server for PDFs. Based on Knoppix, WatchOCR uses cuneiform, and exactimage to create text searchable PDFs from image only PDFs. Using the web interface, WatchOCR can be remotely configured to monitor a watched folder for newly scanned PDFs for OCR conversion. Leveraging the power of a liveCD distro, WatchOCR is the fastest way to get an OCR service up and running for your network."

Yeah, the Aero theme can create problems for Get Pixel Color and similar commands. This is a known problem caused by 3D rendering. You'll face similar difficulties with many games and Flash programs.

Apparently, when there is rendering, the pixel color will change slightly every few seconds. You won't notice the change, but it is enough to break most scripts.

I suggest you create a "pixel dump" test script which will just sample one particular pixel repeatedly, say 10 times in one second. Then report all 10 values in a dialog box. If you see a few different numbers, you could adjust your script to check for those too.

Mine works surprisingly quick so it isn't inherent. I seem to remember someone once saying that the Aero themes in Vista and 7, could that be it?

%N1%, %T1%, %D1% are the only arrays available for Macro Express 3.x. You cannot specify %N[1]% in ME3; you're probably thinking about Macro Express Pro.

To add to the confusion, if you import an ME3 script into MEP, all the %N1%, %T1% variables get converted to %N[1]%, %T[1]%

Anyway, I'm glad there are named variables in MEP!

Your variable names are a bit odd! Instead of %N1%, %T1%, etc., you ought to be using %N[1]%, %T[1]%! Your %N1% is a different variable to %N2%, whereas I'm using the first 2 elements of an array called N or T.

HOW THE SCRIPTS WORK

The two external programs are doing all the heavy lifting; these scripts merely launch them.

The first part of the script launches IrfanView with the following parameters:

/capture=2 /crop=(%Xposi%,%Yposi%,180,150) /convert=%ScreenCaptFile%

On a command line (command prompt), this would be equivalent to:

irfanview.exe /capture=2 /crop=(4,21,180,150) /convert=CapturedSegment.jpg

/capture=2 means capture the foreground window

/crop=(4,21,180,150) means crop a 180x150 segment starting at 4,21

/jpgq=81 sets Jpg quality at 81

/convert=CapturedSegment.jpg means convert the captured image to Jpg

The “Capture Active Win” script is almost the same as the "Capture 180x150" except it leaves out the crop parameter (no cropping), so it is basically:

irfanview.exe /capture=2 /convert=CapturedWindow.jpg

Both versions of the script then launch md5.exe:

md5.exe CapturedSegment.jpg

which will calculate and spit out the MD5 checksum for the file. An MD5 checksum serves as a “fingerprint” to uniquely identify a file’s contents. If you know the MD5 checksum for two files, you can get a definitive answer to the question: Are these two files different?

Other stuff to note

The scripts will save captured images to the folder listed in the %USERPROFILE% Environment Variable. On Windows Vista/7 this usually means C:\Users\{your username}, while on Windows XP/2000, it is usually C:\Documents and Settings\{ your username}.

The captured images are in Jpg format. You can view them using any image viewer, including IrfanView.

The scripts have built-in benchmarking; they will report their Start and End times.

A full list of IrfanView command-line options can be obtained from the IrfanView help file, which is installed along with the program.

More info about MD5 at: Wikipedia.org/wiki/MD5

More info about checksums at: Wikipedia.org/wiki/Checksum

INSTALLATION GUIDE AND HOW-TO

Please do not skip any of these steps.

1. Download and install IrfanView from www.irfanview.com. Take note of the installation folder, which is usually C:\Program Files\IrfanView\

2. Download md5.zip from www.fourmilab.ch/md5/

3. Extract md5.exe from the zip file. There is no installer; just copy md5.exe into C:\Program Files\

4. Download the scripts below (attached) and import the scripts into Macro Express Pro. If IrfanView and md5.exe are not in their default folder locations, pls modify the scripts accordingly.

5. Press Ctrl-Alt-9 to capture a segment of the active window, or Ctrl-Alt-0 to capture the full active window.

6. The script will report the MD5 checksum for the captured segment or active window, and the location for the saved file.

Image capture and MD5 checksum calc.mex

I created these scripts partly in response to feature requests from Strahan et al. who asked for an “image capture and compare” solution.

These scripts will capture images and generate MD5 checksums, which you can then use for comparison.

There are already excellent programs for capturing screen images and calculating MD5 checksums, so I don’t want to reinvent the wheel.

For these scripts, I am using IrfanView and md5.exe. Both are free and fast programs. I wanted to create an MEP-only solution, but my initial efforts showed that MEP is too slow for this purpose.

On my 3-year-old notebook, these scripts (using external programs) take 1.1 sec to execute, which I think would not bother most users. An MEP-only solution would probably be 5 to 10 times slower.

These scripts can serve as building blocks for screen logging, image comparison, or image matching scripts. Please modify to suit to your purposes. A full installation guide and how-to follows in the next post.

I am actually working on an image search solution for MEP, which is a tougher problem. I’ll probably make use of IrfanView again.

Hi, what exactly are you planning to do with the file? Are you moving/copying to another folder? Launching it?

If so, there is a more straightforward way.

The Desktop is merely another folder in your system. It is usually located in C:\Users\"Your Username"\Desktop

Once you've figured out the exact path for your Desktop, You can use commands like If File Exists to check if the file is present. After that, there are the File/Folders commands that allow you to move, copy, delete, rename, etc.

If you need to open the file using a specific program, you can try the Program Launch command. Most programs will accept a filename as a parameter, e.g.

Program/Path name:

C:\Program Files\IrfanView\i_view32.exe

Program Parameters:

"C:\Users\Your Username\Desktop\Image file you want to open.jpg"

e.g. 2:

Program/Path name:

notepad.exe

Program Parameters:

"C:\Users\Your Username\Desktop\Text file you want to open.txt"

No mousing needed ;-)

Hello all,

 

I wanted to know if ME Pro can search the desktop folder for a specific file(using Repeat) and once it comes up with the file, drag the mouse to the file, click once and drag the file(unopened) to another area of the screen?

 

I have been trying a few things and all have failed me. I am probably missing something.

 

Here is the sequence:

 

A)Directory command senses added file to Desktop folder

B)Repeat through the folder to find the newest file

C)???? This is the part that I cannot figure out??? How can you drag the mouse to the file?

D)Single left click and hold down mouse

E)Keep the left mouse button down

f)Provide coordinates to where the mouse should go(while the left mouse is held down)

g)Move mouse with file to coordinate

h)Release mouse

 

Pat

I wrote an Image Search script a few years ago that does what you describe.

Look for the "Image Search For Macex" topic from Jan 2006 in the Macro Express 3.x Third Party Tools category. The "Acquire Search Pixels" macro is the one that does it.

-Lemming

Anyone know of software that will basically take a screen print, map all the pixels and their colors and then spit that out into a file? I've setup a macro to scan for 30 contiguous and unique pixels. Unfortunately, it takes forever. I'm trying to speed up this process.

Ugh, intentional obsfucation.

Are the documents in image format, or do they merely have DRM restrictions that limit copying/editing/printing? If it's the latter, you may be able to get past the DRM by using a non-Adobe pdf reader:

http://www.whoismadhur.com/2009/04/03/5-free-alternatives-to-adobe-reader/

For more advanced DRM removal and password recovery, there's:

Advanced PDF Password Recovery from Elcomsoft

Yeah it does seem a long way around doesn’t it? I thought of that straight off but it's not an option. These are being generated by a big payroll company who has a competing service so they are purposefully putting it into a format to make it difficult to automate.

Hi Terry, thanks for pointing out the typo, and thanks for testing the scripts. Yes, I would agree they are slow. I believe it is a limitation of MEP, and not my coding ;-) I am guessing that ME and MEP are slow at comparisons, which would really impact comparison sort algorithms. Bubble, Gnome and Insertion Sort are all comparison sorts and they have to compare items thousands of times a second.

Anyway, I wrote the scripts mainly as a proof-of-concept and to improve my understanding of sorting algorithms. Just goes to show that once you have understood an algorithm, you can implement it even with "rudimentary" script languages.

For heavy-duty sorting, I would recommend using TextPad like you did, or Windows' built-in sort.exe. Both are faster than my code by many orders of magnitude.

I'm actually finishing up a Comb Sort script, but am still not happy with its performance. It is faster than Insertion Sort (about 50% faster), but it is also a comparison sort, so I don't think I can optimize it any more.

As for the error messages, are the paths/filenames are correct in your script? Also, if you're using Vista/Win 7, the OS will not let scripts access certain folders, e.g. C:\ is usually off-limits. All four scripts are almost identical. The only difference is the %SourceFile% they point to, and the size of the array for sorting.

- Lemming

Thanks for sharing that and presenting it so well.

 

I've installed and tested the macros, but they're incredibly slow, yes?

 

Using my text editor (TextPad), the 1000 item file took only milliseconds for the sort. But the macro took 3.5 s for the 500 and over 26 s for the 1000.

 

The 100 macro ran successfully.

The 250 and 500 macros gave error messages at lines 84 and 83 respectively: "The file could not be opened.

I/O error 105"

The 1000 macro would not finish (had to be terminated manually).

 

 

 

I think that should be

Change the lines that start with ... “Variable Set String %ResultsFile%”.

 

--

Terry, East Grinstead, UK

I've implemented 3 sorting algorithms for Macro Express Pro - Bubble Sort, Gnome Sort, and Insertion Sort.

You can search the forum for those terms. However, their performance is poor if you need to sort more than 200 items. I believe it's a limitation of MEP and not my coding ;-)

BTW, your use of Windows' sort.exe already works very well. It is faster than my code by many orders of magnitude. Anyway, I only wrote the scripts to improve my understanding of sorting algorithms.

Using sort.exe may not be "elegant" but it certainly works and it is fast. Plus, sort.exe is installed with every copy of Windows (I believe it dates back to DOS days!).

-Lemming

I'd thought about using a bubble sort like that, but I haven't actually written a real bubble sort since I was in highschool. It's amazing how quickly you become dependent on functions like sort() when they're built right into the languages you use. I sort arrays all the time - without actually having written a function for it in over 20 years (yeah, highschool was a long time ago). When I thought about doing it this morning I quickly discarded the idea because I wasn't confident I'd even remember how it goes, much less how to make it work in Macro Express scripting. (Back then, I think I wrote it in Turbo Pascal.)

 

Maybe I discarded it too quickly. It's always good for the brain to go back and study the fundamentals.

 

If you decide today is rainy enough for you, then I'll gladly and happily skip the work because I'm not sure it's raining quite that hard here (especially since I already put together something that works). But if I do get a chance to go at it some time soon I'll be sure to post my results here.

 

(Actually, I'm looking forward to the challenge now. If only I could justify shoving the rest of my work aside for an hour or two.)

 

wayne

 

(Oh, and yes: It was your post about sorting that got the ball rolling. So thanks!)