Script to perform Regex operations?

[terrypin]

Has anyone used ME3/MEP to do the sort of text editing you can do with Regular Expressions? If so, to avoid much potential re-invention, I'd appreciate your sharing please.

 

As a solid example, I have a large text file, placed in a variable, and I want to find and copy occurrences of strings like this:

<Placemark>
                       <name>DisneyLand - Balloon</name>

 

That's a Return (EOL) directly after '<Placemark>', followed by an indeterminate number of spaces, before '<name>'.

 

I could do this by opening the whole file in my text editor (TextPad) and using its Find/Replace tool with a target using Regex, like this:

<placemark>\n +<name>(.*)</name>

 

The '\n' finds the EOL, the ' +' finds one or more spaces, and the '(.*)' finds any number of normal characters.

 

TextPad highlights all of that string ready for copying to the clipboard or whatever. Then I repeat the Find to locate subsequent occurrences.

 

What's the neatest way to tackle that in ME Pro please? The part that has me flummoxed at the moment is including the unknown number of spaces in the result. I can envisage a lot of code. It may even be worth pasting the whole lot into TextPad after all.

 

--

Terry, East Grinstead, UK

[paul]

MEP has, as far as I know, no Regex capabilities. So whatever solution you adopt has to be done via an external piece of software.

 

AutoIt provides 2 regex built-in commands, StringRegExp and StringRegExpReplace.

See RegEx in AutoIt for more details.

 

In any case, you'll have to write the logic yourself.

[terrypin]

MEP has, as far as I know, no Regex capabilities. So whatever solution you adopt has to be done via an external piece of software.

 

AutoIt provides 2 regex built-in commands, StringRegExp and StringRegExpReplace.

See RegEx in AutoIt for more details.

 

In any case, you'll have to write the logic yourself.

 

Thanks Paul. As I'm still a raw rookie with AutoIt, I reckon I'll try TextPad as my next step. I was rather hoping someone might have already developed macros for a few basic subroutines.

 

--

Terry, East Grinstead, UK

[Cory]

Terry I do this stuff all the time and it's very simple. Sometimes it's and download of an HTML file, sometimes it's XML which uses tags like HTML, or even text file extracts from reports and such. There are several techniques I use and they vary widely depending on the application and consistency of the data structure. In some cases I've used the Split command with great success. Other times I've had to chew thru the file item by item. I have written several examples here abou thow to do this so you might want to read a few of my posts on the subject.

 

But in the example you provided here's how I would do it. Assume %Source% is a large string var with everything.

1. First set the integer %Position% to the position of the text "<name>" in %Source%.
   
2. Next add 5 to %Position%.
   
3. Now delete part of %Source% from 1 to %Position%.
   
4. Next set %Position% to the location of "</name>.
   
5. Subtract 1 from %Position%
   
6. Copy Part of text from 1 to %Position% from %Source% to %My Text%.
   

Now odds are you need to grab several things so this might be in a repeat. I usually but an "If %Position%=0 then break" section in there. IE if it's zero there are no more to be found. And of course you can include a counter and load an array using the counter as the element pointer. But I left all that out for clarity. In my experience I can plow thru huge amounts of data in a small amount of time.

 

If you have several instance of <name> in the source check out the Split command. In your example if there were hundreds of theme parks I would split on "<name>" then run thru the array and deleting everything after, and including "</name>". If you give me a better example of the data you're playing with I can give you better techniques.

[terrypin]

Thanks Cory, appreciate the help. However, as mentioned, the part that's challenging me isn't covered in your reply. I'm comfortable getting strings using the technique you described. But I need to parse ALL of the example I gave, not just the easy bit inside the <name> xxx </name> tags. I need <Placemark> too, because there are other sections of this KML file that contain strings like these:

 

<Document>	<name>xxxxxxxxxxxx</name>

 

<Folder>            <name>xxxxxxxxxxxx</name>

 

<Path>   <name>xxxxxxxxxxxx</name>

 

etc, which I do not want to process.

 

It's the varying number of spaces between <Placemark> and <name> that tempt me to switch to my text editor and use Regex.

 

--

Terry, East Grinstead, UK

[Cory]

Maybe I'm not understanding you. You say "parse" but maybe we're not in agreement as to the meaning. When I parse something I split it into logical chunks. Is that what you want to do? Variable number of spaces is not a problem as you suggest because I am setting a variable number of places with the Set Integer command. Perhaps you could say it again but this time be less explicit and give an example. Tell me what you want me to do as if I was a person who was going to manually open these fields and copy out what you wanted. Terry I am confident I can do whatever it is you want. I'm just not sure what that is.

[terrypin]

Maybe I'm not understanding you. You say "parse" but maybe we're not in agreement as to the meaning. When I parse something I split it into logical chunks. Is that what you want to do? Variable number of spaces is not a problem as you suggest because I am setting a variable number of places with the Set Integer command. Perhaps you could say it again but this time be less explicit and give an example. Tell me what you want me to do as if I was a person who was going to manually open these fields and copy out what you wanted. Terry I am confident I can do whatever it is you want. I'm just not sure what that is.

 

I'm assuming you mean more explicit, Cory! OK, I'll try. I've attached a small example source file. From that I want to end up with just 3 lines or variables:

 

Perpignan Airports

Untitled Path

Baena

 

Also, sometimes, instead of tabs on the left the file will have spaces instead, although your Trim macro should handle both. As I said, the part that I'm struggling with a bit is locating strings like <name>Baena</name> and <name>Perpignan Airports</name> without also finding unwanted strings like <name>KmlFile</name> and <name>Test folder</name>.

 

--

Terry, East Grinstead, UK

ExampleForCory.txt

[Cory]

Actually I did mean less explicit;-: You see sometimes people presume the solution takes a certain form and their info is based on that presumption. My hope was that you would give me instructions as if I was a human doing it. EG "Open the file, look for something, find the next something, and copy that to a file..." kind of thing. But now that I have some real data to look at I'm understanding more clearly. I'll assume you always want the name value after each placemark. Attached is my sample macro. Copy the text to your clipboard before running it.

Terry Parse Sample.mex

[terrypin]

Actually I did mean less explicit;-: You see sometimes people presume the solution takes a certain form and their info is based on that presumption. My hope was that you would give me instructions as if I was a human doing it. EG "Open the file, look for something, find the next something, and copy that to a file..." kind of thing. But now that I have some real data to look at I'm understanding more clearly. I'll assume you always want the name value after each placemark. Attached is my sample macro. Copy the text to your clipboard before running it.

Terry Parse Sample.mex

 

Excellent, thanks a bunch Cory, works fine and got me over that mental block!

 

--

Terry, East Grinstead, UK

[Cory]

It's a fine feeling when one overcomes conceptual hurdles. I hope a few come my way soon!

[lemming]

Yeah, I would second the addition of regex support. A search-n-replace operation that needs dozens of lines of code can often be distilled into a single regular expression.

 

Macro Express developers, might I suggest PCRE - Perl Compatible Regular Expressions, a free library that is widely used to add regex support.

 

Terry, in the meantime, you might want to check out:

 

GREP Command for Windows XP

http://malektips.com/xp_dos_0011.html

 

UPDATE: Here's a direct link to the Windows Server 2003 Resource Kit Tools which includes Qgrep.exe.

http://www.microsoft.com/downloads/details.aspx?FamilyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd&DisplayLang=en

 

The download is about 12MB but you really only need Qgrep.exe which is located in C:\Program Files\Windows Resource Kits\Tools

[paul]

Yeah, I would second the addition of regex support. A search-n-replace operation that needs dozens of lines of code can often be distilled into a single regular expression.

You know, I'd far, far rather Insight fixed all the outstanding MEP bugs before they start to add new functionality!

I still experience screen display problems, freezes, running men when they should be stationary, extreme slowness in many operations, etc., etc.

 

And I would really like the Run Macro In Variable command to work properly, i.e. with user-defined variable names.

 

Recently I'm not receiving acknowledgements of new bug reports, nor of bugs fixed. Anyone else experiencing this?

[Cory]

I've received trackers for bug I've reported but not to much in the way of 'resolved' reports. And I know they are aware of the freezing problem and a few other issues we have been having and are taking it very seriously. Hopefully that will equate to some fixes soon.

[terrypin]

You know, I'd far, far rather Insight fixed all the outstanding MEP bugs before they start to add new functionality!

I still experience screen display problems, freezes, running men when they should be stationary, extreme slowness in many operations, etc., etc.

 

Me too.

 

And I would really like the Run Macro In Variable command to work properly, i.e. with user-defined variable names.

 

Recently I'm not receiving acknowledgements of new bug reports, nor of bugs fixed. Anyone else experiencing this?

 

As far as recall I have received acknowledgements of my recent bug reports. Last was [iSS8625], 'ME Pro Script Editor opens the wrong tab when there are tabs hidden on the left' on 25th May.

 

But the last report of a bug fix I had was 6 months ago, on 17th Dec 2009, for [iSS6686]:

--------------------

Macro Express Pro Script Editor - Access violation at address 00404AFA in module 'macscript.exe'. Read of address 00000008 0x00404AFA

 

Related: [iSS5974] [iSS6058] [iSS6148] [iSS6244] [iSS6365] [iSS6541] [iSS6686] [iSS7180]

Same or similar: [iSS6686] [iSS7332] [iSS7494]

The change will be available with the next release of Macro Express Pro

--------------------

Presumably those were all included in the subsequent release(s)?

 

 

--

Terry, East Grinstead, UK

[terrypin]

My email on return from holiday includes this automatic notification:

 

ailj1668 has just posted a reply to a topic that you have subscribed to titled "Script to perform Regex operations?".

 

I don't see any such post in that thread - do others?

 

--

Terry, East Grinstead, UK

[paul]

This forum was "visited" by some alien who posted a lengthy email containing various rants - nothing to do with MEP. Perhaps this email has been removed from all the places it was posted to?

[terrypin]

This forum was "visited" by some alien who posted a lengthy email containing various rants - nothing to do with MEP. Perhaps this email has been removed from all the places it was posted to?

 

Thanks Paul. That explains the other similar puzzle as well.

 

--

Terry, East Grinstead, UK

[lemming]

Hi Terry, I've been testing GNU utilities for Win32 (unxutils.sourceforge.net), which includes classic Unix utilities like Grep, Sed, and Awk. These 3 support regular expressions, and I got them to work with Macro Express Pro.

 

For your text problem, I wrote an MEP script which calls Sed.

 

The MEP script is only 6 lines long - all it does is set a working folder, launch Sed, and report the results.

 

The Sed script which is doing the actual heavy lifting is only 3 lines long! (excluding comments)

 

All thanks to regex ;-)

 

QUICK START

 

1. Download Run Sed.mex and Run Sed.zip from here

 

2. Unzip Run Sed.zip and put the 3 files (sed.exe, Sed_Script.txt, ExampleForCory.txt) into a folder which MEP can read. Take note of the folder path. Also note that Windows 7/Vista will block access to certain folders.

 

3. Import Run Sed.mex into Macro Express Pro

 

4. Change the WorkingFolder variable in the script to the full path of the folder which contains the 3 files from Step #2

 

5. Run the script, and it will process ExampleForCory.txt and report the results

 

The Sed_Script.txt file has comments which explain the Sed commands. I also learned a lot from this site:

 

Famous Sed One-Liners Explained

 

 

-Lemming

Run Sed.mex

Run Sed.zip

[aarons]

I know this thread is 8 years old, but it still shows up on searches, so I will mention that Macro Express Pro can utilize RegEx through external scripts (VBscript and Javascript both have RegEx support).

Passing text into the external scripts can be a little cumbersome, and usually requires creating a temporary file which is then opened from the script.

Passing it back to Macroexpress is easier thanks to console output.

[terrypin]

Thanks aarons, stiil appreciate the feedback, despite the eight year delay. But as my need for regex is relatively infrequent I guess I'll stick to what I know and use either MEP itself (along the lines of Cory's example) or deploy the text in Textpad, rather than invest time learning new stuff.

Lemming: Only just seen your eight year old post. If you ever see this, my apologies! Probably missed it as it was about 10 weeks after my enquiry and the automatic email prompting system failed, assuming it existed then. But  my reply is probably as above . However,  I have downloaded both files and when I finish a couple of pressing projects I'll experiment in case it's easier than I suspect.

[kunkel321]

Found this via forum search for "regular expression."    I think it would be great if MacroExpressPro could utilize RegExs natively.  Has anyone formally requested this as a feature?  Do you guys think that it would be a desirable feature for them to add?  As others have indicated, PCRE is what I would recommend.    It does occur to me that writing regex code might be overwhelming for some people (I'm a total novice myself).  Still, I think it would be a welcome ability for the MacEx arsenal.  What do you guys think?  

Here's a visual of how it might appear in the GUI. 

 

[acantor]

I've used regular expressions occasionally, and found them as daunting as I have found them indispensable when needed.

 

Why not make a feature request?

[terrypin]

Nice to have but probably a major effort and I would suggest outstanding bugs, flaws and other features get higher priority.

Despite my good intentions nine years ago, I never did get around to trying Lemming’s approach. Perhaps that’s a reflection  of its relative importance to me. I do use Regex quite often, but usually within my text editor, TextPad, quickly activated from ME Pro. Possible analogy: MEP is you at your work bench. You reach for the hammer to do task X, for the screwdriver for task Y, etc.

Terry, UK

[Cory]

I will write you a simple RegEx application to do RegEx that you can control with MEP.

Also, that looks like XML. I could do it with a .NET XMLReader class. I do JSON also. 

[kunkel321]

Acantor and Terry,

I probably will go ahead and submit a formal feature request so they can get it on the "wish list."   Though I will refer Insight to the last page of this thread too, because it does make sense for them to focus on existing bugs and long-standing feature requests first.   

Cory,

Thank you for your offer.  I'm inclined to take you up on it, but I'm afraid that any XML, .NET, or JSON integrations would just go right over my head.  Have you worked with AutoHotKey at all?  I've done a little bit of stuff with AHK, but nothing that passes data between AHK and another application.  I'm guessing that there would be some similarities between the setups, whether using AHK, XML, or whatever.  My guess is that MacExp would send the text and the regex to a Windows environmental variable, then the other application would read that variable?  I don't know.