Data Array Technique

[Les Hazlett]

Hi,

 

I need to collect frequency statics on nearly 100 items. When an item of a particular type is found, I want to increment an integer counter for that type of item. I am considering using the D Registers and sequentially converting values back to integer as I am summarizing the results. I sure wish, if the item was a 52 or 93 or 21, that I could add one to %D52% or %D93% or %D21% without using multi-page long case statements.

 

Any ideas?

 

Les

[Linda]

EDITED FOR CLARIFICATION:

 

Have you considered using an INI file? I've found them a useful way to get around ME's limited variable-space and lack of arrays.

 

* Create an INI file with key names 1-99 (or whatever) , and set them all equal to 0.

 

Then, say %N1% is the variable that comes back as 52 or 93 or whatever, depending on what you want to incriment. Each time this value comes back:

 

* Convert %N1% to a text string, %T1%

* Read a vairable, %T2% from an INI file (using %T1% as the key-name)

* Convert %T2% to an integer, %N2%

* Increment %N2%

* Convert %N2% back to a text string, and save to the INI file at key-name %T1%.

 

Not the most elegant peice of code ever invented, but it beats a switch statement with dozens of cases!

[Les Hazlett]

Linda you're wonderful!

 

It works just like you said.

 

It also works without initializing the values in the INI file. Starting with a blank INI file, blank values are returned for any requested field. The blanks are changed to zero in the conversion to integer.

 

It also works using the %N??% directly for a numerical keyname.

 

Also, I think that I could avoid the conversion-increment-conversion by appending a character to the INI field and counting the characters later with the Variable Set Integer to the text length for the field. This should be faster and I have more memory than time. I plan to do a timing comparison to learn how much faster.

 

This brings up the subject of running timing tests. Has this been discussed on either the new forum or the old newsgroup? I hate to spend the time to write code if someone has documented the execution time for various commands.

 

I am grateful that you share your knowledge and experience.

 

Les

[Linda]

I'm glad it worked! And I'm glad all the text-conversions/initializations wern't neccesary- I wanted to err on the side of caution and give the most conservative case.

 

I've only been following the newsgroup since it moved to the new format, and haven't worked with timing the functions myself, though if this information is out there, it would be of interest to me as well.

 

I know that the company that hosts this site has a collection of timing functions in the macro library that they sell, but I've never used it myself, and can't speak to it.

[Les Hazlett]

A few message back, I said:

 

"Also, I think that I could avoid the conversion-increment-conversion by appending a character to the INI field and counting the characters later with the Variable Set Integer to the text length for the field. This should be faster and I have more memory than time. I plan to do a timing comparison to learn how much faster."

 

That was sure a bad idea. As the length of the field grew, the time required to append to it grew - a lot. In 10,000 iterations, conversion-increment-conversion was faster than appending.

 

If anyone has any guidance on timing commands, please provide here or as a separate topic.

 

Thanks,

 

Les

[Nicolas]

talking about macros performances is really interesting ! sometimes I'm really surprised to see a macro being as performant as a dedicated C++ program but some of my macros can be really slow (but i'm too lazy to develop a C++ program when good performances are not really needed)

 

I think reading and writing this INI file so many times is really time consuming ... its main advantage (in my point of view) is its persistancy : even if your computer crashes, you'll have the latest results in the INI file ...

 

did you try to create "dynamic macros" (I don't know if such macros have a name) ?

 

for instance, as variable modify integer : inc(%N1%) is coded as <NMVAR:08:01:0:0000001:0:0000000> (you can see this switching between scripting editor and direct editor)

 

so if you need to increment the n-th variable (I'll use %N99% to store this value "n"), you could maybe try something like that :

Variable Set String %T1% "<NMVAR:08:%N99%:0:0000001:0:0000000>"
Run Macro in Variable %T1%

 

I can't tell you if it works and if it is better, as I never used such a trick ... but if you try this, please keep us informed !

[floyd]

A couple points of interest:

 

Timing tests:

The PGM Functions Library has a Developer Tools category that contains the necessary functions to do timing tests. Here is a link to more information on those timing tools. They were invaluable to us during the development phase of the Library. Simply highlight the section of code that you want to time and then hit the HotKey that calls the timer Ctrl+Alt+Shft+T.

 

Incrementing counters dynamically:

Following is a sample playable that increments random integer variables 10,000 times in a little over 3 seconds:

Repeat Start (Repeat 10000 times)
 // Create a random integer from 10 to 99
 Repeat Until %N1% >= 10
   Variable Set Integer %N1% with a Random Number
 Repeat End
 // Create a dynamic string containing the random integer and run it
 Variable Set String %T1% "<NMVAR:08:%N1%:0:0000001:0:0000000>"
 Run Macro in Variable %T1%
Repeat End
Macro Return

<REP3:01:000001:000001:10000:0:01:><REM2:Create a random integer from 10 to 99><REP3:08:000005:000002:0001:0:01:10><IVAR2:01:06:99><ENDREP><REM2:Create a dynamic string containing the random integer and run it><TVAR2:01:01:<NMVAR:08:%N1%:0:0000001:0:0000000>><RUNMACVAR:1><ENDREP><MRETURN>

I have also attached it to this message. Each time through the loop a random integer between 10 and 99 is picked. The value is then plugged into the Increment Integer command string, which is then run using the Run Macro in Variable command.

DynamicIncrement.zip

[Linda]

This is much more elegant!

 

Floyd- does this using variables-inside-vairables only work on the newer versions of ME?

 

I had tried to get it to work before, and could not. When you posted a source example for it the other day (thanks!) I tried to get it to work based on that, and could not.

 

Am I doing something wrong, or is it the version of ME I'm using?

 

I tried the script you posted above, but substituted a "Text Box Display" for the "Macro Run" call. The text box displays %T1%, and comes up as (three successive runs through the loop):

 

<NMVAR:08:89:0:0000001:0:0000000>

<NMVAR:08:96:0:0000001:0:0000000>

<NMVAR:08:63:0:0000001:0:0000000>

 

i.e, it's just inserting the variable within the string, not reading the resulting string as a variable.

 

(As always, institutionally stuck with 3.0.4.1 for the itermediate-term future, at least)

[floyd]

The Run Macro in Variable command was not added to Macro Express until version 3.4.0.0 so you do not have it in your version. However, you do have the Load Macro Text File command, which is similar. It runs a macro from a file rather than a variable. Here is the adjusted code:

Repeat Start (Repeat 10000 times)
 // Create a random integer from 10 to 99
 Repeat Until %N1% >= 10
   Variable Set Integer %N1% with a Random Number
 Repeat End
 // Create a dynamic string containing the random integer and run it
 Variable Set String %T1% "<NMVAR:08:%N1%:0:0000001:0:0000000>"
 Variable Modify String: Append %T1% to %T2%
Repeat End
// Save the whole string to a file
Variable Set String %T1% from Environment Variable
Variable Modify String: Append "Me3Test.txt" to %T1%
Variable Modify String: Save %T2% to Text File
Delay 250 Milliseconds
Wait for File Ready: "
// Run the command string
Load Macro Text File: "%T1%"
Macro Return

<REP3:01:000001:000001:10000:0:01:><REM2:Create a random integer from 10 to 99><REP3:08:000005:000002:0001:0:01:10><IVAR2:01:06:99><ENDREP><REM2:Create a dynamic string containing the random integer and run it><TVAR2:01:01:<NMVAR:08:%N1%:0:0000001:0:0000000>><TMVAR2:08:02:01:000:000:><ENDREP><REM2:Save the whole string to a file><TVAR2:01:11:Temporary Path><TMVAR2:07:01:00:000:000:Me3Test.txt><TMVAR2:17:02:00:000:000:%T1%F><MSD:250><WFREADY:000000:000005:000000%T1%><REM2:Run the command string><TFILE:%T1%><MRETURN>

The difference is that you are just appending one string to another each time through the loop. The final string is saved to a temporary file after the loop ends, then the file is run by simply loading it. In this example you have saved 10,000 separate commands to the file. On my system it took about 15 seconds to run this example. I would estimate that 95% of that time was spent just writing to and then launching the file.

 

This could also have been done by writing one line to a file from within the loop, then running that file each time (like the Run Macro in Variable command), however, it would have taken almost 1-1/2 hours to run through 10,000 loops.

DynamicIncrement2.zip

[Linda]

Oh! That's the peice I was somehow missing- that it's not translating it directly, but that the saving it and loading it were a crucial part of the picture. Thanks!

[Les Hazlett]

Thanks to you all for the excellent coverage of this topic. I have been slow to respond because of requirements of my "real" work. I finished the essential part of the macro capability I was doing and have now used it successfully.

 

I had to make several concessions to meet my required production date. One of the features I delayed was the reporting section where I needed to collect the statistics on the frequency of occurence. I just got back to writing that section and have written it using the "Run Macro in Variable" technique.

 

It works great and FAST. Thanks for the very useful method.

 

Les