OK. I think I understand what you want.

In this context a source file is any one of the input files that matches either your Train set pattern or your Test set pattern.

The construct ${var%%pattern} expands to the contents of the shell variable var with the longest string that matches pattern at the end of the string removed. Similarly ${var%pattern} expands to the contents of the shell variable var with the shortest string that matches pattern at the end of the string removed, ${var##pattern} expands to the contents of the shell variable var with the longest string that matches pattern at the start of the string removed, and ${var#pattern} expands to the contents of the shell variable var with the shortest string that matches pattern at the start of the string removed. If the given pattern doesn't match the appropriate part of the expansion of $var, $var is expanded in full.

So, for example if $src is set to

Code:

puredate/64Xtest14.txt-James-Maggie.txt

or to

Code:

/home/dwc/test/puredate/64Xtest14.txt-James-Maggie.txt

then the command:

Code:

sf=${src##*/}

will set sf to 64Xtest14.txt-James-Maggie.txt, and then the command:

Code:

df="${sf%%.txt*}"

will set df to 64Xtest14, and then the commands:

Code:

df=${df#64[A-Z]train}
df=${df#64[A-Z]test}

will set df to 14 (with the 1st command leaving df unchanged and the 2nd command removing the leading 64Xtest. (With a source filename matching the pattern with train in it, the 1st command would remove the leading part of the string up to and including train and the 2nd command would leave the value unchanged.)

If you save the following script in a file, name it consolidate, make it executable, and execute it; it will consolidate all text in the files in and under the current working directory that match the pattern 64[A-Z]test[0-9][0-9].txt-*.txt or the pattern 64[A-Z]train[A-Z][0-9].txt-*.txt into files named 64[A-Z]/[A-Z][0-9][0-9].txt or 64[A-Z]/[A-Z][A-Z][0-9].txt under the current working directory, respectively:

Code:

#!/bin/ksh
# Usage: consolidate
#  The consolidate utility copies the contents of source files with
#  names matching one of two patterns in or under the current working
#  directory into summary files in directories (with the directory
#  name and file name derived from the name of the source file).
#   */64[A-Z]test[0-9][0-9].txt-*.txt -> 64[A-Z]/[A-Z][0-9][0-9].txt
#   */64[A-Z]train[A-Z][0-9].txt-*.txt -> 64[A-Z]/[A-Z][A-Z][0-9].txt
ec=0    # Script exit code.
find .  -name '64[A-Z]test[0-9][0-9].txt-*.txt' -o \
        -name '64[A-Z]train[A-Z][0-9].txt-*.txt' | while read src
do
        # Get last component of pathname of source file ($sf).
        sf="${src##*/}"
        # Target directory ($dir) will be "64x" (where x is a single upper case
        # letter) after throwing away train* or test*.
        dir="${sf%%t*}"
        # Create the target directory if it doesn't already exist.
        if [ ! -d "$dir" ]
        then    mkdir "$dir"
                rc=$?
                if [ $rc -ne 0 ]
                then    ec=1
                        printf "%s: \"%s\" not processed.\n" "$0" "$src" >&2
                        continue
                fi
        fi
        # Change source filename ($sf) to destination filename ($df):
        df="${sf%%.txt*}"       # Get rid of trailing ".txt-*.txt"
        df="${df#64[A-Z]train}" # Get rid of leading "64[A-Z]train" or
        df="${df#64[A-Z]test}"  #   "64[A-Z]test".
        df="${dir#64}$df.txt"   # Put back the "[A-Z]" removed in last step and
                                #   add trailing ".txt".
        cat "$src" >> "$dir"/"$df"
        rc=$?
        if [ $rc -eq 0 ]
        then    ;# printf "%s: cat %s >> %s succeeded\n" "$0" "$src" "$dir/$df"
                # rm "$src"
        else    ec=1
                printf "%s: cat %s >> %s failed (%d)\n" \
                        "$0" "$src" "$dir/$df" "$rc" >&2
        fi
done
exit $ec

This was written and tested using ksh, but only uses shell features specified by the POSIX standards and the Single UNIX Specifications (so it should work the same with any shell that conforms to these standards). It could be made a little more efficient using features that are only available in more recent versions of ksh, but the script shown here should work with any version of ksh as well as any other standards conforming shell.

If you would like to see a status report of the files successfully processed while this script is running, remove the ;# from the then clause of the last if command.

If you want to remove the source files after they have been successfully written into one of the consolidation files, remove the # in front of the rm command if the same then clause. Note that if you do this, you should also check the exit status of this rm command like the script does with the mkdir and cat commands.

You could also add options to be interpreted by this script to enable removing the source files that have been successfully copied, to enable printing of successfully completed copies, to set a different source directory, and to set a different destination directory, but I'll leave that as an exercise for the reader.

Hope this helps,
Don