Shell Programming and Scripting

Code:

awk 'NR==1{for(i=1;i<=NF;i++)col[i]=i}{for(i=1;i<=NF;i++){ printf "%s%s",$i,FS > col[i]".txt" } }' myfile
paste -s ?.txt

Hi, kurumi:

This is a good approach for avoiding my code's memory appetite and for also avoiding your original approach's appetite for fork-exec. However, there a few fatal bugs that will not yield a correct matrix transposition.

* '?.txt' would not match properly since the '?' only matches a single character but the headers shown -- from which the file names are taken -- are all multicharacter; '*.txt' is necessary.

* The shell globbing used to build the paste command line may not (will not with the data given in the original post) yield the header names in the same order as they appear in the file. The file names need to be named in a way that preserves the order.

* The use of paste -s on multiple files without a terminating newline produces a one line file, when what's wanted is one line per file. cat would be correct if not for the absence of a newline at the end of each file.

* The command line generated by all those file names may possibly exceed the system's command line length limit.

* The AWK process will require many file descriptors (600,003 or so .... one per output file in addition to stdin, stdout, and stderr). Depending on how the system is configured, this may exceed the login account's imposed descriptor limits. If that's the case, then the limits need to be bumped up or AWK needs to close each descriptor after writing to it.

Regards,
Alister

P.S. Please don't delete proposed solutions that have been discussed in a thread. Disappearing posts cause confusion and are an obstacle to continued dialogue and learning.

---------- Post updated at 05:55 PM ---------- Previous update was at 05:53 PM ----------

Hi genehunter:

The following code is a reworking of Kurumi's approach which fixes the bugs I mentioned above and makes conservative assumptions regarding ARG_MAX and file descriptor limits. To invoke it, save the code to a file, make it executable, and invoke it with one argument, the name of the data file:

Code:

#!/bin/sh

infile="$1"

awk '
    BEGIN {
        getline
        l=length(NF)
        for (i=1; i<=NF; i++) {
            f[i]=sprintf("%"l"s", i) ".col"
            gsub(" ", "0", f[i])
            printf("%s", $i) > f[i]
            close(f[i])
        }
    }

    {
        for (i=1; i<=NF; i++) {
            printf("%s%s", FS, $i) >> f[i]
            close(f[i])
        }
    }

    END {
        for (i in f) {
            printf("\n") >> f[i]
            close(f[i])
        }
    }' "$infile"

for f in *.col; do
    echo "$f"
done | xargs cat > outfile

If the file descriptor limit were sufficiently large, then all the close() calls could be deleted. And if ARG_MAX were sufficiently large, a simple "cat * > outfile" would suffice.

Regards,
Alister