Originally Posted by Don Cragun

The 1st post in this thread explicitly requested that every field in your input files be searched for PVALUE=number. But, the sample data provided never shows more than once such string on an input line and, on lines that do have something matching that pattern, it always appears in the last field on the line. But, we have no indication of whether or not the sample data provided in post #5 in this thread is representative of the actual data that needs to be processed. From the code samples posted, it appears that the submitter wants one output file produced for each input file that contains matched lines. The submitter seems to also want to have 15 copies of awk running in parallel (which only makes sense if those 15 awk commands won't be thrashing CPU and/or disk accesses.

Assuming that parallel processing won't really help much here (and might actually slow down processing), avoiding xargs completely, and assuming that an input line may contain more than one of the patterns above; I would try something more like:

Code:

awk -F';' '
FNR == 1 {
	if(of != "")
		close(of)
	of = FILENAME ".fail"
}
{	for(i = 1; i <= NF; i++)
		if($i ~ /^PVALUE=/ && (substr($i, 8) + 0) > .05) {
			print > of
			next
		}
}' *.txt

Note that this doesn't create an output file for every input file; it only creates an output file if one or more lines in the corresponding input file meets your criteria.

If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.