Shell Programming and Scripting

Hi I have multiple files for which I want to use awk for the following:

Read each line in each file- if any of the columns match "PVALUE=" followed by the number, then print the line in case the number following "PVALUE=" is greater than 0.05.

I did the following:

Code:

ls *.txt | xargs -I @ -P15 sh -c "awk {if /PVALUE=(\d+)/)){if ($1 > 0.05){print $_}}}' @ - >@.fail"

but i get an error message:

Code:

sh: -c: line 0: syntax error near unexpected token `('

I appreciate suggestion in correcting the one liner for accommodating all of the above.

thanks

Moderator's Comments:

Please use CODE tags when displaying sample input, sample output, and code segments as required by Forum Rules.

doesn't the expression in the if statement have to be enclosed in square brackets?

Should it be like this ? It still gives me an error message

Code:

ls *.txt| xargs -I @ -P15 sh -c "awk {if [/PVALUE=(\d+)/)]{if ($1 >=0.05){print $_}}}' @ - >@.fail"

I'm not at all sure that I understand what you're trying to do here. And saying you're getting an error message without showing us the exact error you're getting leaves us with a lot of guess work. But whatever you do is going to need matched single quotes surrounding your awk script, and xargs is not likely to replace @ inside a double quoted string with the filenames you seem to want.

But even after you get the quotes fixed (and getting rid of the unneeded sh -c and the extra set of double quotes needed for that, you still have several syntax errors in your awk script. Please show us a sample of the data in one of these text files and the output you are hoping this awk script will produce from that sample input file.

Your first post implies that there are multiple fields in your text files, and that you want to examine every column, so we also need to know that your text files are using as field delimiters.

And, especially since you're depending on non-standard options in the xargs utility you're using, we also need to know what operating system and shell you're using.

Hi Don
The following is the file sample.

I have edited the command as following:

Code:

ls *.txt | xargs -I {} -P15 sh -c awk {if [/PVALUE=(\d+)/){if ($1 >=0.05){print $_}}]}' {} >{}.fail

The following is the error message:
I am using the x86_64 GNU/Linux

How about (untested):

Code:

awk 'match ($0, /PVALUE=[0-9.]*/) && substr($0, RSTART+7, RLENGTH-7) > 0.05' *.txt

Thanks a lot.
Although tThe command works, individual output files are empty.No error message.

Code:

ls *.txt |xargs -I {} -P15 sh -c "awk 'match ($0, /PVALUE=[0-9.]*/) && substr($0, RSTART+7, RLENGTH-7) > 0.05' {} >{}.fail"