Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Gnu parallel and awk syntax quotes

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Gnu parallel and awk syntax quotes
# 1  
Old 02-08-2012
Power Gnu parallel and awk syntax quotes
I have many files that has the following structure.
Code:
CHR         SNP         BP   A1       TEST    NMISS         OR       SE      L95      U95         STAT            P 
   1   rs2980319     766985    A        ADD     4948      1.068  0.08356   0.9065    1.258       0.7853       0.4323
   1   rs2980319     766985    A     VAR1     4948      1.109  0.02484    1.057    1.165        4.177    2.952e-05
   1   rs2980319     766985    A        VAR2     4948  1.567e+27    2.441 1.31e+25 1.875e+29        25.65   4.009e-145
   1   rs2980319     766985    A        VAR3     4948  3.031e+12    2.221 3.902e+10 2.355e+14        12.94     2.63e-38

I want to output certain columns and lines which only contain "ADD" in the 5th column.
I am trying to use GNU parallel and am having no luck getting output.
Code:
>parallel ""awk 'NR==1{print $2, $1, $3, $4, $7, $9, $10, $12; next} $5=="ADD"{print $2, $1, $3, $4, $7, $9, $10, $12 | "sort -gk8"}' OFS="\t" {1}_fileA.assoc > {1}_fileA.assoc.ADD

The syntax seems too dense for me.
Any help is appreciated.
# 2  
Old 02-08-2012
It looks extremely bizzare to me, too. I have no idea why they'd "parallel" a single file, or even how...
# 3  
Old 02-08-2012
I just gave an exmple of a single file .
The {1} can be substituted by any number of variables that act as prefixes for FileA

parallel - build and execute shell command lines from standard input in parallel

I gind GNU parallel to be very useful especially without access to a cluster for processing big data. However, yet to learn to make awk work with it. Smilie

Thanks for looking.
# 4  
Old 02-08-2012
the redirection would be for the entire run of 'parallel' (which i have no experience with). you'd be better off creating a small shell script to run, or possibly something like

Code:
parallel sh -c 'awk '\''program''\' OFS="\t" $1_fileA.assoc > $1_fileA.assoc.ADD' _ {1}

maybe... :/
# 5  
Old 02-08-2012
Try this:

Code:
... parallel awk \'NR == 1 \{ \
  print \$2, \$1, \$3, \$4, \$7, \$9, \$10, \$12 \
  next \
  \} \
  \$5 == \"ADD\" \{ \
    print \$2, \$1, \$3, \$4, \$7, \$9, \$10, \$12 \| \"sort -gk8\" \
    \}' OFS=\'\\t\' {1}_fileA.assoc \> {1}_fileA.assoc.ADD

Or just put your awk code in a script file and make your life easier Smilie
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Python GNU parallel single command on multiple cores

Hello, I have a 4 core machine. Here is my initial script cd /work/ python script.py input.txt output.txt 1 2 3 This script runs for 1.5hrs. So I read across the web and figured out that you can use GNU parallel to submit multiple jobs using parallel. But I am not sure if I can run... (4 Replies)
Discussion started by: jacobs.smith
4 Replies

2. Shell Programming and Scripting

Using quotes in awk

Hello, i had a problem running a script , and after investigation found its all to do with the quotes: cat file1 line1 val1 val2 line2 val1 val2 line3 val1 val2 awk 'BEGIN {RS="\n\n"; FS="\n";} {print $1 $2}' file1 This gives me the wrong output: (5 Replies)
Discussion started by: andy391791
5 Replies

3. Shell Programming and Scripting

awk - Print where value is in quotes

Hi All, I have input data like follows: "1234"|"ABC" "1234"|"CBA" "1222"|"ZZZ" I am trying to awk print all records where Col1 = "1234". Below is the code I have so far: Var1=1 Var2=1234 awk -F "|" "$ ${Var1} == "\"${Var2}\"" { print; }' inputfile However when the AWK... (2 Replies)
Discussion started by: RichZR
2 Replies

4. Shell Programming and Scripting

Converting from Linux bash (GNU) to Solaris script syntax errors

Original script written on CentOS 6.3 with GNU bash 4.1.2 Destination system is Solaris 9 with GNU bash 2.05 (not changeable by me) I have a script written on the linux side but now we need to provide a version to another site that "doesn't like linux". I've been going through changing the ] or... (13 Replies)
Discussion started by: oly_r
13 Replies

5. Shell Programming and Scripting

awk without quotes

I want to execute awk command without quotes. who am i | awk {'print $2'} above code should be something like: who am i | awk {print $2} Why such weird requirement? Im assigning command to a variable, hence i need to escape the quotes. e.g: x='who am i | awk {\'print $2\'}' I want... (11 Replies)
Discussion started by: Arun_Linux
11 Replies

6. Shell Programming and Scripting

syntax issue with quotes in mysql command for a bash script

i'm trying to write a bash script that executes a mysql statement mysql -sN -e INSERT INTO "$database"."$tableprefix"users (var1, var2,var3) VALUES (123, '1','') i don't know where to put the quotes it doesnt work with this one: ` it seems i can only put double quotes around the... (0 Replies)
Discussion started by: vanessafan99
0 Replies

7. Shell Programming and Scripting

quotes using awk

i want to print the statement below using awk,but i am unable to get the quotes ("22345",1,"Thank you"); How can i do this (5 Replies)
Discussion started by: tomjones
5 Replies

8. UNIX for Dummies Questions & Answers

| help | unix | grep (GNU grep) 2.5.1 | advanced regex syntax

Hello, I'm working on unix with grep (GNU grep) 2.5.1. I'm going through some of the newer regex syntax using Regular Expression Reference - Advanced Syntax a guide. ls -aLl /bin | grep "\(x\)" Which works, just highlights 'x' where ever, when ever. I'm trying to to get (?:) to work but... (4 Replies)
Discussion started by: MykC
4 Replies

9. UNIX for Dummies Questions & Answers

Parallel processing using AWK program

Hi All, could any expert tell me about writing a parallel processing program using AWK program is possible or not? Any example would be much appreciated... With Regards / Mysore Ganapati :confused: (5 Replies)
Discussion started by: ganapati
5 Replies

10. Shell Programming and Scripting

Single v. double quotes in sed (GNU utilities for Win32 version)

I'm using sed for Windows at a WinNT command prompt. Wrapping the expressions with single quotes doesn't work. I have to use double quotes. Is this the norm for the Windows implementation? (2 Replies)
Discussion started by: daddydojo
2 Replies
Login or Register to Ask a Question