Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk to print range of fields

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to print range of fields
# 1  
Old 12-06-2011
awk to print range of fields
Hi

file.in and file.out are in csv format. the code I have now is,
Code:
cat file.in | awk -F"," '!($1$2$3$4$5$6$7$8 in a){a[$1$2$3$4$5$6$7$8];print $0}' > file.out

Here, I am printing entire line using $0. however, I want to print $1 to $150 and it should be in csv format. Cut -d is not good in performace. Using for loop within awk is not producing output as expected.

file.in
Code:
hi,there,how,are,you,123
hi,there,how,are,you,124
hi,there,how,are,you,125
hi,there,how,are,you,126

file.out (please note i have reduced no.of columns for convinience, i need 150 columns)
Code:
hi,there,how,are
hi,there,how,are
hi,there,how,are
hi,there,how,are

Thanks in advance
K


Moderator's Comments:
Mod Comment Please use code tags!
Last edited by zaxxon; 12-06-2011 at 12:44 PM.. Reason: code tags, see PM
# 2  
Old 12-06-2011
That's a useless use of cat. If you're worried about performance, that should be the first thing to go.

cut's performance ought to better than awk's, being a more specialized tool. This is exactly the sort of problem cut was made for. If you use it in a silly way, like running it once per line, it will of course run slowly, but so would awk.

Code:
cut -d "," -f 1-150 < input > output

# 3  
Old 12-06-2011
Hi Corona,

Code:
cat file.in | awk -F"," '!($1$2$3$4$5$6$7$8 in a){a[$1$2$3$4$5$6$7$8];print $0}' | cut -d, -f1-150  > file.out

I use the above code, ie., using cut within awk. Do you think its effective way of using it like that.

Thanks, K.
# 4  
Old 12-06-2011
You're still doing that useless use of cat.

Getting rid of that cat will probably save you about enough CPU power to pay for that entire cut.

The code otherwise looks good. I don't think there's an efficient way to specify entire ranges of fields in awk itself.

Code:
<file.in awk -F"," '!($1$2$3$4$5$6$7$8 in a){a[$1$2$3$4$5$6$7$8];print $0}' | cut -d, -f1-150  >file.out

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to print text in field if match and range is met

In the awk below I am trying to match the value in $4 of file1 with the split value from $4 in file2. I store the value of $4 in file1 in A and the split value (using the _ for the split) in array. I then strore the value in $2 as min, the value in $3 as max, and the value in $1 as chr. If A is... (6 Replies)
Discussion started by: cmccabe
6 Replies

2. Shell Programming and Scripting

awk to print out lines that do not fall between range in file

In the awk below I am trying to print out those lines in file2 that are no between $2 and $3 in file1. Both files are tab-delimeted and I think it's close but currently it is printeing out the matches. The --- are not part of the files they are just to show what lines match or fall into the range... (6 Replies)
Discussion started by: cmccabe
6 Replies

3. Shell Programming and Scripting

Print multiple fields with awk

so its common knowledge one can print multiple fields with simple commands like this: echo 12 44 45 552 24 | awk '{print $1,$4,$3}' but suppose i want to avoid specifying the "$" symbol. is that possible? can something like this be done: echo 12 44 45 552 24 | awk '{print $(1,4,3)}' ... (9 Replies)
Discussion started by: SkySmart
9 Replies

4. Shell Programming and Scripting

awk to remove range of fields

I am trying to cut a range of fields in awk. The below seems to work for removing field 50, but what is the correct syntax for removing a range ($50-$62). Thank you :). awk awk 'BEGIN{FS=OFS="\t"}{$50=""; gsub(/\t\t/,"\t")}1' test.vcf.hg19_multianno.txt > output.csv Maybe: awk... (6 Replies)
Discussion started by: cmccabe
6 Replies

5. Shell Programming and Scripting

awk print even fields of file

Hello: I want to print out the even number of fields plus the first column as row identifiers. input.txt ID X1 ID X2 ID X3 ID X4 A 700 A 1200 A 400 A 1300 B 2000 B 1000 B 2000 B 600 C 1400 C 200 C 1000 C 1200 D 1300 D 500 D 600 D 200and the output is: output.txt ID X1 X2 X3... (3 Replies)
Discussion started by: yifangt
3 Replies

6. Shell Programming and Scripting

awk sort based on difference of fields and print all fields

Hi I have a file as below <field1> <field2> <field3> ... <field_num1> <field_num2> Trying to sort based on difference of <field_num1> and <field_num2> in desceding order and print all fields. I tried this and it doesn't sort on the difference field .. Appreciate your help. cat... (9 Replies)
Discussion started by: newstart
9 Replies

7. UNIX for Dummies Questions & Answers

How to print specific range using awk?

I want to print specific range of rows and then its columns using awk command. lets say if a file contain 15 line then i need to print line 5 to 9. awk '{print;for( NR>=5&&NR<=9); do; print "<tr>\n<td>"$1"</td><td>"$2"</td><td>"$3"</td>\n</tr" done}' xyz.csv not sure what's wrong... (9 Replies)
Discussion started by: amyt1234
9 Replies

8. Shell Programming and Scripting

How to print 1st field and last 2 fields together and the rest of the fields after it using awk?

Hi experts, I need to print the first field first then last two fields should come next and then i need to print rest of the fields. Input : a1,abc,jsd,fhf,fkk,b1,b2 a2,acb,dfg,ghj,b3,c4 a3,djf,wdjg,fkg,dff,ggk,d4,d5 Expected output: a1,b1,b2,abc,jsd,fhf,fkk... (6 Replies)
Discussion started by: 100bees
6 Replies

9. Shell Programming and Scripting

AWK Script - Print a column - within a Row Range

Hi, Please read the whole thread. I have been working on this script below. It works fine, feel free to copy and test with the INPUT File below as well. example: PACKET DATA PROTOCOL CONTEXT DATA APNID PDPADD EQOSID VPAA PDPCH PDPTY PDPID 10 ... (6 Replies)
Discussion started by: panapty
6 Replies

10. Shell Programming and Scripting

How to print first matching range in awk?

following is input - <Schema> <schema_name>admin</schema_name> <Version>1.1</Version> <schema_name>admin</schema_name> <Version>1.2</Version> </Schema> ... (12 Replies)
Discussion started by: thearpit
12 Replies
Login or Register to Ask a Question