Awk: subset of fields as variable with sprint Post: 302972397

Sponsored Content

Top Forums UNIX for Beginners Questions & Answers Awk: subset of fields as variable with sprint Post 302972397 by RavinderSingh13 on Thursday 5th of May 2016 01:08:30 AM

05-05-2016

Moderator

Hello beca123456,

Not sure if in output shown A_(5) and D_(12) where _ is required, if not then following may help you in same. If you are not considering the order of the output then following may help you.

Code:

awk -F"|" '{XYZ[$2 FS $3 FS $3 FS $4]=XYZ[$2 FS $3 FS $3 FS $4]?XYZ[$2 FS $3 FS $3 FS $4]";"$1"("$NF")":$1"("$NF")"} END{for(i in XYZ){print XYZ[i] FS i}}'  Input_file
OR
awk -F"|" '{Q=$2 FS $3 FS $3 FS $4;XYZ[Q]=XYZ[Q]?XYZ[Q]";"$1"("$NF")":$1"("$NF")"} END{for(i in XYZ){print XYZ[i] FS i}}'  Input_file

Output will be as follows then.

Code:

D(12)|1|9|9|3
A(5);B(3);C(1)|1|2|2|3

If you need the correct order in output too as per your Input_file then following may help you in same.

Code:

awk -F"|" 'FNR==NR{XYZ[$2 FS $3 FS $4 FS $5]=XYZ[$2 FS $3 FS $4 FS $5]?XYZ[$2 FS $3 FS $4 FS $5]";"$1"("$NF")":$1"("$NF")";next} (($2 FS $3 FS $4 FS $5) in XYZ){print XYZ[$2 FS $3 FS $4 FS $5] OFS $2 FS $3 FS $4 FS $5;delete XYZ[$2 FS $3 FS $4 FS $5]}' Input_file OFS="|" Input_file
OR
awk -F"|" 'FNR==NR{Q=$2 FS $3 FS $4 FS $5;XYZ[Q]=XYZ[Q]?XYZ[Q]";"$1"("$NF")":$1"("$NF")";next} {P=$2 FS $3 FS $4 FS $5} (P in XYZ){print XYZ[P] OFS P;delete XYZ[P]}' Input_file OFS="|"  Input_file

Output will be as follows then.

Code:

A(5);B(3);C(1)|1|2|3|4
D(12)|1|9|3|4

If above is not completing your requirements then please show us more Input_file and expected output with more specific details on same, hope this helps you.
EDIT: Adding a non-one liner form of solutions now.

Code:

awk -F"|" '{
                XYZ[$2 FS $3 FS $3 FS $4]=XYZ[$2 FS $3 FS $3 FS $4]?XYZ[$2 FS $3 FS $3 FS $4]";"$1"("$NF")":$1"("$NF")"
           }
           END{
                for(i in XYZ){
                                print XYZ[i] FS i
                             }
              }
          ' Input_file
  
 
awk -F"|" 'FNR==NR{
                        Q=$2 FS $3 FS $4 FS $5;
                        XYZ[Q]=XYZ[Q]?XYZ[Q]";"$1"("$NF")":$1"("$NF")";
                        next
                  }
                  {
                        P=$2 FS $3 FS $4 FS $5
                  }
                  (P in XYZ){
                                print XYZ[P] OFS P;
                                delete XYZ[P]
                            }
          ' Input_file OFS="|" Input_file

Thanks,
R. Singh

Last edited by RavinderSingh13; 05-05-2016 at 02:18 AM.. Reason: Added non-one liner form of solution now.

RavinderSingh13

View Public Profile for RavinderSingh13

Find all posts by RavinderSingh13

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk sed cut? to rearrange random number of fields into 3 fields

I'm working on formatting some attendance data to meet a vendors requirements to upload to their system. With some help on the forums here, I have the data close. But they've since changed what they want. The vendor wants me to submit three fields to them. Field 1 is the studentid field,...

2. Shell Programming and Scripting

awk print variable then fields in variable

i have this variable: varT="1--2--3--5" i want to use awk to print field 3 from this variable. i dont want to do the "echo $varT". but here's my awk code: awk -v valA="$varT" "BEGIN {print valA}" this prints the entire line. i feel like i'm so close to getting what i want. i...

3. Shell Programming and Scripting

Join fields comparing 4 fields using awk

Hi All, I am looking for an awk script to do the following Join the fields together only if the first 4 fields are same. Can it be done with join function in awk?? a,b,c,d,8,,, a,b,c,d,,7,, a,b,c,d,,,9, a,b,p,e,8,,, a.b,p,e,,9,, a,b,p,z,,,,9 a,b,p,z,,8,, desired output: ...

4. Shell Programming and Scripting

How to print 1st field and last 2 fields together and the rest of the fields after it using awk?

Hi experts, I need to print the first field first then last two fields should come next and then i need to print rest of the fields. Input : a1,abc,jsd,fhf,fkk,b1,b2 a2,acb,dfg,ghj,b3,c4 a3,djf,wdjg,fkg,dff,ggk,d4,d5 Expected output: a1,b1,b2,abc,jsd,fhf,fkk...

5. Shell Programming and Scripting

Concatenating awk fields with variable

Hi, I am trying to do this:- FILE=application.log PID=12345 FILE=`echo $FILE | awk -F "." '{print $1 "$PID" $2}'` echo $FILE application$PIDlog I need the output to be application12345.log but I am not sure how to get the $PID variable into the output. I have tried various things...

6. Shell Programming and Scripting

awk - compare 1st 15 fields of record with 20 fields

I'm trying to compare 2 files for differences in a selct number of fields. When differnces are found it will write the whole record of the second file including appending '|C' out to a delta file. Each record will have 20 fields, but only want to do comparison of 1st 15 fields. The 1st field of...

7. Shell Programming and Scripting

awk processing of variable number of fields data file

Hy! I need to post-process some data files which have variable (and periodic) number of fields. For example, I need to square (data -> data*data) the folowing data file: -5.34281E-28 -3.69822E-29 8.19128E-29 9.55444E-29 8.16494E-29 6.23125E-29 4.42106E-29 2.94592E-29 1.84841E-29 ...

8. Shell Programming and Scripting

awk sort based on difference of fields and print all fields

Hi I have a file as below <field1> <field2> <field3> ... <field_num1> <field_num2> Trying to sort based on difference of <field_num1> and <field_num2> in desceding order and print all fields. I tried this and it doesn't sort on the difference field .. Appreciate your help. cat...

9. Shell Programming and Scripting

awk to filter file using another working on smaller subset

In the below awk if I use the attached file as the input, I get no results for TCF4. However, if I just copy that line from the attached file and use that as input I get results for TCF4. Basically the gene file is a 1 column list that is used to filter $8 of the attached file. When there is a...

10. UNIX for Beginners Questions & Answers

Check if string variable is a subset of another string variable

Below is my ksh shell script where I need to check if variable fileprops is a subset of $1 argument. echo "FILE PROPERTY: $fileprops" echo "PARAMETER3: $1" if ; then echo "We are Good. $line FILE is found to be INTACT !! " else echo...

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk sed cut? to rearrange random number of fields into 3 fields

Discussion started by: axo959

2. Shell Programming and Scripting

awk print variable then fields in variable

Discussion started by: SkySmart

3. Shell Programming and Scripting

Join fields comparing 4 fields using awk

Discussion started by: aksijain

4. Shell Programming and Scripting

How to print 1st field and last 2 fields together and the rest of the fields after it using awk?

Discussion started by: 100bees

5. Shell Programming and Scripting

Concatenating awk fields with variable

Discussion started by: sniper57

6. Shell Programming and Scripting

awk - compare 1st 15 fields of record with 20 fields

Discussion started by: sljnk

7. Shell Programming and Scripting

awk processing of variable number of fields data file

Discussion started by: radudownload

8. Shell Programming and Scripting

awk sort based on difference of fields and print all fields

Discussion started by: newstart

9. Shell Programming and Scripting

awk to filter file using another working on smaller subset

Discussion started by: cmccabe

10. UNIX for Beginners Questions & Answers

Check if string variable is a subset of another string variable

Discussion started by: mohtashims