Visit Our UNIX and Linux User Community


Appending different columns of multiple files in awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Appending different columns of multiple files in awk
# 1  
Old 10-22-2017
Appending different columns of multiple files in awk

Hello All,
I have three input files
Code:
cat file1
col1|col2|col3
a|1|A
b|2|B

cat file2
col1|col2|col3
c|3|C

cat file3
col1|col2|col3
d|4|D
e|5|E

i want below output
file4
Code:
col1|col2
a|1
3|C
D|4

there is no logic inside it. I am just looking a way to append different fields of different files in awk

there is one error giving psuedo code i could write.
Code:
awk -F"|" '{ print $1,$2 }' file1 '{print $2,$3}' file2 '{print $3,$1}' file3

please help me on this..
# 2  
Old 10-22-2017
Quote:
Originally Posted by looney
Code:
cat file1
col1|col2|col3
a|1|A
b|2|B

cat file2
col1|col2|col3
c|3|C

cat file3
col1|col2|col3
d|4|D
e|5|E

i want below output
file4
Code:
col1|col2
a|1
3|C
D|4

Where does the D|4 come from?
What happened to b|2|B?

---------- Post updated at 10:20 AM ---------- Previous update was at 09:52 AM ----------

Quote:
Originally Posted by looney
there is one error giving psuedo code i could write.
Code:
awk -F"|" '{ print $1,$2 }' file1 '{print $2,$3}' file2 '{print $3,$1}' file3

please help me on this..
The format of an AWK command looks as follows:
Code:
awk 'pattern {action}' input-file ...

Which means until here everything will go well:
Code:
awk -F"|" '{ print $1,$2 }' file1

The rest is not correct, unless all of them are files.
Code:
'{print $2,$3}' file2 '{print $3,$1}' file3

Quote:
Originally Posted by looney
there is no logic inside it. I am just looking a way to append different fields of different files in awk
That sounds a bit like what's the answer to the ultimate question of Life, The Universe, and Everything.
# 3  
Old 10-22-2017
Try:
Code:
awk '
  BEGIN {
    FS=OFS="|"
    print "col1|col2"
  }

  FNR==n+1 {
    print $p1, $p2
  }
' n=1 p1=1 p2=2 file1 p1=2 p2=3 file2 p1=3 p2=2 file3

Code:
col1|col2
a|1
3|C
D|4

This User Gave Thanks to Scrutinizer For This Post:
# 4  
Old 10-22-2017
@Aia Thanks for looking into it. As i said there is no logic , i have randomly selected different columns of different files. I know the code i wrote was not correct that is the reason i mentioned pseudo code just to make it understand. Perhaps it created more confusion. Smilie

@Scrutinizer Thanks , this is what i was looking for. So the magic lies in p1=1 p2=2 file1 p1=2 p2=3 file2 p1=3 p2=2 file3.
I don't know why thanks button is disable!
# 5  
Old 10-22-2017
Code:
Code:
awk -v answer42='col1|col2' '
  BEGIN {
    FS=OFS="|"
    print answer42
  }

  FNR==universe+1 {
    print $mouse1, $mouse2
  }
' FS=\| OFS=\| universe=1 mouse1=1 mouse2=2 file1 mouse1=2 mouse2=3 file2 mouse1=3 mouse2=2 file3

Output:
Code:
col1|col2
a|1
3|C
D|4

This User Gave Thanks to Aia For This Post:
# 6  
Old 10-22-2017
Hello looney,

Could you please try following approach too(with function) and let me know if this helps you.
Code:
awk -F"|" '
function print_val(var){
  num=split(var, array,",");
  for(i=1;i<=num;i++){
    printf("%s%s",$array[i],i==num?RS:"|")
}
}
BEGIN{
  print "col1|col2"
}
FNR==1{
  ++file;
  next
}
file==1{
  print_val("1,2");
  nextfile
}
file==2{
  print_val("2,3");
  nextfile
}
file==3{
  print_val("3,2");
  nextfile
}
'   Input_file1  Input_file2  Input_file3

Output will be as follows.
Code:
col1|col2
a|1
3|C
D|4

You need to pass number of fields which you want to print to function named eg--> print_val("1,3") and it should do the trick, since you want to read only very first line of each Input_file so I have used nextfile so that it will NOT go further for Input_file reading and will switch to next Input_file, moreover you need NOT to create many variables in case number of Input_file(s) are too much(tested this in GNU awk).

Thanks,
R. Singh
# 7  
Old 10-22-2017
Quote:
Originally Posted by looney
.
.
.
there is no logic inside it. I am just looking a way to append different fields of different files in awk.
.
.
.
If there's no logics to it, you can't program it. Programming means cast logics into code and apply repeatedly onto structurally identical but contentswise differing data, like everyday's log files, or multiple customer entries in a DB.
All above proposals convert your input data exactly as given into output exactly as in post#1. Why don't you do it in a one off editor session?
This User Gave Thanks to RudiC For This Post:

Previous Thread | Next Thread
Test Your Knowledge in Computers #786
Difficulty: Medium
In 2019, Bill Gates had a net worth estimated to be over $200 billion USD.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Appending multiple files through UNIX

Hi All, I have requirement where multiple csv files are present in a directory and each file contains a header.I need to append the contents of all the files into one file by removing header. Once the data is merged in one file ,I need to remove duplicates on nth column to find out distinct... (2 Replies)
Discussion started by: STCET22
2 Replies

2. UNIX for Dummies Questions & Answers

Appending multiple files

I am trying to append multiple files in a directory cat /a/file1.txt /a/file2.txt /a/file3.txt /a/file4.txt > /a/file.txt Except file2 every other file is appending. I interchanged file names and ran the command. Whatever file repeating in the second position is missing in output... (6 Replies)
Discussion started by: eskay
6 Replies

3. Shell Programming and Scripting

awk arrays comparing multiple columns across two files.

Hi, I'm trying to use awk arrays to compare values across two files based on multiple columns. I've attempted to load file 2 into an array and compare with values in file 1, but success has been absent. If anyone has any suggestions (and I'm not even sure if my script so far is on the right lines)... (4 Replies)
Discussion started by: hubleo
4 Replies

4. Shell Programming and Scripting

awk script to split file into multiple files based on many columns

So I have a space delimited file that I'd like to split into multiple files based on multiple column values. This is what my data looks like 1bc9A02 1 10 1000 FTDLNLVQALRQFLWSFRLPGEAQKIDRMMEAFAQRYCQCNNGVFQSTDTCYVLSFAIIMLNTSLHNPNVKDKPTVERFIAMNRGINDGGDLPEELLRNLYESIKNEPFKIPELEHHHHHH 1ku1A02 1 10... (9 Replies)
Discussion started by: viored
9 Replies

5. UNIX for Dummies Questions & Answers

Appending columns at the end of output using awk/sed

Hi , I have the below ouput, =====gopi===== assasassaa adsadsadsdsada asdsadsadasdsa sadasdsadsd =====kannan=== asdasdasd sadasddsaasd adasdd =====hbk=== asasasssa .... .. I want the output like as below, not able paste here correctly. (2 Replies)
Discussion started by: eeegopikannan
2 Replies

6. Shell Programming and Scripting

Appending columns of two files using shell script

Hi, I am using ksh, I want to read one csv file and append the columns of another file with new column. My input file: col1,col2 --------- siri,886 satya,890 priya,850 Another file with the below date:(test.csv) col3 ----- 321 333 442 (1 Reply)
Discussion started by: siri_886
1 Replies

7. Shell Programming and Scripting

Extracting columns from multiple files with awk

hi everyone! I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next} {print a, $2}' file1 file2 I added the file3, file4 and... (10 Replies)
Discussion started by: orcaja
10 Replies

8. UNIX for Dummies Questions & Answers

Extracting columns from multiple files with awk

hi everyone! I already posted it in scripts, I'm sorry, it's doubled I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next}... (1 Reply)
Discussion started by: orcaja
1 Replies

9. Shell Programming and Scripting

appending several columns with awk and paste

Hello, I am trying to solve for a couple of hours now the following problem: I have n files and would like to add the third column of each file to a new file: temp1.txt 1 2 3 1 2 3 1 2 3 temp2.txt 1 2 4 1 2 4 1 2 4 1 2 4 temp3.txt (2 Replies)
Discussion started by: creamcheese
2 Replies

10. Shell Programming and Scripting

awk 3 files to one based on multiple columns

Hi all, I have three files, one is a navigation file, one is a depth file and one is a file containing the measured field of gravity. The formats of the files are; navigation file: 2006 320 17 39 0 0 *nav 21.31542 -157.887 2006 320 17 39 10 0 *nav 21.31542 -157.887 2006 320 17 39 20 0... (2 Replies)
Discussion started by: andrealphus
2 Replies

Featured Tech Videos