Map new file to old file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Map new file to old file
# 1  
Old 05-21-2015
Map new file to old file

Hellos,

Some columns of a file have undergone name changes from old to new format,
to run some of the scripts I need to revert back and rearrange these columns from the new to the old format.


oldfile has the column headers as following, data is not important


Code:
colA	colB	colC	colD	colE
10	11	12	13	15
14	14	14	14	16

newfile looks like the following

Code:
col1 	col2 	col3	col4	col5	col6
1	2	3	4	9	45
5	6	7	8	9	46

there is a names table which tells us which name changes to what

names

Code:
col5	colA
col4	colC
col2 	colD
col1	colB

I need to make the newfile look like the oldfile, by rearranging and renaming the columns, extra columns in the newfile appended to the end,ignoring the extra columns in the oldfile.

Code:
colA	colB	colC	colD	col3	col6
9	1	4	2	3	45
9	5	8	6	7	46




Here is my attempt, I`m struggling to rearrange the columns. Please assist.


Code:
awk '  FILENAME=="names" { n[$1]=$2;next }
       FILENAME=="oldfile"  && NR==1 { for (i=1;i<=NF;i++) o[i]=$i ; next }
       FILENAME=="newfile" && NR==1 {  s=x; for (i=1;i<=NF;i++) { 
       						for ($i in n) {
       							$i=n[$i] }
       						s=s FS n[$i]
       						{
       				       print s
       				     }' names oldfile newfile

# 2  
Old 05-21-2015
Quite a complex problem, here is my solution:

Code:
awk '
FNR==1 { file++ }
file==1 { new[$1]=$2; next }
file==2 {
  if(FNR==1) for(i=1;i<=NF;i++) old[i]=$i
  next
}
file==3&&FNR==1 {
    for(i=1;i<=NF;i++)
        if($i in new) {
            check=h
            for(j=1;j in old;j++) if(old[j]==new[$i]) { pos[j]=i; h++}
            if(check==h) {
                print "Error: cannot update field \"" $i "\": field \"" new[$i] "\" not found in oldfile" > "/dev/stderr"
                exit 1
            }
        }
        for(i=1;i<=NF;i++)
        if($i in new) $i=new[$i]
        else pos[++h]=i
}
{
   out=$(pos[1])
   for(i=2;i<=NF;i++) out= out FS $(pos[i])
   print out
}' names oldfile newfile

Edit: Added code in red, this is just for safety and ensures if you list a field in "names" and it cannot be found in "oldfile" we report a problem and abort. Without this check you get a corrupt output file with no error reported.

Last edited by Chubler_XL; 05-21-2015 at 08:45 PM..
This User Gave Thanks to Chubler_XL For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script (sh file) logic to compare contents of one file with another file and output to file

Shell script logic Hi I have 2 input files like with file 1 content as (file1) "BRGTEST-242" a.txt "BRGTEST-240" a.txt "BRGTEST-219" e.txt File 2 contents as fle(2) "BRGTEST-244" a.txt "BRGTEST-244" b.txt "BRGTEST-231" c.txt "BRGTEST-231" d.txt "BRGTEST-221" e.txt I want to get... (22 Replies)
Discussion started by: pottic
22 Replies

2. UNIX for Advanced & Expert Users

Map snps into a ref gene file

I have the following data set about the snps ID txt file POS ID 78599583 rs987435 33395779 rs345783 189807684 rs955894 33907909 rs6088791 75664046 rs11180435 218890658 rs17571465 127630276 rs17011450 90919465 rs6919430 and a gene... (7 Replies)
Discussion started by: marwah
7 Replies

3. AIX

Passing user/pass credentials to automountd map file.

How could I pass credentials to the automountd daemon through it's map files? Tried credentials=/some/file.txt - Didn't work. Tried SHARE.domain.com/user/pass/SHARE_NAME in map file - Didn't work Tried user=USER,pass=PASS - Didn't work. I can see the mounts created but when I access them... (4 Replies)
Discussion started by: Devyn
4 Replies

4. UNIX for Dummies Questions & Answers

Lookup field in map file

Hi, I have two questions which I would massively appreciate help with. 1. I am trying to insert a field into a file similar to the vlookup function in excel. In column 2 is a gene id for which i would like to insert the full name in the adjacent column. I have a map file (map.file) which... (1 Reply)
Discussion started by: genehersh
1 Replies

5. UNIX for Advanced & Expert Users

cannot find map file

Hi, all: My writed network device driver works fine when the transmitted file is under several MegaBytes, but above this size, especially dozens of or hundreds of MegaBytes, the kernel panic ocurres! I check the kern.log and find this error : 522 Nov 14 19:35:32 liklstar-server kerneNov 14... (2 Replies)
Discussion started by: liklstar
2 Replies

6. Shell Programming and Scripting

mappin strings of two different file and finding the mapped string and then map other fields.

As i am new to unix so facing some problems in scripting: here is my question: i m having two files. 1st file say a.txt contain 3 column like SPECIALITY|UMP_CODE|SPECIALTY_CODE Addictive Diseases|25ADD|ADD Addictive Diseases/Family Practice|25ADD|ADD/FP Aerospace Medicine|1.041666667|AM... (4 Replies)
Discussion started by: dsh007
4 Replies

7. Shell Programming and Scripting

Making a hash map from a records file

Greets again. Let me first say that this forum and all of you participants are amazing. You give of your time and knowledge to those of us who aspire and have not reached your level (yet). My thanks. I have created a file of records and wish to make a hash map from it. Here is an example of what... (6 Replies)
Discussion started by: rdettwyler
6 Replies

8. Linux

Auto map network drive using SAMBA with batch file

Hi everyone. I have several shares (see smb.conf below). Each setup fairly similarly, and several groups. My end goals are: 1) to have the share automatically map as the P: drive to members of the employees group 2) to have the share automatically map as the t: drive to members of the tech... (0 Replies)
Discussion started by: unassassinable
0 Replies

9. Shell Programming and Scripting

Script to map to .csv file

Hello Guys, I want to write a script which will search for all the *.c,*.h,*.txt in the file. The result will be again search in a .csv file. If that .c file or .h or .txt is existing in the .csv,then put a * mark corresponding to the row in .csv file. ---------- Post updated at 05:19 AM... (2 Replies)
Discussion started by: suvenduperl
2 Replies

10. UNIX for Dummies Questions & Answers

samba user.map file

I am using Samba 3.0.33. The user.map file is not referred anywhere is Smb.conf. Infact, I couldn't see the user.map file any where in samba. But everything seems to be working. Is there an alternative way to refer to the users list. (2 Replies)
Discussion started by: sagar_evc
2 Replies
Login or Register to Ask a Question