Joining fixed width files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Joining fixed width files
# 1  
Old 01-31-2018
Joining fixed width files

Hi All,

I need to join fixed width files on a column which is position 1 to 3 and need to have all the records from file1

file1.txt
Code:
Cu1nullL1L2
Cu2nullL1L2
Cu3nullL1L2

file2.txt
Code:
Cu1B1B2
Cu3B1B2

output.txt
Code:
Cu1L1B1L2B2
Cu2L1L2
Cu3L1B1L2B3

I tried but not getting the expected resuls.

Any inputs please.

Thanks
Shashi
# 2  
Old 01-31-2018
What did you try?

What operating system are you using?

What shell are you using?

Is output.txt supposed to be a fixed-width file too? If so, what fill character is supposed to be used to fill the "empty" space at the end of the Cu2 record?
# 3  
Old 01-31-2018
Hi Don,

I tried to print but unable to join the files and getting error.

Code:
awk '{print substr($0,1,3),substr($0,8,2),substr($0,10,2)}' file1.txt
awk '{print substr($0,1,3),substr($0,4,2),substr($0,6,2)}' file2.txt

I 'm using bash.
output.txt should be fixed width file and if there is no matching record then it should have white spaces at the end.

Thanks
Shash
# 4  
Old 01-31-2018
How about - for exactly your sample files given in post#1 -
Code:
awk '
                {$0 = substr($0,1,3) " " substr($0,length-3,2) " " substr($0,length-1,2)
                }

NR == FNR       {T1[$1] = $2
                 T2[$1] = $3
                 next
                }

                {$0 = $1 $2 T1[$1] $3 T2[$1]
                 $0 = sprintf ("%s%*s", $0, 11-length, "")
                }

1

' file2 file1
Cu1L1B1L2B2
Cu2L1L2    
Cu3L1B1L2B2

This User Gave Thanks to RudiC For This Post:
# 5  
Old 01-31-2018
Hi Shash,
Your desired output doesn't make any sense to me. When you find lines in both files for a given key, the data from the fields in both files are intermixed. When data is only found in one file, why isn't the output supposed to have the data from each field in that input file in the output columns related to those input fields. In other words, with your sample input data, why isn't the output:
Code:
Cu1L1B1L2B2
Cu2L1  L2  
Cu3L1B1L2B2

instead of the output you said you want:
Code:
Cu1L1B1L2B2
Cu2L1L2    
Cu3L1B1L2B3

especially since there is no B3 anywhere in either of your sample input files?

To get the output shown above, you could use something like:
Code:
awk '
FNR == NR {
	data1[key = substr($0, 1, 3), 1] = substr($0, 8, 2)
	data1[key, 2] = substr($0, 10, 2)
	keys[key]
	next
}
{	data2[key = substr($0, 1, 3), 1] = substr($0, 4, 2)
	data2[key, 2] = substr($0, 6, 2)
	keys[key]
}
END {	for(key in keys)
		printf("%s%2.2s%2.2s%2.2s%2.2s\n", key, data1[key, 1],
		    data2[key, 1], data1[key, 2], data2[key, 2])
}' file1.txt file2.txt > output.txt

but note that the order of the output lines may vary. If the output order matters, you need to clearly state how the output order should be determined when:
  • file1.txt contains keys that do not appear in file2.txt (as in your example),
  • file2.txt contains keys that do not appear in file1.txt, and
  • both files contain keys that do not appear in the other file.
Note that I asked what operating system you're using and you didn't answer.

If you're using a Solaris/SunOS system and want to try the above code, change awk to /usr/xpg4/bin/awk or nawk.

The missing 2nd components of the 2nd data1[] and data2[] assignments have been fixed as noted by RudiC in post #5.

Last edited by Don Cragun; 01-31-2018 at 06:40 PM.. Reason: Fix typos.
This User Gave Thanks to Don Cragun For This Post:
# 6  
Old 01-31-2018
Hmmm - I'm afraid the second index is missing in the
Quote:
data1[key] = substr($0, 10, 2)
and
Quote:
data2[key] = substr($0, 6, 2)
assignments, or do I get this wrong?
This User Gave Thanks to RudiC For This Post:
# 7  
Old 01-31-2018
Quote:
Originally Posted by RudiC
Hmmm - I'm afraid the second index is missing in the and assignments, or do I get this wrong?
Yes, indeed. You got it right.

Post #5 has been fixed.

Thanks,
Don
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Alter Fixed Width File

Thank u so much .Its working fine as expected. ---------- Post updated at 03:41 PM ---------- Previous update was at 01:46 PM ---------- I need one more help. I have another file(fixed length) that will get negative value (ex:-00000000003000) in postion (98 - 112) then i have to... (6 Replies)
Discussion started by: vinus
6 Replies

2. UNIX for Dummies Questions & Answers

Length of a fixed width file

I have a fixed width file of length 53. when is try to get the lengh of the record of that file i get 2 different answers. awk '{print length;exit}' <File_name> The above code gives me length 50. wc -L <File_name> The above code gives me length 53. Please clarify on... (2 Replies)
Discussion started by: Amrutha24
2 Replies

3. Shell Programming and Scripting

variable fixed-width fields

Hi there, CTL Port IO Rate(IOPS) Read Rate(IOPS) Write Rate(IOPS) Read Hit(%) Write Hit(%) Trans. Rate(MB/S) Read Trans. Rate(MB/S) Write Trans. Rate(MB/S) 09:36:48 0 A 136 0 135 97 100 ... (6 Replies)
Discussion started by: gray380
6 Replies

4. Shell Programming and Scripting

Comparing two fixed width file

Hi Guys I am checking the treads to get the answer but i am not able to get the answer for my question. I have two files. First file is a pattern file and the second file is the file i want to search in it. Output will be the lines from file2. File1: P2797f12af 44751228... (10 Replies)
Discussion started by: anshul_er
10 Replies

5. Shell Programming and Scripting

Fixed-Width file from Oracle

Hi All, I have created a script which generates FIXED-WIDTH file by executing Oracle query. SELECT RPAD(NVL(col1,CHR(9)),20)||NVL(col2,CHR(9))||NVL(col3,CHR(9) FROM XYZ It generates the data file with proper alignment. But if same file i transfer to windows server or Mainframe... (5 Replies)
Discussion started by: Amit.Sagpariya
5 Replies

6. Shell Programming and Scripting

Printing Fixed Width Columns

Hi everyone, I have been working on a pretty laborious shellscript (with bash) the last couple weeks that parses my firewall policies (from a Juniper) for me and creates a nifty little columned output. It does so using awk on a line by line basis to pull out the appropriate pieces of each... (4 Replies)
Discussion started by: cixelsyd
4 Replies

7. Shell Programming and Scripting

awk: creating a fixed-width single file from 2 different files

I have to create a single file from three files, Please see below for samples: day.txt 20090101 20090102 item.txt 123456789101 12345678910209 1234567891 str.txt 1 12 123 output.txt 20090101123456789101 1 0 2009010112345678910209 12 ... (2 Replies)
Discussion started by: tamahomekarasu
2 Replies

8. Shell Programming and Scripting

Removing \n within a fixed width record

I am trying to remove a line feed (\n) within a fixed width record. I tried the tr -d ‘\n' command, but it also removes the record delimiter. Is there a way to remove the line feed without removing the record delimiter? (10 Replies)
Discussion started by: CKT_newbie88
10 Replies

9. UNIX Desktop Questions & Answers

Help with Fixed width File Parsing

I am trying to parse a Fixed width file with data as below. I am trying to assign column values from each record to variables. When I parse the data, the spaces in all coumns are dropped. I would like to retain the spaces as part of the dat stored in the variables. Any help is appreciated. I... (4 Replies)
Discussion started by: sate911
4 Replies

10. UNIX for Dummies Questions & Answers

Fixed Width file using AWK

I am using the following command at the Unix prompt to make my 'infile' into a fixed width file of 100 characters. awk '{printf "%-100s\n",$0}' infile > outfile However, there are some records with a special character "©" These records are using 3 characters in place of one and my record... (2 Replies)
Discussion started by: alok.benjwal
2 Replies
Login or Register to Ask a Question