Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Adding a delimiter to a variable length file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Adding a delimiter to a variable length file
# 1  
Old 09-02-2012
Adding a delimiter to a variable length file
Hi,

I'm new to unix, i have a variable length file like below,
Code:
01|Test|Test1|Sample|
02|AA|BB|CC|DD|
03|AAA|BBB|CCC|DDD|EEE|RRR|TTT|SSS|YYY|

I need to make this as a fixed length file. Assume that i have 10 columns in the DAT file.

for ex: the first 01 record is having 4cols - there i need to add six "|" delimiter and for 02 record need to add five delimiters. Like wise i need to read few record number and add the delimiter correspondingly..

Can anyone help me on this regard.

your help would be much appreciated..

Thanks in advance,
Mohankumar.V
Last edited by radoulov; 09-02-2012 at 01:27 PM..
# 2  
Old 09-02-2012
I think this might work:

Code:
awk -F "|" '{ printf( "%s%s\n", $0, substr( "||||||||||", 1, 11-NF ) ) }'  input-file >output-file

# 3  
Old 09-02-2012
Code:
$ awk -F'|' '{NF=11}1' OFS='|' input
01|Test|Test1|Sample|||||||
02|AA|BB|CC|DD||||||
03|AAA|BBB|CCC|DDD|EEE|RRR|TTT|SSS|YYY|

# 4  
Old 09-02-2012
Quote:
Originally Posted by neutronscott
Code:
$ awk -F'|' '{NF=11}1' OFS='|' input
01|Test|Test1|Sample|||||||
02|AA|BB|CC|DD||||||
03|AAA|BBB|CCC|DDD|EEE|RRR|TTT|SSS|YYY|

Just to add that in some awk implementations the assignment to NF won't trigger the reconstruction of the current record:

Code:
zsh-5.0.0[t]% awk -F\| 'NF=11' OFS=\| infile            
01|Test|Test1|Sample|||||||
02|AA|BB|CC|DD||||||
03|AAA|BBB|CCC|DDD|EEE|RRR|TTT|SSS|YYY|
zsh-5.0.0[t]% nawk -F\| 'NF=11' OFS=\| infile
01|Test|Test1|Sample|
02|AA|BB|CC|DD|
03|AAA|BBB|CCC|DDD|EEE|RRR|TTT|SSS|YYY|
zsh-5.0.0[t]% nawk --version
awk version 20070501

The above nawk is the GnuWin32 nawk.

In such cases something like this could also be used as a workaround:

Code:
awk F\| '{
  for (i = 0; ++i < = c;)
    printf "%s", ($i (i < c ? OFS : ORS))
  }' OFS=\| c=11 infile

Last edited by radoulov; 09-02-2012 at 01:50 PM..
# 5  
Old 09-02-2012
Adding a delimiter to a variable length file
Hi,

Many thanks for the response. i'm able to get the desired output. but i have a question. i need to write this in .ksh file, wouldn't i need to write a do while loop? bcoz i have some 80k records in the file. Would the command check all the lines?

don't mistake me if i wrong anywhere..

Regards,
Mohankumar.V
# 6  
Old 09-02-2012
Yes the above mentioned commands will check for all the records..no need of any loop..

my version:
Code:
 awk -F\| 'BEGIN{OFS=FS;}{$11="";}1' input_file

Last edited by msabhi; 09-02-2012 at 04:05 PM..
# 7  
Old 09-02-2012
Or:

Code:
awk -F\| '(NF < 11 && $11 = x) || 1' OFS=\| infile

This will work even if $11 == 0 Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert variable length record to fixed length

Hi Team, I have an issue to split the file which is having special chracter(German Char) using awk command. I have a different length records in a file. I am separating the files based on the length using awk command. The command is working fine if the record is not having any... (7 Replies)
Discussion started by: Anthuvan
7 Replies

2. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

3. Shell Programming and Scripting

changing a variable length text to a fixed length

Hi, Can anyone help with a effective solution ? I need to change a variable length text field (between 1 - 18 characters) to a fixed length text of 18 characters with the unused portion, at the end, filled with spaces. The text field is actually field 10 of a .csv file however I could cut... (7 Replies)
Discussion started by: dc18
7 Replies

4. Shell Programming and Scripting

Split variable length and variable format CSV file

Dear all, I have basic knowledge of Unix script and her I am trying to process variable length and variable format CSV file. The file length will depend on the numbers of Earnings/Deductions/Direct Deposits. And The format will depend on whether it is Earnings/Deductions or Direct Deposits... (2 Replies)
Discussion started by: chechun
2 Replies

5. Shell Programming and Scripting

Make variable length record a fixed length

Very, very new to unix scripting and have a unique situation. I have a file of records that contain 3 records types: (H)eader Records (D)etail Records (T)railer Records The Detail records are 82 bytes in length which is perfect. The Header and Trailer records sometimes are 82 bytes in... (3 Replies)
Discussion started by: jclanc8
3 Replies

6. Shell Programming and Scripting

Adding a delimiter to a text file

Im writing a KSH script to read a simple text file and add a delimiter. Ive written the following script but it runs very slow. I initially used the cut command to substring the input record then switched to this version using awk to substring... both run too slow. Any ideas how to make this more... (2 Replies)
Discussion started by: lock
2 Replies

7. UNIX for Dummies Questions & Answers

Convert a tab delimited/variable length file to fixed length file

Hi, all. I need to convert a file tab delimited/variable length file in AIX to a fixed lenght file delimited by spaces. This is the input file: 10200002<tab>US$ COM<tab>16/12/2008<tab>2,3775<tab>2,3783 19300978<tab>EURO<tab>16/12/2008<tab>3,28523<tab>3,28657 And this is the expected... (2 Replies)
Discussion started by: Everton_Silveir
2 Replies

8. Shell Programming and Scripting

Pivot variable record length file and change delimiter

Hi experts. I got a file (500mb max) and need to pivot it (loading into ORCL) and change BLANK delimiter to PIPE |. Sometimes there are multipel BLANKS (as a particular value may be BLANK, or simply two BLANKS instead of one BLANK). thanks for your input! Cheers, Layout... (3 Replies)
Discussion started by: thomasr
3 Replies

9. Shell Programming and Scripting

adding delimiter to a fixed width file

Hi , I have a file : CSCH74000.00 CSCH74000.00 CSCH74100.00 CSCH74000.00 CSCH74100.00 CSCH74000.00 CSCH74000.00 CSCH74100.00 CSCH74100.00 CSCH74100.00 I have to put a delimiter( say comma) in between after 6th character: CSCH74,000.00 CSCH74,000.00 CSCH74,100.00 (2 Replies)
Discussion started by: sumeet
2 Replies

10. Shell Programming and Scripting

creating a fixed length output from a variable length input

Is there a command that sets a variable length? I have a input of a variable length field but my output for that field needs to be set to 32 char. Is there such a command? I am on a sun box running ksh Thanks (2 Replies)
Discussion started by: r1500
2 Replies
Login or Register to Ask a Question