Aligning text files by max field length Post: 302546514

Sponsored Content

Top Forums Shell Programming and Scripting Aligning text files by max field length Post 302546514 by physalis2099 on Thursday 11th of August 2011 10:55:33 AM

08-11-2011

Registered User

Aligning text files by max field length

Hello,
Is there anyway that I can align a pipe delimited text file by the maxium field length where the field is separated out by pipes for large text files with more than
100,000 rows?

So, far I have searched other forums and google about aligning text files in unix and I have noticed that several other users use the awk utility. Since I am new to awk
I have attempted in writing my own code after reading some of the awk utility syntax, but I am getting stuck.
If awk is not the best utility to achieve this is there any way to code this???

My test code:

#!/bin/ksh
awk 'BEGIN {FS = "|"}
{
for(i=1;i<=NF;i++)
{
if (length($i) > max)
max = length($i)
maxlen($i) = max
}
}
END
{
for (i in max) print (i,max)
}
' $(find . -name "testfile.txt")

Below is a sample of the text file that I have:

Pipe Delimited Text file
YEAR|NAME|PRODUCT_ID|ORDER_ID|CUSTOMER_ID
2001|Unix book|12354|01587|5487651484
2002|Programming|65487|6564548|654365146
2003|Airsoft Guns|6544888|548|65498
2004|Video Games|101100018|44|648
2010|Wayside Stories from wayside school|5487454|4|64645646
.
.
.

Desired Output:

YEAR|NAME |PRODUCT_ID|ORDER_ID|CUSTOMER_ID
2001 |Unix book |12354 |01587 |5487651484
2002 |Programming |65487 |6564548 |654365146
2003 |Airsoft Guns |6544888 |548 |65498
2004 |Video Games |101100018 |44 |648
2010 |Wayside Stories from wayside school|5487454 |4 |64645646
.
.
.

Thanks,

Desired Output.txt (626 Bytes)

physalis2099

View Public Profile for physalis2099

Find all posts by physalis2099

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Length of a Unix filepath max length

Hi Guys, Could anyone shed some light on the length of a Unix filepath max length pls ? thanks ! Wilson

2. Shell Programming and Scripting

what is the max length of args i can pass in shell?

i have a shell script which takes several args. what is the maximum length of string i can give as argument?

3. UNIX for Dummies Questions & Answers

Read file and remove max length

Hi all, I tried to write a shell to read huge file and eliminate max length record which is wrong generated record. But I get an error remove_sp.sh: line 27: syntax error near unexpected token `else' remove_sp.sh: line 27: ` else $LINE >> REJFILE' My shell is here: #!/bin/sh...

4. Shell Programming and Scripting

Counting the max length of string

Hi all, I have a flat file of 1000 rows. I want to check the length of the 5th column. The one having the longest length , I want to set it as DEFINED PARAMETER. So later I can check others with that particular number only. Any ideas ??

5. UNIX for Advanced & Expert Users

How to increase max username length?

Hi, This is my first post to this site. So kindly forgive if I am writing in a wrong section. My query is that... I want to modify the max username length size. I guess it is 32/64 on CentOS. Now I want to change it to 128. Is there any way to do that? Thanks in advance!! :)

6. UNIX for Dummies Questions & Answers

Modify the max username length

Hey Any one... Do u know any way I can modify the max username length in unix? I guess it is 32/64 characters by default. Suppose I want to increase it to 128. i hav tried /etc/skel but no use... How can I do that?

7. Shell Programming and Scripting

Flat file-make field length equal to header length

Hello Everyone, I am stuck with one issue while working on abstract flat file which i have to use as input and load data to table. Input Data- ------ ------------------------ ---- ----------------- WFI001 Xxxxxx Control Work Item A Number of Records ------ ------------------------...

8. Shell Programming and Scripting

Aligning columns in a text file using Perl

Hi All, I am new to perl and was trying to write a simple program which will generate a text file as output.. now the output which i am getting is something like this.. ================================================================================================== Col1 ...

9. Shell Programming and Scripting

Read text between regexps and write into files based on a field in the text

Hi, I have a huge file that has data something like shown below: huge_file.txt start regexp Name=Name1 Title=Analyst Address=Address1 Department=Finance end regexp some text some text start regexp Name=Name2 Title=Controller Address=Address2 Department=Finance end regexp

10. Shell Programming and Scripting

Find max length of the field and then replace zero

hai guys, pick the 1st field and calculate max length. if suppose max length is 2, then compare the all records if <2 then add zero's prefix of the record. for ex: s.no,sname 1,djud 37,jtuhe in this max length of the 1st field is 2 right the output wil be s.no,sname 01,djud...

LEARN ABOUT SUSE

history

HISTORY(5)							File Formats Manual							HISTORY(5)

NAME

       history - record of current and recently expired Usenet articles

DESCRIPTION

       The  file  <pathdb in inn.conf>/history keeps a record of all articles currently stored in the news system, as well as those that have been
       received but since expired.  In a typical production environment, this file will be many megabytes.

       The file consists of text lines.  Each line corresponds to one article.	The file is normally kept sorted in the order  in  which  articles
       are  received,  although  this  is not a requirement.  Innd(8) appends a new line each time it files an article, and expire(8) builds a new
       version of the file by removing old articles and purging old entries.

       Each line consists of two or three fields separated by a tab, shown below as 	:
	      [Hash]	     	   date
	      [Hash]	     	   date	 	   token

       The Hash field is the ASCII representation of the hash of the Message-ID header.  This is directly used for the key of the dbz(3).

       The date field consists of three sub-fields separated by a tilde.  All sub-fields are the text representation  of  the  number  of  seconds
       since  the  epoch -- i.e., a time_t; see gettimeofday(2).  The first sub-field is the article's arrival date.  If copies of the article are
       still present then the second sub-field is either the value of the article's Expires header, or a hyphen if no expiration date  was  speci-
       fied.   If  an article has been expired then the second sub-field will be a hyphen.  The third sub-field is the value of the article's Date
       header, recording when the article was posted.

       The token field is a token of the article.  This field is empty if the article has been expired.

       For example, an article whose Message-ID was <7q2saq$sal$1@isrv4.pa.vix.com>, posted on 26 Aug 1999 08:02:34 GMT and  recieved  at  26  Aug
       1999 08:06:54 GMT, could have a history line (broken into three lines for display) like the following:
	      [E6184A5BC2898A35A3140B149DE91D5C]  	
		  935678987~-~935678821  	
		  @030154574F00000000000007CE3B000004BA@

       In  addition to the text file, there is a dbz(3) database associated with the file that uses the Message-ID field as a key to determine the
       offset in the text file where the associated line begins.  For historical reasons, the key includes the trailing    byte  (which  is  not
       stored in the text file).

HISTORY

       Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews.  This is revision 3782, dated 2000-08-17.

SEE ALSO

       dbz(3), expire(8), inn.conf(5), innd(8), makehistory(8).

																	HISTORY(5)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Length of a Unix filepath max length

Discussion started by: wilsontan

2. Shell Programming and Scripting

what is the max length of args i can pass in shell?

Discussion started by: senthilk615

3. UNIX for Dummies Questions & Answers

Read file and remove max length

Discussion started by: mr_bold

4. Shell Programming and Scripting

Counting the max length of string

Discussion started by: ganesh123