Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk record limition

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk record limition
# 1  
Old 01-27-2012
awk record limition
I read from my awk program book that awk has limitation of record length as 3000.
Can anyone tell me whether this is true? Is it possible to check awk limit setup?

The reason I ask is that I tested awk with input file a lot longer than 3000 characters and it is OK to generate output.

Thanks in advance.
# 2  
Old 01-27-2012
If you tried that on solaris, it would barf with a message about the line being too long.

But if you used nawk on solaris instead, it'd work fine.

So awk can be that limited on some systems, just for the sake of backwards compatibility with someone's 70's-era shell script. But those systems usually have better versions available as nawk or gawk..
# 3  
Old 01-27-2012
Thanks
I run awk on AIX. Do you have any idea about AIX?
# 4  
Old 01-27-2012
My knowledge on AIX is very limited, but in general, the older, more commercial, and more expensive a distribution of UNIX is, the more likely they are to hide the better utilities in strange places so somebody abusing 30-year-old awk bugs won't have the floor ripped out from under him.

If you have gawk or nawk, use them in preference to plain 'awk'. Most distributors supply their own 'nawk', occasionally in weird paths. gawk is an open-source thing, and could perhaps be installed by the sysadmin on request.

If you're working in a linux desktop or server, you're set, because awk and gawk are the same thing there -- GNU awk, with all the features of nawk and more besides.
# 5  
Old 01-27-2012
Quote:
Originally Posted by menglm
Thanks
I run awk on AIX. Do you have any idea about AIX?
The awk installed on AIX and HPUX is nawk...only Solaris maintains that distinction where awk is the original awk and nawk is the new awk...and hope this helps
# 6  
Old 01-28-2012
I am not sure about AIX. But for gawk, the source code has a file on limitations:
Code:
# cat gawk-4.0.0/LIMITATIONS 
  Copyright (C) 2005, 2006 Free Software Foundation, Inc.
  
  Copying and distribution of this file, with or without modification,
  are permitted in any medium without royalty provided the copyright
  notice and this notice are preserved.

This file describes limits of gawk on a Unix system (although it
is variable even then).  Non-Unix systems may have other limits.

# of fields in a record:  MAX_LONG
Length of input record:  MAX_INT 
Length of output record:  unlimited
Size of a field:  MAX_INT 
Size of a printf string:  MAX_INT 
Size of a literal string:  MAX_INT 
Characters in a character class:  2^(# of bits per byte)
# of file redirections:  unlimited
# of pipe redirections:  min(# of processes per user, # of open files)
double-precision floating point
Length of source line:  unlimited
Number of input records in one file: MAX_LONG
Number of input records total: MAX_LONG

If you want to find out how your AIX's awk limitation, try this:
Code:
# cat limit.py
col=1
while True:
        count=col
        while count>0:
                print count,
                count-=1
        print
        col+=1


# python limit.py | awk '{print NF}'

On my 32-bit netbook running Ubuntu:
Code:
awk: program limit exceeded: maximum number of fields size=32767
	FILENAME="-" FNR=32768 NR=32768
Traceback (most recent call last):
  File "./limit.py", line 8, in <module>
    print count,
IOError: [Errno 32] Broken pipe

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk record matching

ok. so i have a list of country names which have been abbreviated. we'll call this list A i have another list that which contains the what country each abbreviated name means. we'll call this list B. so example of the content of list B: #delimited by tabs #ABBR COUNTRY COUNTRY... (2 Replies)
Discussion started by: SkySmart
2 Replies

2. Shell Programming and Scripting

How to compare current record,with next and previous record in awk without using array?

Hi! all can any one tell me how to compare current record of column with next and previous record in awk without using array my case is like this input.txt 0 32 1 26 2 27 3 34 4 26 5 25 6 24 9 23 0 32 1 28 2 15 3 26 4 24 (7 Replies)
Discussion started by: Dona Clara
7 Replies

3. UNIX for Dummies Questions & Answers

awk: record too long

Hi All , I am getting record too long for the below command . nawk -F\" '{a=a" "$2} END{for(i in a) print i,a }' test|sort|awk '{for(i=1;i<=NF;i++) t=t"\t"$i;if(NF>max)max=NF} END{for(i=1;i<=max;i++)print t }' File test has 850 records ... Please help.. (2 Replies)
Discussion started by: saj
2 Replies

4. Shell Programming and Scripting

Sorting within a record using AWK

Hello, I have a file which has the following format: I have to do is sort individual records in the file based on the 4th field. Each record starts with "Module". Is there an easy way to do this using awk. I have tried piping output from awk to sort and also using "sort" inside awk but... (8 Replies)
Discussion started by: fifteate
8 Replies

5. Shell Programming and Scripting

Trouble getting the next to last record with awk

Hello all, I'm a beginner to shell/ awk script writing, and I'm trying to do something that looks like it shouldn't be (too) hard at all to do, but unfortunately, I can't seem to be able to find the right way to do it with awk. I need to look for the time several processes start & end in a... (5 Replies)
Discussion started by: Muadib
5 Replies

6. UNIX for Dummies Questions & Answers

Record splitting with AWK

Hi all ! I need your help as quick as possible. My input file like this: bạc těnh （ 薄情） 1 . 薄情な．２． 夫婦或いは男女の不貞を指す。 bách (百，迫)1.100ﾄﾞｿ. tr a m b a c ともいう． ２．柏（ｶヽしわ）･　3.圧迫する．４．差し迫った， My propose is take the value in the firt bracket. I used the command like : ...if (index(... (6 Replies)
Discussion started by: maixu134
6 Replies

7. Shell Programming and Scripting

AWK - if last line/record do something

Hello: I am trying to perform a certain action if the current record is the last line of the input file. But I am unable to figure out how to determine the last line of a file in awk. I need to do something like this: awk '{ if (lastline == NR) Do Something}' myfile.txt I have tried the... (3 Replies)
Discussion started by: PacificWonder
3 Replies

8. Shell Programming and Scripting

help with awk delimited by |~| in a record

I have a file which contains 1 million records of the following format. Each field is delimited by a pipe tilda pipe "|~|" show below. I would like to print only one column ie the CARDDESC value. for example here it says CARDDESC=A8T1. so anything after CARDDESC= and before |~|CARDTYPE is what... (11 Replies)
Discussion started by: knijjar
11 Replies

9. Shell Programming and Scripting

awk: record has too many fields

Hi, I'm trying this command - but get this error. Do you guys have any workaround for this? cat tf|sed 's/{//g'|sed 's/,//g'|awk '{for (i=1;i<=NF;i++) {if ($i == "OPTIME") {k = i + 2; print $i,$k}}}' awk: record `2005 Jul 28 17:35:29...' has too many fields record number 15 This is how... (3 Replies)
Discussion started by: chaandana
3 Replies

10. UNIX for Dummies Questions & Answers

Record too long for awk

I am trying to generate a small report with the help of awk. The contents are present in a file whose last line is very long. I can't shorten this line as its generated after a lot of processing. On reading this file awk says record "starting of line ..." too long record number 30 Now... (2 Replies)
Discussion started by: vibhor_agarwali
2 Replies
Login or Register to Ask a Question