Nawk Problem - nawk out of space in tostring on


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Nawk Problem - nawk out of space in tostring on
# 1  
Nawk Problem - nawk out of space in tostring on

Hi.. i am running nawk scripts on solaris system to get records of file1 not in file2 and find duplicate records in a while with the following scripts -compare
Code:
nawk 'NR==FNR{a[$0]++;next;} !a[$0] {print"line"FNR $0}' file1 file2

duplicate -
Code:
nawk '{a[$0]++}END{for(i in a){if(a[i]-1)print i,a[i]}}' file1

in the middle of script I get an error message saying nawk: out of space in tostring on record 971360... I am using a file having 2 million records. Please suggest.. It is very very important...

I searched and came to know that gawk can solve this, but it won't run on Solaris..

Last edited by Scrutinizer; 02-17-2014 at 12:57 PM.. Reason: copied wrong code; mod: spelling
# 2  
Two suggestions -
1. gawk is available for Solaris 8->11 Sharing /opt/csw OpenCSW Solaris packages
2. to find duplicates in a single file use an online method
Code:
  sort file | nawk 'NR==1 {old=$0; next}  {if (old==$0) {print $0}; old=$0}'

# 3  
Thanks Jim.. But I want to avoid using sort as that would reorganise my file and hence display of records which I want to avoid.. Is there not any other solution except using gawk as I don't have much control on my machine..

Last edited by Scrutinizer; 02-17-2014 at 12:52 PM.. Reason: SPELLING!
# 4  
Hi.
The nawk and Solaris I use doesn't like the code:
Code:
$ nawk 'NR==FNR{a[$0]++;next;} !a[$0] {print"line"FNR"$0}' file1 file2
nawk: syntax error at source line 1
 context is
        NR==FNR{a[$0]++;next;} !a[$0] >>>  {print"line"FNR"$0} <<< 
nawk: illegal statement at source line 1
        missing }

Possibly an un-matched double quote?

For this system:
Code:
OS, ker|rel, machine: SunOS, 5.10, i86pc
Distribution        : Solaris 10 10/08 s10x_u6wos_07b X86
bash GNU bash 3.00.16
nawk - ( /usr/bin/nawk, Jan 8 2007 )

I was trying to convert this to perl, which generally has better memory management.

cheers, drl
# 5  
Yes drl.. Double quote mismatch was there.. Please let me know if you can convert this to perl

Last edited by Scrutinizer; 02-17-2014 at 12:55 PM.. Reason: Language and spelling
# 6  
Hi.
Quote:
Originally Posted by Abhiraj Singh
yes drl..double quote mismatch was thr.. pls let me knw if u can convert this to perl
Please post the corrected code ... cheers, drl
# 7  
code editted

Quote:
Originally Posted by drl
Hi.

Please post the corrected code ... cheers, drl
Code is edited in the original post

Last edited by Scrutinizer; 02-17-2014 at 12:55 PM.. Reason: Spelling
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #364
Difficulty: Medium
vi first appeared in 1BSD.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Nawk if logic problem

nawk '{ fmt="%3s %22s %48s %35s %21s\n"; if ($3==$6 && $1=="STOPLOSS") { tpy="Successful Match"; jnme=$1; sts="File will be loaded"; cntrl=$3; audit=$6; printf (fmt, tpy,jnme,sts,cntrl,audit) >> "'${AUDIT_DATA_FILE}/${AUDIT36}'" }else if ($3!=$6 && $1=="STOPLOSS") { tpy="Mis-Match ";... (4 Replies)
Discussion started by: wawa
4 Replies

2. Shell Programming and Scripting

problem in redirecting records using nawk

I am trying to redirect record to two files using nawk if-else. #Identify good and bad records and redirect records using if-then-else nawk -F"|" '{if(NF!=14){printf("%s\n",$0) >> "$fn"_bad_data}else{printf("%s\n",$0) >> $fn}}' "$fn".orig "$fn".orig is the source file name bad... (7 Replies)
Discussion started by: siteregsam
7 Replies

3. Shell Programming and Scripting

nawk and space in the parameter

Hi, Could you please tell me how nawk command works when there is a asterisk <*> or space with asterisk < *> or <* > in the parameter. I am just trying to read line by line and fetch fourth parameter separated by delimiter (|). But if there is a * or < *> or <* > in the fourth parameter it... (7 Replies)
Discussion started by: nram_krishna@ya
7 Replies

4. Shell Programming and Scripting

problem in nawk : case insensitive pattern matching

HI, My file contains data something like 034500,5,B5004946544EB185,DEFAULT,0 Now i want to do a pettern match for DEFAULT and remove that particular line from file and transfer the rest contents to temp file.But my req is i want to do case insensitive matching ie DEFAULT / default. I... (4 Replies)
Discussion started by: centurion_13
4 Replies

5. Shell Programming and Scripting

Nesting - two nawk into one nawk

hi people; this is my two awk code: nawk '/cell+-/{r=(NF==8) ? $4FS$5FS$6 : NF==7 ? $4FS$5 : $4 ;c=split(r,rr);for (i=1;i<=c;i++){if(rr != "111111"){printf($3" %d ""\n",(i+3))}}printf("")}' /home/gc_sw/str.txt > /home/gc_sw/predwn.txt nawk -F'*' '{gsub(/ *$/,"")}$0=$1$($NF-2)'... (2 Replies)
Discussion started by: gc_sw
2 Replies

6. Shell Programming and Scripting

Multiple array problem in NAWK

I HAD these 2 files: file1 pictures.txt 5 ref2313 4 ref2345 3 ref5432 2 ref4244 1 dance.txt 6 ref2342 5 ref2352 4 ref0695 3 ref5738 2 ref4948 1 treehouse.txt 6 ref8573 5 ref3284 4 ref5838 3 ref4738 2 ref4573 1 file2 pictures.txt 1 3 dance.txt 2 4 treehouse.txt 3 5 what I... (1 Reply)
Discussion started by: linuxkid
1 Replies

7. Shell Programming and Scripting

Problem in splitiing file based on regex using awk/nawk

I have a file tmp.txt as shown below: Controller: 0 Disk: 0.0.0 Disk: 0.1.0 Disk: 0.2.0 Controller: 1 Volume:c1t2d0 Disk: 0.0.0 Disk: 0.1.0 Disk: 0.2.0 Controller: 2 Volume:c2t2d0 Disk: 0.2.0 Now I want to split... (4 Replies)
Discussion started by: durbam2002
4 Replies

8. Shell Programming and Scripting

nawk help

Hi Gurus, I am using a script as under : read string nawk -v search="$string" ' /a/,/z/ { block = (block ? block ORS : "") $0; } /z/ { if (block ~ search) print block; } ' <File> nawk -v search="$string" ' /b/,/z/ { ... (1 Reply)
Discussion started by: vanand420
1 Replies

9. Shell Programming and Scripting

how to access values of awk/nawk variables outside the awk/nawk block?

i'm new to shell scripting and have a problem please help me in the script i have a nawk block which has a variable count nawk{ . . . count=count+1 print count } now i want to access the value of the count variable outside the awk block,like.. s=`expr count / m` (m is... (5 Replies)
Discussion started by: saniya
5 Replies

10. Shell Programming and Scripting

nawk problem

How are ya, Heres the problem. I have a line of data that can either be in this format. "20" or "20kg" for the 20kg one i need to be able to read that their is kg on the end of this field and then ignore it and move on to the next line. Can anyone help. Cheers (3 Replies)
Discussion started by: ben_shark
3 Replies