Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Nawk Problem - nawk out of space in tostring on

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Nawk Problem - nawk out of space in tostring on
# 1  
Old 02-17-2014
Nawk Problem - nawk out of space in tostring on
Hi.. i am running nawk scripts on solaris system to get records of file1 not in file2 and find duplicate records in a while with the following scripts -compare
Code:
nawk 'NR==FNR{a[$0]++;next;} !a[$0] {print"line"FNR $0}' file1 file2

duplicate -
Code:
nawk '{a[$0]++}END{for(i in a){if(a[i]-1)print i,a[i]}}' file1

in the middle of script I get an error message saying nawk: out of space in tostring on record 971360... I am using a file having 2 million records. Please suggest.. It is very very important...

I searched and came to know that gawk can solve this, but it won't run on Solaris..
Last edited by Scrutinizer; 02-17-2014 at 11:57 AM.. Reason: copied wrong code; mod: spelling
# 2  
Old 02-17-2014
Two suggestions -
1. gawk is available for Solaris 8->11 Sharing /opt/csw » OpenCSW Solaris packages
2. to find duplicates in a single file use an online method
Code:
  sort file | nawk 'NR==1 {old=$0; next}  {if (old==$0) {print $0}; old=$0}'

# 3  
Old 02-17-2014
Thanks Jim.. But I want to avoid using sort as that would reorganise my file and hence display of records which I want to avoid.. Is there not any other solution except using gawk as I don't have much control on my machine..
Last edited by Scrutinizer; 02-17-2014 at 11:52 AM.. Reason: SPELLING!
# 4  
Old 02-17-2014
Hi.
The nawk and Solaris I use doesn't like the code:
Code:
$ nawk 'NR==FNR{a[$0]++;next;} !a[$0] {print"line"FNR"$0}' file1 file2
nawk: syntax error at source line 1
 context is
        NR==FNR{a[$0]++;next;} !a[$0] >>>  {print"line"FNR"$0} <<< 
nawk: illegal statement at source line 1
        missing }

Possibly an un-matched double quote?

For this system:
Code:
OS, ker|rel, machine: SunOS, 5.10, i86pc
Distribution        : Solaris 10 10/08 s10x_u6wos_07b X86
bash GNU bash 3.00.16
nawk - ( /usr/bin/nawk, Jan 8 2007 )

I was trying to convert this to perl, which generally has better memory management.

cheers, drl
# 5  
Old 02-17-2014
Yes drl.. Double quote mismatch was there.. Please let me know if you can convert this to perl
Last edited by Scrutinizer; 02-17-2014 at 11:55 AM.. Reason: Language and spelling
# 6  
Old 02-17-2014
Hi.
Quote:
Originally Posted by Abhiraj Singh
yes drl..double quote mismatch was thr.. pls let me knw if u can convert this to perl
Please post the corrected code ... cheers, drl
# 7  
Old 02-17-2014
code editted
Quote:
Originally Posted by drl
Hi.

Please post the corrected code ... cheers, drl
Code is edited in the original post
Last edited by Scrutinizer; 02-17-2014 at 11:55 AM.. Reason: Spelling
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Nawk if logic problem

nawk '{ fmt="%3s %22s %48s %35s %21s\n"; if ($3==$6 && $1=="STOPLOSS") { tpy="Successful Match"; jnme=$1; sts="File will be loaded"; cntrl=$3; audit=$6; printf (fmt, tpy,jnme,sts,cntrl,audit) >> "'${AUDIT_DATA_FILE}/${AUDIT36}'" }else if ($3!=$6 && $1=="STOPLOSS") { tpy="Mis-Match ";... (4 Replies)
Discussion started by: wawa
4 Replies

2. Shell Programming and Scripting

problem in redirecting records using nawk

I am trying to redirect record to two files using nawk if-else. #Identify good and bad records and redirect records using if-then-else nawk -F"|" '{if(NF!=14){printf("%s\n",$0) >> "$fn"_bad_data}else{printf("%s\n",$0) >> $fn}}' "$fn".orig "$fn".orig is the source file name bad... (7 Replies)
Discussion started by: siteregsam
7 Replies

3. Shell Programming and Scripting

nawk and space in the parameter

Hi, Could you please tell me how nawk command works when there is a asterisk <*> or space with asterisk < *> or <* > in the parameter. I am just trying to read line by line and fetch fourth parameter separated by delimiter (|). But if there is a * or < *> or <* > in the fourth parameter it... (7 Replies)
Discussion started by: nram_krishna@ya
7 Replies

4. Shell Programming and Scripting

problem in nawk : case insensitive pattern matching

HI, My file contains data something like 034500,5,B5004946544EB185,DEFAULT,0 Now i want to do a pettern match for DEFAULT and remove that particular line from file and transfer the rest contents to temp file.But my req is i want to do case insensitive matching ie DEFAULT / default. I... (4 Replies)
Discussion started by: centurion_13
4 Replies

5. Shell Programming and Scripting

using nawk

help out with code. two files aaa bbb contains some records..output file xyz should be like this..see below i/p file:aaa 08350|60521|0000|505|0000|1555|000|NYCMT|Pd_1 |-11878 i/p file: bbb 60521|60510 o/p file :xyz 60510|08350|60521|0000|505|0000|1555|000|NYCMT|Pd_1 |-11878 (5 Replies)
Discussion started by: Diddy
5 Replies

6. Shell Programming and Scripting

Nesting - two nawk into one nawk

hi people; this is my two awk code: nawk '/cell+-/{r=(NF==8) ? $4FS$5FS$6 : NF==7 ? $4FS$5 : $4 ;c=split(r,rr);for (i=1;i<=c;i++){if(rr != "111111"){printf($3" %d ""\n",(i+3))}}printf("")}' /home/gc_sw/str.txt > /home/gc_sw/predwn.txt nawk -F'*' '{gsub(/ *$/,"")}$0=$1$($NF-2)'... (2 Replies)
Discussion started by: gc_sw
2 Replies

7. Shell Programming and Scripting

Multiple array problem in NAWK

I HAD these 2 files: file1 pictures.txt 5 ref2313 4 ref2345 3 ref5432 2 ref4244 1 dance.txt 6 ref2342 5 ref2352 4 ref0695 3 ref5738 2 ref4948 1 treehouse.txt 6 ref8573 5 ref3284 4 ref5838 3 ref4738 2 ref4573 1 file2 pictures.txt 1 3 dance.txt 2 4 treehouse.txt 3 5 what I... (1 Reply)
Discussion started by: linuxkid
1 Replies

8. Shell Programming and Scripting

Problem in splitiing file based on regex using awk/nawk

I have a file tmp.txt as shown below: Controller: 0 Disk: 0.0.0 Disk: 0.1.0 Disk: 0.2.0 Controller: 1 Volume:c1t2d0 Disk: 0.0.0 Disk: 0.1.0 Disk: 0.2.0 Controller: 2 Volume:c2t2d0 Disk: 0.2.0 Now I want to split... (4 Replies)
Discussion started by: durbam2002
4 Replies

9. Shell Programming and Scripting

how to access values of awk/nawk variables outside the awk/nawk block?

i'm new to shell scripting and have a problem please help me in the script i have a nawk block which has a variable count nawk{ . . . count=count+1 print count } now i want to access the value of the count variable outside the awk block,like.. s=`expr count / m` (m is... (5 Replies)
Discussion started by: saniya
5 Replies

10. Shell Programming and Scripting

nawk problem

How are ya, Heres the problem. I have a line of data that can either be in this format. "20" or "20kg" for the 20kg one i need to be able to read that their is kg on the end of this field and then ignore it and move on to the next line. Can anyone help. Cheers (3 Replies)
Discussion started by: ben_shark
3 Replies
Login or Register to Ask a Question