Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk scripting
Top Forums Shell Programming and Scripting awk scripting Post 302321747 by kiranmosarla on Tuesday 2nd of June 2009 04:24:56 AM
Old 06-02-2009
awk scripting
Hi

I have 2 files of large size( 10 Miilions) , i want to join 2 files based on some condition .
for this taking lot of time and 100 cpu .i want to iterate the based on some value (like 1 lakh)
I put the 2 files in the associative arrays . if the array limit reaches the 1 lach join the with the second file . this process is repeate at last line of the file .


BEGIN{
FS="|"
while((getline line < analyzer ) > 0 ){
split( line,analyzerInfo,"|")
domainName =analyzerInfo[1]
domainPurpose=analyzerInfo[3]
if(domainPurpose ~ /^PPC_PARKED$/){
if( domainName in dnameArrayanalyzer){
#dnameArrayanalyzer[domainName]=line
}
else{
dnameArrayanalyzer[domainName]=analyzerInfo[1]
}
}
}
}
{
domainNameRegi =$1
if( domainNameRegi in dnameArrayanalyzer){
RegiId=$2
RagiName=$3
if( domainNameRegi in Regiinfo ){
}
else{
#print $0 > RegiId"Domains" #it prints the domains into RegiIdDomains if the RegiID same
print $0 > "Domains" #it prints the domains into Domains
if(RagiName in Regiinfo){
Regiinfo[RagiName] = Regiinfo[RagiName] + 1
}
else{
Regiinfo[RagiName]=1
}
}
}
else{
#print $0 >"domainNotMatchRegi"
}
}
END{
print "########################################" > "Domains"
print " NoofDomains RegistrarWise " >"Domains"
print "########################################" > "Domains"
for( var in Regiinfo ){
print var "| " Regiinfo[var] > "Domains" #"NoOfDomainsRegistrarWise"
}
}

when i am executing lot of time for 10 miilion records .
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Awk scripting

Hi, I'm new to unix and i am kind of familiar with the basic commands. can anyone suggest some good books especially for AWK scripting and SHELL scripting thanks, Hari (2 Replies)
Discussion started by: rharee
2 Replies

2. Shell Programming and Scripting

AWK scripting

I have a text file in which the text has been divided into paragraphs (two line breaks or tab marks a new paragraph) and I want to make a script which output would delete line breaks within the paragraph and the different paragraphs would be separated by two line breaks. So, if my input file... (14 Replies)
Discussion started by: Muki101
14 Replies

3. Shell Programming and Scripting

Scripting via awk

Hi, I am trying to understand what is happening here, new to scripting: I have a couple of these, but if I knew what was going on in one I can figure out the rest: awk '/rpc-100083/ { $2 = "enable -r" } $3 ~ /.NOS99dtlogin/ { $t = $2; $2 = $3; $3 = $t } { print }' /var/svc/profile/upgrade... (2 Replies)
Discussion started by: ulemsee
2 Replies

4. Shell Programming and Scripting

scripting/awk help : awk sum output is not comming in regular format. Pls advise.

Hi Experts, I am adding a column of numbers with awk , however not getting correct output: # awk '{sum+=$1} END {print sum}' datafile 2.15291e+06 How can I getthe output like : 2152910 Thank you.. # awk '{sum+=$1} END {print sum}' datafile 2.15079e+06 (3 Replies)
Discussion started by: rveri
3 Replies

5. Shell Programming and Scripting

Need help with awk scripting.

Hi, i am newbie to this site and hope to learn but problem is s but need help urgently. Plz pm me if you are good at this. Help will be appreciated. (11 Replies)
Discussion started by: Rookie80
11 Replies

6. Shell Programming and Scripting

Help with AWK and Scripting!

Hi, This is the first time I am working with awk and I am not familiar with any commands in it. But I managed to do most of my work just left with one more. Needing your help! I have to extract only the matrix (written within ) from a text file. For example: 1JTJ_0006_ACGC_NPNP_A_12_15.pdb ... (17 Replies)
Discussion started by: SriJit
17 Replies

7. Shell Programming and Scripting

Need help with awk scripting

hi all, I am working on awk scripting.I have created two awk files and now have a requirement of replacing the contents of first file with some contents of second file. Please find below the two files created.File1 has 3 records and File2 has 4 records. cat File1 111,0165,CB21031251,0165,... (3 Replies)
Discussion started by: csrohit
3 Replies

8. Shell Programming and Scripting

Need help in awk scripting

Hi I am beginner of shell/AWK scripting , can you please help me in select particular column and column between two pattern from a multiple column file. file1.txt number status date1 date2 description category ... (7 Replies)
Discussion started by: vijay_rajni
7 Replies

9. Shell Programming and Scripting

awk Scripting

Hey guys, I want to get all the columns in this input file tab-delimited, i need to get the column send them to a variable. From there i could print them in shuffle and pick and select columns i want. Here is the input sample 2013/08/05 06:50:38:067 MINOR SiteScope ... (9 Replies)
Discussion started by: ryandegreat25
9 Replies

10. Shell Programming and Scripting

New at scripting awk with variable

I am trying to write a shell script that will add a date to and awk command from the command prompt. here is the line I am having difficulties. zgrep -i playback $dir/$1-*-errors.out.gz |cut -d '|' -f 1,11,12,15 | awk -v start=${start} -v end=${end} -F '|' '{$1>=start && $1 <=end} {print $2... (7 Replies)
Discussion started by: infinity0880
7 Replies

LEARN ABOUT SUNOS

lpq

lpq(1B) 					     SunOS/BSD Compatibility Package Commands						   lpq(1B)

NAME

       lpq - display the content of a print queue

SYNOPSIS

       /usr/ucb/lpq [-P destination] [-l] [ + [interval]] [ request-ID...] [user...]

DESCRIPTION

       The lpq utility displays the information about the contents of a print queue. A print queue is comprised of print requests that are waiting
       in the process of being printed.

       lpq displays the following information to the standard output:

	 o  the username of the person associated with a print request,

	 o  the position of a print request in the print queue,

	 o  the name of file or files comprising a print request,

	 o  the job number of a print request, and

	 o  the size of the file requested by a print request. File size is reported in bytes.

       Normally, only as much information as will fit on one line is displayed. If the name of the input file associated with a print  request	is
       not available, the input file field indicates the standard input.

       The  print  client  commands  locate  destination information using the printers database in the name service switch. See nsswitch.conf(4),
       printers(4), and printers.conf(4) for details.

OPTIONS

       The following options are supported:

       -P destination  Displays information about printer or class of printers (see lpadmin(1M)) . Specify destination using atomic or POSIX-style
		       (server:destination)  names.  See  printers.conf(4)  for  information regarding the naming conventions for atomic names and
		       standards(5) for information regarding POSIX.

       -l	       Displays information in long format. Long format includes the name of the host from which a print request originated in the
		       display.

       + [interval]    Displays  information  at  specific  time intervals. Stops displaying information when the print queue is empty. Clears the
		       screen before reporting displaying the print queue. Specify interval as the number of seconds between displays. If   inter-
		       val is not specified only executes once.

OPERANDS

       The following operands are supported:

       request-ID      The job number associated with a print request.

       user	       The name of the user about whose jobs  lpq reports information. Specify user as a valid username.

EXIT STATUS

       The following exit values are returned:

       0	       Successful completion.

       non-zero        An error occurred.

FILES

       /var/spool/print/[cd]f*		       Spooling directory and request files for jobs awaiting transfer.

       /var/spool/lp/*			       LP print queue

       $HOME/.printers			       User-configurable printer database

       /etc/printers.conf		       System printer configuration database

       printers.conf.byname		       NIS version of /etc/printers.conf

       printers.org_dir 		       NIS+ version of /etc/printers.conf

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWscplp 		   |
       +-----------------------------+-----------------------------+

SEE ALSO

       lp(1), lpc(1B), lpr(1B), lprm(1B), lpstat(1), lpadmin(1M), nsswitch.conf(4), printers(4), printers.conf(4), attributes(5), standards(5)

SunOS 5.10							    23 May 2003 							   lpq(1B)
All times are GMT -4. The time now is 05:30 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy