awk scripting Post: 302321747

Sponsored Content

Top Forums Shell Programming and Scripting awk scripting Post 302321747 by kiranmosarla on Tuesday 2nd of June 2009 04:24:56 AM

06-02-2009

Registered User

awk scripting

Hi

I have 2 files of large size( 10 Miilions) , i want to join 2 files based on some condition .
for this taking lot of time and 100 cpu .i want to iterate the based on some value (like 1 lakh)
I put the 2 files in the associative arrays . if the array limit reaches the 1 lach join the with the second file . this process is repeate at last line of the file .

BEGIN{
FS="|"
while((getline line < analyzer ) > 0 ){
split( line,analyzerInfo,"|")
domainName =analyzerInfo[1]
domainPurpose=analyzerInfo[3]
if(domainPurpose ~ /^PPC_PARKED$/){
if( domainName in dnameArrayanalyzer){
#dnameArrayanalyzer[domainName]=line
}
else{
dnameArrayanalyzer[domainName]=analyzerInfo[1]
}
}
}
}
{
domainNameRegi =$1
if( domainNameRegi in dnameArrayanalyzer){
RegiId=$2
RagiName=$3
if( domainNameRegi in Regiinfo ){
}
else{
#print $0 > RegiId"Domains" #it prints the domains into RegiIdDomains if the RegiID same
print $0 > "Domains" #it prints the domains into Domains
if(RagiName in Regiinfo){
Regiinfo[RagiName] = Regiinfo[RagiName] + 1
}
else{
Regiinfo[RagiName]=1
}
}
}
else{
#print $0 >"domainNotMatchRegi"
}
}
END{
print "########################################" > "Domains"
print " NoofDomains RegistrarWise " >"Domains"
print "########################################" > "Domains"
for( var in Regiinfo ){
print var "| " Regiinfo[var] > "Domains" #"NoOfDomainsRegistrarWise"
}
}

when i am executing lot of time for 10 miilion records .

kiranmosarla

View Public Profile for kiranmosarla

Find all posts by kiranmosarla

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Awk scripting

Hi, I'm new to unix and i am kind of familiar with the basic commands. can anyone suggest some good books especially for AWK scripting and SHELL scripting thanks, Hari

2. Shell Programming and Scripting

AWK scripting

I have a text file in which the text has been divided into paragraphs (two line breaks or tab marks a new paragraph) and I want to make a script which output would delete line breaks within the paragraph and the different paragraphs would be separated by two line breaks. So, if my input file...

3. Shell Programming and Scripting

Scripting via awk

Hi, I am trying to understand what is happening here, new to scripting: I have a couple of these, but if I knew what was going on in one I can figure out the rest: awk '/rpc-100083/ { $2 = "enable -r" } $3 ~ /.NOS99dtlogin/ { $t = $2; $2 = $3; $3 = $t } { print }' /var/svc/profile/upgrade...

4. Shell Programming and Scripting

scripting/awk help : awk sum output is not comming in regular format. Pls advise.

Hi Experts, I am adding a column of numbers with awk , however not getting correct output: # awk '{sum+=$1} END {print sum}' datafile 2.15291e+06 How can I getthe output like : 2152910 Thank you.. # awk '{sum+=$1} END {print sum}' datafile 2.15079e+06

5. Shell Programming and Scripting

Need help with awk scripting.

Hi, i am newbie to this site and hope to learn but problem is s but need help urgently. Plz pm me if you are good at this. Help will be appreciated.

6. Shell Programming and Scripting

Help with AWK and Scripting!

Hi, This is the first time I am working with awk and I am not familiar with any commands in it. But I managed to do most of my work just left with one more. Needing your help! I have to extract only the matrix (written within ) from a text file. For example: 1JTJ_0006_ACGC_NPNP_A_12_15.pdb ...

7. Shell Programming and Scripting

Need help with awk scripting

hi all, I am working on awk scripting.I have created two awk files and now have a requirement of replacing the contents of first file with some contents of second file. Please find below the two files created.File1 has 3 records and File2 has 4 records. cat File1 111,0165,CB21031251,0165,...

8. Shell Programming and Scripting

Need help in awk scripting

Hi I am beginner of shell/AWK scripting , can you please help me in select particular column and column between two pattern from a multiple column file. file1.txt number status date1 date2 description category ...

9. Shell Programming and Scripting

awk Scripting

Hey guys, I want to get all the columns in this input file tab-delimited, i need to get the column send them to a variable. From there i could print them in shuffle and pick and select columns i want. Here is the input sample 2013/08/05 06:50:38:067 MINOR SiteScope ...

10. Shell Programming and Scripting

New at scripting awk with variable

I am trying to write a shell script that will add a date to and awk command from the command prompt. here is the line I am having difficulties. zgrep -i playback $dir/$1-*-errors.out.gz |cut -d '|' -f 1,11,12,15 | awk -v start=${start} -v end=${end} -F '|' '{$1>=start && $1 <=end} {print $2...

LEARN ABOUT LINUX

apr-config

APR-1-CONFIG(1) 						   User Commands						   APR-1-CONFIG(1)

NAME

       apr-1-config - Return metainformation about apr library

SYNOPSIS

       apr-1-config [OPTION]

DESCRIPTION

       The  apr-1-config  program   is used to retrieve information about the apr library in the system.  It is typically used to compile and link
       against the library.

       When linking with libtool, an application should do something like:

	      APR_LIBS="`apr-1-config --link-libtool --libs`"

       or when linking directly:

	      APR_LIBS="`apr-1-config --link-ld --libs`"

       An application should use the results of --cflags, --cppflags, --includes, and --ldflags in their build process.

OPTIONS

       Known values for OPTION are:

       --prefix[=DIR]
	      change prefix to DIR

       --bindir
	      print location where binaries are installed

       --includedir
	      print location where headers are installed

       --cc   print C compiler name

       --cpp  print C preprocessor name and any required options

       --cflags
	      print C compiler flags

       --cppflags
	      print C preprocessor flags

       --includes
	      print include information

       --ldflags
	      print linker flags

       --libs print additional libraries to link against

       --srcdir
	      print APR source directory

       --installbuilddir print APR build helper directory

       --link-ld
	      print link switch(es) for linking to APR

       --link-libtool
	      print the libtool inputs for linking to APR

       --shlib-path-var
	      print the name of the shared library path env var

       --apr-la-file
	      print the path to the .la file, if available

       --apr-so-ext
	      print the extensions of shared objects on this platform

       --apr-lib-target
	      print the libtool target information

       --apr-libtool
	      print the path to APR's libtool

       --version
	      print the APR's version as a dotted triple

       --help print the help

AUTHOR

       This manual page was written by Vincent Danjean <Vincent.Danjean@ens-lyon.org> for the Debian project (but may be used by others).

apr-1-config 1.2.2						    March 2006							   APR-1-CONFIG(1)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Awk scripting

Discussion started by: rharee

2. Shell Programming and Scripting

AWK scripting

Discussion started by: Muki101

3. Shell Programming and Scripting

Scripting via awk

Discussion started by: ulemsee

4. Shell Programming and Scripting

scripting/awk help : awk sum output is not comming in regular format. Pls advise.

Discussion started by: rveri