Sponsored Content
Top Forums Shell Programming and Scripting Performance issue with awk script. Post 302212740 by RRVARMA on Tuesday 8th of July 2008 08:18:12 AM
Old 07-08-2008
PHP Performance issue with awk script.

Hi,

The below awk script is taking about 1 hour to fetch just 11 records(columns). There are about 48000 records. The script file name is take_first_uniq.sh

Code:
#!/bin/ksh  

if [ $# -eq 2 ] 
then  

while read line 
do 
first=`echo $line | awk -F"|" '{print $1$2$3}'`
while read line2
do
second=`echo $line2 | awk -F"|" '{print $7$13$14}'`
if [ ${first} == ${second} ] 
then 
echo $line2
fi 
done < $2

done < $1  
fi

I call this script this way..

Code:
ksh take_first_uniq.sh file_3uniq_fields.out file_sort_all_fields.out > file_uniq_master.out


Please suggest me how to increase the performance.. I'm new to awk scripting.

Thanks,
RRVARMA
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

performance issue using gzcat, awk and sort

hi all, I was able to do a script to gather a few files and sort them. here it is: #!/usr/bin/ksh ls *mainFile* |cut -c20-21 | sort > temp set -A line_array i=0 file_name='temp' while read file_line do line_array=${file_line} let i=${i}+1 (5 Replies)
Discussion started by: naoseionome
5 Replies

2. Shell Programming and Scripting

Performance issue with ftp script.

Hi All, I have written a script to FTP files from local server to remote server. When i try it for few number of files the scripts runs successfully. But the same script when i run for 200-300 files it gives me performanace issue by aborting the connection. Please help me out to improve the... (7 Replies)
Discussion started by: Shiv@jad
7 Replies

3. UNIX for Advanced & Expert Users

FTP-Shell Script-Performance issue

Hello All, Request any one of Unix/Linux masters to clarify on the below. How far it is feasible to open a new ftp connection for transferring each file when there are multiple files to be sent. I have developed shell script to send all files at single stretch but some how it doesnt suit to... (3 Replies)
Discussion started by: RSC1985
3 Replies

4. Solaris

Performance issue

Hi Gurus, I am beginner in solaris and want to know what are the things we need to check for performance monitoring on our solairs OS. for DISK,CPU and MEMORY. Also how we do ipforwarding in slaris Many thanks for your help Pradeep P (4 Replies)
Discussion started by: ppandey21
4 Replies

5. Shell Programming and Scripting

Script performance issue

hi i have written a shell script which comapare a text file data with files within number of different directories. example. Text File: i have a file /u02/abc.txt which have almost 20000 file names Directories: i have a path /u03 which have some subdirectories like a,b,c which have almost... (2 Replies)
Discussion started by: malikshahid85
2 Replies

6. UNIX for Dummies Questions & Answers

Performance issue

hi I am having a performance issue with the following requirement i have to create a permutation and combination on a set of three files such that each record in each file is picked and the output is redirected in a specific format but it is taking around 70 odd hours to prepare a combination... (7 Replies)
Discussion started by: mad_man12
7 Replies

7. Shell Programming and Scripting

awk performance issue

Hi, I have the code below as cat <filename> | tr '~' '\n' | sed '/^$/ d' | sed "s/*/|/g" > <filename> awk -F\| -vt=`date +%m%d%y%H%M%S%s` '$1=="ST",$1=="SE"{if($1=="ST"){close(f);f="214_edifile_"t"" ++i} ; $1=$1; print>f}' OFS=\| <filename> This script replaces some characters and... (4 Replies)
Discussion started by: atlantis_yy
4 Replies

8. UNIX for Dummies Questions & Answers

awk script performance issue

Hello All, I have the below excerpt of code in my shell script and it taking long time to complete, though it prints the output quickly. Is there a way to make it come out once it finds the first instance as the file size of 4.7 GB it could be going through all lines of the data file to find for... (3 Replies)
Discussion started by: Ariean
3 Replies

9. Shell Programming and Scripting

Performance issue in shell script

Hi All, I am facing performance issue while rinning the LINUX shell script. I have file1 and file 2. File one is the source file and file 2 is lookup file. Need to replace if the pattern is matching in file1 with file2. The order of lookup file is important as if any match then exit... (8 Replies)
Discussion started by: ureddy
8 Replies

10. Shell Programming and Scripting

Performance Issue - Shell Script

Hi, I am beginner in shell scripting. I have written a script to parse file(s) having large number of lines each having multiple comma separated strings. But it seems like script is very slow. It took more than 30mins to parse a file with size 120MB (523564 lines), below is the script code ... (4 Replies)
Discussion started by: imrandec85
4 Replies
print(1)							   User Commands							  print(1)

NAME
print - shell built-in function to output characters to the screen or window SYNOPSIS
ksh print [ -Rnprsu [n]] [arg...] DESCRIPTION
ksh The shell output mechanism. With no flags or with flag - or -, the arguments are printed on standard output as described by echo(1). OPTIONS
The following options are supported: -n suppresses new-line from being added to the output. -R -r (raw mode) ignore the escape conventions of echo. The -R option will print all subsequent arguments and options other than -n. -p causes the arguments to be written onto the pipe of the process spawned with |& instead of standard output. -s causes the arguments to be written onto the history file instead of standard output. -u [ n ] flag can be used to specify a one digit file descriptor unit number n on which the output will be placed. The default is 1. EXIT STATUS
The following exit values are returned: 0 Successful operation. >0 Output file is not open for writing. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
echo(1), ksh(1), attributes(5) SunOS 5.10 15 Apr 1994 print(1)
All times are GMT -4. The time now is 06:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy