Visit Our UNIX and Linux User Community


shell script performance issues --Urgent


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting shell script performance issues --Urgent
# 8  
Old 12-13-2007
Quote:
Originally Posted by icefish
example of line
COL001 | P | 2007-02-01-00.00.00.000000 | | sam | babu | | | M | 1949-01-04-00.00.00.000000 | INDIA | | C60 | | 110 S | | ENNIS | IN | 46563 | INDIA |
ok, I assume this is all ONE line.
you didn't say what part of it you wanted....
In the future pls use vB Codes when posting sample data and/or quoting others.
# 9  
Old 12-13-2007
Sure vgersh99,
I will do that from next time.
Can you please give code for doing that?

Thanks & Regards,
# 10  
Old 12-13-2007
Code:
echo 'COL001 | P | 2007-02-01-00.00.00.000000 | | sam | babu | | | M | 1949-01-04-00.00.00.000000 | INDIA | | C60 | | 110 S | | ENNIS | IN | 46563 | INDIA |' | awk -F'|' '{gsub("[^0-9]", "", $3); print "[" $3 "]"}'

But I'd advise to do EVERYTHING in awk - including reading line/records - no need for the shell 'while read line....'
# 11  
Old 12-13-2007
I have changed the script to use awk only
example

awk -F "|" '{

#ENTITY_TYPE=`print "$line" | cut -d'|' -f2 | awk '{gsub(/^[ \t]+|[ \t]+$/,"");print}'`
ENTITY_TYPE=$2

if [ ${ENTITY_TYPE} == "O" ]
then
ENTITY_TYPE="B"
else
ENTITY_TYPE="P"
fi
#CUSTOMER_ID=`print "$line" | cut -d'|' -f1 | awk '{gsub(/^[ \t]+|[ \t]+$/,"");print}'`
CUSTOMER_ID=$1

BIRTH_DATE=`print $10 | sed 's/[^0-9]//g' | cut -c1-8`

}' inquiry.txt

My question is can I use sed and cut inside awk?
If now can you please tell me the alternative way to do this?

Thanks in advance.
# 12  
Old 12-13-2007
Can someone please help me on this?
Its kind of urgent..

Thanks a lot in advance.
# 13  
Old 12-13-2007
you can use ANY utility from within awk, but it's not recommended as you're defeating the purpose of using just ONE tool capabale of doing it all natively.
Furthermore, I've provided a sample awk code in my previous post. Is there something in the sample code that you're stuck with?
# 14  
Old 12-13-2007
Can you send some format of lines of your Input file..

Although I am also new to scripting but I think..
You are using awk inside while statement . The number of times the awk appears the programme will take that much time. Thats the problem I think.

you no need to put awk inside while..instead process all your data once you inside awk .

Here your all the lines of your input file is processed as many times the awk appears..



user_prady

Last edited by user_prady; 12-13-2007 at 10:36 PM..

Previous Thread | Next Thread
Test Your Knowledge in Computers #405
Difficulty: Medium
Cygwin consists of three parts: a dynamic-link library (DLL) as an API compatibility layer in the form of a C standard library providing a substantial part of the POSIX API functionality, an extensive collection of software tools and applications that provide a Unix-like look and feel, and an MOTIF-based window manager.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

URGENT Reading a file and assessing the syntax shell script URGENT

I am trying to write a shell script which takes an input file as an arguement in the terminal e.g. bash shellscriptname.sh input.txt. I would like for the file to be read line by line each time checking if the .txt file contains certain words or letters(validating the syntax). If the line being... (1 Reply)
Discussion started by: Gurdza32
1 Replies

2. What is on Your Mind?

Baiduspider and Forum Performance Issues

For years we blocked Baiduspider due to the fact their bots do not obey the robots.txt directive and can really hurt site performance when they unleash 100 bots on the site each pulling pages many times per second. Last year, I unblocked Baiduspider's IP addresses, and now the problem is back. ... (1 Reply)
Discussion started by: Neo
1 Replies

3. Shell Programming and Scripting

Performance problem in Shell Script

Hi, I am Shell script beginner. I wrote a shell programming that will take each line of a file1 and search for it in another file2 and give me the output of the lines that do not exist in the file2. I wrote it using do while nested loop but the problem here is its running for ever . Is there... (12 Replies)
Discussion started by: sakthisivi
12 Replies

4. AIX

AIX 6.1 Memory Performance issues

Good Day Everyone, Just wonder anyone has encounter AIX 6.1 Memory Performance issues ? What I have in my current scenario is we have 3 datastage servers (Segregate server and EE jobs - for those who know Datastage achitect) and 2 db servers(running HA to load balance 4 nodes partitions for... (3 Replies)
Discussion started by: ckwan
3 Replies

5. Solaris

Getcwd performance issues

Hello everyone, recently we have been experiencing performance issues with chmod. We managed to narrow it down to getcwd. The following folder exists: /Folder1/subfol1/subfol2/subfol3 cd /Folder1/subfol1/subfol2/subfol3 truss -D pwd 2>&1 | grep getcwd 0.0001... (4 Replies)
Discussion started by: KotekBury
4 Replies

6. AIX

Performance issues for LPAR with GPFS 3.4

Hi, We have GPFS 3.4 Installed on two AIX 6.1 Nodes. We have 3 GPFS Mount points: /abc01 4TB (Comprises of 14 x 300GB disks from XIV SAN) /abc02 4TB (Comprises of 14 x 300GB disks from XIV SAN) /abc03 1TB ((Comprises of Multiple 300GB disks from XIV SAN) Now these 40... (1 Reply)
Discussion started by: aixromeo
1 Replies

7. UNIX for Dummies Questions & Answers

Awk Performance Issues

Hi All, I'm facing an issue in my awk script. The script is processing a large text file having the details of a number of persons, each person's details being written from 100 to 250 tags as given below: 100 START| 101klklk| ... 245 opr| 246 55| 250 END| 100 START| ... 245 pp| 246... (4 Replies)
Discussion started by: pgp_acc1
4 Replies

8. Shell Programming and Scripting

Script Performance problem . urgent frnds

HI frnds I have one flat with data and am loading the data into oracle table. While loading , rejected records are captured in log file. Now I want to read the log file and get the all rejected records and the reason for the rejection. I developed the script . its finding 5000 rejected... (7 Replies)
Discussion started by: Gopal_Engg
7 Replies

9. Solaris

raidctl performance issues

using the internal 2 drives mirror was created using raidctl on 100's of our servers . sometime when one drive fails we dont face any issue & we replace the drive with out any problem . but sometimes when one drive fails , system becomes unresponsive and doesnot allow us to login , the only way to... (1 Reply)
Discussion started by: skamal4u
1 Replies

10. UNIX for Advanced & Expert Users

Performance of a shell script

Hiii, I wrote a shell script for testing purpose. I have to test around 200thousand entries with the script.When i am doing only for 6000 entries its taking almost 1hour.If i test the whole testingdata it will take huge amount of time. I just want to know is it something dependent on the... (2 Replies)
Discussion started by: namishtiwari
2 Replies

Featured Tech Videos