Visit Our UNIX and Linux User Community


Performance Tuning


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Performance Tuning
# 1  
Old 10-24-2008
Performance Tuning

Hi All,

In last one week, i have posted many questions in this portal. At last i am succeeded to make my 1st unix script.

following are 2 points where my script is taking tooooo long.

1. Print the total number of records excluding header & footer. I have found that awk 'END{print NR - 2}' <filename> will give exact result. But my file has 520 columns seperated with ^~^. while using this command, error is 'too long...... record'. so i have used like this.

a=` cut -d~ -f1 NAM2008101601.OUT |awk 'END{print NR - 2}' `

But this is taking too long.

2. i need sum of amount column. So first i have cut that column & paste it in a file. Then i have calculated sum. But it is also taking long time.

cut -d~ -f27 $FILE_NAME | cut -c2-23 > amount
tot_val=`awk '{a+=$0}END{printf "%.5f\n",a}' amount`

Appriciate if someone can help me to get alternatives which speed up this process.

Thnaks
Amit
# 2  
Old 10-24-2008
For the point 1, since you are assuming that there will be always header and footer, not sure why you need to read whole file NAM2008101601.OUT to get record count.

Can you try using file size and record length to get record count, subtract 2 to get data records count?

1) Take file size from ls -l NAM2008101601.OUT
2) Get the record length, if it is not constant, then use wc to get record length
3) Total records = file size/record length
4) Subtract 2 from step 3, to take header/trailer counts.

Hope this might work.
# 3  
Old 10-24-2008
What version of awk are you using? GNU awk (gawk) has no predefined field count or record length limit.

Previous Thread | Next Thread
Test Your Knowledge in Computers #166
Difficulty: Easy
The Internet is a packet-switched network.
True or False?

9 More Discussions You Might Find Interesting

1. AIX

IBM AIX I/O Performance Tuning

I have a IBM Power9 server coupled with a NVMe StorWize V7000 GEN3 storage, doing some benchmarks and noticing that single thread I/O (80% Read / 20% Write, common OLTP I/O profile) seems slow. ./xdisk -R0 -r80 -b 8k -M 1 -f /usr1/testing -t60 -OD -V BS Proc AIO read% IO Flag IO/s ... (8 Replies)
Discussion started by: c3rb3rus
8 Replies

2. Solaris

System Check Performance Tuning

Hello Forum, Well I am fairly new to this Solaris os thing. One thing I would like to check for system health and performance. I know the codes like prstat,vmstat,sar,iostat,netstat,prtdiag -v, What else does a want to be sys admin have to look for when checking a solaris box? I know... (3 Replies)
Discussion started by: br1an
3 Replies

3. Solaris

Solaris Performance tuning

Dear all, I have a Local zone , where users feel that performance is not good. Is it wise to collect the inputs from the local zone rather than taking from the global zone. And also Can I tune from Global zone , so that it will reflect in local zone. Rgds rj (2 Replies)
Discussion started by: jegaraman
2 Replies

4. AIX

Performance issue / tuning advice

Please take a look at this system and give your analysis / advice. Can it be tuned to get a better performance? We are not getting more hardware ressources at the moment. We have to live with what we have. Application running on the system is SAS. OS is AIX 6.1 Let me know if you need output of... (7 Replies)
Discussion started by: firefox111
7 Replies

5. AIX

How to do Performance monitoring and tuning in AIX

How to do Performance monitoring and tuning in AIX. (2 Replies)
Discussion started by: AIXlearner
2 Replies

6. Shell Programming and Scripting

Oracle-performance tuning

Sorry, This is out of scope of this group.But I require the clarification pretty urgently. My Oracle database is parallely enabled. Still,in a particular table queries do not work "parallely" always. How is this? (9 Replies)
Discussion started by: kthri
9 Replies

7. UNIX for Dummies Questions & Answers

Performance Tuning

Hi to all, I'm interested in finding an introduction about Performance Tuning under Unix (or Linux); can somebody please point me in the right direction? Best regards (1 Reply)
Discussion started by: domyalex
1 Replies

8. Filesystems, Disks and Memory

EXT3 Performance tuning

Hi all, long time ago I posted something, but now, it is needed again :( Currently, I am handling with a big NFS Server for more than 200 clients, this sever has to work with 256 NFSDs. Because of this huge amount of NFSDs, there are thousands of small write accesses down to the disk and... (3 Replies)
Discussion started by: malcom
3 Replies

9. UNIX for Dummies Questions & Answers

Performance tuning.

can someone tell me a good site to go to in order to learn this. please do not recommen nay books because i dont have interest in that. if you know of any good sites with good straight forward explanation on how to split loads on machines that has excessive loading, please let me know Also,... (1 Reply)
Discussion started by: TRUEST
1 Replies

Featured Tech Videos