Sponsored Content
Full Discussion: Awk Performance Issues
Top Forums UNIX for Dummies Questions & Answers Awk Performance Issues Post 302350089 by pgp_acc1 on Wednesday 2nd of September 2009 06:54:32 PM
Old 09-02-2009
Question Awk Performance Issues

Hi All,

I'm facing an issue in my awk script.
The script is processing a large text file having the details of a number of persons, each person's details being written from 100 to 250 tags as given below:
100 START|
101klklk|
...
245 opr|
246 55|
250 END|
100 START|
...
245 pp|
246 65|
250 END|
In the code it writes each individual person's info into a temporary file and then process it for customization

We increased the tags from 250 to 450 leaving almost 200 tags blank.
like from 245 tag to 250 where moved to 445 to 450.(tags from 245-444 will not be coming in the file for now)
But after doing this rearrangment awk script has performance issues of almost double the time. So did changing these tags/identifiers degraded performance?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

shell script performance issues --Urgent

I need help in awk please help immediatly. This below function is taking lot of time Please help me to fine tune it so that it runs faster. The file count is around 3million records # Process Body processbody() { #set -x while read line do ... (18 Replies)
Discussion started by: icefish
18 Replies

2. Solaris

raidctl performance issues

using the internal 2 drives mirror was created using raidctl on 100's of our servers . sometime when one drive fails we dont face any issue & we replace the drive with out any problem . but sometimes when one drive fails , system becomes unresponsive and doesnot allow us to login , the only way to... (1 Reply)
Discussion started by: skamal4u
1 Replies

3. Programming

performance issues of calling a function in if condition

Hi, I have written a program in C and have to test the return value of the functions. So the normal way of doin this wud b int rc rc=myfunction(input); if(rc=TRUE){ } else{ } But instead of doing this I have called the function in the if() condition. Does this have any... (2 Replies)
Discussion started by: sidmania
2 Replies

4. Solaris

Solaris 11 Express NAT performance issues

Hi all, I decided to replace my linux router/firewall with Solaris 11 express. This is a pppoe connection directly to my server...no router boxes. I got everything setup, but the performance is terrible on the NAT....really slow. A web page that loads on the server instantly will take... (3 Replies)
Discussion started by: vectox
3 Replies

5. AIX

Performance issues for LPAR with GPFS 3.4

Hi, We have GPFS 3.4 Installed on two AIX 6.1 Nodes. We have 3 GPFS Mount points: /abc01 4TB (Comprises of 14 x 300GB disks from XIV SAN) /abc02 4TB (Comprises of 14 x 300GB disks from XIV SAN) /abc03 1TB ((Comprises of Multiple 300GB disks from XIV SAN) Now these 40... (1 Reply)
Discussion started by: aixromeo
1 Replies

6. Solaris

zfs send receive performance issues

I 'm trying to clone a zfs file system pool/u01 to a new file system called newpool/u01 using following commands zfs list zfs snapshot pool/u01@new zfs send pool/u01@new | zfs -F receive newpool/u01 Its a 100G file system snapshot and copied to same server on different pool and... (9 Replies)
Discussion started by: fugitive
9 Replies

7. Solaris

Getcwd performance issues

Hello everyone, recently we have been experiencing performance issues with chmod. We managed to narrow it down to getcwd. The following folder exists: /Folder1/subfol1/subfol2/subfol3 cd /Folder1/subfol1/subfol2/subfol3 truss -D pwd 2>&1 | grep getcwd 0.0001... (4 Replies)
Discussion started by: KotekBury
4 Replies

8. AIX

AIX 6.1 Memory Performance issues

Good Day Everyone, Just wonder anyone has encounter AIX 6.1 Memory Performance issues ? What I have in my current scenario is we have 3 datastage servers (Segregate server and EE jobs - for those who know Datastage achitect) and 2 db servers(running HA to load balance 4 nodes partitions for... (3 Replies)
Discussion started by: ckwan
3 Replies

9. UNIX for Dummies Questions & Answers

Hard Disk Performance issues Suse 11 SP1

hi guys right now I have 6 Virtual Machines (VMs) running on Vmware ESXi 5.1 and attached to Storage SAN. All these run Suse Linux 11 SP1 x64. All of a sudden 1 of these VMs is running very slow making high CPU usage and I see al wait % kinda high 40-50%. Apparently since I don't own this... (5 Replies)
Discussion started by: karlochacon
5 Replies

10. What is on Your Mind?

Baiduspider and Forum Performance Issues

For years we blocked Baiduspider due to the fact their bots do not obey the robots.txt directive and can really hurt site performance when they unleash 100 bots on the site each pulling pages many times per second. Last year, I unblocked Baiduspider's IP addresses, and now the problem is back. ... (1 Reply)
Discussion started by: Neo
1 Replies
PMDATXMON(1)						      General Commands Manual						      PMDATXMON(1)

NAME
pmdatxmon - txmon performance metrics domain agent (PMDA) SYNOPSIS
$PCP_PMDAS_DIR/txmon/pmdatxmon [-d domain] [-l logfile] [-U username] tx_type ... $PCP_PMDAS_DIR/txmon/txrecord [-l] $PCP_PMDAS_DIR/txmon/txrecord tx_type servtime [tx_type servtime ... ] $PCP_PMDAS_DIR/txmon/genload DESCRIPTION
pmdatxmon is an example Performance Metrics Domain Agent (PMDA) which exports a small number of performance metrics from a simulated trans- action monitor. The txmon PMDA is shipped as both binary and source code and is designed to be an aid for PMDA developers; the txmon PMDA demonstrates how performance data can be exported from an application (in this case txrecord) to the PCP infrastructure via a shared memory segment. As a matter of convenience, pmdatxmon creates (and destroys on exit) the shared memory segment. The tx_type arguments are arbitrary unique tags used to identify different transaction types. The txrecord application simulates the processing of one or more transactions identified by tx_type and with an observed service time of servtime . With the -l option, txrecord displays the current summary of the transaction activity from the shared memory segment. genload is a shell and awk(1) script that acts as a front-end to txrecord to generate a constant load of simulated transaction activity. A brief description of the pmdatxmon command line options follows: -d It is absolutely crucial that the performance metrics domain number specified here is unique and consistent. That is, domain should be different for every PMDA on the one host, and the same domain number should be used for the same PMDA on all hosts. -l Location of the log file. By default, a log file named txmon.log is written in the current directory of pmcd(1) when pmdatxmon is started, i.e. $PCP_LOG_DIR/pmcd. If the log file cannot be created or is not writable, output is written to the standard error instead. -U User account under which to run the agent. The default is the unprivileged "pcp" account in current versions of PCP, but in older versions the superuser account ("root") was used by default. INSTALLATION
If you want access to the names, help text and values for the txmon performance metrics, do the following as root: # cd $PCP_PMDAS_DIR/txmon # ./Install You will be prompted for the tx_type tags. If you want to undo the installation, do the following as root: # cd $PCP_PMDAS_DIR/txmon # ./Remove pmdatxmon is launched by pmcd(1) and should never be executed directly. The Install and Remove scripts notify pmcd(1) when the agent is installed or removed. FILES
$PCP_PMCDCONF_PATH command line options used to launch pmdatxmon $PCP_PMDAS_DIR/txmon/help default help text file for the txmon metrics $PCP_PMDAS_DIR/txmon/Install installation script for the pmdatxmon agent $PCP_PMDAS_DIR/txmon/Remove undo installation script for the pmdatxmon agent $PCP_LOG_DIR/pmcd/txmon.log default log file for error messages and other information from pmdatxmon PCP ENVIRONMENT
Environment variables with the prefix PCP_ are used to parameterize the file and directory names used by PCP. On each installation, the file /etc/pcp.conf contains the local values for these variables. The $PCP_CONF variable may be used to specify an alternative configura- tion file, as described in pcp.conf(5). SEE ALSO
PCPIntro(1), pmcd(1), pmdasample(1), pmdatrivial(1), txmonvis(1) and PMDA(3). Performance Co-Pilot PCP PMDATXMON(1)
All times are GMT -4. The time now is 02:01 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy