Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Linux shell programming performance issue
Top Forums Shell Programming and Scripting Linux shell programming performance issue Post 302914986 by ureddy on Saturday 30th of August 2014 02:10:51 AM
Old 08-30-2014
Hammer & Screwdriver Linux shell programming performance issue
Hi All,

can any one help me on this please.

Replace sting in FILE1.txt with FILE2.txt. FILE1.txt record must have at least one state is repeated once.But need to replace only from second occurrence in record in FILE1.txt

Condition: order of searching the records in FILE2.txt is impartent.

So when the FILE2.txt record entry is matched with FILE1.txt then break the loop and don't search again the FILE2.txt next record. see below exampe.

FILE1.txt
----------
Code:
TEXAS CALIFORNIA TEXAS
DALLAS CALIFORNIA CALIFORNIA DALLAS DALLAS TEXAS

FILE2.txt
------------
Code:
TEXAS,TX
DALLAS,DA
CALIFORNIA,CA
NEWYORK,NY


output:
-------
Code:
TEXAS CALIFORNIA TX

(TEXAS is matched so replaced TEXAS with TX in 2nd occurrence)
Code:
DALLAS CALIFORNIA CALIFORNIA DA DA TEXAS

(DALLAS and CALIFORNIA is matched more than once but as the order in FILE2.txt is impartent so DALLAS is coming first than CALIFORNIA so replacing DALLS only and breaking the loop and not searching again for CALIFORNIA)

I have implemented this using while loop and its working as expected but as the FILE1.txt have Millions of records and FILE2.txt have 50 records, so its taking Hours to complete. Any AWK solution for this to speed up the performance please ?
Last edited by Don Cragun; 08-30-2014 at 04:06 AM.. Reason: Remove QUOTE tags; add CODE tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Can somebody advise any free Linux sever for shell programming?

Hi, everybody. I just wonder whether there are a couple of free Linux servers running as terminals where people can practice Unix Shell Programming? I'd like to set up one myself but unfortunatly can't do it. I can't switch to Linux now coz I run a couple of servers on my machine. Cygwin is... (3 Replies)
Discussion started by: belgampaul
3 Replies

2. News, Links, Events and Announcements

Announcing collectl - new performance linux performance monitor

About 4 years ago I wrote this tool inspired by Rob Urban's collect tool for DEC's Tru64 Unix. What makes this tool as different as collect was in its day is its ability to run at a low overhead and collect tons of stuff. I've expanded the general concept and even include data not available in... (0 Replies)
Discussion started by: MarkSeger
0 Replies

3. Shell Programming and Scripting

Sed issue in K Shell programming

I am doing the following script in k shell sed -i 's/FILENAME/$i/g' TEST/test$j.ctl > TEST/control$j.ctl In the file it replaces $i for all FILENAME, it doesnot replace with the value of i. I put single quotes like below sed -i 's/FILENAME/'$i'/g' TEST/test$j.ctl > TEST/control$j.ctl I... (9 Replies)
Discussion started by: toshidas2000
9 Replies

4. UNIX for Advanced & Expert Users

run win app on Linux -performance issue

We develop software for diagnostic tools for cars. we a use a portable PC(x86) runs Win98 to run our applications. Hence the working environment in the company is Windows, specifically we use BASIC to develop the GUI, communication functions, DLL, etc. and run them on the Win98 PC. We suggested... (1 Reply)
Discussion started by: raedbenz
1 Replies

5. UNIX for Advanced & Expert Users

FTP-Shell Script-Performance issue

Hello All, Request any one of Unix/Linux masters to clarify on the below. How far it is feasible to open a new ftp connection for transferring each file when there are multiple files to be sent. I have developed shell script to send all files at single stretch but some how it doesnt suit to... (3 Replies)
Discussion started by: RSC1985
3 Replies

6. UNIX for Dummies Questions & Answers

Linux machine performance issue.

One of our database server is suddenly became very slow and i have no clue what to do .Please help. I m sharing the performance inforamtion regarding cpu,harddisk,ram . ########CPU Information######## Machine Uptime Information: uptime 10:25:06 up 16:50, 1 user, load average: 5.84, 5.65,... (10 Replies)
Discussion started by: pinga123
10 Replies

7. Shell Programming and Scripting

Shell programming in Linux

Hi, I have been working on Sun Solaris since a long time. Recently I got to work on RH Linux. My Linux version details are: Linux 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux I have a simple command in my shell script: export BKPTAG=`date... (3 Replies)
Discussion started by: sagarparadkar
3 Replies

8. Red Hat

Performance issue in Linux

IN solaris, for network high-availability we are using IPMP concept, can u tell me in REDHAT LINUX what we are using... also pls share good step to read & understand the that concept... Also performance issue in linux what are step & cmd can u tell me??? (2 Replies)
Discussion started by: tiger09
2 Replies

9. Shell Programming and Scripting

Performance issue in shell script

Hi All, I am facing performance issue while rinning the LINUX shell script. I have file1 and file 2. File one is the source file and file 2 is lookup file. Need to replace if the pattern is matching in file1 with file2. The order of lookup file is important as if any match then exit... (8 Replies)
Discussion started by: ureddy
8 Replies

10. Shell Programming and Scripting

Performance Issue - Shell Script

Hi, I am beginner in shell scripting. I have written a script to parse file(s) having large number of lines each having multiple comma separated strings. But it seems like script is very slow. It took more than 30mins to parse a file with size 120MB (523564 lines), below is the script code ... (4 Replies)
Discussion started by: imrandec85
4 Replies

LEARN ABOUT DEBIAN

gmetad

GMETAD(1)							   User Commands							 GMETAD(1)

NAME

       gmetad - manual page for Ganglia Meta Daemon

SYNOPSIS

       gmetad [OPTIONS]...

DESCRIPTION

       The  Ganglia  Meta  Daemon  (gmetad)  collects  information  from  multiple  gmond  or  gmetad data sources, saves the information to local
       round-robin databases, and exports XML which is the concatentation of all data sources

       -h, --help
	      Print help and exit

       -V, --version
	      Print version and exit

       -c, --conf=STRING
	      Location of gmetad configuration file (default='/etc/ganglia/gmetad.conf')

       -d, --debug=INT
	      Debug level. If greater than zero, daemon will stay in foreground.  (default='0')

       -p, --pid-file=STRING
	      Write process-id to file

AUTHOR

       Matt Massie <massie@cs.berkeley.edu>

REPORTING BUGS

       Report all bugs to the Ganglia Bugzilla Page at

	 http://bugzilla.ganglia.info/cgi-bin/bugzilla/index.cgi

COPYRIGHT

       Copyright (c) 2001, 2002, 2003, 2004, 2005 by The Regents of the University of California.  All rights reserved.

       Permission to use, copy, modify, and distribute this software and its documentation for any  purpose,  without  fee,  and  without  written
       agreement  is  hereby granted, provided that the above copyright notice and the following two paragraphs appear in all copies of this soft-
       ware.

       IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR  CONSEQUENTIAL  DAMAGES
       ARISING	OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF
       SUCH DAMAGE.

       THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF  MERCHANTABIL-
       ITY  AND FITNESS FOR A PARTICULAR PURPOSE.  THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO
       OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

gmetad								    March 2008								 GMETAD(1)
All times are GMT -4. The time now is 03:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy