Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Speeding up search and replace in a for loop
Top Forums Shell Programming and Scripting Speeding up search and replace in a for loop Post 302657711 by alister on Monday 18th of June 2012 09:21:40 AM
Old 06-18-2012
Quote:
Originally Posted by pbluescript
Sure. The actual commands I ran were slightly different than what I posted as there are two places per line that could be changed, but I only wanted one of them to change.

sed in a for loop version:
Code:
sed -i "s#gene_id \"$OLD\"#gene_id \"$NEW\"#g" file.txt

alister's version:

Code:
awk -F, '{print "%s#gene_id \""$1"\"#gene_id \""$2"\"#g"} END {print "x"}' conversion.csv | ex -s file.txt

If you only want to change the first occurence per line, remove the trailing g (global) flag in the sed substitute command. Otherwise, after the first substitution in each line, the sed/ex regular expression engine will continue scanning the remainder of the line for a second possible match (in your sample commands, "gene_id..."), instead of immediately exiting the substitute command and moving on to the next line. I wouldn't expect much improvement, but who knows.


sed in a for loop version:
Code:
sed -i "s#\"$OLD\"#\"$NEW\"#" file.txt

alister's version:

Code:
awk -F, '{print "%s#\""$1"\"#\""$2"\"#"} END {print "x"}' conversion.csv | ex -s file.txt

Perhaps you may want to re-include the "gene_id" text to tighten the match, but you definitely don't want to use the global substitution flag.

I edited those right here in the forum's webpage textbox, so, if you are going to test them, it would be prudent to use a small data set just to rule out a non-fatal typo (with all those double quotes hanging around, I may have accidentally deleted one).

Regards,
Alister
Last edited by alister; 06-18-2012 at 10:49 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl: Search for string on line then search and replace text

Hi All, I have a file that I need to be able to find a pattern match on a line, search that line for a text pattern, and replace that text. An example of 4 lines in my file is: 1. MatchText_randomNumberOfText moreData ReplaceMe moreData 2. MatchText_randomNumberOfText moreData moreData... (4 Replies)
Discussion started by: Crypto
4 Replies

2. UNIX for Dummies Questions & Answers

Speeding up a Shell Script (find, grep and a for loop)

Hi all, I'm having some trouble with a shell script that I have put together to search our web pages for links to PDFs. The first thing I did was: ls -R | grep .pdf > /tmp/dave_pdfs.outWhich generates a list of all of the PDFs on the server. For the sake of arguement, say it looks like... (8 Replies)
Discussion started by: Dave Stockdale
8 Replies

3. Shell Programming and Scripting

awk - replace number of string length from search and replace for a serialized array

Hello, I really would appreciate some help with a bash script for some string manipulation on an SQL dump: I'd like to be able to rename "sites/WHATEVER/files" to "sites/SOMETHINGELSE/files" within the sql dump. This is quite easy with sed: sed -e... (1 Reply)
Discussion started by: otrotipo
1 Replies

4. Programming

PERL, search and replace inside foreach loop

Hello All, Im a Hardware engineer, I have written this script to automate my job. I got stuck in the following location. CODE: .. .. ... foreach $key(keys %arr_hash) { my ($loc,$ind,$add) = split /,/, $arr_hash{$key}; &create_verilog($key, $loc, $ind ,$add); } sub create_verilog{... (2 Replies)
Discussion started by: riyasnr007
2 Replies

5. UNIX for Dummies Questions & Answers

Speeding/Optimizing GREP search on CSV files

Hi all, I have problem with searching hundreds of CSV files, the problem is that search is lasting too long (over 5min). Csv files are "," delimited, and have 30 fields each line, but I always grep same 4 fields - so is there a way to grep just those 4 fields to speed-up search. Example:... (11 Replies)
Discussion started by: Whit3H0rse
11 Replies

6. Shell Programming and Scripting

perl search and replace - search in first line and replance in 2nd line

Dear All, i want to search particular string and want to replance next line value. following is the test file. search string is tmp,??? ,10:1 "???" may contain any 3 character it should remain the same and next line replace with ,10:50 tmp,123 --- if match tmp,??? then... (3 Replies)
Discussion started by: arvindng
3 Replies

7. Shell Programming and Scripting

search replace with loop and variable

Hi, could anyone help me with this, tried several times but still not getting it right or having enough grounding to do it outside of javascript: Using awk or sed or bash: need to go through a text file using a for next loop, replacing substrings in the file that consist of a potentially multi... (3 Replies)
Discussion started by: wind
3 Replies

8. Shell Programming and Scripting

Nested search in a file and replace the inner search

Hi Team, I am new to unix, please help me in this. I have a file named properties. The content of the file is : ##Mobile props east.url=https://qa.east.corp.com/prop/end west.url=https://qa.west.corp.com/prop/end south.url=https://qa.south.corp.com/prop/end... (2 Replies)
Discussion started by: tolearn
2 Replies

9. Shell Programming and Scripting

Speeding up substitutions

Hi all, I have a lookup table from which I am looking up values (from col1) and replacing them by corresponding values (from col2) in another file. lookup file a,b c,d So just replace a by b, and replace c by d. mainfile a,fvvgeggsegg,dvs a,fgeggefddddddddddg... (7 Replies)
Discussion started by: senhia83
7 Replies

10. Shell Programming and Scripting

Help speeding up script

This is my first experience writing unix script. I've created the following script. It does what I want it to do, but I need it to be a lot faster. Is there any way to speed it up? cat 'Tax_Provision_Sample.dat' | sort | while read p; do fn=`echo $p|cut -d~ -f2,4,3,8,9`; echo $p >> "$fn.txt";... (20 Replies)
Discussion started by: JohnN6
20 Replies

LEARN ABOUT SUSE

irsend

IRSEND(1)								FSF								 IRSEND(1)

NAME

       irsend - basic LIRC program to send infra-red commands

SYNOPSIS

       irsend [options] DIRECTIVE REMOTE CODE [CODE...]

DESCRIPTION

       Asks the lircd daemon to send one or more CIR (Consumer Infra-Red) commands. This is intended for remote control of electronic devices such
       as TV boxes, HiFi sets, etc.

       DIRECTIVE can be:
	  SEND_ONCE	    - send CODE [CODE ...] once
	  SEND_START	    - start repeating CODE
	  SEND_STOP	    - stop repeating CODE
	  LIST		    - list configured remote items
	  SET_TRANSMITTERS  - set transmitters NUM [NUM ...]
	  SIMULATE	    - simulate IR event

       REMOTE is the name of a remote, as described in the lircd configuration file.

       CODE is the name of a remote control key of REMOTE, as it appears in the lircd configuration file.

       NUM is the transmitter number of the hardware device.

       For the LIST DIRECTIVE, REMOTE and/or CODE can be empty:

	  LIST	 ""    ""   - list all configured remote names
	  LIST REMOTE  ""   - list all codes of REMOTE
	  LIST REMOTE CODE  - list only CODE of REMOTE

       The SIMULATE command only works if it has been explicitly enabled in lircd.

       -h --help
	      display usage summary

       -v --version
	      display version

       -d --device
	      use given lircd socket [/var/run/lirc/lircd]

       -a --address=host[:port]
	      connect to lircd at this address

       -# --count=n
	      send command n times

EXAMPLES

	  irsend LIST DenonTuner ""
	  irsend SEND_ONCE  DenonTuner PROG-SCAN
	  irsend SEND_ONCE  OnkyoAmpli VOL-UP VOL-UP VOL-UP VOL-UP
	  irsend SEND_START OnkyoAmpli VOL-DOWN ; sleep 3
	  irsend SEND_STOP  OnkyoAmpli VOL-DOWN
	  irsend SET_TRANSMITTERS 1
	  irsend SET_TRANSMITTERS 1 3 4
	  irsend SIMULATE "0000000000000476 00 OK TECHNISAT_ST3004S"

FILES

       /etc/lirc/lircd.conf
	      Default lircd configuration file.  It should contain all the remotes, their infra-red codes and the corresponding timing	and  wave-
	      form details.

DIAGNOSTICS

       If lircd is not running (or /var/run/lirc/lircd lacks write permissions) irsend aborts with the following diagnostics:
       "irsend: could not connect to socket"
       "irsend: Connection refused" (or "Permission denied").

SEE ALSO

       The documentation for lirc is maintained as html pages. They are located under html/ in the documentation directory.

       lircd(8), mode2(1), smode2(1), xmode2(1), irrecord(1), irw(1), http://www.lirc.org.

irsend 0.8.7pre1						     May 2010								 IRSEND(1)
All times are GMT -4. The time now is 03:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy