Perl script to read string from file#1 and find/replace in file#2


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Perl script to read string from file#1 and find/replace in file#2
# 8  
Old 08-09-2015
Quote:
Originally Posted by pchang
thanks guys for all your suggestions.

I'm starting to think outside the Unix utilities and maybe a Java program will speed things up? What do you think?

Unfortunately, I don't have any experience with Java coding.
Could you tell us how the run time using sed compared when you tried my suggestion in post #4 in this thread using ed and/or ex?

I take it that my intuition was off, but for future reference, I'd appreciate hearing how the run times compared.
# 9  
Old 08-09-2015
Hi Don.

When I have some time, I will test out your suggestion.

Right now, we are just trying to find a quick solution - we are going live in a few weeks.

Thanks.
# 10  
Old 08-09-2015
Quote:
Originally Posted by pchang
Hi Don.

When I have some time, I will test out your suggestion.

Right now, we are just trying to find a quick solution - we are going live in a few weeks.

Thanks.
Never mind. My intuition was off. At least on OS X 10.10.4 on a 2 year old MacBook Pro with abc.txt containing about 4k lines and abc.sed containing 144 substitutions, sed averaged about .07 seconds, ex averaged about .11 seconds and ed took about .45 seconds. Your mileage may vary with different data, different hardware, and/or a different OS. If you want to try the script I used to convert abc.sed to an equivalent abc.ed and time sed and ed, it is at the end of this script. The abc.ed script it produces will work with both ed and ex. (Note that it was only tested with alphanumeric BREs in the sed substitute commands. Further work may be required if BRE special characters (especially /) or commas are included in the BREs in abc.se.


Code:
#!/bin/ksh
rm -f abc.ed
time sed -f abc.sed abc.txt > new_abc.txt
ed abc.sed <<-"EOF"
g/s\/\([^\/]*\)/s,,g/\1/s/,
$a
w _new_abc.txt
q
.
w abc.ed
q
EOF
time ed -s abc.txt  < abc.ed
echo comparing results
diff _new_abc.txt new_abc.txt

# 11  
Old 08-10-2015
I did some benchmarks, too.
With sed the runtime grows exponentionally with abc.sed.
GNU sed, when abc.sed is >90 lines, is already slower than the perl solution in post#2.
For my tests I have embedded the perl code in a shell script, so it can read from a pipe or from an argument, just like sed.
Code:
#!/bin/sh

perl -pe '
BEGIN {
open $fh, "<", "abc.sed" or die "$!\n";
@gsubs = <$fh>;
close $fh;
@gsubs = map{s#^s/|/g$|\n##g; split "/"} @gsubs;
%replace = @gsubs;

$search = join "|", keys %replace;
}

# main loop
{
    s/($search)/$replace{$1}/g;
}
' "$@"

This User Gave Thanks to MadeInGermany For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to replace a string with pattern read from a file

I have two files blocks.txt and rules.txt. In blocks.txt i have the following entries Linux1 Linux2 Linux3 ..... Linux10 In rules.txt i have the lines where a filename pattern starts like 'blk-name.*' I want to replace 'blk-name' with the names read from blocks.txt file I tried... (2 Replies)
Discussion started by: Jag02
2 Replies

2. UNIX for Beginners Questions & Answers

Find and replace a string in a text file

Dear all, I want to find all the "," in my text file and then replace the commas to a tab. I found a script online but I don't know how to modify the script for my case. Any one can help? Thank you. @echo off &setlocal set "search=%1" set "replace=%2" set "textfile=Input.txt" set... (2 Replies)
Discussion started by: forevertl
2 Replies

3. Shell Programming and Scripting

[Need help] perl script to find the occurance of string from a text file

I have two files 1. input.txt 2. keyword.txt input.txt has contents like .src_ref 0 "call.s" 24 first 0x000000 0x5a80 0x0060 BRA.l 0x60 .src_ref 0 "call.s" 30 first 0x000002 0x1bc5 RETI .src_ref 0 "call.s" 31 first 0x000003 0x6840 ... (2 Replies)
Discussion started by: acdc
2 Replies

4. Shell Programming and Scripting

How to read file, and replace certain string with another string?

Hi all, the value in the following file is just an example. It could be a different value/network addresses. Here is my example of initial output in a file name net.txt Initial Output, net.txt The goal is to produce the following format which is to convert from CIDR to Netmask... (6 Replies)
Discussion started by: type8code0
6 Replies

5. Shell Programming and Scripting

perl- read search and replace string from the file

Dear all, I have a number of files and each file has two sections separated by a blank line. At the top section, I have lines which describes the values of the alphabetical characters, # s #; 0.123 # p #; 12.3 # d #; -2.33 # f #; 5.68 <blank line> sssssss spfdffff sdfffffff Now I... (4 Replies)
Discussion started by: sasharma
4 Replies

6. Shell Programming and Scripting

find string and replace with string in other file

Dear all, I need your help, I have file like this: file1:23456 01910964830098775635 34567 01942809546554654323 67589 26546854368698023653 09778 58716868568576876878 08675 86178546154065406546 08573 54165843543054354305 . .file2: 23456 25 34567 26 67589 27 (2 Replies)
Discussion started by: attila
2 Replies

7. Shell Programming and Scripting

How to find a certain string in a file and replace it with a value from another file using sed/awk?

Hi Everyone, I am new to this forum and new to sed/awk programming too !! I need to find particular string in file1(text file) and replace it with a value from another text file(file2) the file2 has only one line and the value to be replaced with is in the second column. file 1: (assert (=... (21 Replies)
Discussion started by: paramad
21 Replies

8. Shell Programming and Scripting

Find and replace string from file which contains variable and path - SH

e.g. /home/$USER/.config replace it with "" (empty) Is this possible? I think you should play a bit with sharps ## and sed:b: (2 Replies)
Discussion started by: hakermania
2 Replies

9. Shell Programming and Scripting

find and replace a string in a file without the use of temp file

Hi - I am looking for a replacing a string in a in multiple *.sql files in directory with a new string without using a temporary file Normally I can use sed command as below for W in ls `FILE*.sql` do sed 's/OLD/NEW/g' $W > TEMPFILE.dat mv TEMPFILE.dat $W done But Here in my... (9 Replies)
Discussion started by: raghutapal
9 Replies

10. Shell Programming and Scripting

Find/replace to new file: ksh -> perl

I have korn shell script that genretaets 100 file based on template replacing the number. The template file is as below: $ cat template file number: NUMBER The shell script is as below: $ cat gen.sh #!/bin/ksh i=1; while ((i <= 100)); do sed "s/NUMBER/$i/" template > file_${i} ((... (1 Reply)
Discussion started by: McLan
1 Replies
Login or Register to Ask a Question