Replace String without using sed


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replace String without using sed
# 8  
Old 10-12-2018
Nice idea, delete and create!
One can also use a { code block } that holds the input file open.
In this case one can even use the stdin (descriptor 0) instead of a descriptor 3.
Code:
HIST_DATE="1900-01-01T00:00:00.000Z"
INC_DATE="2018-10-04T09:18:43.000Z"
fn=/tmp/file.tst
{
rm $fn
sed "s/$HIST_DATE/$INC_DATE/" >$fn
} <$fn

This User Gave Thanks to MadeInGermany For This Post:
# 9  
Old 10-13-2018
Nice trick Smilie , but it is not much different from or offers advantages over simply creating a new file and then renaming it to the old file.
Code:
sed "s/$HIST_DATE/$INC_DATE/" "${fn}" > "${fn}.tmp" && mv -- "${fn}.tmp" "${fn}"


--
What the trick with the file descriptor does is:
  1. point a file descriptor (fd) to the inode that is pointed to by the file
  2. remove the filename, but leave the inode (since it is still pointed to by the fd)
  3. create a new file (and thus a new inode) using the old filename
  4. operate sed reading from the fd (the old file) and writing to the new file
  5. close fd and thus remove the old file inode

It is more risky, since if anything goes wrong with writing to the new file, the old file still gets deleted.

Last edited by Scrutinizer; 10-13-2018 at 08:00 AM..
# 10  
Old 10-13-2018
under linux, the risk can be reduced as long as the file descriptor is not closed:
Code:
$ exec 3</tmp/file.tst
$ rm /tmp/file.tst 
$ cat <&3
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$ file /tmp/file.tst
/tmp/file.tst: cannot open `/tmp/file.tst' (No such file or directory)
$ cat <&3
$ cat /proc/self/fd/3
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-01-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$Paramsoqlfilter=Systemmodstamp > 1900-02-01T00:00:00.000Z
$ ls -l /proc/self/fd/3 
lr-x------ 1 disedorgue disedorgue 64 oct.  13 14:04 /proc/self/fd/3 -> /tmp/file.tst (deleted)

Regards.
# 11  
Old 10-14-2018
Quote:
Originally Posted by Scrutinizer
It is more risky, since if anything goes wrong with writing to the new file, the old file still gets deleted.
Quote:
Originally Posted by disedorgue
under linux, the risk can be reduced
With "anything goes wrong" Scrutinizer perhaps meant the system crashing, the process being aborted for some kind of error or something such. In this cases you'd end up with a deleted file or a zero-length results file.

In my experience it pays off to deleted the source only once you are satisfied with the target and not one iota sooner.

bakunin
# 12  
Old 10-14-2018
Quote:
Originally Posted by bakunin
With "anything goes wrong" Scrutinizer perhaps meant the system crashing, the process being aborted for some kind of error or something such. In this cases you'd end up with a deleted file or a zero-length results file.
Ok, but sed -i (or perl -i ) has the same problem, no ?
# 13  
Old 10-14-2018
Quote:
Originally Posted by disedorgue
Ok, but sed -i (or perl -i ) has the same problem, no ?
No, they don't - but they have other problems which is why their usage is discouraged anyway:

What sed -i does is in fact opening a second file, write to it and then deleting the original and moving its temporary file where the original one was - BEHIND YOUR BACK!

The "advantage" is manifold: first you have no control over this. If something happens (crash, aborted task, ...) then the temporary file remains and clutters up space. That might happen to you with writing to a target file "manually" too, but in this case you are aware of the fact and might take cleaning actions. In case of sed -i you are probably not aware of this temporary file because you have been lied to by the program about its existence.

Second, even if you are aware of that: where would start searching for it? If i have some code like this:

Code:
if sed '<something>' /path/to/input > /some/output
     mv /some/output > /path/to/input
else
     print -u2 "Error executing sed ..."
fi

I have control over where the temporary file goes. And because of that i know where to look at to clean up. Do you know where sed -i puts its temporary file? i don't and the man page is busy telling you lies about the non-existence of the file, so it won't tell you either.

Third, and worst, IMHO: because of the mechanism you are most probably not aware that the files inode changes! This does not only mean the inode number but it may also mean that the ownership and/or the filemode may change (in case of a directory with SUID bits set). If one uses an explicit temporary file one is perhaps aware of this and does (from the example above:

Code:
if sed '<something>' /path/to/input > /some/output
     mv /some/output > /path/to/input
     chown someuser:somegroup /path/to/input
     chmod ....
else
     print -u2 "Error executing sed ..."
fi

But with the original file still in place (wink wink) why should one do that?

Finally, and perhaps most importantly: scripts should make as few assumptions about their surroundings as possible. Only GNU-sed understands the -i-option, but POSIX does not. Should i willingly limit my script to systems with a GNU-sed available? I don't think so.

All these reasons are why i strongly suggest not to use the -i option even if the sed in question supports it.

I hope this helps.

bakunin
# 14  
Old 10-14-2018
(GNU) sed -i and perl -i work differently as can be checked with strace.

GNU sed -i creates a new temporary file first and then moves it back to the original file if all is OK
perl -i opens the file, unlinks it and then open a new file with the same name..

So GNU sed -i works in a safer way whereas perl -i does not..

Last edited by Scrutinizer; 10-14-2018 at 09:07 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using sed to replace string

Hello guys, I'm working in a cellular company and i'm trying to do a script to run some commands automaticlly. I'm using Solaris version :SunOS pk-ercuas4 5.10 in my work. I've a file that creates by script named test1.mos that that look like: confb+ gsg+ lt all $date = `date... (1 Reply)
Discussion started by: oferg
1 Replies

2. Shell Programming and Scripting

Replace string in XML file with awk/sed with string from another

Sorry for the long/weird title but I'm stuck on a problem I have. I have this XML file: </member> <member> <name>TransactionID</name> <value><string>123456789123456</string></value> </member> <member> <name>Number</name> ... (9 Replies)
Discussion started by: cozzin
9 Replies

3. Shell Programming and Scripting

sed String Replace

I have this file and variable on my machine $cat /tmp/disk.tmp check_disk.pl -H localhost -D ARG1 -w 75 -c 90 $echo $_ARG1 /,/opt,/tmp,/usr,/var,/boot,/dev/shm how do I replace the string ARG1 in my file (disk.tmp) with the value of my variable _ARG1 and placing them inside quotes while... (4 Replies)
Discussion started by: thursday_g0ne
4 Replies

4. Shell Programming and Scripting

sed or awk command to replace a string pattern with another string based on position of this string

here is what i want to achieve... consider a file contains below contents. the file size is large about 60mb cat dump.sql INSERT INTO `table1` (`id`, `action`, `date`, `descrip`, `lastModified`) VALUES (1,'Change','2011-05-05 00:00:00','Account Updated','2012-02-10... (10 Replies)
Discussion started by: vivek d r
10 Replies

5. Shell Programming and Scripting

replace (sed?) a string in file with multiple lines (string) from variable

Can someone tell me how I can do this? e.g: a=$(echo -e wert trewt ertert ertert ertert erttert erterte rterter tertertert ert) How do i replace the STRING with $a? I try this: sed -i 's/STRING/'"$a"'/g' filename.ext but this don' t work (2 Replies)
Discussion started by: jforce
2 Replies

6. UNIX for Dummies Questions & Answers

sed replace string

Hi, How can i execute the below. I want to replace value1 string value with new/new/newString value sed 's/value1/new/new/newString'/' file.txt ~Vinodh' Kumar (2 Replies)
Discussion started by: vino_hymi
2 Replies

7. Shell Programming and Scripting

How to use sed to replace the a string in the same file using sed?

How do i replace a string using sed into the same file without creating a intermediate file? (7 Replies)
Discussion started by: gomes1333
7 Replies

8. Shell Programming and Scripting

Using sed to replace a string in file with a string in a variable that contains spaces

Hi, i call my shell like: my_shell "my project name" my script: #!/bin/bash -vx projectname=$1 sed s/'PROJECT_NAME ='/'PROJECT_NAME = '$projectname/ <test_config_doxy >temp cp temp test_config_doxy the following error occurres: sed s/'PROJECT_NAME ... (2 Replies)
Discussion started by: vivelafete
2 Replies

9. Shell Programming and Scripting

sed: replace string with another string (with spaces)

Hi I have an XML file with strings XABCD, XEFGHX and XIJKLX. I would like to replace XABCDX with "This is the first string", XEFGHX with "This is the second string" and XIJKLX with "This is the third string". What is the best way to implement this? Should I have a file with the data that is... (4 Replies)
Discussion started by: zmfcat1
4 Replies

10. UNIX for Dummies Questions & Answers

Is it possible to replace more the 1 string with one SED?

i want to replace two phrases in text file originalstringA.1. blah blah.... originalstringB.1. got this code so far: #variables IP=$1 NO=$2 FS=$3 IS=$4 NN=1 #echo variables echo '' echo $IP echo $NO echo $FS echo $IS (1 Reply)
Discussion started by: tuathan
1 Replies
Login or Register to Ask a Question