Substitute a character with sed


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Substitute a character with sed
# 1  
Old 03-14-2019
Substitute a character with sed

hi all,
i'd like to modify a file with sed , i want to substuite a char "-" with "/"
how can i do this?
Thanks for all
regards
Francesco

Last edited by RavinderSingh13; 03-14-2019 at 09:16 PM..
# 2  
Old 03-14-2019
Hi,
the command will overwrite the file
Code:
sed -i 's/-/\//g' file

will bring in stdout
Code:
tr '-' '/' <file


Last edited by nezabudka; 03-14-2019 at 08:02 AM..
# 3  
Old 03-14-2019
Code:
sed -i 's#-#/#g' file

This User Gave Thanks to anbu23 For This Post:
# 4  
Old 03-14-2019
Moderator's Comments:
Mod Comment I edited the threads title to correct the typo ("substite"-> "substitute", "charather" -> "character").


Please note that the -i switch is NOT part of the standard sed and is only understood by GNU-sed. In addition i strongly suggest NOT to use it even if it is available:

Code:
sed -i '....'  /path/to/file                              # not recommended

if sed '....' /path/to/file > /path/to/file.tmp ; then    # recommended way
     mv  /path/to/file.tmp  /path/to/file
     chown <correct ownership> /path/to/file              # optional
     chmod <correct filemode> /path/to/file               # optional
else
     <error handling procedure here>
fi

Basically sed always relies on a second file for output because it cannot work how it does work (that is: as a pipeline) in any other way. The -i switch does not change that at all, it just makes sed do it "behind the curtain". In fact a second (temporary) file is created and then moved over the original file.

The problem with this is that: the inode number (and all the other metadata) of the file is changed, of course. If you do it manually as i suggested this is obvious because you obviously have a new file with which you replace the old one. With the -i switch you seem to have the same file - but you don't! So, if you rely on the file ownership or the filemode staying constant but have a directory with sticky bits set or if you rely that the inode number stays constant you are going to be disappointed.

The even bigger problem (IMHO) is that a temporary file gets created but - unlike with the standard way of doing things - you have no control over the process. In case of an error the temporary file might stay, wasting space. Or it may be created in a filesystem where you don't expect it and this filesystem may be too small. Or....

As a last concern i'd like to offer: it is not portable. The way i sketched out above will work everywhere. The method using the -i swithc will only work on select systems. This usually will not pop up for years but the incompatibility will raise its ugly head in the moment you need that least - and it will inadvertently bite you in the behind.

(Sad truth: this will happen anyway sooner or later. If you can minimise the opportunities it can happen it is likely to hurt a bit less - and i for my part am pretty particular when it comes to parts of me i like to use for sitting.)

I hope this helps.

bakunin
I hope this helps.

Last edited by bakunin; 03-14-2019 at 01:21 PM..
These 2 Users Gave Thanks to bakunin For This Post:
# 5  
Old 03-14-2019
Hi bakunin,

thanks for the remarks about different aspects of sed -i.

Quote:
Originally Posted by bakunin
So, if you rely on the file ownership or the filemode staying constant but have a directory with sticky bits set or if you rely that the inode number stays constant you are going to be disappointed.
Current GNU sed reapplies permissions and ownership after in-place edit. (Inodenumber is changed of course). I was not aware of that until know. Thanks for the hint. Grateful to the GNU-Developers, that they did a good job - here too.

See here(example as root user, so ownership-reapply is demonstrated):

Code:
# sed --version
# sed (GNU sed) 4.4

# ls -l test
-r--r--r-- 1 www-data www-data 5 Mär 14 14:16 test

# strace -o strace.out sed -r -i  's/bla/blub/' test

# ls -l test
-r--r--r-- 1 www-data www-data 5 Mär 14 14:17 test

# relevant parts within strace.out
# grep -E "(chmod|chown)" strace.out 
fchown(4, 33, 33)                       = 0
fchmod(4, 0100444)                      = 0

Quote:
As a last concern i'd like to offer: it is not portable.
Yes, that's true. And as far as I understand, you are working in a more "pure" Unix-Environment (AIX, Solaris, ...) where this is very important to have scripts running on different platforms. And I assume scripts written will run on different platforms very often.

For my part I heavily rely on the GNU-Tools and what affects me, I'm working nearly 100% in a Linux environment(with different distributions) where the gnu collection is the default toolset in all cases. I think compatibility of my many scripts will break in lots of places because there are lots of differences in tools, different commands, same commands/different switches, ... . I think I would have a lot of extra work if I - for one thing - refrain from using the advanced feature set of the gnu tools and - for the other - to care about compatibility with systems I probably will never get in touch with.

Different worlds - Different rules ?

Last edited by stomp; 03-14-2019 at 12:01 PM..
These 2 Users Gave Thanks to stomp For This Post:
# 6  
Old 03-14-2019
Quote:
Originally Posted by stomp
Current GNU sed reapplies permissions and ownership after in-place edit. (Inodenumber is changed of course). I was not aware of that until now.
This is a new development it seems and thank you for the update. Older versions (upon the knowledge of which i drew when i wrote the above) didn't, though.

Quote:
Originally Posted by stomp
And as far as I understand, you are working in a pure "Unix"-Environment (AIX, Solaris, ...) where this is very important to have scripts running on different platforms. And I assume scripts written will run on different platforms very often.
Not always, but most times and, yes, that shaped my habits. It is not about "pure UNIX" versus "Linux", though, it is more about "every UNIX and every Linux too". When i write scripts i write them with absolutely every possibility in mind and if i have only, say, Solaris and AIX to serve but i know something won't work that way in HP-UX i will try to find a way that will work there too. Otherwise chances are you are finished with the script and two days later HP-UX is introduced and you restart from zero.

If i have to deal with things which are different in every system i try to encapsulate them as much as possible. Most times i try to isolate the logic and use layer-functions which then only do the OS-dependent parts like this:

Code:
do_something ()
case "$OS" in
     AIX)
          <AIX-related stuff here>
          ;;

     SunOS)
          <SunOS-related stuff here>
          ;;

     .....
esac

Everything else is decided/processed/triggered/... above and finally the OS-dependent part is called.

Quote:
Originally Posted by stomp
For my part I heavily rely on the GNU-Tools and what affects me, I'm working nearly 100% in a Linux environment [...]. I think I would have a lot of extra work if I - for one thing - refrain from using the advanced feature set of the gnu tools
Well, the work you talk about you have only once. Second, i try to write my scripts like any software developer writes his programs: as general (and generalised) as possible. As a freelancer i regularly change the customer and work in different surroundings. It often happened to me that i was asked to write a certain procedure for systems A, B, and C and i just smiled, took out a USB-stick and said "on this you'll find exactly that". I might have written it for systems D, E and F, but as i write as generalised and OS-agnostic as possible it usually works on A, B and C too or at least does so with minor changes. If it takes you 10 minutes to adapt a script you would have worked two weeks to create anew (because it took you two weeks to write it in first place) you know that the one day you spent making it as independent from the actual surroundings as possible was well invested. It was well invested not only for me but also the customer who got a flexible and extensible solution.

Before doing systems administration i was programmer and i write scripts the same way i wrote programs: with adaptability and maintainability in mind. If you have a 50-lines script it doesn't matter if you produce a well-written piece of software or a kludge. If your scripts hit and break the 1000-lines mark you better have your source-code under control because otherwise you will never get finished. Yes, i have several 1000-lines scripts doing things for me.

And the old if it was hard to write it should be hard to read (and even harder to understand) is a great source of amusement but once you stopped chuckling over it you do as it should be done.

As i said here we are essentially artists as "technics" is derived from the greek word for "art". A painting isn't finished when everything is covered with paint but when you - as the painter - feel that sense of accomplishment and satisfaction with what you did. A program (=script) will be finished not when it runs without an error but when it runs AND has that inherent (and hard to describe) quality of "rightness", is documented, has error handling, will be able to deal with all conceivable conditions and more. In one word, it is finished not when it is running but when it is "good".

The great sculptist Bernini, considered to be one of the greatest of all times didn't stop when the plaster was used up but when he thought there was nothing left to do on a statue. This was what made him "the Bernini" and what took him above other sculptors - not the number of works, not the amount of material used. Write your programs like you do any other art: put your everything, the essence of your being, into it and don't stop until you think you could make your whole existence depending on it.

bakunin
These 2 Users Gave Thanks to bakunin For This Post:
# 7  
Old 03-14-2019
Bakunin's replacement code in post#4 is portable but still creates a new inode for the input file ("/path/to/file")
This not only denotes a repair of the original owner/group/permissions, it will make a previously hard-linked file stand-alone, and this damage cannot be repaired.
Also GNU sed -i (and perl -i) cannot preserve the original linkage.
If we are going for a real improvement then we avoid the mv command!
Code:
if sed '....' /path/to/file > /path/to/file.tmp ; then
     cp  /path/to/file.tmp  /path/to/file
else
     <error handling procedure here>
fi

The cp command preserves the original inode (owner/group/permissions and linkage) - only the timestamp changes, and that's pretty correct.
This User Gave Thanks to MadeInGermany For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed substitute command -- need help

I am trying to do what I thought should be a simple substitution, but I can't get it to work. File: Desire output: I thought I'd start with a sed command to remove the part of the header line preceding the string "comp", then go on to remove the suffix of the target string (e.g. ":3-509(-)"),... (3 Replies)
Discussion started by: pathunkathunk
3 Replies

2. Shell Programming and Scripting

How to substitute variable in sed for special character?

Hi , I have input file like below Hi this is "vinoth". Hi happy to work with 'unix' USA(united states of America) My script variables are below : Dquote=Ộ Squote=&#$567 Obrac=&^986 Cbrac=&^745 I want to read the variables in my SED command to replace the double quote,single... (9 Replies)
Discussion started by: vinothsekark
9 Replies

3. Shell Programming and Scripting

using sed or gsub to substitute characters!

Is there a way to substitute the URL-encoding references of ( & and ` ) with their actual appearance? for example.... %26 is & say I want to convert every %26 in my file to &..... awk '{gsub(/%26/,"&");print}' Is there a way to do this? I also want to be able to convert ` too! (3 Replies)
Discussion started by: puttster
3 Replies

4. Shell Programming and Scripting

Sed question to substitute data in \2

Hi All, Here is what I'm trying to do with sed: Input File: somechangeVariable1=Something I would like to change somechangeVariable2=Something else I would like to change ... Output File: somechangeVariable1=Something I would like to different somechangeVariable2=Something else I would... (6 Replies)
Discussion started by: Peace_Dude1
6 Replies

5. UNIX for Dummies Questions & Answers

Using sed to substitute between quotes.

I'm using sed to perform a simply search and replace. The typical data is: <fig><image href="Graphics/BAV.gif" align="left" placement="break" I need to replace the value in the first set of quotes, keeping the remainder of the line the same. Thus: <fig><image href="NEW_VALUE" align="left"... (3 Replies)
Discussion started by: Steve_altius
3 Replies

6. Shell Programming and Scripting

Using sed to substitute first occurrence

I am trying to get rid of some ending tags but I run into some problems. Ex. How are you?</EndTag><Begin>It is fine.</Begin><New> Just about I am trying to get rid of the ending tags, starts with </ and ending with >. (which is </EndTag> and </Begin>) I tried the following sed... (2 Replies)
Discussion started by: quixoticking11
2 Replies

7. AIX

check for a particular character inside a file and substitute with a given character?

i am a newbie to shell script,so i want a kshell script in which i need to check for a particular character inside a file through conditional looping(like if ,case,while)and if that character exists ,then substitute a given character to that character. consider a file test.txt,inside the file... (1 Reply)
Discussion started by: karthikprasathk
1 Replies

8. Shell Programming and Scripting

Using SED to substitute between two patterns.

Hi All, I'm currently using SED to make various changes to some .xml files I'm working on, but I'm stuck on this particular problem. I want to remove '<placeholder>element-name</placeholder>' from the following: <heading>Element <placeholder>element-name</placeholder> not... (2 Replies)
Discussion started by: Steve_altius
2 Replies

9. UNIX for Dummies Questions & Answers

read a variable character by character, substitute characters with something else

im having trouble doing this: i have a variable with 2 characters repeating e.g. aababbbaaaababaabbaabbba is there a way i can search the variable for a's and b's and then change a's to b's and b's to a's? im guessing its like getting the 1's compliment of the string im doing this in... (2 Replies)
Discussion started by: vipervenom25
2 Replies

10. UNIX for Dummies Questions & Answers

sed substitute situation

I am having a problem executing a sed substitute in a file. I have tried alot of different things I found in previous posts, however non seem to work. I want to substitute this in $FILE: VALUE=33.4 In the script I have tried the following: prev=$(awk -F"=" '{ print $2 }' $FILE ) new=$(echo... (16 Replies)
Discussion started by: newbreed1
16 Replies
Login or Register to Ask a Question