sed or awk editing help


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed or awk editing help
# 15  
Old 10-31-2010
I do not understand what you are referring to? A loop or running sed twice is necessary if you want to do a replace in two adjacent fields. This is because sed starts to do a replace at the character after where it stopped the last time. If you include the second comma in the pattern then the field that follows is not matched because the first comma was matched the previous time.

A couple of suggestions leave out the second comma, but then all whitespace gets deleted even in fields where the are other characters, which is not what the OP was asking.

I added the possibility of doing the replace not only in fields between comma's, but also for the two fields that have only one comma, namely the first and the last. Which is not precisely what the OP asked, but likely what he requires, since he speaks of fields separated by commas.

S.

Last edited by Scrutinizer; 10-31-2010 at 10:04 AM..
# 16  
Old 10-31-2010
I dont understant you too..
i say already when necessary or "not necessary" twice usage ..
and i explain with examples when DGPickett says in this example usage is mandatory..I answer to him related this..I hope this is enough.

And lets come your examples
I said loop is not necessary

with global flag

Code:
echo '  ,          abc,             ,sd ,      ,   ,     ' |sed -r 's/(,|^)  *(,|$)/\1\2/g'
,          abc,,sd ,,   ,

with loop
Code:
echo '  ,          abc,             ,sd ,      ,   ,     ' |sed -r ':a;s/(,|^)  *(,|$)/\1\2/;ta'
,          abc,,sd ,,,


and solution non-loop
Code:
echo '  ,          abc,             ,sd ,      ,   ,     ' |sed -r 's/(,|^)   *|  (,|$)/\1\2/g'
,          abc,,sd ,,,


Regards
ygemici-sed lover team
# 17  
Old 10-31-2010
Your suggestion, take a look at the sd field, it does not stay intact.
Code:
echo '  ,          abc,             ,  sd   ,      ,   ,     ' | sed -r 's/(,|^)   *|  (,|$)/\1\2/g'
,          abc,,sd ,    , ,

With the loop:
Code:
$  echo '  ,          abc,             ,  sd   ,      ,   ,     ' | sed -r ':a;s/(,|^)  *(,|$)/\1\2/;ta'
,          abc,,  sd   ,,,


Last edited by Scrutinizer; 10-31-2010 at 12:21 PM..
# 18  
Old 10-31-2010
Best way non-loop solution every time for me but all the same you wish also..

with loop
Code:
 time sed -r ':a;s/(,|^)  *(,|$)/\1\2/;ta' testfile

Code:
real    0m4.116s
user    0m0.596s
sys     0m0.166s



with non-loop
Code:
 time sed -r 's/(  *,|,|^)  *(,|$)/\1\2/g;s/,  *,/,,/g' testfile

Code:
real    0m1.773s
user    0m0.915s
sys     0m0.110s

# 19  
Old 10-31-2010
It was not a matter of which method is faster, but of which method actually works. Have a look at the 4th field that contains spaces and the letters sd, and what comes after that in post #17. That field should remain unchanged, but it is not, so IMO your original "non-loop" suggestion is not working properly. This needs to be done by either running the sed twice or through a loop like I suggested.

This time you are leaving your single step approach and you are presenting yet another alternative, with two search and replace statements (which you said earlier would not be necessary), which is like a loop with the use of the g flag.

One thing I noticed just now is that in post#11 in there was a g flag used but in the middle the sed -r option the g flag was accidentally missing. It is a little bit more efficient to leave that in as I noted in that same post. When we add the g-flag as was intended:
Code:
sed -r ':a;s/(,|^)  *(,|$)/\1\2/g;ta'

There is no significant difference on my system.

---------- Post updated 01-11-10 at 00:13 ---------- Previous update was 31-10-10 at 23:00 ----------

I tested you suggestion but it does not work properly for the last field:
Code:
$ echo '     ,          abc,             ,  sd   ,      ,   ,     ,      ' | sed -r ':a;s/(,|^)  *(,|$)/\1\2/g;ta' | od -c
0000000   ,                                           a   b   c   ,   ,
0000020           s   d               ,   ,   ,   ,  \n
0000034

cuts the spaces in the last field like it should, whereas
Code:
$ echo '     ,          abc,             ,  sd   ,      ,   ,     ,      ' | sed -r 's/(  *,|,|^)  *(,|$)/\1\2/g;s/,  *,/,,/g' | od -c
0000000   ,                                           a   b   c   ,   ,
0000020           s   d               ,   ,   ,   ,
0000040      \n
0000042

leaves them in.

Last edited by Scrutinizer; 10-31-2010 at 07:56 PM..
# 20  
Old 11-01-2010
(There are spaces in the second line third=final field):
Code:
$ sed '
  s/, *,/,,/g
 ' <<! |cat -e
aaa,   bbb,ccc   ,    ,    ,dddd
    ,eee,   
!
aaa,   bbb,ccc   ,,    ,dddd$
    ,eee,   $
$

See, one pass misses the adjacent field of spaces, and without the (), ^ and $ extended regex, the first and last fields are not done. You can work around that with two passes, space-space-asterisk to ignore the empty fields possibly from the first pass, and with extra commas:
Code:
$ sed '
  s/.*/,&,/
  s/,  *,/,,/g
  s/,  *,/,,/g
  s/,\(.*\),/\1/
 ' <<! |cat -e
aaa,   bbb,ccc   ,    ,    ,dddd
    ,eee,   
!
aaa,   bbb,ccc   ,,,dddd$
,eee,$
$

# 21  
Old 11-01-2010
Allright then why do you try to add sed code with non-loop?

with non-loop
Code:
# echo '     ,          abc,             ,  sd   ,      ,   ,     ,      ' | sed -r 's/(  *,|,|^)  *(,|$)/\1\2/g;s/,  *,/,,/g;s/,  *$/,/' |od -bc
0000000 054 040 040 040 040 040 040 040 040 040 040 141 142 143 054 054
          ,                                           a   b   c   ,   ,
0000020 040 040 163 144 040 040 040 054 054 054 054 012
                  s   d               ,   ,   ,   ,  \n
0000034

with loop
Code:
# echo '     ,          abc,             ,  sd   ,      ,   ,     ,      ' | sed -r ':a;s/(,|^)  *(,|$)/\1\2/g;ta' | od -bc
0000000 054 040 040 040 040 040 040 040 040 040 040 141 142 143 054 054
          ,                                           a   b   c   ,   ,
0000020 040 040 163 144 040 040 040 054 054 054 054 012
                  s   d               ,   ,   ,   ,  \n
0000034

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Editing files with sed or something similar

{ "AFafa": "FAFA","AFafa": "FAFA" "baseball":"soccer","wrestling":"dancing" "rhinos":"crocodiles","roles":"foodchain" } I need to insert a new line before the closing brackets "}" so that the final output looks like this: { "AFafa": "FAFA","AFafa": "FAFA"... (6 Replies)
Discussion started by: SkySmart
6 Replies

2. Shell Programming and Scripting

editing file with awk cut and sed

HI All, I am new to unix. I have a file would like to do some editing by using awk, cut and sed. Could anyone help? This file contain 100 lines. There are one line for example: 2,"102343454",5060,"579668","579668","579668","SIP",,,"825922","035885221283026",1,268,"00:59:00.782 APR 17... (2 Replies)
Discussion started by: mimilaw
2 Replies

3. UNIX for Dummies Questions & Answers

sed help finding and editing

With sed 1. I need to find a line that contains "DVM" and "73069". 2. I need to insert a double quote at the beginning of the first line of the file. These two have been driving me crazy for the last 45 minutes. Any help would be greatly appreciated. Thanks (3 Replies)
Discussion started by: nlassiter
3 Replies

4. UNIX for Dummies Questions & Answers

sed editing help....

Hello all, I need some help with sed. seems like i cant get through it. So here is what i am trying. when i do ps -ef|grep bla blah ...like below...i get /u01/app/oracle/11g/bin/tnslsnr .... but i want to replace that string with something using sed. So basically i want to get rid of... (3 Replies)
Discussion started by: abdul.irfan2
3 Replies

5. Shell Programming and Scripting

Line/Variable Editing for Awk sed Cut

Hello, i have a file, i open the file and read the line, i want to get the first item in the csv file and also teh third+6 item and wirte it to a new csv file. only problem is that using echo it takes TOO LONG: please help a newbie. below is my code: WorkingDir=$1 FileName=`cut -d ',' -f... (2 Replies)
Discussion started by: limamichelle
2 Replies

6. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

7. Shell Programming and Scripting

problem in using sed command in editing a file

Hi all, I have a conf file, i want to update some entries in that conf file. Below is the code for that using a temporary file. sed '/workgroup=/ c\workgroup=Workgroup' /usr/local/netx.conf > /usr/local/netx.conf.tmp mv -f /usr/local/netx.conf.tmp /usr/local/netx.conf Sample contents of... (9 Replies)
Discussion started by: ranj14r
9 Replies

8. Homework & Coursework Questions

String editing using sed? awk?

1. The problem statement, all variables and given/known data: Problem Statement for project: When an account is created on the CS Unix network, a public html directory is created in the account's home directory. A default web page is put into that directory. Some users replace or... (13 Replies)
Discussion started by: peage1475
13 Replies

9. Shell Programming and Scripting

Editing Commas in a textfile using sed

Hi guys task removing the last commas of 5th and 6th columns. The bug in the script is causing effect because of whitespaces around commas. I tried to delete white spaces first and running the above script. but still some where getting the results wrong. I already have a script to do this... (12 Replies)
Discussion started by: repinementer
12 Replies

10. Shell Programming and Scripting

Editing File using awk/sed

Hello Awk Gurus, Can anyone of you help me with the below problem. I have got a file having data in below format pmFaultyTransportBlocks ----------------------- 9842993 pmFrmNoOfDiscRachFrames ----------------------- NULL pmNoRecRandomAccSuccess -----------------------... (4 Replies)
Discussion started by: Mohammed
4 Replies
Login or Register to Ask a Question