Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Tricky sed required

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Tricky sed required
# 8  
Old 05-21-2013
Thanks
Cheers, That certainly does the job. Smilie

Would love to know how to do it in sed if anyone else fancies a crack at it...

Thanks again, that will get me out of the hole.
# 9  
Old 05-21-2013
The following sed-script will first remove any single quote inside "[".."]"-ranges, then remove "[..]" pairs. This should ensure that "'e','e'" would be untouched while "['e','e']" would reduce to "e,e". I hope this is what you wanted:

Code:
sed ':start
      /\[[^]]*\'\]/ {
           s/\(\[[^]]*\)\'/\1/
           b start
      }
      s/\[\([^]]*\)\]/\1/g' /path/to/infile

I hope this helps.

bakunin
# 10  
Old 05-22-2013
Hi Bakunin,

That's certainly the kind of solution I am looking for. Unfortunately it doesn't quite do what I want, probably my fault for not taking a real file home with me.

I am back in work this morning and have just tried it against a real sample file -

Code:
"channel.facebook.com","['unknown', 'productivityloss']","[1001005, 1000041]","['standard', 'web2']"
"channel.facebook.com","['unknown', 'productivityloss']","[1001005, 1000041]","['standard', 'web2']"
"channel.facebook.com","['unknown', 'productivityloss']","[1001005, 1000041]","['standard', 'web2']"
"channel.facebook.com","['unknown', 'productivityloss']","[1001005, 1000041]","['standard', 'web2']"
"channel.facebook.com","['unknown', 'productivityloss']","[1001005, 1000041]","['standard', 'web2']"

Output of sed command -

Code:
"channel.facebook.com","'unknown', 'productivityloss'","1001005, 1000041","'standard', 'web2'"
"channel.facebook.com","'unknown', 'productivityloss'","1001005, 1000041","'standard', 'web2'"
"channel.facebook.com","'unknown', 'productivityloss'","1001005, 1000041","'standard', 'web2'"
"channel.facebook.com","'unknown', 'productivityloss'","1001005, 1000041","'standard', 'web2'"
"channel.facebook.com","'unknown', 'productivityloss'","1001005, 1000041","'standard', 'web2'"

So where I have single quoted strings inside brackets, I'm not losing the brackets.

This might be me, I had to change the outer quotes on your sed to " in order to get it to run -

Code:
sed ":start
      /\[[^]]*\'\]/ {
           s/\(\[[^]]*\)\'/\1/
           b start
      }
      s/\[\([^]]*\)\]/\1/g" $File

Would you be kind enough to explain how the sed works? I am trying to get to grips with sed and would appreciate the insight.

Thanks for the help Smilie

The perl will get me out of trouble for now, but I'm not sure it is installed on all of our servers so would prefer a sed solution if possible for portability.

Brad
# 11  
Old 05-22-2013
Quote:
Originally Posted by steadyonabix
Would you be kind enough to explain how the sed works? I am trying to get to grips with sed and would appreciate the insight.
Code:
sed ':start                         # create a label
      /\[[^]]*\'\]/ {               # execute the following only if the line contains
                                    # this regexp: a "[", followed by optional
                                    # non-"]", followed by "'", followed by "]"
           s/\(\[[^]]*\)\'/\1/      # replace "[ - non-] - '" by itself except for the last '
           b start                  # go back to the label defined above
      }
      s/\[\([^]]*\)\]/\1/g'         # replace "[ - non-] -]" by anything inside the brackets

The first lines set up a loop: one "'" is removed inside "[...]", then "b start" starts over. Once there is no single quote left the regexp in the second line will not match any more and the whole block is skipped. At last the last line removes the brackets by replacing "[-content-]" by "content".

Probably my script failed because the first part has escaped single quotes ("\'") and the escaping didn't work as expected.

I hope this helps.

bakunin
# 12  
Old 05-22-2013
Thanks Smilie

That's really helpful.

Now I can play around with it and get a good understanding...

Cheers

Brad
# 13  
Old 05-23-2013
regular expression in perl is tailor-made for this kind of issue
Code:
while(<DATA>){
	s/(?:[\[\]]|'(?=[^\[]*\]))//xg;
	print;
}
__DATA__
"['1235','3234']","abcde","[1234]","['1235','3234']"
"'","abcde","[1235]","['1236','2234']"
"[1236]","['1237','1234']","","1234"
"'e'","[1237]","['1238','0234']",""

This User Gave Thanks to summer_cherry For This Post:
Klashxx
# 14  
Old 05-24-2013
Thanks Summer, I really should learn perl Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Clarification required on sed

Hi Can some one tell what does this sed command do sed 's/*$//g I am more curious on the highlighted part , can some one explain what does that mean. Thanks Sri (1 Reply)
Discussion started by: Sri3001
1 Replies

2. Shell Programming and Scripting

sed help required

Hi All, I have one file with below type of data in it, $ cat test.txt ###123 ###xyxytuerwb ###2 ###tyupe Here I would like to replace all the characters with "x" after the 3 "###" with the same number of characters. Can you please help me to achieve this. (7 Replies)
Discussion started by: gr8_usk
7 Replies

3. Shell Programming and Scripting

sed tricky problem

Hi, I have a file which contains two strings: AAAAA and BBBBB I have two variables in my script: DATE="03/21/2010" aDate="20100321" I need to replace string AAAAA with variable $DATE and BBBBB with $aDate. Here is what I do sed "s/AAAAA/$DATE/" $BASIC_TMPLT | sed "s/BBBBB/$aDate/" >... (4 Replies)
Discussion started by: axed
4 Replies

4. Shell Programming and Scripting

Another tricky sed or awk question

This post is in reference to https://www.unix.com/shell-programming-scripting/137977-tricky-sed-awk-question-post302428154.html#post302428154 I am trying to go the opposite direction now: I have the following file: a,b,C,f,g a,b,D,f,g a,b,E,f,g h,i,J,k,l m,n,O,t,u m,n,P,t,u m,n,Q,t,u... (3 Replies)
Discussion started by: awayand
3 Replies

5. Shell Programming and Scripting

Tricky sed or awk question

Hello everyone, unfortunately I am no unix nor scripting guru, which is why I am asking for help here. I am trying to reformat a .csv file using sed or awk which has the following format: a,b,C-D-E,f,g h,i,J,k,l m,n,O-P-Q-R-S,t,u v,w,X-Y,z,a It's basically a 5-field text file which has an... (7 Replies)
Discussion started by: awayand
7 Replies

6. Shell Programming and Scripting

Help required with awk/sed

Hi I have a file, with format like: column1|coulumn2|column3|column4 A|X|K|18 L|O|R|31,42,25 G|H|I|55,66 L|E|Q|25,31,94 output required: column1|coulumn2|column3|column4 A|X|K|18 L|O|R|31,25 L|E|Q|25,31 Input File Format: All columns are seperated using |, last column... (8 Replies)
Discussion started by: New to awk
8 Replies

7. Shell Programming and Scripting

Help required in sed or awk.

Hi All, I need to pick up data on both sides of "=" sign. For eg, following is the context that I have. 125.156.125.147=machine1 147.125.185.156=machine2 147.125.185.159=machine3 Can I have the ip address in one variable and machine name in another variable using sed or awk. ... (1 Reply)
Discussion started by: nua7
1 Replies

8. Shell Programming and Scripting

Help in sed required.

Hi All, I am facing a small problem in sed. I want to insert a line in the existing file. Existing code: access to attr=userPassword by self write by * auth access to * by self write by users read by anonymous auth Desired code: access to attr=userPassword by self... (14 Replies)
Discussion started by: nua7
14 Replies

9. UNIX for Dummies Questions & Answers

Sed-- command help required

Hi Gurus, I have a small requirement. Let suppose i have a file test.txt test.txt contains Dispatched date = '2008-04-08' Name = 'Logers' Now i want to add one more line to it as Number of Responses = "$a" $a will be chnaging dynamically which i had grepped it in the script. Now i... (6 Replies)
Discussion started by: pssandeep
6 Replies

10. Shell Programming and Scripting

Tricky Sed

Hello. I am trying to convert occurrences of 'NULL' from a datafile. The 'NULL' occurences appears at this: |NULL| NULL|NULL| NULL|NULL| NULL|NULL| NULL| There should be 52 fields per line. I would like any occurrence of | NULL| or |NULL| to appear as '||' Currently I am using this sed... (2 Replies)
Discussion started by: bestbuyernc
2 Replies
Login or Register to Ask a Question