Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sed: How to modify files in a complex way

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed: How to modify files in a complex way
# 1  
Old 12-04-2009
sed: How to modify files in a complex way
Hello,

I am new to sed and hope that someone can help me with the following task.

I need to modify a txt file which has format like this:
Code:
xy=CreateDB|head.queue|head.source|head.definition|rtf.edit|rtf.task|rft.cut
abc|source|divine|line4|5|true

into something like:
Code:
head.queue=abc
head.definition=source
head.definition=divine
rtf.edit=line4
rtf.task=5
rft.cut=true

Can someone help me with it?
Last edited by radoulov; 12-04-2009 at 10:38 AM.. Reason: Please use code tags!
# 2  
Old 12-04-2009
hope you don't mind a ksh solution:

Code:
num=0
sed -e 's/|/ /g' << EOF |
xy=CreateDB|head.queue|head.source|head.definition|rtf.edit|rtf.task|rft.cut
abc|source|divine|line4|5|true
EOF
while read crud ; do

  (( num +=1 ))

  if [[ $num -eq 1 ]]; then
    set -A col `echo $crud`
    continue
  fi

  ### set -A data `echo $crud` ### this second array not necessary.

  a=1

  ### for datum in `echo ${data[*]}` ; do
  for datum in $crud ; do

    echo ${col[$a]}=$datum
    (( a += 1 ))

  done

done 2>&1 |
  tee $0.log

works for a dynamic number of columns.
Did you notice that your first column is extra?

The only problem I see with this is if you have embedded spaces elsewhere in your data.
In that case, you may have to toggle your IFS variable.

IFS="|"

IFS=" "

depending on where you're at in the script.
# 3  
Old 12-04-2009
Hello quirkasaurus,

Thank you for the code, but what I want more is that how can I rearrange the content into the way I want with SED or AWK solution, is it possible? (cuz I really want to learn it.)
# 4  
Old 12-04-2009
oy...

yes it is... with awk, not sed.

Even easier, sorry, i default to ksh:

Code:
cat << EOF |
xy=CreateDB|head.queue|head.source|head.definition|rtf.edit|rtf.task|rft.cut
abc|source|divine|line4|5|true
EOF

nawk -F\| '{
  if ( NR == 1 ){
    split( $0, headers );
    next;
    }

  for ( x = 1; x <= NF; x++ ){
    print headers[x] "=" $x;
    }
  }'



---------- Post updated at 07:15 AM ---------- Previous update was at 07:10 AM ----------

hmmm... still a problem with that extra first field.
Anyways, here's the fix for that:

Code:
cat << EOF |
xy=CreateDB|head.queue|head.source|head.definition|rtf.edit|rtf.task|rft.cut
abc|source|divine|line4|5|true
EOF

nawk -F\| '{
  if ( NR == 1 ){
    for ( x = 2; x <= NF; x++ ){
      headers[x-1]=$x;
      }
    next;
    }

  for ( x = 1; x <= NF; x++ ){
    print headers[x] "=" $x;
    }
  }'

# 5  
Old 12-04-2009
Hello quirkasaurus,

Thank you for your help. I have a general question, why in this case using AWK is more suitable than SED? Is it because SED is more for modifying data (that is, in my case, data are more like being "rearranged" than being "modified" and therefore AWK is suitable?)
# 6  
Old 12-04-2009
sed is a "stream-editor" and afaik it does not have the ability to remember variables and print them later.

awk -- is a fully functional programming language, containing complete flow control, arrays, hash arrays, regular expressions, system calls, multiple file input and output, etc...

sed is fine for relatively straightforward inline modifications.

awk is more suitable for problems that require logic and flow control and greater complexity.
# 7  
Old 12-04-2009
Sed is Turing-complete. You can do everything in sed.

Code:
sed '
s/^[^|]*|//
N
:o
s/\([^|]*\)|\(.*\n\)\([^|]*\)|/\1=\3\n\2/
P
s/^[^\n]*\n//
to
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Complex Filter using grep, awk or sed

Hi, I'm not very familiar witrh sed or awk and hope the somebody can help me to solve my problem. I need to filter a text report using grep, sed or awk. I would like to cut out text lines with the pattern INFO and if exists the following lines of the pattern DETAILS. I need te keep the lines with... (4 Replies)
Discussion started by: Frankg
4 Replies

2. Shell Programming and Scripting

Complex string operation (awk, sed, other?)

I have a file that contains RewriteRules for 200 countries (2 examples for 1 country below): RewriteRule ^/at(/|/index.html|)$ http://%{HTTP_HOST}/locate/index.html?locale=de_AT #& RewriteRule ^/at_english(/|/index.html|)$ http://%{HTTP_HOST}/locate/index.html?locale=en_AT I have... (5 Replies)
Discussion started by: usshadowop
5 Replies

3. Shell Programming and Scripting

Modify sed script

I'm trying to take out strings from log files and add them to a csv. For example, in the directory now, there are 2 log files. I get the following results: sed -e '/custodian/b' -e '/packaged by/b' -e '/package name/b' -e '/Total Data (MB) Read/b' -e '/Begin Time/b' -e d * packaged by =... (10 Replies)
Discussion started by: chipperuga
10 Replies

4. Shell Programming and Scripting

A complex sed statement

I have following requirement. Say, my text file contains following patterns {2010501005|XXGpvertex|9|0|17|0|{|{30100001|XXparameter_set|@@@@{{30001002|XXparameter|!prototype_path|$AB_COMPONENTS/Sort/Sort.mpc|3|2|Pf$|@{0|}} }}@0|@315000|78500|335000|99000|114000|87000|17|And the Sort|Ab... (8 Replies)
Discussion started by: Shell_Learner
8 Replies

5. Programming

Need to modify contents of file with complex patterns.

hi, my fstab file content is like this along with some other lines: /dev/vg0/var1 /var1 ext3 defaults 0 2 /dev/vg0/flx1 /flx1 ext3 defaults 0 2 /dev/vg0/var /var ext3 defaults 0 1 /dev/vg0/flx /flx ext3 defaults 0 2 I want to remove lines with /dev/vg0/var and... (5 Replies)
Discussion started by: success
5 Replies

6. Shell Programming and Scripting

SED complex string replacement

sed -i 's:"ps -ef | grep $(cat $PID_FILE) | grep -v grep":"ps -C java -o pid,cmd | grep ${SERVER_NAME} | cut -d' ' -f1 | grep -v grep":g' scriptName That's what I'm attempting to do. I'm attempting to replace this: ps -ef | grep $(cat $PID_FILE) | grep -v grep with this: ps -C java -o... (5 Replies)
Discussion started by: cbo0485
5 Replies

7. Shell Programming and Scripting

Double Spacing complex sed pipeline

my script: FILE="$1" echo "You Entered $FILE" if ; then tmp=$(cat $FILE | sed '/./!d' | sed -n '/regex/,/regex/{/regex/d;p}'| sed -n '/---/,+2!p' | sed -n '/#/!p' | sed 's/^*//' | sed -e\ s/*:// | sed -n '/==> /!p' | sed -n '/--> /!p' | sed -n '/regex/,+1!p' | sed -n '/======/!p' | sed -n... (1 Reply)
Discussion started by: omgsomuchppl
1 Replies

8. Shell Programming and Scripting

Complex find grep or sed command

Haven't worked in bash for ages. did a good bit of shell scripting in regular sh, but have forgotten most of it. I have several thousand php files that now include the following line at the end of the file. There is no LF or CR/LF before it begins, it is just concatenated to the final line of... (3 Replies)
Discussion started by: sjburden
3 Replies

9. Shell Programming and Scripting

Complex sed replacement

Hi, I have a file that I would like to translate using sed. I can do some basic sed commands, but Im afraid this level is beyond me. I have this file - ...alter... .. ...65536... ... ...65536... ... ...alter... ... ...65536... etc What I would like to do is replace the first... (11 Replies)
Discussion started by: one_ring99
11 Replies

10. Shell Programming and Scripting

Complex Sed/Awk Question?

Hello, So i have this file called /apps/turnout which looks like that of the contents of the /etc/shadow (but not exactly) the file has a long list in it. basically, the contents of this file looks something similar to the following: jajajajalala:D#$#AFVAdfda lalabavisof:#%R@fafla#$... (3 Replies)
Discussion started by: SkySmart
3 Replies
Login or Register to Ask a Question