Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Replace Block

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replace Block
# 8  
Old 05-11-2010
MySQL
It contains a lot of punctuation and other special characters..

"(div idm1)" and "(/div)" are a line
Code:
 
[root@sistem1lnx ~]# cat testfile
This is a test
This is test
"(div idm1)"
The first line\'/'\.\'.[];][[+_)_()*&^&^&%^#%@#$!~[][[][=-00-99889763412`>?>`,';\';][p][{}]12`//'-0
This is a test
This is a test
This is a test
This is a test
The last line
"(/div)"
This is a test
This is test
......................
............................
................................

the other file contains punctuation and other special characters too..

Code:
[root@sistem1lnx ~]# cat file_to_add
tetreawd0-9-1212e312?"?{}_=--=0-00908][]989*()*&*&^&^%^$%#$$!@#!~@~`\'\/[;[][[]:
test12asas9ie23U&^I*&)*()(_*%&^$!@#~@~@~@~#$^%*&Y)(*_(_*()+)_+_
deneme34534tgergeg56u&*(*)()*_+*işç.",i22414"31<F2><F4>
test123é!'^'^é!'^+%&/())=?_ĞÜ;İ:ÇÖ><>£#${[}\|¨`|<z<z:şö,i

Code:
linecount=$( (sed -n '/"(div idm1)"/,/"(\/div)"/{;/"(div idm1)"/!p;}' testfile | sed -n '$!p' | wc -l) )
while [ $(( linecount -= 1 )) -gt -1 ]
  do
    sed -i '/^"(div idm1)"/ {n;d;q}' testfile
  done
  while read line
     do
       sed -i "/^\"(\/div)\"/ i$line" testfile
     done < file_to_add

Code:
[root@sistem1lnx ~]# cat testfile
This is a test
This is test
"(div idm1)"
tetreawd0-9-1212e312?"?{}_=--=0-00908][]989*()*&*&^&^%^$%#$$!@#!~@~`'/[;[][[]:
test12asas9ie23U&^I*&)*()(_*%&^$!@#~@~@~@~#$^%*&Y)(*_(_*()+)_+_
deneme34534tgergeg56u&*(*)()*_+*işç.",i22414"31<F2><F4>
test123é!'^'^é!'^+%&/())=?_ĞÜ;İ:ÇÖ><>£#${[}|¨`|<z<z:şö,i
"(/div)"
This is a test
This is test
......................
............................
................................

First example Ok!!

for another test
;;;; my testfile

"(div idm1)" and "(/div)" are in a line which contains punctuation and other special characters..

Code:
[root@sistem1lnx ~]# cat testfile
This is a test
This is test
"(div idm1)"The first line\'/'\.\'.[];][[+_)_()*&^&^&%^#%@#$!~[][[][=-00-99889763412`>?>`,';\';][p][{}]12`//'-0
This is a teste$%@#$!@#$!^$^&*))_\'/'/[[;;[,;,\z\d
This is a teste)_()&*&*%#$!~#!!~##!@#@#$#/
This is a test%^%<>?:"{P?{KI@#!@#!@!$~0[-[-][\[{}
This is a test3224234354()(_*)?.,.';';:::'\\]..][}{}312wefef
The last line$%@#$!@#$!^$^&*))_\'/'/[[;;[,;,\z\\\||||><><ALS[;12-3445]"(/div)"
This is a test
This is test
......................
............................
................................


Code:
 
sed -i 's/"(div idm1)"/"(div idm1)"\n/' testfile ; sed -i 's/"(\/div)"/\n"(\/div)"/' testfile
linecount=$( (sed -n '/"(div idm1)"/,/"(\/div)"/{;/"(div idm1)"/!p;}' testfile | sed -n '$!p' | wc -l) )
while [ $(( linecount -= 1 )) -gt -1 ]
do
sed -i '/^"(div idm1)"/ {n;d;q}' testfile
done
while read line
do
sed -i "/^\"(\/div)\"/ i$line" testfile
done < file_to_add
 
linecountaddfile=$( (cat file_to_add | wc -l)) # Other file line count for last line editing on testfile (my orginal file)
 
#Howmany line to read next line for sed and sed process on the org file
 
commandx=""
x="n;" # for read next line
 
while [ $(( linecountaddfile -= 1 )) -gt -1 ]
do
commandx="$commandx$x";
echo $linecount "$commandx"
done
 
# four line (4) in file_to_add so $commandx="n;n;n;n; "

Code:
# And for "(/div)" editing so to merge last line
sed -i " /(div idm1)/{ $commandx N; /\n/ {s///;} } " testfile
 
Now testfile is like seem

Code:
cat testfile
This is a test
This is test
"(div idm1)"
tetreawd0-9-1212e312?"?{}_=--=0-00908][]989*()*&*&^&^%^$%#$$!@#!~@~`'/[;[][[]:
test12asas9ie23U&^I*&)*()(_*%&^$!@#~@~@~@~#$^%*&Y)(*_(_*()+)_+_
deneme34534tgergeg56u&*(*)()*_+*işç.",i22414"31<F2><F4>
test123é!'^'^é!'^+%&/())=?_ĞÜ;İ:ÇÖ><>£#${[}|¨`|<z<z:şö,i"(/div)"
This is a test
This is test
......................
............................
................................


# more after "(div idm1)" editing so merge first line between our patterns ("(div idm1)" and "(/div)" )

Code:
 
sed -i '/"(div idm1)"/{N; /\n/ {s///;} } ' testfile

Now testfile is like seem

Code:
 
 
[root@sistem1lnx ~]# cat testfile
This is a test
This is test
"(div idm1)"tetreawd0-9-1212e312?"?{}_=--=0-00908][]989*()*&*&^&^%^$%#$$!@#!~@~`'/[;[][[]:
test12asas9ie23U&^I*&)*()(_*%&^$!@#~@~@~@~#$^%*&Y)(*_(_*()+)_+_
deneme34534tgergeg56u&*(*)()*_+*işç.",i22414"31<F2><F4>
test123é!'^'^é!'^+%&/())=?_ĞÜ;İ:ÇÖ><>£#${[}|¨`|<z<z:şö,i"(/div)"
This is a test
This is test
......................
............................
................................


The result is succesfull Smilie

Best Regards
ygemici
Last edited by ygemici; 05-11-2010 at 03:00 PM..
# 9  
Old 05-12-2010
Hi, Dieter5.

Apologies for the long post.

I think that we often see a tool and a few features, and we think it will fit for a solution to a problem. It's possible that we see, for example, that sed has a range selection for lines:
Code:
line1,line2

or
Code:
/pattern1/,/pattern2/

and we think that we can perform some task based on that. However,we may be over-looking how a stream editor works.

The sed command, in normal operation, reads a single line, and applies all the operations on that line that are appropriate. So, if an operation has a range selection that includes the current line, then the operation proceeds. When all the operations are completed, the line is written out, and the next line is read in, operated upon, etc. So there are constraints on what can be done, and certainly on what can be done easily.

On the other hand, awk is flexible tool that may seem like it takes longer to master, but most people who take that time find it to be well spent.

The one advantage that sed has over awk is in the area of efficiency -- sed is 1/6 the size of awk (machine efficiency), and simple substitutions are easily coded (human efficiency).

I don't know if the code from ygemici is the best sed solution, but it is certainly better than I could do without a lot of extra work.

A few simple awk rules in the core of the solution below, seem simple and straight-forward. I'm sure that the awk can be improved upon, but this solution works as it is.
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate block-replacement, preserve delimiters, awk.

# Uncomment to run script as external user.
# export PATH="/usr/local/bin:/usr/bin:/bin"
# Infrastructure details, environment, commands for forum posts. 
set +o nounset
pe() { for i;do printf "%s" "$i";done; printf "\n"; }
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe ; pe "Environment: LC_ALL = $LC_ALL, LANG = $LANG"
pe "(Versions displayed with local utility \"version\")"
c=$( ps | grep $$ | awk '{print $NF}' )
version >/dev/null 2>&1 && s=$(_eat $0 $1) || s=""
[ "$c" = "$s" ] && p="$s" || p="$c"
version >/dev/null 2>&1 && version "=o" $p specimen awk
set -o nounset

FILE1=${1-data1}
shift
FILE2=${1-data2}

# Sample data files with head / tail if specimen fails.
pe
specimen 7 $FILE1 $FILE2 \
|| { pe "(head/tail)"; head -n 5 $FILE1 $FILE2; pe " ||" ;\
     tail -n 5 $FILE1 $FILE2; }

pe
pe " Results:"
awk -v inside="$FILE2" '
BEGIN                     { looking = 0 }
looking == 1 && /(\/div)/ { looking = 0 ;
                            while ( ( getline t1 < inside > 0 ) ) {
                              print t1
                            }
                            close(inside)
                            print
                            next
                          }
/(div id=m1)/             { print ; looking = 1 ; next }
looking == 0              { print }
' $FILE1

exit 0

produces, by inserting file data2 into data1 between the markers:
Code:
% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0 
GNU bash 3.2.39
specimen (local) 1.17
GNU Awk 3.1.5

Whole: 7:0:7 of 13 lines in file "data1"
foo
(div id=m1)
bar
baz
(/div)
corge
warg
(div id=m1)
fred
plugh
xyzzy
(/div)
thud

Whole: 7:0:7 of 6 lines in file "data2"
  A little song
  A little dance
  A little seltzer down your pants
  Lots of specials: \'/'\.\'.[];][[+_)_()
  more: *&^&^&%^#%@#$!~[][[][=-00-99889763412
  still more: `>?>`,';\';][p][{}]12`//'-0

 Results:
foo
(div id=m1)
  A little song
  A little dance
  A little seltzer down your pants
  Lots of specials: \'/'\.\'.[];][[+_)_()
  more: *&^&^&%^#%@#$!~[][[][=-00-99889763412
  still more: `>?>`,';\';][p][{}]12`//'-0
(/div)
corge
warg
(div id=m1)
  A little song
  A little dance
  A little seltzer down your pants
  Lots of specials: \'/'\.\'.[];][[+_)_()
  more: *&^&^&%^#%@#$!~[][[][=-00-99889763412
  still more: `>?>`,';\';][p][{}]12`//'-0
(/div)
thud

Quote:
"Different tools may rely on different assumptions about their
context--e.g., surrounding infrastructure, control model, data
model, communication protocols, etc.--which can lead to an
architectural mismatch between the application and the tools.
Such a mismatch leads to hacks and workarounds that will make the
code more complex than necessary."

-- 97 Things Every Programmer Should Know

"Sed has a few rudimentary programming constructs that can be
used to build more complicated scripts. It also has a limited
ability to work on more than one line at a time."

-- sed & awk, 2nd Edition
cheers, drl
Last edited by drl; 05-12-2010 at 01:23 PM.. Reason: ( Edit 1: typo, clarify )
# 10  
Old 05-12-2010
I took the message (awk) and got the solution (see above).
And now I know that there is no sed-awk program that can interpret

sed-awk s/(div id=m1).*?(\/div)/(div id=m1)$replacetextfromfile(\/div)/"

Thanks!
# 11  
Old 08-14-2010
A little bit simple solution :

Code:
cat $file | grep -A $nb_line_to_suppress "$motif" > /tmp/tempfile.tmp
diff --suppress-common-lines --unchanged-line-format='' $file /tmp/tempfile.tmp > file.suppressed_block
rm -f /tmp/tempfile.tmp
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk script to extract a column, replace one of the header and replace year(from ddmmyy to yyyy)

I have a csv which has lot of columns . I was looking for an awk script which would extract a column twice. for the first occurance the header and data needs to be intact but for the second occurance i want to replace the header name since it a duplicate and extract year value which is in ddmmyy... (10 Replies)
Discussion started by: Kunalcurious
10 Replies

2. Shell Programming and Scripting

Bash shell: Replace a text block by another

Hello, I'm an starter in Bash scripting. I would like to write a script in Bash shell that replaces a specific text block (a function) by another text block in a file: for example in my file --> $HOME/myFile.js replacing following function between other functions in the file: function ABC()... (6 Replies)
Discussion started by: om1559
6 Replies

3. UNIX for Advanced & Expert Users

Move a block of lines to file if string found in the block.

I have a "main" file which has blocks of data for each user defined by tags BEGIN and END. BEGIN ID_NUM:24879 USER:abc123 HOW:47M CMD1:xyz1 CMD2:arp2 STATE:active PROCESS:id60 END BEGIN ID_NUM:24880 USER:def123 HOW:4M CMD1:xyz1 CMD2:xyz2 STATE:running PROCESS:id64 END (7 Replies)
Discussion started by: grep_me
7 Replies

4. Shell Programming and Scripting

using sed/awk to replace a block of text in a file?

My apologies if this has been answered in a previous post. I've been doing a lot of searching, but I haven't been able to find what I was looking for. Specifically, I am wondering if I can utilize sed and/or awk to locate two strings in a file, and replace everything between those two strings... (12 Replies)
Discussion started by: kiddsupreme
12 Replies

5. Shell Programming and Scripting

replace block of text with content of another file

Hello, file1: not to be changed not to be changed <start> old stuff old stuff old stuff <end> not to be changed not to be changed file2: new text new text desired output: (3 Replies)
Discussion started by: ripat
3 Replies

6. Shell Programming and Scripting

Replace text block in multiple files

I need to replace (delete) a text block in a bunch of files, its a html table, almost at the end of pages but the location varies. In Windows I used Filemonkey, but nothing like that in Unix? There is replace from mysql, but how does it deal with newlines? sed only works with single lines,... (6 Replies)
Discussion started by: eiland
6 Replies

7. Shell Programming and Scripting

Need to find a multiple line block and replace with a multiple line block

I would perfer to use cut and paste to do this but I can't find a GUI to do this with. What I want to do is to find a multiple line block of code like Exit Sub Log_Handler: then replace it with GoTo RSLogRtn Exit Sub Log_Handler: Basically it is just an insert, but I may want to... (8 Replies)
Discussion started by: Randem
8 Replies

8. Shell Programming and Scripting

finding a block in a file and replace with another file block.

(1) Yes but how is this block different from the other 24? You will need this information in order to identify and replace this block correctly (out of the 25). Ans: The 1st line and last line of this block are unique from other block. The 1st line is “rem Subset Rows (&&tempName.*) and The... (1 Reply)
Discussion started by: Zaheer.mic
1 Replies

9. Shell Programming and Scripting

awk - replace number of string length from search and replace for a serialized array

Hello, I really would appreciate some help with a bash script for some string manipulation on an SQL dump: I'd like to be able to rename "sites/WHATEVER/files" to "sites/SOMETHINGELSE/files" within the sql dump. This is quite easy with sed: sed -e... (1 Reply)
Discussion started by: otrotipo
1 Replies

10. UNIX for Dummies Questions & Answers

Search/replace using visual block

Hi, how would we replace a few patterns on the same line with a change of it's own..using visual block on vim editor. ie a file contains lines such as the following: abccss (dfrss)) emailkk abdcss (dfrss)) dmailkk Using visual block once, replacement is indeed to get the following output:... (0 Replies)
Discussion started by: Manan
0 Replies
Login or Register to Ask a Question