Visit Our UNIX and Linux User Community


Sed/awk join lines once pattern found


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Sed/awk join lines once pattern found
# 1  
Old 04-11-2017
Sed/awk join lines once pattern found

Hi all

OS - RHEL6.4

I have input file -f1.txt
Code:
 [=129Dytt.....===..
  
 Sometxt
 \Start\now\fine step
 watch this space for tools 
 ends here 
  
 =--2323UT;;;====.......
  
 \Start\then\go language again
 cubernates space 
 sign in done 
 line again
 do it now

I need to search line which starts with
Code:
\Start

and read next line till it gets blank line and join them all. I need to trim any trailing spaces for each line.So output.txt should be..
Code:
 \Start\now\fine stepwatch this space for toolsends here 
 \Start\then\go language againcubernates spacesign in doneline againdo itnow

I tried ..
Code:
 sed -n '^//Start/{
 s/[ \t]*$// # trim trailing spaces
 N # Move to next line
 s/\n//p #join line and print them

But it goes to next line not beyond. Not sure how to keep lines in buffer till blank line comes.

Tried AWK
Code:
sed -i 's/[ \t]*$//g' f1.txt ; gawk '/^\\Start/{gsub("[\n]","",$0);print $0}RS=""' f1.txt

This gives me only last part
Code:
\Start\then\go language againcubernates spacesign in doneline againdo itnow

Can someone help me out here please?

Thanks
# 2  
Old 04-11-2017
You can try this, it is probably a lot more complicated that in needs to be but it seems to work. It trims the leading spaces before \Start as well so if that is undesired then it can be changed.

Code:
awk '/\\Start/{f=1};!NF{if (f) print A[f];split("",A);f=0};{gsub(/^ /,"");A[f]=A[f]?A[f] $0:$0};END{if (f) print A[f]}'

This User Gave Thanks to pilnet101 For This Post:
# 3  
Old 04-12-2017
Try:
Code:
awk '/\\Start/{f=1} !NF{if(f)print s; s=x; f=0} f{s=s $0} END{if(f) print s}'  file

Code:
 \Start\now\fine step watch this space for tools  ends here 
 \Start\then\go language again cubernates space  sign in done  line again do it now


--
Note: the input file has leading spaces on every line
You can get rid of those, like this
Code:
awk 'NF{$1=$1} /^\\Start/{f=1} !NF{if(f)print s; s=x; f=0} f{s=s $0} END{if(f) print s}'

and then it will be more like you output sample:
Code:
\Start\now\fine stepwatch this space for toolsends here
\Start\then\go language againcubernates spacesign in doneline againdo it now

This User Gave Thanks to Scrutinizer For This Post:
# 4  
Old 04-12-2017
Thanks both

Scrutinizer could you please help me to understand few things ..
1. What this code is doing
Code:
!NF{if(f)print s; s=x; f=0}

. I understand here we are processing blank lines and if f is true(1) then print the string in buffer and reset the s. so here s=x is same as s=""?

2. AS i know we can in sed or awk we can grab the pattern like ..

Code:
awk '/Start/,/^$/ {action}

here we get set of lines between start and blank line. Can we not join them some way ...This is just clear my concept.
# 5  
Old 04-12-2017
Hi krsnadasa,

1. Yes that is exactly right.
2. Yes that could be done, but then you need to test inside as well which is the case, so here I think it is not that practical. I did not use /^$/ since that would not work with your sample, seeing as its "empty lines" contain spaces. So I used !NF instead .

Last edited by Scrutinizer; 04-12-2017 at 06:25 PM..
# 6  
Old 04-13-2017
thanks much
Code:
Solved AWK/Sed Join

# 7  
Old 04-13-2017
sed only:
Code:
sed -rn 's/^ *| *$//; / *\\Start/,/^$/ {H; /^$/ {g;s/\n//g;p;s/.*//;h; };}; $ {g;s/\n//g;p;}' file
\Start\now\fine stepwatch this space for tools ends here 
\Start\then\go language againcubernates space sign in done line againdo it now

This User Gave Thanks to RudiC For This Post:

Previous Thread | Next Thread
Test Your Knowledge in Computers #136
Difficulty: Easy
The IEEE named Linus Torvalds as the recipient of the IEEE Computer Society's Computer Pioneer Award in 2014.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Join lines using sed or awk

Hi, I have text file that looks like this: blabla bla PATTERN LINE1 LINE2 bla bla bla PATTERN LINE1 LINE2 bla PATTERN LINE1 LINE2 bla (9 Replies)
Discussion started by: hench
9 Replies

2. Shell Programming and Scripting

Using awk or sed to find a pattern that has lines before and after it

Dear gurus, Please help this beginner to write and understand the required script. I am looking for useing awk for sed. I have a few thousand lines file whose contain are mostly as below and I am trying to achieve followings. 1. Find a string, say user1. Then hash the line containing the... (6 Replies)
Discussion started by: ran_bon_78
6 Replies

3. Shell Programming and Scripting

Sed/awk/perl command to replace pattern in multiple lines

Hi I know sed and awk has options to give range of line numbers, but I need to replace pattern in specific lines Something like sed -e '1s,14s,26s/pattern/new pattern/' file name Can somebody help me in this.... I am fine with see/awk/perl Thank you in advance (9 Replies)
Discussion started by: dani777
9 Replies

4. Shell Programming and Scripting

awk to print all lines after a pattern is found

Is there a way with aw to print all lines after a string is found There is a file like this ....... ........ 2012/19/11 :11.58 PM some data lne no date 2012/19/11 :11.59 PM some other data 2012/20/11 :12.00 AM some other data some line without dates some more lines without dates... (8 Replies)
Discussion started by: swayam123
8 Replies

5. Shell Programming and Scripting

Getting lines before and until next pattern in file /awk, sed

Hi, I need to get specific parts in a large file. I need to: Get a line containing an IP address, and read from there to another line saying ***SNMP-END*** So, I have the start and the end well defined, but the problem is that apparently the awk command using the -F option doesn't work... (17 Replies)
Discussion started by: ocramas
17 Replies

6. Shell Programming and Scripting

How to print the lines between the pattern using awk/grep/sed?

Hi, I need a help to search a pattern and print the multiple lines between them. Input file: Tue May 29 12:30:33 EDT 2012:threadWebContainer : 357:com.travimp.hotelierlinks.abba.service.RequestHandler.requestService(String, ITICSDataSet): hotelCancelReservation request: ... (4 Replies)
Discussion started by: aroragaurav.84
4 Replies

7. UNIX for Dummies Questions & Answers

sed, join lines that do not match pattern

Hello, Could someone help me with sed. I have searched for solution 5 days allready :wall:, but cant find. Unfortunately my "sed" knowledge not good enough to manage it. I have the text: 123, foo1, bar1, short text1, dat1e, stable_pattern 124, foo2, bar2, long text with few lines, date,... (4 Replies)
Discussion started by: petrasl
4 Replies

8. Shell Programming and Scripting

sed/awk : how to delete lines based on IP pattern ?

Hi, I would like to delete lines in /etc/hosts on few workstations, basically I want to delete all the lines for a list of machines like this : for HOST in $(cat stations.lst |uniq) do # echo -n "$HOST" if ping -c 1 $HOST > /dev/null 2>&1 then HOSTNAME_val=`rsh $HOST "sed... (3 Replies)
Discussion started by: albator1932
3 Replies

9. Shell Programming and Scripting

sed/awk to insert multiple lines before pattern

I'm attempting to insert multiple lines before a line matching a given search pattern. These lines are generated in a separate function and can either be piped in as stdout or read from a temporary file. I've been able to insert the lines from a file after the pattern using: sed -i '/pattern/... (2 Replies)
Discussion started by: zksailor534
2 Replies

10. Shell Programming and Scripting

How to awk/sed/grep lines which contains a pattern at a given position

Dear friends I am new to linux and was trying to split some files userwise in our linux server. I have a data file of 156 continuous columns named ecscr final. I want the script to redirect all the lines containing a pattern of 7 digits to separate files. I was using grep to do that,... (2 Replies)
Discussion started by: anoopvraj
2 Replies

Featured Tech Videos