Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Getting multiple messy lines into one single line

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Getting multiple messy lines into one single line
# 1  
Old 11-13-2009
Getting multiple messy lines into one single line
I have a file that contains the following:

Code:
:@:176:@:4:@:name:@:file:@:this is 
a summary:@:description can be
long but who knows
can even have <br> tags.:@:how to:@:type:@:18544:@:550:@:400:END:
          
                
   

:@:177:@:9:@:name:@:file:@:summary:@:this 
will containg a long
description
that couple be broken
line.:@:how to:@:type:@:3925:@:650:@:250:END:

And it continues to about 10K lines. Every description can take more/less lines etc, but each field should be separated by :@: until it reaches the end of the line :END: to start an entire new line.

I'm having trouble creating a script using php/shell/awk/sed or something similar to go through the file and organize it so that the multiple lines become one line until it reaches the :END: ending parameter.

The pattern here is the :@: every line has it, execept for the blank lines which are many. And each line ends with :END: So every single line should end with :END: and then a new line will start.

So that the above file should end like this:

Code:
:@:176:@:4:@:name:@:file:@:this is a summary:@:description can be long but who knows can even have <br> tags.:@:how to:@:type:@:18544:@:550:@:400:END:
:@:177:@:9:@:name:@:file:@:summary:@:this will containg a long description that couple be broken line.:@:how to:@:type:@:3925:@:650:@:250:END:

With only 2 long lines.

I'm having trouble even starting this Smilie it might be the time but, I'm coming for help here.
One thing to be careful is when concatenating the lines to not eat any spaces between words, so that it doesn't read like "this isasummary" instead of "this is a summary".

Please let me know how this can be done or if you can share some ideas and how-tos or some coding.

Thanks in advance.
# 2  
Old 11-13-2009
Code:
awk   '/^:@/{print ""}NF{printf $0" "}' file

next time show what you have tried.
# 3  
Old 11-13-2009
Code:
sed -n '/^:@:/{
1{h;}
1!{x;s/\n//g;p;}
}
/^:@:/!{
/^$/!{H;}
}
${x;s/\n//g;p;}'

# 4  
Old 11-13-2009
ghostdog74, you script worked great for that little file above, but when I run awk in my file wich now how about 10K lines I get an error:

Code:
awk: (FILENAME=to-organize.txt FNR=3397) fatal: not enough arguments to satisfy format string
	`:@:2168:@:18:@:Dig An:@:dig-an.s:@:100% de dança para você:@:100% de dança para você:@:Mouse - comand 'o jog:@:SWF:@:1217:@:540:@:340:END:
	                                                  ^ ran out for this one

I didn't post anything because I didn't know where to start and was kind of late here.

summer_cherry, your sed worked for some lines as well, but the big file has many extra spaces and also unicode characters like ê é ç, etc it also has special characters & % $

I was thinking in something like, IF the first 3 characters of a line is :@: it would look for the :END: strings and everything before it would be put to one single line
OR
we can put everything on the file into one huge line and then run, sed 's/:END:/:END:\n/g' file which would break the line after :END: and so on.

let me know your ideas, and thank you in advance
# 5  
Old 11-13-2009
Try this

Code:
nawk '
/^:@/{l++}
(NF){ _[l] = _[l]$0" " }
END{
    for( i = 1; i <= l; i++ )
      printf("%s\n", _[i])
} ' infile

Last edited by steadyonabix; 11-13-2009 at 02:25 PM.. Reason: typo
# 6  
Old 11-13-2009
nice steadyonabix, your sed worked somehow, but we're almost there, I've noticed that some lines are like:

Code:
:@:6:@:1:@:Shortcut:@:shortcut.s:@:Leve o carr:@:Você terá que construir uma pista.
:@:Todos os movimentos com:@:SWF:@:157916:@:346:@:476:END:
:@:7:@:1:@:SLOY 3:@:SLOY3.s:@:Mova todos as cores do bloco:@:Para vencer este  azuis para cima, as lado direito e o lado esquerdo.
:@:Todos os movimentos com o:@:SWF:@:46176:@:510:@:400:END:

So your awk is getting each line that starts with :@: and in that case, the line is broken and the :@: seems to be in the second line but its not a new line since the :END: didn't reach, can we add something like I mentioned before, until it doesn't reach :END: it does't creat a new line \n ?

Thank you very much
# 7  
Old 11-13-2009
This seems to do the job, try it on your large file

Code:
nawk '
/^:@/ && ( ! n ){ l++; n++ }
(NF){ _[l] = _[l]$0" " }
/:END:/{ n = 0 }
END{
    for( i = 1; i <= l; i++ )
      printf("%s\n", _[i])
} ' infile
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Multiple lines to single line

I have code as below # create temporary table `temp4277`(key(waybill_no)) select waybill_no,concat_ws('',card_type,card_series_no) cardinfo from rfid_temp_ticket where waybill_no='4277' group by... (4 Replies)
Discussion started by: kaushik02018
4 Replies

2. Shell Programming and Scripting

Merge multiple lines into a single line

Hi all, I'm relatively new to scripting, I can do pretty basic things. I have a daily log file that looks like: timestamp=2017-06-28-01.01.35.080576; event status=0; userid=user1; authid=user1; application id=10.10.10.10.11111.12345678901; application name=GUI; ... (29 Replies)
Discussion started by: dwdnet
29 Replies

3. Shell Programming and Scripting

Coverting multiple lines to a single line

Hi all, I have a requirement to covert multiple lines in a comma delimited file to a single line through shell scripting. We should compare the data in the first column in each line. If it is same, then the other data should be put in the same line.Below is the sample input and expected output:... (4 Replies)
Discussion started by: Bobby_2000
4 Replies

4. Shell Programming and Scripting

Combine multiple lines into single line

Hi All , I have a file with below data # User@Host: xyz @ # Query_time: t1 Lock_time: t2 Rows_sent: n1 Rows_examined: n2 SET timestamp=1396852200; select count(1) from table; # Time: 140406 23:30:01 # User@Host: abc @ # Query_time: t1 Lock_time: t2 Rows_sent: n1 Rows_examined:... (6 Replies)
Discussion started by: rakesh_411
6 Replies

5. Shell Programming and Scripting

merging multiple lines into single line

Hi, 1. Each message starts with date 2. There is blank line between each message 3. Each message does not contain same number of lines. Any help in merging multiple lines in each message to a single line is much appreciated. AIX: Korn Shell Error log file looks like below. ... (5 Replies)
Discussion started by: bala123
5 Replies

6. Shell Programming and Scripting

Multiple lines in a single column to be merged as a single line for a record

Hi, I have a requirement with, No~Dt~Notes 1~2011/08/1~"aaa bbb ccc ddd eee fff ggg hhh" Single column alone got splitted into multiple lines. I require the output as No~Dt~Notes 1~2011/08/1~"aaa<>bbb<>ccc<>ddd<>eee<>fff<>ggg<>hhh" mean to say those new lines to be... (1 Reply)
Discussion started by: Bhuvaneswari
1 Replies

7. Shell Programming and Scripting

Combine multiple lines in single line

This is related to one of my previous post but now with a slight difference: I need the "Updated:" to be in one line as well as the "Information:" on one line as well. These are in multiple lines right now as seen below. These can have 2 or more lines that needs to be in one line. System name:... (8 Replies)
Discussion started by: The One
8 Replies

8. Shell Programming and Scripting

Multiple lines into a single line on Ubuntu 10.04

Hi, I've some files with the following data and i need to convert the lines between the separator ---, into a single line. I've tried with the paste cmd but my main problem is that the number of lines between the separator is not fix, it can vary between 1-4 lines. Input --- 2010-02-22... (8 Replies)
Discussion started by: RickyC9999
8 Replies

9. Shell Programming and Scripting

Multiple lines into a single line

Hi, I've some files with the following data and i need to convert the lines between the separator ---, into a single line. I've tried with the paste cmd but my main problem is that the number of lines between the separator is not fix, it can very between 1-4 lines. Input --- 2010-02-22... (4 Replies)
Discussion started by: RickyC9999
4 Replies

10. Shell Programming and Scripting

Splitting a single line into multiple lines

I have a case where, I need to look into a file. Go to each line of the file, find the length of the line, if the length of the line is more than 75 chars, I need to split the line into multiple lines of 75chars max. If the length of the line is less than 75, we need not do anything. So at the... (4 Replies)
Discussion started by: thanuman
4 Replies
Login or Register to Ask a Question