Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Append in one line

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Append in one line
# 1  
Old 05-04-2011
Power Append in one line
Hello,

I have a very huge file having below data:-
Code:
subsD,00 05 02 70 DB 3D 4A B8 47 5B 38 00,919030055007,,22,
,,,,23,
,,,,21,
subsD,00 05 02 FD DE 3D 4A B8 47 5D 38 00,919030055007,,23,
,,,,47,
,,,,49,
subsD,00 05 02 BA 01 3E 4A B8 47 67 38 00,919030055007,,22,
,,,,23,
,,,,21,
subsD,00 05 02 F8 0F 3E 4A B8 47 6C 38 00,919030055007,,22,
,,,,23,
,,,,21,

I want output like below:-
Code:
subsD,00 05 02 70 DB 3D 4A B8 47 5B 38 00,919030055007,,22,23,21,
subsD,00 05 02 FD DE 3D 4A B8 47 5D 38 00,919030055007,,23,47,49,
subsD,00 05 02 BA 01 3E 4A B8 47 67 38 00,919030055007,,22,23,21,
subsD,00 05 02 F8 0F 3E 4A B8 47 6C 38 00,919030055007,,22,23,21

Means all lines starting with ',,,,' should be appended to previous line.

I tried using sed replacing ',,,,' with dual backspaces but its not working out.
Please help!!

Moderator's Comments:
Mod Comment Please use [code] and [/code] tags when posting code, data or logs etc. to preserve formatting and enhance readability, thanks.
Last edited by zaxxon; 05-04-2011 at 09:20 AM.. Reason: code tags
# 2  
Old 05-04-2011
Code:
awk 'NR>1&&/^sub/{sub($0,"\n"$0,$0)}{printf $0}END{print}' infile

Use nawk instead of awk if you run Solaris/SunOS

Code:
nawk 'NR>1&&/^sub/{sub($0,"\n"$0,$0)}{printf $0}END{print}' infile

Last edited by ctsgnb; 05-04-2011 at 09:08 AM..
# 3  
Old 05-04-2011
Quote:
Originally Posted by ctsgnb
Code:
awk 'NR>1&&/^sub/{sub($0,"\n"$0,$0)}{printf $0}END{print}' infile

Use nawk instead of awk if you run Solaris/SunOS

Code:
nawk 'NR>1&&/^sub/{sub($0,"\n"$0,$0)}{printf $0}END{print}' infile

The above commands give as output:
Code:
subsD,00 05 02 70 DB 3D 4A B8 47 5B 38 00,919030055007,,22,,,,,23,,,,,21,
subsD,00 05 02 FD DE 3D 4A B8 47 5D 38 00,919030055007,,23,,,,,47,,,,,49,
subsD,00 05 02 BA 01 3E 4A B8 47 67 38 00,919030055007,,22,,,,,23,,,,,21,
subsD,00 05 02 F8 0F 3E 4A B8 47 6C 38 00,919030055007,,22,,,,,23,,,,,21,

Try this one:
Code:
awk -F, '!$1{$0=$(NF-1) FS}{ORS=NR%3?"":RS}1' file

# 4  
Old 05-04-2011
Quote:
Originally Posted by Franklin52
Code:
awk -F, '!$1{$0=$(NF-1) FS}{ORS=NR%3?"":RS}1' file

I don't get the
Code:
!$1{$0=$(NF-1) FS}

This should then be sufficient :
Code:
awk '{ORS=NR%3?z:RS}1' infile

but it won't be able to handle files whose ^sub... occurrences' interval do vary (sometime 2 line inbetween sometimes more, sometime less ...)

But ok, in the given example it look like this interval doesn't change .
# 5  
Old 05-04-2011
This removes the 4 leading field separators of a record if the first field is empty:
Code:
!$1{$0=$(NF-1) FS}

This:
Code:
awk '{ORS=NR%3?z:RS}1' infile

gives the same output as your first solution.
# 6  
Old 05-14-2011
sed and awk shell script and command line examples
I noticed you didn't seem to get a response that addressed the case of the logical CSV line being broken over some number other than 3 lines.
The code I provide works with the logical CSV broken across any number of lines as long as the continued lines start with a comma and have the same format.

Here is a sed version:
Code:
sed -ne '1{h;d};/^,/!{x;p};/^,/{s/,*\([^,]*,\)$/\1/;H;x;s/\n//g;x};${x;p}' append_csvlines.input

Running awk shell script (source provided below).
Code:
./append_csvlines.awk append_csvlines.input

awk shell script output
Code:
subsD,00 05 02 70 DB 3D 4A B8 47 5B 38 00,919030055007,,22,23,21,
subsD,00 05 02 FD DE 3D 4A B8 47 5D 38 00,919030055007,,23,47,49,
subsD,00 05 02 BA 01 3E 4A B8 47 67 38 00,919030055007,,22,23,21,
subsD,00 05 02 F8 0F 3E 4A B8 47 6C 38 00,919030055007,,22,23,21,

Example input you supplied.
Code:
subsD,00 05 02 70 DB 3D 4A B8 47 5B 38 00,919030055007,,22,
,,,,23,
,,,,21,
subsD,00 05 02 FD DE 3D 4A B8 47 5D 38 00,919030055007,,23,
,,,,47,
,,,,49,
subsD,00 05 02 BA 01 3E 4A B8 47 67 38 00,919030055007,,22,
,,,,23,
,,,,21,
subsD,00 05 02 F8 0F 3E 4A B8 47 6C 38 00,919030055007,,22,
,,,,23,
,,,,21,

Example output that you requested.
Code:
subsD,00 05 02 70 DB 3D 4A B8 47 5B 38 00,919030055007,,22,23,21,
subsD,00 05 02 FD DE 3D 4A B8 47 5D 38 00,919030055007,,23,47,49,
subsD,00 05 02 BA 01 3E 4A B8 47 67 38 00,919030055007,,22,23,21,
subsD,00 05 02 F8 0F 3E 4A B8 47 6C 38 00,919030055007,,22,23,21

Single line awk command.
Code:
awk 'BEGIN{FS=","} $1&&NR>1{$0=RS$0} !$1{$0=$(NF-1)FS} {printf $0} END{printf RS}' append_csvlines.input

The output of this script differs from your example output by a comma on the final output line. I assumed that was a typo on your part.
Code:
subsD,00 05 02 70 DB 3D 4A B8 47 5B 38 00,919030055007,,22,23,21,
subsD,00 05 02 FD DE 3D 4A B8 47 5D 38 00,919030055007,,23,47,49,
subsD,00 05 02 BA 01 3E 4A B8 47 67 38 00,919030055007,,22,23,21,
subsD,00 05 02 F8 0F 3E 4A B8 47 6C 38 00,919030055007,,22,23,21,

Here is the shell script append_lines.awk. You have to chmod +x it or it won't work.
Code:
#!/usr/bin/awk -f
BEGIN{FS=","}
$1&&NR>1{$0=RS$0}
!$1{$0=$(NF-1)FS}
{printf $0}
END{printf RS}

You can turn the sed command into a shell script as well:
Code:
#!/bin/sed -nf
1{h;d}
/^,/!{x;p}
/^,/{s/,*\([^,]*,\)$/\1/;H;x;s/\n//g;x}
${x;p}

Both the sed and awk versions have a similar pattern. The first line and last lines are special cases and you have to manually output new lines.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove new line starting with a numeric value and append it to the previous line

Hi, i have a file with multiple entries. After some tests with sed i managed to get the file output as follows: lsn=X-LINK-IN0,apc=661:0,state=avail,avail/links=1/1, 00,2110597,2094790,0,81,529,75649011,56435363, lsn=TM1ITP1-AM1ITP1-LS,apc=500:0,state=avail,avail/links=1/1,... (5 Replies)
Discussion started by: nms
5 Replies

2. Shell Programming and Scripting

Append Next line with current Line bassed on condition

Hi, I have an XML file and I am tring to extract some data form it, after lot of data cleaning process, I ended up with an issue, and need your urgent support. my current input data in below format: <Node>xxxxxx <Node>yyyyy</Node> <Node>zzzzzz <Node>12345</node> I need... (9 Replies)
Discussion started by: rramkrishnas
9 Replies

3. UNIX for Dummies Questions & Answers

How to remove fields space and append next line to previous line.?

awk 'BEGIN{FS = "Ç"} NR == 1 {p = $0; next} NF > 1 {print p; p = $0} NF <= 1 {p = (p " " $0)} END {print p}' input.txt > output.txt This is what the input data file looks like with broken lines Code: 29863 Ç890000000 Ç543209911 ÇCHNGOHG Ç000000001 Ç055 ... (4 Replies)
Discussion started by: cumeh1624
4 Replies

4. Shell Programming and Scripting

find a certain line and append text to the end of the line

After I create printer queues in AIX, I have to append a filter file location within that printers custom file. within lets say test_queue.txt I need to find the row that starts with :699 and then I need to append on the end the string /usr/local/bin/k_portrait.sh. Now I've gotten the sed... (2 Replies)
Discussion started by: peachclift
2 Replies

5. Shell Programming and Scripting

Append next line to previous line when one pattern not found

Hi, I need help for below scenario.I have a flat file which is having records seperated by delimiters which will represent each record for oracle table.My Control file will consider each line as one record for that table. Some of the lines are aligned in two/three lines so that records are... (4 Replies)
Discussion started by: kannansr621
4 Replies

6. Shell Programming and Scripting

shell script to read a line in gps receiver log file and append that line to new file

Hi, I have gps receiver log..its giving readings .like below Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GPSD,R=1 $GPGSV,3,1,11,08,16,328,40,11,36,127,00,28,33,283,39,20,11,165,00*71... (3 Replies)
Discussion started by: gudivada213
3 Replies

7. Shell Programming and Scripting

Append each line to next previous line in a file

Hi all, Please help me in providing sample code to append the following 4 lines in one row. Input : A1/EXT "BAPBSC10/07B/00" 523 090530 0115 RXOCF-430 HY1711 1 EXTERNAL ALARM DOOR ALARM Output should be : A1/EXT "BAPBSC10/07B/00" 523 090530 0115 ... (8 Replies)
Discussion started by: sudhakaryadav
8 Replies

8. Shell Programming and Scripting

Append line that does not contain pipe to it previous line

Hi All, I have a file which contains data as below When we see no pipe character in the line. append those lines to the previous line with pipe character till we get the next line with pipe character with ~(concat with ~) Input file looks like: 1080530944|001|john.l.bonner|Acknowledge|CN... (11 Replies)
Discussion started by: ainuddin
11 Replies

9. Shell Programming and Scripting

Joining lines in reverse. append line 1 to line 2.

Hi I have used many times the various methods to append two lines together in a file. This time I want to append the 1st line to the second and repeat for the complete file.... an example This is the file owns the big brown dog joe owns the small black dog jim What I want is ... (7 Replies)
Discussion started by: dwalley
7 Replies

10. UNIX Desktop Questions & Answers

append a line to the last line in a file

Suppose i have a file "xyz.txt" which contains abcdef ghijklm nop Now in want to add "qrst" to the last line such that the file becomes abcdef ghijklm nopqrst P.S:The o/p i need is abcdef ghijklm nopqrst (and not) nop qrst (5 Replies)
Discussion started by: subhrap.das
5 Replies
Login or Register to Ask a Question