Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sed - merge lines bw 2 specific characters

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sed - merge lines bw 2 specific characters
# 1  
Old 03-29-2011
Sed - merge lines bw 2 specific characters
Hi, I have a bash script and I am looking for a command that will merge specific lines together.

Sample Data:

Code:
registration time = 1300890272
Id = 1
setd = 0
tagunt = 26
tagId=6, length=8, value=[ 30 30 2d 30 36 ]
tagId=9, length=5, value=[ 35 30 31 ]
tagId=7, length=2, value=[ 00 8c ]
tagId=16, length=2, value=[ 00 00 ]
tagId=32, length=1, value=[ 19 ]
tagId=33, length=1, value=[ 62 ]
tagId=34, length=1, value=[ 15 ]
tagId=35, length=16, value=[ 00 01 00 03 55 ea 00 00 00 00 00 00 00 00 00 00
]
tagId=35, length=16, value=[ 00 04 00 00 4a 93 00 15 00 00 00 00 29 45 55 00
]
tagId=35, length=16, value=[ 00 01 00 03 6b 22 00 00 00 00 00 00 00 00 00 00
]
tagId=35, length=16, value=[ 00 01 00 02 35 d4 00 00 00 00 00 00 00 00 00 00
]
tagId=35, length=16, value=[ 00 04 00 0b 80 84 00 3d 00 00 00 00 47 dd b4 80
]
tagId=35, length=16, value=[ 00 04 00 0b 80 84 00 24 00 00 00 00 45 7c 9f 80
]
tagId=35, length=16, value=[ 00 04 00 0b 80 84 00 25 00 00 00 00 45 6a 2a 80
]
tagId=35, length=16, value=[  ]
tagId=35, length=16, value=[  ]
tagId=35, length=16, value=[  ]
tagId=48, length=4, value=[ 00 00 00 0b ]
tagId=64, length=4, value=[ 00 00 00 21 ]
tagId=65, length=1, value=[ 00 ]
tagId=80, length=2, value=[ 00 00 ]
tagId=81, length=2, value=[ 00 00 ]
tagId=144, length=2, value=[ 00 02 ]
tagId=4, length=1, value=[ 02 ]
tagId=128, length=32, value=[ 34 67 23 65 21 87 45 98 34 31 30 33 31 30
31 34 35 31 5f 90 30 38 30 35 38 30 34
]

I need each tagId with data to be on 1 line. As you can see tagid=35 is split into 2 lines and tagId 128 is split into 3 lines. Also, tagid=35 does not always have anything in the value, so I cant always append the nextline.

Im trying to find a way to search for a specific tagid, check if there is an end ] and if not, then append the nextline.

Since my script will be doing multiple other sed edits on the file output, this does not need to be output to a new file, instead it can just edit the original file.

I need the output look like the following:

Code:
tagId=6, length=8, value=[ 30 30 2d 30 36 ]
tagId=9, length=5, value=[ 35 30 31 ]
tagId=7, length=2, value=[ 00 8c ]
tagId=16, length=2, value=[ 00 00 ]
tagId=32, length=1, value=[ 19 ]
tagId=33, length=1, value=[ 62 ]
tagId=34, length=1, value=[ 15 ]
tagId=35, length=16, value=[ 00 01 00 03 55 ea 00 00 00 00 00 00 00 00 00 00]
tagId=35, length=16, value=[ 00 04 00 00 4a 93 00 15 00 00 00 00 29 45 55 00]
tagId=35, length=16, value=[ 00 01 00 03 6b 22 00 00 00 00 00 00 00 00 00 00]
tagId=35, length=16, value=[ 00 01 00 02 35 d4 00 00 00 00 00 00 00 00 00 00]
tagId=35, length=16, value=[ 00 04 00 0b 80 84 00 3d 00 00 00 00 47 dd b4 80]
tagId=35, length=16, value=[ 00 04 00 0b 80 84 00 24 00 00 00 00 45 7c 9f 80]
tagId=35, length=16, value=[ 00 04 00 0b 80 84 00 25 00 00 00 00 45 6a 2a 80]
tagId=35, length=16, value=[  ]
tagId=35, length=16, value=[  ]
tagId=35, length=16, value=[  ]
tagId=48, length=4, value=[ 00 00 00 0b ]
tagId=64, length=4, value=[ 00 00 00 21 ]
tagId=65, length=1, value=[ 00 ]
tagId=80, length=2, value=[ 00 00 ]
tagId=81, length=2, value=[ 00 00 ]
tagId=144, length=2, value=[ 00 02 ]
tagId=4, length=1, value=[ 02 ]
tagId=128, length=32, value=[ 34 67 23 65 21 87 45 98 34 31 30 33 31 30 31 34 35 31 5f 90 30 38 30 35 38 30 34]

Thanks for any help you can give.
Last edited by Scott; 03-30-2011 at 02:49 PM.. Reason: Code tags for code, quote tags for quotes
# 2  
Old 03-29-2011
Code:
sed '4,${:a;/]/!N;s/\n//;ta}' file

# 3  
Old 03-29-2011
Quote:
Originally Posted by yinyuemi
Code:
sed '4,${:a;/]/!N;s/\n//;ta}' file

Great idea, but little bug. Smilie
Code:
sed '5,${:a;/]/!N;s/\n//;ta}'  file

by awk:

Code:
awk 'NR<=4 {print;next}{printf $0 (/]/?RS:FS)}' infile

Last edited by rdcwayx; 03-29-2011 at 11:48 PM.. Reason: fixed, a little different with Chubler_XL's code.
This User Gave Thanks to rdcwayx For This Post:
Winsarc
# 4  
Old 03-29-2011
Quote:
Originally Posted by rdcwayx
Great idea, but little bug. Smilie
Code:
sed '5,${:a;/]/!N;s/\n//;ta}'  file

by awk:

Code:
awk 'NR<=4 {print;next}{printf $0 (/]/?RS:X)}' infile

Thanks for debugging the code.Smilie
This User Gave Thanks to yinyuemi For This Post:
Winsarc
# 5  
Old 03-29-2011
Both those solutions give this for the list line:

Code:
tagId=128, length=32, value=[ 34 67 23 65 21 87 45 98 34 31 30 33 31 3031 34 35 31 5f 90 30 38 30 35 38 30 34]

Fixes:
Code:
sed '5,${:a;/]/!N;s/\n/ /;ta}'  file
awk 'NR<=4 {print;next}{printf $0 (/]/?RS:" ")}' infile

These 2 Users Gave Thanks to Chubler_XL For This Post:
Winsarc yinyuemi
# 6  
Old 03-29-2011
Code:
$ ruby -ne 'print (/]$/) ? $_ : $_.chomp  if $.>4' file
tagId=6, length=8, value=[ 30 30 2d 30 36 ]
tagId=9, length=5, value=[ 35 30 31 ]
tagId=7, length=2, value=[ 00 8c ]
tagId=16, length=2, value=[ 00 00 ]
tagId=32, length=1, value=[ 19 ]
tagId=33, length=1, value=[ 62 ]
tagId=34, length=1, value=[ 15 ]
tagId=35, length=16, value=[ 00 01 00 03 55 ea 00 00 00 00 00 00 00 00 00 00]
tagId=35, length=16, value=[ 00 04 00 00 4a 93 00 15 00 00 00 00 29 45 55 00]
tagId=35, length=16, value=[ 00 01 00 03 6b 22 00 00 00 00 00 00 00 00 00 00]
tagId=35, length=16, value=[ 00 01 00 02 35 d4 00 00 00 00 00 00 00 00 00 00]
tagId=35, length=16, value=[ 00 04 00 0b 80 84 00 3d 00 00 00 00 47 dd b4 80]
tagId=35, length=16, value=[ 00 04 00 0b 80 84 00 24 00 00 00 00 45 7c 9f 80]
tagId=35, length=16, value=[ 00 04 00 0b 80 84 00 25 00 00 00 00 45 6a 2a 80]
tagId=35, length=16, value=[ ]
tagId=35, length=16, value=[ ]
tagId=35, length=16, value=[ ]
tagId=48, length=4, value=[ 00 00 00 0b ]
tagId=64, length=4, value=[ 00 00 00 21 ]
tagId=65, length=1, value=[ 00 ]
tagId=80, length=2, value=[ 00 00 ]
tagId=81, length=2, value=[ 00 00 ]
tagId=144, length=2, value=[ 00 02 ]
tagId=4, length=1, value=[ 02 ]
tagId=128, length=32, value=[ 34 67 23 65 21 87 45 98 34 31 30 33 31 3031 34 35 31 5f 90 30 38 30 35 38 30 34]

Last edited by kurumi; 03-30-2011 at 12:44 AM..
# 7  
Old 03-29-2011
Don't know ruby all that well, but I doubt that will work for the last 3 lines in the example and can't see how it would remove the first few lines as required (those without a [ in them).

Edit: OK looks like you've got it fixed up now
Last edited by Chubler_XL; 03-30-2011 at 02:21 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge lines with varying characters

Hi, I have a large set of data (firewall logs) that I'm trying to summarize. I've been able to write a script to consolidate the ports, now am looking to conslidate even further, based on IP. Source Destination Type Port 192.168.5.108 192.168.11.12 TCP 1, 2, 3, 4, 5, 15 192.168.5.109... (6 Replies)
Discussion started by: umang2382
6 Replies

2. Shell Programming and Scripting

ksh sed - Extract specific lines with mulitple occurance of interesting lines

Data file example I look for primary and * to isolate the interesting slot number. slot=`sed '/^primary$/,/\*/!d' filename | tail -1 | sed s'/*//' | awk '{print $1" "$2}'` Now I want to get the Touch line for only the associate slot number, in this case, because the asterisk... (2 Replies)
Discussion started by: popeye
2 Replies

3. Shell Programming and Scripting

Replacing specific characters using sed

Hi, I have a text file which is output from a server and it lists all the files in a specific volume. However, the volume name appears as volume_name:. I would like to replace this with \\volume_name\volume_name. This is not a problem in itself as I can use sed to globally look for the... (8 Replies)
Discussion started by: vnayak
8 Replies

4. Shell Programming and Scripting

How delete characters of specific line with sed?

Hi, I have a text file with some lines like this: /MEDIA/DISK1/23568742.MOV /MEDIA/DISK1/87456321.AVI /MEDIA/DISK2/PART1/45753131.AVI /IMPORT/44452.WAV ... I want to remove the last 12 characters in each line that it ends "AVI". Should look like this: /MEDIA/DISK1/23568742.MOV... (12 Replies)
Discussion started by: inaki
12 Replies

5. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum using sed, awk

Hi friends, This is sed & awk type question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers whenever i find it and produce an output file with the sum. For example ###start of input text file #### abc def ghi 1 2 3 4 kjld random... (3 Replies)
Discussion started by: kaaliakahn
3 Replies

6. Shell Programming and Scripting

sed replacing specific characters and control characters by escaping

sed -e "s// /g" old.txt > new.txt While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies

7. Shell Programming and Scripting

Merge two lines using sed

Hi, I am trying to merge two lines, first line starts with a particular pattern and second line ends with a particular pattern in a file. Something like: First line starts with say ABC Second line ends with say XYZ After a merge, the line should become ABC.......XYZ I tried... (14 Replies)
Discussion started by: Sunny Arora
14 Replies

8. UNIX Desktop Questions & Answers

grep lines with two specific characters somewhere in the line

I'm having trouble with extracting certain lines from a file based on whether they have all the required fields. Original file: snt:594:Sam N This bpt:2342:Bob P That lr:123 wrp:23:Whoever Person cor:794 Desired output: snt:594:Sam N This bpt:2342:Bob P That wrp:23:Whoever Person ... (3 Replies)
Discussion started by: Chthonic
3 Replies

9. Shell Programming and Scripting

Merge lines in Flat file based on first 5 characters

Hi I have the fixed width flat file having the following data 12345aaaaaaaaaabbbbbbbbbb 12365sssssssssscccccccccc 12365sssss 12367ddddddddddvvvvvvvvvv 12367 vvvvv Here the first column is length 5 second is length 10 third is length 10 if the second or third column exceeds... (3 Replies)
Discussion started by: Brado
3 Replies

10. Shell Programming and Scripting

Use sed to merge multiple lines

Hi all: I have a file in which the contents are as following: ... This is a test ONE TWO Hello, world! XXX YYY CCC test again three, four five six seven world AAA BBB QQQ test eight, nine world (3 Replies)
Discussion started by: xb88
3 Replies
Login or Register to Ask a Question