concatenate lines using shell scripting


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting concatenate lines using shell scripting
# 1  
Old 09-06-2009
concatenate lines using shell scripting

i have a mega file in this format:

a,
b,
c,
d,
=
a2,
b2,
c2,
d2,
=
a3,
b3
=

i want to combine lines until = meet. the result should be :
a,b,c,d,
a2,b2,c3,d2
a3,b3

need your help. thanks
# 2  
Old 09-06-2009
You can of course do this via shell scripting, but using sed would be a faster and by far clearer way to do it.

Why do you want to do it with a shell script?

bakunin
# 3  
Old 09-06-2009
sorry, i didn't mean that. sed or awk are all good.

---------- Post updated at 01:11 PM ---------- Previous update was at 12:58 PM ----------

would you please show me how to do it in sed or awk? thx
# 4  
Old 09-06-2009
Code:
awk '/=/{$0="\n"}1' ORS="" file

# 5  
Old 09-06-2009
perhaps.....bur certainly better....

Code:
awk '{if($0 !~ /=/) ORS=""}{if($0 ~/=/) ORS="\n"} {print}' | sed -e 's/=//g' -e 's/,$//g' file

# 6  
Old 09-06-2009
in sed it is easy: we have two types of lines, the ones reading "=" and the others. When we encounter a "="-line, we want to print out what we have so far, minus the newlines. If we encounter one of the other lines we want to store its contents until we encounter a "="-line.

sed has a so-called "hold space", think of it as a variable, where you can store things until you need them. We append everything to this hold space until we encounter a "="-line, then we recall the hold space, filter out all embedded newlines and print it, then start over.

In the following script i have put in comment for your understanding, remove them, because sed doesn't allow inline commenting in scripts. Furthermore, you can put the whole script on one line, replacing linefeeds with semicolons:

Code:
sed -n '/^=/ {                   # if a line starts with "="
          s/.*//                 # delete this lines content
          x                      # exchange the pattern space (empty) and the hold space
          s/\n//g                # delete newlines
          p                      # then print what you have
     }
     /^=/ ! {                    # if a line doesn't start with "="
          H                      # append it to the hold space
     }' /your/file > newfile

for short:

Code:
sed -n '/^=/{s/.*//;x;s/\n//gp};/^=/!{H}' /your/file > newfile

I hope this helps.

bakunin
# 7  
Old 09-06-2009
Quote:
Originally Posted by protocomm
perhaps.....bur certainly better....
Did you check the OP requirements Smilie , certainly not.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete all CONSECUTIVE text lines from file shell scripting

Hi I have a text file like below. THe content of the text will vary. Entire text file have four consecutive lines followed with blank line. I want to delete the occurrence of the two consicutive lines in the text file. I don't have pattern to match and delete. Just i need to delete all... (5 Replies)
Discussion started by: RJSKR28
5 Replies

2. UNIX for Dummies Questions & Answers

Concatenate three lines into one

Hi. I'm new to this forum. I am attempting to parse an Audit Log from Cognos/TM1, selecting only Event IDs of "client" which are found on the "start-tag" record. These Logs are in a pseudo-XML format but not a true XML format. I want to FTP an Audit Log File from the Cognos server to our UNIX... (7 Replies)
Discussion started by: FredAtArrow
7 Replies

3. UNIX for Dummies Questions & Answers

Shell script to read lines in a text file and filter user data Shell Programming and Scripting

sxsaaas (3 Replies)
Discussion started by: VikrantD
3 Replies

4. UNIX for Advanced & Expert Users

Concatenate lines in file shell script

Hi colleagues, I have a file in this format. "/cccc/pppp/dddd/ggg/prueba.txt". ERROR" THE error bbbbbbbbbb finish rows. "/kkkk/mmmm/hhhh/jjj/ejemplo.txt". ERROR This is other error rows.I need my file in this format. "/cccc/pppp/dddd/ggg/prueba.txt". ERROR" THE error bbbbbbbbbb finish rows.... (3 Replies)
Discussion started by: systemoper
3 Replies

5. Programming

Concatenate two lines in a fIle

Hi All, Can any one help me in finding the solution for concatenating two or more lines in a file and writing them to a temporary file. for Example: He is a wise student. So he got first rank. This is in a file i want the output as He is a wise student so he got first rank. into a file... (3 Replies)
Discussion started by: uday.sena.m
3 Replies

6. Shell Programming and Scripting

searching multiple lines and replacing in shell scripting

Hi, I have a file with below contents, ssenthil = rw anilkg = rw I want to search for "ssenthil" and need to delete line 1 and 2 , if the third line starts with "" respectively and blank line immediately and third line starts with " anilkg = rw Please help me . Great day... (5 Replies)
Discussion started by: anil8103
5 Replies

7. Shell Programming and Scripting

Concatenate lines between lines starting with a specific pattern

Hi, I have a file such as: --- >contig00001 length=35524 numreads=2944 gACGCCGCGCGCCGCGGCCAGGGCTGGCCCA CAGGCCGCGCGGCGTCGGCTGGCTGAG >contig00002 length=4242 numreads=43423 ATGCCGAAGGTCCGCCTGGGGCTGG CGCCGGGAGCATGTAGCG --- I would like to concatenate the lines not starting with ">"... (9 Replies)
Discussion started by: s052866
9 Replies

8. Shell Programming and Scripting

concatenate lines in pairs

Hi, I have a text file with the following contents /C=IT/O=INFN/OU=Personal Certificate/L=Napoli/CN=Some guy /C=IT/O=INFN/CN=INFN CA /O=Grid/O=NorduGrid/OU=uninett.no/CN=Another guy /O=Grid/O=NorduGrid/CN=NorduGrid Certification Authority /C=TW/O=AP/OU=GRID/CN=Someone else... (5 Replies)
Discussion started by: kerl
5 Replies

9. Shell Programming and Scripting

Concatenate two lines in one

Does anyone know a way I can fix my file below where it has lines that are cut? Just like the (notice the space before ms,c=PH).. dn: cn=english,ou=Messaging,ou=lang,o=subject,t=j ms,c=PH which should be dn: cn=english,ou=Messaging,ou=lang,o=subject,t=jms,c=PH The whole text file... (5 Replies)
Discussion started by: Orbix
5 Replies

10. UNIX Desktop Questions & Answers

How to concatenate consecutive lines

I have a few lines like -- feature 1, subfeat 0, type 3, subtype 1, value 0, -- feature 1, subfeat 0, type 1, subtype 1, value 0, I would like to concatenate the... (1 Reply)
Discussion started by: shivi707
1 Replies
Login or Register to Ask a Question