Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Break one line to many lines using awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Break one line to many lines using awk
# 1  
Old 05-29-2009
Break one line to many lines using awk
Break one line to many lines using awk

The below code works but i want to implement without combining field 2 and 3 and then splitting i would like to do this in one command
instead of writing multiple commands and creating multiple lines.

Code:
nawk -F"|" '{print $1,$2SUBSEP$3}' OFS="|"  file > file1
nawk -F"|" '{gsub(",", " ",$2);gsub(SUBSEP, " ",$2);print}' OFS="|" file1 > file2
nawk -F"|" '{if ($2 ~ " ") {split ($2,a," "); for (i in a) {print $1,a[i]}}  else print}' OFS="|"  file2 > file3

file:
Quote:
abc|IND INDX DXO|IND IJK VERMONT
def|AMC,LOWES|IND TMC
ghi|IND TMC BMW|KJL,TNT,CHK,OPEN

output:
Quote:
abc|INDX
abc|DXO
abc|IND
abc|IJK
abc|VERMONT
abc|IND
def|LOWES
def|IND
def|TMC
def|AMC
ghi|TMC
ghi|BMW
ghi|KJL
ghi|TNT
ghi|CHK
ghi|OPEN
ghi|IND


Help in completing as this is not working:

Code:
nawk -F"|" '{for (i=2;i<=NF;i++) {if ($1 ~ " ") {RS=[, ] print}}' file

# 2  
Old 05-29-2009
if you have Python
Code:
#!/usr/bin/env python

for line in open("file2"):
    line=line.strip().split("|")
    for item in line[1:]:
        if "," in item:
            item=item.split(",")
        else:
            item=item.split()
        for i in item:
            print "%s|%s" %(line[0],i)

output
Code:
# ./test.py
abc|IND
abc|INDX
abc|DXO
abc|IND
abc|IJK
abc|VERMONT
def|AMC
def|LOWES
def|IND
def|TMC
ghi|IND
ghi|TMC
ghi|BMW
ghi|KJL
ghi|TNT
ghi|CHK
ghi|OPEN

# 3  
Old 05-30-2009
this will do i guess
Code:
awk -F"[|, ]" '{for(i=2;i<=NF;i++){print $1"|"$i}}' filename

# 4  
Old 05-30-2009
Code:
awk '{gsub(/[|,]/," ");for (i=2;i<=NF;i++)print $1,$i}' OFS="|" filename

If you want to get only the uniq combination, pass it to sort -u

Code:
awk '{gsub(/[|,]/," ");for (i=2;i<=NF;i++)print $1,$i}' OFS="|" filename | sort -u

-Devaraj Takhellambam
# 5  
Old 05-30-2009
Quote:
Originally Posted by devtakh
Code:
awk '{gsub(/[|,]/," ");for (i=2;i<=NF;i++)print $1,$i}' OFS="|" filename

If you want to get only the uniq combination, pass it to sort -u

Code:
awk '{gsub(/[|,]/," ");for (i=2;i<=NF;i++)print $1,$i}' OFS="|" filename | sort -u

-Devaraj Takhellambam
Thanks devtakh
The above solution works for the particular file i posted.
But in case i have say file with 10 fields and i want to break line on basis of last 2 columns as we did for the earlier file then this solution doesnt work because if the first 8 columns if they have a space or comma in them then they will be considered as another field.



Would appreciate if you can make the code versatile for this kind of file as well
# 6  
Old 05-30-2009
Did you try the other solutions suggested in this thread?
# 7  
Old 05-30-2009
Quote:
Originally Posted by Franklin52
Did you try the other solutions suggested in this thread?
Code:
awk -F"[|, ]" '{for(i=2;i<=NF;i++){print $1"|"$i}}' filename

The problem with solution is my input file is pipe delimited and can have comma "," as data in this.
If my first 8 columns has comma "," in it this will consider it as a different field and breaks it into another line
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. UNIX for Beginners Questions & Answers

awk script for pattern match and line break

Hi, I have input which reads like 9089.00 ----- kl jkjjljk lkkk; (909099) 9097.00 ----- HGJJHHJ jcxkjlkjvhvlk jhdkjksdfkhfskd 898.00 ----- HHHH I am trying to do something like this - As soon as I found pattern match "XYZ.00-----" it will insert a line break to the input and will go to... (3 Replies)
Discussion started by: Indra2011
3 Replies

3. Shell Programming and Scripting

Break line content into multiple lines using delimiter

I need to break the line after every 3rd semi colon(;) using Unix shell scripting Input.txt ABC;DEF;JHY;LKU;QWE;BVF;RGHY; Output.txt ABC;DEF;JHY; LKU;QWE;BVF; RGHY; (1 Reply)
Discussion started by: meet_calramz
1 Replies

4. Shell Programming and Scripting

Break lines that appear together

Hello to all May you help me with a sed or awk script to fix the text below please. I have a file where all lines should have 67 characters but from time to time 2 lines appear together like below. * First column always has 15 characters * Second column always has 32 characters * 3rd,... (5 Replies)
Discussion started by: Ophiuchus
5 Replies

5. Shell Programming and Scripting

Enumerate lines until each line break using awk

Hi, I have the following data: This this DT 0.99955 0 4 is be VBZ 1 5 7 sentence sentence NN 0.916667 8 16 one one NN 0.545078 17 20 . . Fp 1 20 21 This this DT 0.99955 22 26 is be VBZ 1 27 29 the the DT 1 30 33 second 2 JJ 0.930556 34 40 sentence sentence NN 0.916667 41 49... (1 Reply)
Discussion started by: owwow14
1 Replies

6. Shell Programming and Scripting

How to calculate mean in AWK? line by line several files, thousands of lines

I'm kinda stuck on this one, I have 7 files with 30.000 lines/file like this 050 0.023 0.504336 050 0.024 0.529521 050 0.025 0.538908 050 0.026 0.537035 I want to find the mean line by line of the third column from the files named like this: Stat-f-1.dat .... Stat-f-7.dat Stat-s-1.dat... (8 Replies)
Discussion started by: AriasFco
8 Replies

7. Shell Programming and Scripting

BASH: Break line, read, break again, read again...

...when the lines use both a colon and commas to separate the parts you want read as information. The first version of this script used cut and other non-Bash-builtins, frequently, which made it nice and zippy with little more than average processor load in GNOME Terminal but, predictably, slow... (2 Replies)
Discussion started by: SilversleevesX
2 Replies

8. Shell Programming and Scripting

Awk to Break lines to multiple lines.

Input File: nawk -F "|" '{ for(i=1;i<=NF;i++) { if (i == 2) {gsub(",","#",$i);z=split($i,a,"")} else if (i == 3) {gsub(",","#",$i);z=split($i,b,"")} } if(z > 0) for(i=1;i<=z;i++) print $1,a,"Test"; if(w > 0) for(j=1;j<=w;j++) ... (1 Reply)
Discussion started by: pinnacle
1 Replies

9. Shell Programming and Scripting

Break lines up into single lines after each space in every line

It sounds a bit confusing but what I have is a text file like the example below (without the Line1, Line2, Line3 etc. of course) and I want to move every group of characters into a new line after each space. Example of text file; line1 .digg-widget-theme2 ul { background: rgb(0, 0, 0) none... (7 Replies)
Discussion started by: lewk
7 Replies

10. Shell Programming and Scripting

join lines on line break in files

i had a file where lines appear to be broken when they shouldn't eg Line 1. kerl abc sdskd sdsjkdlsd sdsdksd \ Line 2. ksdkks sdnjs djsdjsd i can do a shift join to combine the lines but i there are plenty of files with this issue Line 1. kerl abc sdskd sdsjkdlsd sdsdksd ksdkks sdnjs... (6 Replies)
Discussion started by: mad_man12
6 Replies
Login or Register to Ask a Question