Visit The New, Modern Unix Linux Community


How to remove fields space and append next line to previous line.?


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers How to remove fields space and append next line to previous line.?
# 1  
How to remove fields space and append next line to previous line.?

Code:
awk 'BEGIN{FS = "«"}
NR == 1 {p = $0; next}
NF > 1 {print p; p = $0}
NF <= 1 {p = (p " " $0)}
END {print p}' input.txt > output.txt

This is what the input data file looks like with broken lines

Code:
Code:
29863       «890000000       «543209911           «CHNGOHG
«000000001   «055                  « 2014 04 24 00 00 00PM«
 
01 89765              «708364680             «736454                «CHNGDONG
«093737464« 2014 04 24 01 12 00PM«

Expected format after transformation should look like this

Code:
Code:
29863«890000000«543209911«CHNGOHG«000000001«055«2014 04 24 00 00 00PM«
 
89765«708364680«736454«CHNGDONG«093737464«2014 04 24 01 12 00PM«

Thanks,
cumeh1624

Last edited by Scrutinizer; 04-25-2014 at 08:55 AM.. Reason: CODE tags
# 2  
Are there empty lines between records? In that case try:
Code:
awk '{$1=$1 x}1' FS='[[:space:]]*«' OFS=« RS= ORS='\n\n' file

Otherwise, try something like:
Code:
awk '{while(NF<8 &&getline n){$0=$0n} $1=$1 x}1' FS='[[:space:]]*«' OFS=« file

--PS--
I read in another thread that you are using AIX . There is a bug in awk on AIX, which is why an empty string (by using the empty variable x) needs to be appended to $1...

Last edited by Scrutinizer; 04-25-2014 at 09:45 AM..
# 3  
Quote:
Originally Posted by cumeh1624
Code:
awk 'BEGIN{FS = "«"}
NR == 1 {p = $0; next}
NF > 1 {print p; p = $0}
NF <= 1 {p = (p " " $0)}
END {print p}' input.txt > output.txt

This is what the input data file looks like with broken lines

Code:
Code:
29863       «890000000       «543209911           «CHNGOHG
«000000001   «055                  « 2014 04 24 00 00 00PM«
 
01 89765              «708364680             «736454                «CHNGDONG
«093737464« 2014 04 24 01 12 00PM«

Expected format after transformation should look like this

Code:
Code:
29863«890000000«543209911«CHNGOHG«000000001«055«2014 04 24 00 00 00PM«
 
89765«708364680«736454«CHNGDONG«093737464«2014 04 24 01 12 00PM«

Thanks,
cumeh1624
I'm still lost. Please describe in English the transformations that you are trying to make.

It appears that you want to remove leading and trailing spaces in a field, but keep spaces in the middle of a field.

It appears that you want to join all sets of adjacent lines that contain any field separators. (Is it all adjacent input lines, or pairs of input lines that are to be joined?)

Will blank or empty lines separate each set of lines that you want to join?

It appears that you want to leave lines with no field separators unchanged. (In your example there is a blank line. Does this also apply to empty lines?)

I see no pattern that would explain why the 01<space> marked in red in input.txt was removed in output.txt. Please explain in English what the logic is for removing non-<space> characters from some fields!

If two lines to be joined don't have a field separator as the last character of the first line or as the first character of the second line, is anything supposed to be added to the line where the lines are joined? (Perhaps a <space> character?)
# 4  
I created a sed command to remove the field's space and use the previous code to join next lines to previous code when NF is less than one and that took care of.
Code:
sed 's/[[:space:]]\{2,\}\«/«/g' input.txt.out>input.txt.out_temp

awk 'BEGIN{FS = "«"}
NR == 1 {p = $0; next}
NF > 1 {print p; p = $0}
NF <= 1 {p = (p " " $0)}
END {print p}' input.txt.out_temp > output.txt

Thanks everyone that contributed into this.
Moderator's Comments:
Mod Comment Please use CODE tags to display code segments; NOT color and font tags!

Last edited by Don Cragun; 04-25-2014 at 09:02 PM.. Reason: Fix tags.
# 5  
Quote:
Originally Posted by cumeh1624
I created a sed command to remove the field's space and use the previous code to join next lines to previous code when NF is less than one and that took care of.
Code:
sed 's/[[:space:]]\{2,\}\«/«/g' input.txt.out>input.txt.out_temp

awk 'BEGIN{FS = "«"}
NR == 1 {p = $0; next}
NF > 1 {print p; p = $0}
NF <= 1 {p = (p " " $0)}
END {print p}' input.txt.out_temp > output.txt

Thanks everyone that contributed into this.
I'm very glad that you got something that works for you. When I try this code with the sample input you provided, I get:
Code:
29863«890000000«543209911«CHNGOHG
«000000001«055« 2014 04 24 00 00 00PM«  
01 89765«708364680«736454«CHNGDONG
«093737464« 2014 04 24 01 12 00PM«

(note the two trailing spaces on the 2nd line of output, the 01<space> at the start of the third line, the <space> before 2014 on the 2nd and 4th lines, and four lines of output containing your field delimiter). This doesn't seem to be at all like what you said you wanted in the first message in this thread:
Code:
29863«890000000«543209911«CHNGOHG«000000001«055«2014 04 24 00 00 00PM«
 
89765«708364680«736454«CHNGDONG«093737464«2014 04 24 01 12 00PM«

This User Gave Thanks to Don Cragun For This Post:
 

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #788
Difficulty: Medium
Steve Wozniak was expelled from the University of California, Berkeley in his first year for hacking the university's computer system.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove new line starting with a numeric value and append it to the previous line

Hi, i have a file with multiple entries. After some tests with sed i managed to get the file output as follows: lsn=X-LINK-IN0,apc=661:0,state=avail,avail/links=1/1, 00,2110597,2094790,0,81,529,75649011,56435363, lsn=TM1ITP1-AM1ITP1-LS,apc=500:0,state=avail,avail/links=1/1,... (5 Replies)
Discussion started by: nms
5 Replies

2. Shell Programming and Scripting

Issue while append to previous line

Hi, I have data as below. 36578019,005-923887317,UNMDL,20151230,2C3CCAAG4GH135448,L,TX,20160108,62,"030916 PPT TX AFF RPRT VALID AFF IN PDP WLL FWD TO RYAN ON 031116 CB1619 ",, 36580219,611-923785453,FC,20151209,ZACCJABT9FPC19274,L,TX,20160108,83,,,... (4 Replies)
Discussion started by: JSKOBS
4 Replies

3. Shell Programming and Scripting

Append next line to previous lines when NF is less than 0

Hi All, This is very urgent, I've a data file with 1.7 millions rows in the file and the delimiter is cedilla and I need to format the data in such a way that if the NF in the next row is less than 1, it will append that value to previous line. Any help will be appricated. Thanks,... (17 Replies)
Discussion started by: cumeh1624
17 Replies

4. Shell Programming and Scripting

Remove previous line if next & previous lines have same 4th character.

I want to remove commands having no output. In below text file. bash-3.2$ cat abc_do_it.txt grpg10so>show trunk group all status grpg11so>show trunk group all status grpg12so>show trunk group all status GCPKNYAIGT73IMO 1440 1345 0 0 94 0 0 INSERVICE 93% 0%... (4 Replies)
Discussion started by: Raza Ali
4 Replies

5. Shell Programming and Scripting

Append next line to previous line when one pattern not found

Hi, I need help for below scenario.I have a flat file which is having records seperated by delimiters which will represent each record for oracle table.My Control file will consider each line as one record for that table. Some of the lines are aligned in two/three lines so that records are... (4 Replies)
Discussion started by: kannansr621
4 Replies

6. Shell Programming and Scripting

Append specific lines to a previous line based on sequential search criteria

I'll try explain this as best I can. Let me know if it is not clear. I have large text files that contain data as such: 143593502 09-08-20 09:02:13 xxxxxxxxxxx xxxxxxxxxxx 09-08-20 09:02:11 N line 1 test line 2 test line 3 test 143593503 09-08-20 09:02:13... (3 Replies)
Discussion started by: jesse
3 Replies

7. Shell Programming and Scripting

Append each line to next previous line in a file

Hi all, Please help me in providing sample code to append the following 4 lines in one row. Input : A1/EXT "BAPBSC10/07B/00" 523 090530 0115 RXOCF-430 HY1711 1 EXTERNAL ALARM DOOR ALARM Output should be : A1/EXT "BAPBSC10/07B/00" 523 090530 0115 ... (8 Replies)
Discussion started by: sudhakaryadav
8 Replies

8. Shell Programming and Scripting

SED or AWK "append line to the previous line"

Hi, How can I remove the line beak in the following case if the line begin with the special char ď;Ē? TEXT Text;text ;text Text;text;text I want to convert the text to: Text;text;text Text;text;text I have already tried to use... (31 Replies)
Discussion started by: research3
31 Replies

9. UNIX for Advanced & Expert Users

append the line with the previous if it not start with 1=

How to append the line with the previous if it not start with 1=. 1=ttt, 2=xxxxxx, 3=4545 44545, 4=66666, 1=ttt, 2=xxxxxx, 3=34434 3545, 4=66666, 5=ffffff 6=uuuuuuu, 7=ooooooo 1=ttt, 2=xxxxxx, 3=311343545, 4=66666 1=ttt, 2=xxxxxx, 5=XAXAXA, 7=FDFD (3 Replies)
Discussion started by: palsevlohit_123
3 Replies

10. Shell Programming and Scripting

Append line that does not contain pipe to it previous line

Hi All, I have a file which contains data as below When we see no pipe character in the line. append those lines to the previous line with pipe character till we get the next line with pipe character with ~(concat with ~) Input file looks like: 1080530944|001|john.l.bonner|Acknowledge|CN... (11 Replies)
Discussion started by: ainuddin
11 Replies

Featured Tech Videos