Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need to remove first character from every third line (or revised nawk).

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need to remove first character from every third line (or revised nawk).
# 1  
Old 10-06-2008
Need to remove first character from every third line (or revised nawk).
Here's the data I'm starting with (example output of two combined queries, filesize: 284k)
Code:
3000877|555-55-1111|2|7/30/2008|TEST|P.O. BOX 1111|PALM DESERT|CA|92211||5555555555||||||||48|||1||1|3|||2|||||||||||||1|3||2|2
||||2||9||3|1|2|2|2|1|3|0||5|2|||||||||88||3|2||3|2||||2|1|||6|5/31/2008|2||||9|AD|||42|42||||||Y|555-55-1111|SMITH|JOHN|||12
/23/1960|2|WH|||||||||Y
3000178|555-55-1112|2|7/23/2008|TEST|P.O. BOX 1112|TEMECULA|CA|92591||5555555555||||||||33|||1||1|3|||2|||||||||||||3|3||2|
2||||2||9||2|1|2|2|2|2|3|0||5|2|||||||||88|||2||3|2||||2|9|||||2||||9|A|||42|42||||||Y|555-55-1112|SMITH|JACK|||12/8/1975|2|BL|
||||||||Y
3000317|555-55-1113|2|7/29/2008|TEST|P.O. BOX 1113|MORENO VALLEY|CA|92556||5555555555||||||||55|||1||4|1|||2|||||||||||||1|3||2|2||||
2||9||1|0|2|2|2|2|3|0||5|2|||||||||88|||2||3|2||||2|9|||||2||||9|A|||42|42||||||Y|555-55-1113|SMITH|JOE|||11/28/1953|2|AO|||||||
||Y

Then I run the following nawk script as a \n is needed after the #103 and #120 entries. (nawk -f scriptname > filename1)
Code:
[
BEGIN {
   FS="|"
       }
 
{
  OFS="|"
}
 
{
print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,
$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,
$41,$42,$43,$44,$45,$46,$47,$48,$49,$50,$51,$52,$53,$54,$55,$56,$57,$58,$59,$60,
$61,$62,$63,$64,$65,$66,$67,$68,$69,$70,$71,$72,$73,$74,$75,$76,$77,$78,$79,$80,
$81,$82,$83,$84,$85,$86,$87,$88,$89,$90,$91,$92,$93,$94,$95,$96,$97,$98,$99,
$100,$101,$102,$103,"\n",
$104,$105,$106,$107,$108,$109,$110,$111,$112,$113,$114,$115,$116,
$117,$118,$119,$120,"\n"
}
END {}

The nawk script output (i.e. filename1) is:
Code:
3000877|555-55-1111|2|7/30/2008|TEST|P.O. BOX 1111|PALM DESERT|CA|92211||5555555555||||||||48|||1||1|3|||2|||||||||||||1|3||2|2
||||2||9||3|1|2|2|2|1|3|0||5|2|||||||||88||3|2||3|2||||2|1|||6|5/31/2008|2||||9|AD|||42|42||||||Y|
|555-55-1111|SMITH|JOHN|||12/23/1960|2|WH|||||||||Y|

3000178|555-55-1112|2|7/23/2008|TEST|P.O. BOX 1112|TEMECULA|CA|92591||5555555555||||||||33|||1||1|3|||2|||||||||||||3|3||2|
2||||2||9||2|1|2|2|2|2|3|0||5|2|||||||||88|||2||3|2||||2|9|||||2||||9|A|||42|42||||||Y|
|555-55-1112|SMITH|JACK|||12/8/1975|2|BL|||||||||Y|

3000317|555-55-1113|2|7/29/2008|TEST|P.O. BOX 1112|MORENO VALLEY|CA|92556||555555555||||||||55|||1||4|1|||2|||||||||||||1|3||2|2||||
2||9||1|0|2|2|2|2|3|0||5|2|||||||||88|||2||3|2||||2|9|||||2||||9|A|||42|42||||||Y|
|555-55-1113|SMITH|JOE|||11/28/1953|2|AO|||||||||Y|

The problem is on the third line of each entry, the nawk script has inserted an additional first character pipe (i.e. |) and it's causing a great deal of havoc given my import requirements. Now I've tried various sed methods of removal based on the first character of every 3rd line throughout the entire file, sadly without success. Lastly, I've revised the nawk script in attempt to exclude the additional pipe character, all without success.
Code:
|555-55-1113|SMITH|JOE|||11/28/1953|2|AO|||||||||Y|

It needs to be:
Code:
555-55-1113|SMITH|JOE|||11/28/1953|2|AO|||||||||Y|

Review, thoughts and suggestions are truly welcomed.

Thanks!
# 2  
Old 10-06-2008
try using "\b" after "\n"
# 3  
Old 10-06-2008
Quote:
Originally Posted by vidyadhar85
try using "\b" after "\n"
Here's the output:
Code:
3000877|555-55-1111|2|7/30/2008|TEST|P.O. BOX 1111|PALM DESERT|CA|92211||555555555||||||||48|||1||1|3|||2|||||||||||||1|3||2|2
||||2||9||3|1|2|2|2|1|3|0||5|2|||||||||88||3|2||3|2||||2|1|||6|5/31/2008|2||||9|AD|||42|42||||||Y|
^H|555-55-1111|SMITH|JOHN|||12/23/1960|2|WH|||||||||Y|
^H

# 4  
Old 10-06-2008
Code:
BEGIN { FS = OFS = "|" }
{
 print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,
     $16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,
     $29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,
     $42,$43,$44,$45,$46,$47,$48,$49,$50,$51,$52,$53,$54,
     $55,$56,$57,$58,$59,$60,$61,$62,$63,$64,$65,$66,$67,
     $68,$69,$70,$71,$72,$73,$74,$75,$76,$77,$78,$79,$80,
     $81,$82,$83,$84,$85,$86,$87,$88,$89,$90,$91,$92,$93,
     $94,$95,$96,$97,$98,$99,$100,$101,$102,$103
 print ""
 print $104,$105,$106,$107,$108,$109,$110,$111,$112,$113,
       $114,$115,$116,$117,$118,$119,$120
 print ""
}

# 5  
Old 10-06-2008
Quote:
Originally Posted by cfajohnson
Code:
BEGIN { FS = OFS = "|" }
{
 print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,
     $16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,
     $29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,
     $42,$43,$44,$45,$46,$47,$48,$49,$50,$51,$52,$53,$54,
     $55,$56,$57,$58,$59,$60,$61,$62,$63,$64,$65,$66,$67,
     $68,$69,$70,$71,$72,$73,$74,$75,$76,$77,$78,$79,$80,
     $81,$82,$83,$84,$85,$86,$87,$88,$89,$90,$91,$92,$93,
     $94,$95,$96,$97,$98,$99,$100,$101,$102,$103
 print ""
 print $104,$105,$106,$107,$108,$109,$110,$111,$112,$113,
       $114,$115,$116,$117,$118,$119,$120
 print ""
}

This did the trick:
Code:
BEGIN { FS = OFS = "|" }
{
 print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,
     $16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,
     $29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,
     $42,$43,$44,$45,$46,$47,$48,$49,$50,$51,$52,$53,$54,
     $55,$56,$57,$58,$59,$60,$61,$62,$63,$64,$65,$66,$67,
     $68,$69,$70,$71,$72,$73,$74,$75,$76,$77,$78,$79,$80,
     $81,$82,$83,$84,$85,$86,$87,$88,$89,$90,$91,$92,$93,
     $94,$95,$96,$97,$98,$99,$100,$101,$102,$103
 print $104,$105,$106,$107,$108,$109,$110,$111,$112,$113,
       $114,$115,$116,$117,$118,$119,$120
 print ""
}

The first print="" statement was removed as it was creating an additional/unnecessary linefeed (i.e. separator).

Thank you!
# 6  
Old 10-07-2008
try this....tell me if this works fine.....

awk '{
if(NR%3!=0)
{
print
}
else
{
for(i=1;i<=NF;i++)
{
if(i==1)
{
printf("%s ",substr($1,2,length($1)-1))
}
else
{
printf("%s ",$i)
}
}
printf"\n"
}
}' file
Last edited by vijay_0209; 10-07-2008 at 01:09 AM..
# 7  
Old 10-07-2008
after ur old o/p just give pipe sign and sed 's/^|//g'
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove character from a column in each line

Hi, I am a newbie to shell scripting (.sh). Please guide me on how to do the below issue. My input file has below data. I want to remove $ sysmbol from the fourth column of each line. (ie, between 4th and 5th pipe symbol) ABC25160|51497|06/02/2010|$32,192.07|MARK|$100|A... (3 Replies)
Discussion started by: rsreejithmenon
3 Replies

2. UNIX for Dummies Questions & Answers

How to remove $ or new line character in a file?

Hi All, Could any one suggest how to remove $ symbol in a text file when i am opening in vi editor. Scenario; For example iam having a file name aaa.txt the data inside the file is like sample name when i am opening in vi editor The same file resembles like below when i am... (1 Reply)
Discussion started by: Chandru_Raj
1 Replies

3. UNIX for Dummies Questions & Answers

Remove last character in each line

Hi guys, Does anyone know how to remove the last character in each of the line? This is what I have: ABCDE.1 GLSJD.2 HIJPL.2 HKAGB.3 IUBWQ.1 What I want (remove the dot and number): ABCDE GLSJD HIJPL HKAGB IUBWQ I tried to use this: sed 's/.*//' But I'm not sure if that is... (3 Replies)
Discussion started by: narachaid
3 Replies

4. Shell Programming and Scripting

How to remove , if first character on line

Hi, I have a file with lines such as the below. I want to remove the comma only if it is the first character on a line. I can't work out how to do this using sed. *ELSET, ELSET=WHEEL_TD2 63, 64, 65, 72, 82, 88, 89, 92, 120, 121, 152, 181, 190, 221, 252, 259 , 260, 282, 283, 285, 286,... (2 Replies)
Discussion started by: carlr
2 Replies

5. Shell Programming and Scripting

remove line feeds followed by character

Hi everyone, I'm very new to using sed, run through some tutorials and everything but I've hit a problem that I'm unable to solve by myself. I need to remove all linefeeds that are followed by a particular character (in this case a semicolon). So basically, all lines starting with a semicolon... (5 Replies)
Discussion started by: fluffdasheep
5 Replies

6. HP-UX

How to remove new line character and append new line character in a file?

Hi Experts, I have data coming in 4 columns and there are new line characters \n in between the data. I need to remove the new line characters in the middle of the row and keep the \n character at the end of the line. File is comma (,) seperated. Eg: ID,Client ,SNo,Rank 37,Airtel \n... (8 Replies)
Discussion started by: sasikari
8 Replies

7. UNIX for Advanced & Expert Users

To remove new line character

Hi, I am facing one interesting problem : I have a file which contains data like this 459,|1998-11-047|a |b |c \n efg | d|e | \n 459,|1998-11-047|a \n c|b |c \n efg | d|e | \n Basically what I have to do is , I have to remove all \n which is coming ( enclosed ) in between... (7 Replies)
Discussion started by: shihabvk
7 Replies

8. Shell Programming and Scripting

sed: remove first character from particular line

Hello Experts, I have a file "tt.txt" which is like: #a1=a2 b1=b2 #c1=c2 I need to remove the pound (#) sign from a particular line. In this case let us assume it's 3rd line : "#c1=c2" I can do it through: sed "s/#c1=c2/c1=c2/" tt.txtbut it is possible that I may not know the value... (6 Replies)
Discussion started by: hkansal
6 Replies

9. UNIX for Dummies Questions & Answers

Trying to remove single character from a line

Here is a sample code grep '903' -i user.txt | tail -2 | awk '{print $2}' | sed 's/B//g' the input file has data as such 903-xxx-xxxxB 903-xxx-xxxxB It is a dialer file i want to remove the "B" any help thanks (5 Replies)
Discussion started by: Iz3k34l
5 Replies

10. Shell Programming and Scripting

Remove Last Character of Line

Hi, I need to put the single line contents of a file into a variable, but remove the last character, for example the file would have this sort of contents: 2;4;3;10;67;54;96; And I want the variable to be: 2;4;3;10;67;54;96 (notice the last ";" has gone). Unfortunately I can't just... (4 Replies)
Discussion started by: danhodges99
4 Replies
Login or Register to Ask a Question