Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Trying to use sed to remove last FF from file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Trying to use sed to remove last FF from file
# 1  
Old 01-23-2014
Trying to use sed to remove last FF from file
I have a script that I am trying to apply on files that have form feeds between pages but I am trying to replace the last form feed, with carriage return so that when I convert it to a PDF file it won't generate a blank page.

My script looks like this
Code:
sed '$ s/\^L/\^M/' invoice.txt >invoice.dat

My file looks like this, keep in mind some files have more than one page so I have to only remove the last form feed.

Code:
^M^M^[[51t^M^M       ABC COMPANY                             ABC SHIPPING COMPAN
Y            ^M
       123 BAKER STREET                        123 DOCK STREET                 ^
M
       ANYTOWN USA 99999                       ANYTOWN USA 08277               ^
M
                                                                               ^
M
^M
                                                                        PAGE   1
^M
^M
  155 498  MS                          07:43 AM     J  08/01/13 01/23/14*C428230
^M
^M
   1       PLA T237          WHITE PRIMER            10.13    6.58     6.58 T
^M
   1       PS  STRAINERS     PAINT SUPPLYS
^M
   1       SEM 39683         GRAY SELF ETC                   20.57    20.57
^M
   1       SEM 39693         ETCH PRIMER                     20.57    20.57
^M
   1       NAS QUART 421-19  SELECTPRIME 2                   22.75    22.75 T
^M
   1       NAS 1/2 PT 483-87 SELECTPRIME A                   15.60    15.60 T
^M
   1       NAS QUART 441-21  MED FB REDUCE                   11.44    11.44 T
^M
   1       NAS 1/2 PT 483-11 FUL-CRYL CAT.                   33.86    33.86 T
^M
   2       PS  MC32          QT MIX CUP                       0.50     1.00 T
^M
   1       DUP M64           DUP. TAC RAG                     1.13     1.13 T
^M
   2       MMM 7447          SCOTCHBRITE                      1.25     2.50 T
^M
   1       PS  UG18          3/4" MASKING                     1.43     1.43 T
^M
   1       PS  PINT CANS     PAINT SUPPLYS
^M
   1       BAT 10U1R         MOWER BATTERY                   43.78    43.78
^M
   1       GAT 6984          FRACT.HP BELT           63.08   37.77    37.77
^M
   1       GAT A100          HEA.DUTY BELT           34.65   20.75    20.75
^M
   1       WIX 42299         WIX AIR FILT.           15.26    9.87     9.87
^M
   1       GAT A100          HEA.DUTY BELT           34.65   20.75    20.75
^M
   1       WIX 51521         WIX OIL FILT.           14.50    8.91     8.91
^M
^M
^M
                                                                     279.26 ^M
                                                                            ^M
                                                                       6.74 ^M
                                                                            ^M
                                                                            ^M
                                                                     286.00 ^M^L
^M^M^[[66t^M

My results are as follows in which the last line is completely dropped.

Code:
       ABC COMPANY                             ABC SHIPPING COMPANY
       123 BAKER STREET                        123 DOCK STREET
       ANYTOWN USA 99999                       ANYTOWN USA 08277
 
                                                                        PAGE   1
   155 498  MS                          07:43 AM     J  08/01/13 01/23/14*C428230
    1       PLA T237          WHITE PRIMER            10.13    6.58     6.58 T
   1       PS  STRAINERS     PAINT SUPPLYS
   1       SEM 39683         GRAY SELF ETC                   20.57    20.57
   1       SEM 39693         ETCH PRIMER                     20.57    20.57
   1       NAS QUART 421-19  SELECTPRIME 2                   22.75    22.75 T
   1       NAS 1/2 PT 483-87 SELECTPRIME A                   15.60    15.60 T
   1       NAS QUART 441-21  MED FB REDUCE                   11.44    11.44 T
   1       NAS 1/2 PT 483-11 FUL-CRYL CAT.                   33.86    33.86 T
   2       PS  MC32          QT MIX CUP                       0.50     1.00 T
   1       DUP M64           DUP. TAC RAG                     1.13     1.13 T
   2       MMM 7447          SCOTCHBRITE                      1.25     2.50 T
   1       PS  UG18          3/4" MASKING                     1.43     1.43 T
   1       PS  PINT CANS     PAINT SUPPLYS
   1       BAT 10U1R         MOWER BATTERY                   43.78    43.78
   1       GAT 6984          FRACT.HP BELT           63.08   37.77    37.77
   1       GAT A100          HEA.DUTY BELT           34.65   20.75    20.75
   1       WIX 42299         WIX AIR FILT.           15.26    9.87     9.87
   1       GAT A100          HEA.DUTY BELT           34.65   20.75    20.75
   1       WIX 51521         WIX OIL FILT.           14.50    8.91     8.91
 
                                                                     279.26
                                                                        6.74

# 2  
Old 01-23-2014
Your results file doesn't show the CRs - is that the actual file, or a cat (without -v) to the terminal?

(i.e. is the last line actually missing, or is it just wrapping on your display?)
# 3  
Old 01-23-2014
My result was in cat form,

here it is in raw form

Code:
^M^M^[[51t^M^M       ABC COMPANY                             ABC SHIPPING COMPAN
Y            ^M
       123 BAKER STREET                        123 DOCK STREET                 ^
M
       ANYTOWN USA 99999                       ANYTOWN USA 08277               ^
M
                                                                               ^
M
^M
                                                                        PAGE   1
^M
^M
  155 498  MS                          07:43 AM     J  08/01/13 01/23/14*C428230
^M
^M
   1       PLA T237          WHITE PRIMER            10.13    6.58     6.58 T
^M
   1       PS  STRAINERS     PAINT SUPPLYS
^M
   1       SEM 39683         GRAY SELF ETC                   20.57    20.57
^M
   1       SEM 39693         ETCH PRIMER                     20.57    20.57
^M
   1       NAS QUART 421-19  SELECTPRIME 2                   22.75    22.75 T
^M
   1       NAS 1/2 PT 483-87 SELECTPRIME A                   15.60    15.60 T
^M
   1       NAS QUART 441-21  MED FB REDUCE                   11.44    11.44 T
^M
   1       NAS 1/2 PT 483-11 FUL-CRYL CAT.                   33.86    33.86 T
^M
   2       PS  MC32          QT MIX CUP                       0.50     1.00 T
^M
   1       DUP M64           DUP. TAC RAG                     1.13     1.13 T
^M
   2       MMM 7447          SCOTCHBRITE                      1.25     2.50 T
^M
   1       PS  UG18          3/4" MASKING                     1.43     1.43 T
^M
   1       PS  PINT CANS     PAINT SUPPLYS
^M
   1       BAT 10U1R         MOWER BATTERY                   43.78    43.78
^M
   1       GAT 6984          FRACT.HP BELT           63.08   37.77    37.77
^M
   1       GAT A100          HEA.DUTY BELT           34.65   20.75    20.75
^M
   1       WIX 42299         WIX AIR FILT.           15.26    9.87     9.87
^M
   1       GAT A100          HEA.DUTY BELT           34.65   20.75    20.75
^M
   1       WIX 51521         WIX OIL FILT.           14.50    8.91     8.91
^M
^M
^M
                                                                     279.26 ^M
                                                                            ^M
                                                                       6.74 ^M
                                                                            ^M
                                                                            ^M

# 4  
Old 01-23-2014
Hello,

Not sure if this will help.

Code:
sed 's/\^M//g;s/\^L//g;s/\^\[\[66t//g;s/\^\[\[51t//g' file_name

If these are not meta chars.

Output is as follows.

Code:
       ABC COMPANY                             ABC SHIPPING COMPAN
Y
       123 BAKER STREET                        123 DOCK STREET                 ^
M
       ANYTOWN USA 99999                       ANYTOWN USA 08277               ^
M
                                                                               ^
M
                                                                        PAGE   1

  155 498  MS                          07:43 AM     J  08/01/13 01/23/14*C428230

   1       PLA T237          WHITE PRIMER            10.13    6.58     6.58 T
   1       PS  STRAINERS     PAINT SUPPLYS
   1       SEM 39683         GRAY SELF ETC                   20.57    20.57
   1       SEM 39693         ETCH PRIMER                     20.57    20.57
   1       NAS QUART 421-19  SELECTPRIME 2                   22.75    22.75 T
   1       NAS 1/2 PT 483-87 SELECTPRIME A                   15.60    15.60 T
   1       NAS QUART 441-21  MED FB REDUCE                   11.44    11.44 T
   1       NAS 1/2 PT 483-11 FUL-CRYL CAT.                   33.86    33.86 T
   2       PS  MC32          QT MIX CUP                       0.50     1.00 T
   1       DUP M64           DUP. TAC RAG                     1.13     1.13 T
   2       MMM 7447          SCOTCHBRITE                      1.25     2.50 T
   1       PS  UG18          3/4" MASKING                     1.43     1.43 T
   1       PS  PINT CANS     PAINT SUPPLYS
   1       BAT 10U1R         MOWER BATTERY                   43.78    43.78
   1       GAT 6984          FRACT.HP BELT           63.08   37.77    37.77
   1       GAT A100          HEA.DUTY BELT           34.65   20.75    20.75
   1       WIX 42299         WIX AIR FILT.           15.26    9.87     9.87
   1       GAT A100          HEA.DUTY BELT           34.65   20.75    20.75
   1       WIX 51521         WIX OIL FILT.           14.50    8.91     8.91
 
                                                                     279.26
                                                                       6.74

                                                                     286.00

Thanks,
R. Singh
# 5  
Old 01-29-2014
Quote:
Originally Posted by RavinderSingh13
Hello,

Not sure if this will help.

Code:
sed 's/\^M//g;s/\^L//g;s/\^\[\[66t//g;s/\^\[\[51t//g' file_name

If these are not meta chars.

Output is as follows.

Code:
       ABC COMPANY                             ABC SHIPPING COMPAN
Y
       123 BAKER STREET                        123 DOCK STREET                 ^
M
       ANYTOWN USA 99999                       ANYTOWN USA 08277               ^
M
                                                                               ^
M
                                                                        PAGE   1

  155 498  MS                          07:43 AM     J  08/01/13 01/23/14*C428230

   1       PLA T237          WHITE PRIMER            10.13    6.58     6.58 T
   1       PS  STRAINERS     PAINT SUPPLYS
   1       SEM 39683         GRAY SELF ETC                   20.57    20.57
   1       SEM 39693         ETCH PRIMER                     20.57    20.57
   1       NAS QUART 421-19  SELECTPRIME 2                   22.75    22.75 T
   1       NAS 1/2 PT 483-87 SELECTPRIME A                   15.60    15.60 T
   1       NAS QUART 441-21  MED FB REDUCE                   11.44    11.44 T
   1       NAS 1/2 PT 483-11 FUL-CRYL CAT.                   33.86    33.86 T
   2       PS  MC32          QT MIX CUP                       0.50     1.00 T
   1       DUP M64           DUP. TAC RAG                     1.13     1.13 T
   2       MMM 7447          SCOTCHBRITE                      1.25     2.50 T
   1       PS  UG18          3/4" MASKING                     1.43     1.43 T
   1       PS  PINT CANS     PAINT SUPPLYS
   1       BAT 10U1R         MOWER BATTERY                   43.78    43.78
   1       GAT 6984          FRACT.HP BELT           63.08   37.77    37.77
   1       GAT A100          HEA.DUTY BELT           34.65   20.75    20.75
   1       WIX 42299         WIX AIR FILT.           15.26    9.87     9.87
   1       GAT A100          HEA.DUTY BELT           34.65   20.75    20.75
   1       WIX 51521         WIX OIL FILT.           14.50    8.91     8.91
 
                                                                     279.26
                                                                       6.74

                                                                     286.00

Thanks,
R. Singh
When I ran the statement as you created it I still lost the last line. I did some more testing and no matter what with the SED command the last line was dropped so I am thinking I have meta characters on that last line causing the issue. I decided to do a test using awk and it read the entire file and wrote out the entire file something sed was not doing so what I need to know how to do is use AWK to replace the last form feed on the last line. Not sure of the syntax, something like this
Code:
awk '{sub(/$/,"\r");print}'

Thanks for the help
# 6  
Old 01-29-2014
The problem comes from discovering what the 'last line' is. sed et al can't look back in time to see that. awk can read the file twice however, once to determine, once to substitute:

Code:
awk 'NR==FNR { LL=NR ; next }; FNR == LL { sub(/\014/, "\r"); }' inputfile inputfile > outputfile

\014 is the octal for a line feed, \r is its substitute. Use whatever you want for it.
# 7  
Old 01-29-2014
Before trying to speculate whether or not you have any particular format why not try
a hexdump and have a deeper look inside the file so that you know exactly what you
are up against.
Code:
hexdump -C /full/path/to/filename

Once we know exactly what we are looking at then we can give informed replies.
Hope this helps...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed to remove text from file

Trying to use sed to, in-place, remove specific text from a file. Since there are / in the text I use | to escape that character. Thank you :). sed -i -e 's|xxxx://www.xxx.com/xx/xx/xxx/.*/|' /home/cmccabe/list sed: -e expression #1, char 51: unterminated `s' command (4 Replies)
Discussion started by: cmccabe
4 Replies

2. Shell Programming and Scripting

Remove the first match only in a file using sed

I'm trying to remove the first match only of 2Z694 from an xml file and replace with a blank File Example: </Phoenix_Response_Data> <Bundle_Name_Primary>2Z694</Bundle_Name_Primary> <Bundle_Name>2Z694</Bundle_Name> </Phoenix_Response_Data> tried using: sed -e 's/'2Z694'/''/1' but this... (15 Replies)
Discussion started by: cillmor
15 Replies

3. Shell Programming and Scripting

remove particular line from a file using sed

Hi i need to remove all the lines staring with 'printf("\n' from a file, example : the file tmp.txt contains printf("\n "); printf("\n good"); printf("\n "); printf("\n "); printf(""); printf( m_sprintf(for printf("\n "); i have tried with following commands but... (5 Replies)
Discussion started by: mprakasheee
5 Replies

4. Shell Programming and Scripting

sed to remove braces from a file

i need to search for user belonging to group 'macusr' and the extract the user name . i am able to write a oneliner for this using awk + sed + tr i am using tr to chop off '()' from the output. but i want to use it in sed itself . can someone please help me with that file contents ... (7 Replies)
Discussion started by: chidori
7 Replies

5. UNIX for Dummies Questions & Answers

Remove part of file name with sed & mv

Ok, so I have bunch of files that are named "orange__file_name.asm" and I want to batch rename them to "file_name.asm" I know that using "ls | sed s/orange__//" will get rid of the part of the file name I do not want. But how do I combine that with the mv command to actually do it? Thanks JG (5 Replies)
Discussion started by: john galt
5 Replies

6. Shell Programming and Scripting

sed -e remove line from file

Hi Trying to remove line from file log_January_1_array and code below doesn't work. $(sed -e '/"$n"/d' <log_January_1_array >log_January_1_array_1) sed doesn't know what is in $n variable and nth happens. Please advice how to make sed running this. thx (2 Replies)
Discussion started by: presul
2 Replies

7. Shell Programming and Scripting

Remove text from a csv file using sed

I am trying to remove the ita from this file: "1234ita","john","smith" "56789ita","jim","thomas" the command i am using is: sed '/^ita/d' infile.csv > outfile.csv it is running but doing nothing, very rarely use sed so trying to learn it any help would be appreciated (2 Replies)
Discussion started by: Pablo_beezo
2 Replies

8. Shell Programming and Scripting

sed over writes my original file (using sed to remove leading spaces)

Hello and thx for reading this I'm using sed to remove only the leading spaces in a file bash-280R# cat foofile some text some text some text some text some text bash-280R# bash-280R# sed 's/^ *//' foofile > foofile.use bash-280R# cat foofile.use some text some text some text... (6 Replies)
Discussion started by: laser
6 Replies

9. Shell Programming and Scripting

Remove pipes in file using SED

Hello, I am writing a script to remove multiples pipes (|) from a file. It needs to look like this data|data|data instead of data|||data||data||||data I have found the correct sed command to do this, but I need to do it in a loop and a for loop has not worked for me. It needs to be done on... (3 Replies)
Discussion started by: TL56
3 Replies

10. Shell Programming and Scripting

how to remove ^@ from a file using sed...or anything

i tried the following:- sed -e file 's/^@//g' > temp also tried sed -e file 's///g' > temp nothing happened....can someone please tell me wht is wrong??? also someinformation abt the character "^@"(it is ONLY ONE character and NOT TWO characters) thanx in advance.. (13 Replies)
Discussion started by: sayonm
13 Replies
Login or Register to Ask a Question