Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Trying to use sed to remove last FF from file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Trying to use sed to remove last FF from file
# 8  
Old 01-29-2014
So if I wanted to replace the FF with LF would this be the correct format? Also did you mean to have the input file twice in your example? I tried it both ways with input file twice and once and my result file is blank
Code:
awk 'NR==FNR { LL=NR ; next }; FNR == LL { sub(/\014/,/\012 ); }' inv >inv.test

Thanks

---------- Post updated at 04:22 PM ---------- Previous update was at 04:21 PM ----------

wisecracker, I did that when I first started working on this but didn't include my hex dump, here it is
Code:
0000    0d 0d 20 20 20 20 20 20  20 41 42 43 20 43 4f 4d   ..       ABC COM
0010    50 41 4e 59 20 20 20 20  20 20 20 20 20 20 20 20   PANY
0020    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
0030    20 41 42 43 20 53 48 49  50 50 49 4e 47 20 43 4f    ABC SHIPPING CO
0040    4d 50 41 4e 59 20 20 20  20 20 20 20 20 20 20 20   MPANY
0050    20 0d 0a 20 20 20 20 20  20 20 31 32 33 20 42 41    ..       123 BA
0060    4b 45 52 20 53 54 52 45  45 54 20 20 20 20 20 20   KER STREET
0070    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
0080    20 20 31 32 33 20 44 4f  43 4b 20 53 54 52 45 45     123 DOCK STREE
0090    54 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20   T
00a0    20 20 0d 0a 20 20 20 20  20 20 20 41 4e 59 54 4f     ..       ANYTO
00b0    57 4e 20 55 53 41 20 39  39 39 39 39 20 20 20 20   WN USA 99999
00c0    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
00d0    20 20 20 41 4e 59 54 4f  57 4e 20 55 53 41 20 30      ANYTOWN USA 0
00e0    38 32 37 37 20 20 20 20  20 20 20 20 20 20 20 20   8277
00f0    20 20 20 0d 0a 20 20 20  20 20 20 20 20 20 20 20      ..
0100    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
*
0140    20 20 20 20 0d 0a 0d 0a  20 20 20 20 20 20 20 20       ....
0150    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
*
0190    50 41 47 45 20 20 20 31  0d 0a 0d 0a 20 20 31 35   PAGE   1....  15
01a0    35 20 34 39 39 20 20 4c  53 20 20 20 20 20 20 20   5 499  LS
"inv.hd" 60 lines, 3743 characters
0190    50 41 47 45 20 20 20 31  0d 0a 0d 0a 20 20 31 35   PAGE   1....  15
01a0    35 20 34 39 39 20 20 4c  53 20 20 20 20 20 20 20   5 499  LS
01b0    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
01c0    20 20 20 30 39 3a 31 39  20 41 4d 20 20 20 20 20      09:19 AM
01d0    4a 20 20 30 31 2f 32 37  2f 31 34 20 30 31 2f 32   J  01/27/14 01/2
01e0    37 2f 31 34 2a 43 34 32  38 32 33 31 0d 0a 0d 0a   7/14*C428231....
01f0    20 20 20 31 20 20 20 20  20 20 20 57 49 58 20 35      1       WIX 5
0200    31 35 31 35 20 20 20 20  20 20 20 20 20 57 49 58   1515         WIX
0210    20 4f 49 4c 20 46 49 4c  54 2e 20 20 20 20 20 20    OIL FILT.
0220    20 20 20 20 20 31 30 2e  34 34 20 20 20 20 36 2e        10.44    6.
0230    34 31 20 20 20 20 20 36  2e 34 31 20 20 20 20 20   41     6.41
0240    0d 0a 20 20 20 31 20 20  20 20 20 20 20 57 49 58   ..   1       WIX
0250    20 35 31 35 31 35 52 20  20 20 20 20 20 20 20 57    51515R        W
0260    49 58 20 52 41 43 49 4e  47 20 46 49 20 20 20 20   IX RACING FI
0270    20 20 20 20 20 20 20 31  37 2e 33 38 20 20 20 31          17.38   1
0280    30 2e 36 38 20 20 20 20  31 30 2e 36 38 20 54 20   0.68    10.68 T
0290    20 20 0d 0a 0d 0a 0d 0a  0d 0a 0d 0a 0d 0a 0d 0a     ..............
02a0    0d 0a 0d 0a 0d 0a 0d 0a  0d 0a 0d 0a 0d 0a 0d 0a   ................
02b0    0d 0a 0d 0a 0d 0a 0d 0a  0d 0a 20 20 20 20 20 20   ..........
02c0    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
*
0300    31 37 2e 30 39 20 0d 0a  20 20 20 20 20 20 20 20   17.09 ..
0310    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
 0300    31 37 2e 30 39 20 0d 0a  20 20 20 20 20 20 20 20   17.09 ..
0310    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
*
0350    20 20 20 20 0d 0a 20 20  20 20 20 20 20 20 20 20       ..
0360    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
*
0390    20 20 20 20 20 20 20 20  20 20 20 20 20 30 2e 37                0.7
03a0    35 20 0d 0a 20 20 20 20  20 20 20 20 20 20 20 20   5 ..
03b0    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
*
03f0    0d 0a 20 20 20 20 20 20  20 20 20 20 20 20 20 20   ..
0400    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
*
0430    20 20 20 20 20 20 20 20  20 20 20 20 20 20 0d 0a                 ..
0440    20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20
*
0480    20 20 20 20 20 20 31 37  2e 38 34 20 0d 0c 0d            17.84 ...
048f

# 9  
Old 01-29-2014
Pardon me, missing one crucial character from my code. Also, the output string should be "\012", since it's not a regular expression like the input is.

Actually, just "\n" will do.

And you forgot to list the filename twice. I even explained why it has to read the file twice -- it needs to find out what the last line is.

Code:
awk 'NR==FNR { LL=NR ; next }; FNR == LL { sub(/\014/,"\n" ); } 1' inv inv >inv.test

These 2 Users Gave Thanks to Corona688 For This Post:
wisecracker ziggy6
# 10  
Old 01-29-2014
Corona688, that worked! Thanks....
# 11  
Old 02-10-2014
Quote:
Originally Posted by Corona688
The problem comes from discovering what the 'last line' is. sed et al can't look back in time to see that.
This is true but looking back in time is not needed - as is reading a file twice.

The logic to apply in sed is: starting with a form-feed character read lines until you encounter another form-feed (in this case dump the lines already read into output and start over) or the last line. In case of the last line replace the first character in the pattern space (the FF) with a newline and dump the result to output. All other lines are simply added to the pattern space ("N") until one of the two conditions above is reached.

The script for this looks like this (<FF> is a form-feed character and supposed to be on a line of its own, that is: preceeded by a newline):

Code:
sed -n '$ {
               s/^./\n/p
          }
        /<FF>/ {
               s/.$//p
               s/^.*$/<FF>/
          }
          N' /path/to/file

I hope this helps.

bakunin
# 12  
Old 02-10-2014
Quote:
Originally Posted by bakunin
The logic to apply in sed is: starting with a form-feed character read lines until you encounter another form-feed (in this case dump the lines already read into output and start over) or the last line. In case of the last line replace the first character in the pattern space (the FF) with a newline and dump the result to output. All other lines are simply added to the pattern space ("N") until one of the two conditions above is reached.
Ah, but it does look back -- it needs to wait until the entire file's already read to decide whether to substitute or not. It has a buffer, which either gets printed or not depending. That kind of logic -- "read until x, buffer until y, print" has a lot of special cases(what if x happens without y?) which makes the code larger and more bugprone, I found it more straightforward to read the file twice since it appeared to be small. It was a design choice.
# 13  
Old 02-10-2014
Hi Corona how about using tac

Code:
$ tac file | awk 'FNR==1{sub(/\014/,"\n" )}1' | tac

---------- Post updated at 09:42 AM ---------- Previous update was at 09:28 AM ----------

Untested version

Code:
$ awk 'function last(){sub(/\014/,"\n" );return $0}{p=$0; print getline == 0 ? last() : p RS $0}' file
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed to remove text from file

Trying to use sed to, in-place, remove specific text from a file. Since there are / in the text I use | to escape that character. Thank you :). sed -i -e 's|xxxx://www.xxx.com/xx/xx/xxx/.*/|' /home/cmccabe/list sed: -e expression #1, char 51: unterminated `s' command (4 Replies)
Discussion started by: cmccabe
4 Replies

2. Shell Programming and Scripting

Remove the first match only in a file using sed

I'm trying to remove the first match only of 2Z694 from an xml file and replace with a blank File Example: </Phoenix_Response_Data> <Bundle_Name_Primary>2Z694</Bundle_Name_Primary> <Bundle_Name>2Z694</Bundle_Name> </Phoenix_Response_Data> tried using: sed -e 's/'2Z694'/''/1' but this... (15 Replies)
Discussion started by: cillmor
15 Replies

3. Shell Programming and Scripting

remove particular line from a file using sed

Hi i need to remove all the lines staring with 'printf("\n' from a file, example : the file tmp.txt contains printf("\n "); printf("\n good"); printf("\n "); printf("\n "); printf(""); printf( m_sprintf(for printf("\n "); i have tried with following commands but... (5 Replies)
Discussion started by: mprakasheee
5 Replies

4. Shell Programming and Scripting

sed to remove braces from a file

i need to search for user belonging to group 'macusr' and the extract the user name . i am able to write a oneliner for this using awk + sed + tr i am using tr to chop off '()' from the output. but i want to use it in sed itself . can someone please help me with that file contents ... (7 Replies)
Discussion started by: chidori
7 Replies

5. UNIX for Dummies Questions & Answers

Remove part of file name with sed & mv

Ok, so I have bunch of files that are named "orange__file_name.asm" and I want to batch rename them to "file_name.asm" I know that using "ls | sed s/orange__//" will get rid of the part of the file name I do not want. But how do I combine that with the mv command to actually do it? Thanks JG (5 Replies)
Discussion started by: john galt
5 Replies

6. Shell Programming and Scripting

sed -e remove line from file

Hi Trying to remove line from file log_January_1_array and code below doesn't work. $(sed -e '/"$n"/d' <log_January_1_array >log_January_1_array_1) sed doesn't know what is in $n variable and nth happens. Please advice how to make sed running this. thx (2 Replies)
Discussion started by: presul
2 Replies

7. Shell Programming and Scripting

Remove text from a csv file using sed

I am trying to remove the ita from this file: "1234ita","john","smith" "56789ita","jim","thomas" the command i am using is: sed '/^ita/d' infile.csv > outfile.csv it is running but doing nothing, very rarely use sed so trying to learn it any help would be appreciated (2 Replies)
Discussion started by: Pablo_beezo
2 Replies

8. Shell Programming and Scripting

sed over writes my original file (using sed to remove leading spaces)

Hello and thx for reading this I'm using sed to remove only the leading spaces in a file bash-280R# cat foofile some text some text some text some text some text bash-280R# bash-280R# sed 's/^ *//' foofile > foofile.use bash-280R# cat foofile.use some text some text some text... (6 Replies)
Discussion started by: laser
6 Replies

9. Shell Programming and Scripting

Remove pipes in file using SED

Hello, I am writing a script to remove multiples pipes (|) from a file. It needs to look like this data|data|data instead of data|||data||data||||data I have found the correct sed command to do this, but I need to do it in a loop and a for loop has not worked for me. It needs to be done on... (3 Replies)
Discussion started by: TL56
3 Replies

10. Shell Programming and Scripting

how to remove ^@ from a file using sed...or anything

i tried the following:- sed -e file 's/^@//g' > temp also tried sed -e file 's///g' > temp nothing happened....can someone please tell me wht is wrong??? also someinformation abt the character "^@"(it is ONLY ONE character and NOT TWO characters) thanx in advance.. (13 Replies)
Discussion started by: sayonm
13 Replies
Login or Register to Ask a Question