Remove lines ending with a certain character


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Remove lines ending with a certain character
# 1  
Old 02-22-2019
Remove lines ending with a certain character

I have a file of a content like this:

Code:
abc_bla -def 800
abc_bla -def 802
abc_bla -def 804
abc_bla -def 806
abc_bla -def 808
abc_bla -def 810
abc_bla -def 812
abc_bla -def 814
...
abc_bla -def 898
abc_bla -def 900
abc_bla -def 902
abc_bla -def 904
...
abc_bla -def 990
abc_bla -def 992
abc_bla -def 994
abc_bla -def 996
abc_bla -def 998
abc_bla -def 1000
abc_bla -def 1002
abc_bla -def 1004
abc_bla -def 1006
...
...
abc_bla -def 1800

I would like to delete all lines in a file that end with a 0 (zero).

I tried this:

Code:
sed '/0/d' file

but this deletes all the lines containing a zero, and I want to delete only those *ending* with a zero.
Then I tried

Code:
grep '0$' file

but that gives me as output only the last line.

How can I select that it chooses the pattern when it is *only* at the end of the line?
So I want my output to be all the lines not ending with a 0.
Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input and sample output as well as when displaying code segments.

Last edited by Don Cragun; 02-22-2019 at 08:02 AM.. Reason: Add missing CODE tags.
# 2  
Old 02-22-2019
you were pretty close. Try this:
Code:
sed '/0$/d' file

# 3  
Old 02-22-2019
Welcome to the forum.


You weren't too far off. Why didn't you apply what you had in your grep approach to the sed command, i.e. anchor the regex at line end? Like
Code:
sed '/0$/d' file

Or, add grep's (man grep)
Quote:
-v, --invert-match
Invert the sense of matching, to select non-matching lines.
option ?


How come your grep "gives me as output only the last line." ?
# 4  
Old 02-22-2019
Quote:
Originally Posted by maya3
I have a file of a content like this:

Code:
abc_bla -def 800
abc_bla -def 802
abc_bla -def 804
abc_bla -def 806
abc_bla -def 808
abc_bla -def 810
abc_bla -def 812
abc_bla -def 814
...
abc_bla -def 898
abc_bla -def 900
abc_bla -def 902
abc_bla -def 904
...
abc_bla -def 990
abc_bla -def 992
abc_bla -def 994
abc_bla -def 996
abc_bla -def 998
abc_bla -def 1000
abc_bla -def 1002
abc_bla -def 1004
abc_bla -def 1006
...
...
abc_bla -def 1800

I would like to delete all lines in a file that end with a 0 (zero).

I tried this:

Code:
sed '/0/d' file

but this deletes all the lines containing a zero, and I want to delete only those *ending* with a zero.
Then I tried

Code:
grep '0$' file

but that gives me as output only the last line.

How can I select that it chooses the pattern when it is *only* at the end of the line?
So I want my output to be all the lines not ending with a 0.
Moderator's Comments:
Mod Comment Please use CODE tags when displaying sample input and sample output as well as when displaying code segments.
Hi maya3,
One would expect that the command:
Code:
grep '0$' file

would print out all lines with a 0 at the end. Since that is not happening, we have to wonder what actually follows the 0 in the lines that are not being printed. Might this be a DOS format text file that has <carriage-return><newline> character pairs at the end of most lines instead of a UNIX format text file that just has a <newline> character at the end of each line? Please show us the output from the command:
Code:
head file | od -bc

where file is the name of the sample file you showed us in post #1 in this thread.

Note that grep '0$' file will give you the lines you want to delete (not the lines you want to keep) if file really does have lines with a trailing 0 character. To get the lines you want to keep, you probably want to use:
Code:
grep -v '0$' file > new_file

after we clean up whatever is keeping the lines with a 0 at the end of a line from being recognized.
# 5  
Old 02-22-2019
Thank you for the answers.

It seemed that I had \r\n at the end of the line and I replaced it with \n but the outputs are still the same.
This is what I get when I write
Code:
head file | od -bc

(the lines are actually longer so this is adapted to my sample file)

Code:
0000000 147 156 141 141 143 137 144 141 147 137 163 165 142 155 151 164
          a   b   c   _   b   l   a   _   s   u   b   m   i   t
0000020 040 055 147 167 145 145 153 040 070 060 062 040 012 147 156 141
              -   d   e   f       8   0   0      \n   a   b   c
0000040 141 143 137 144 141 147 137 163 165 142 155 151 164 040 055 147
          _   b   l   a  etc.

# 6  
Old 02-22-2019
That output doesn't seem to originally belong to the command you issued, and it certainly doesn't match the sample in post #1. Please apply utmost care when posting code and data!


Still from the octal dump I infer you don't have digits (incl. zeroes) at line end, but digits plus spaces. Try
Code:
sed '/0 *$/d' file

or
Code:
grep -v '0 *$' file

This User Gave Thanks to RudiC For This Post:
# 7  
Old 02-22-2019
Quote:
Originally Posted by maya3
Thank you for the answers.

It seemed that I had \r\n at the end of the line and I replaced it with \n but the outputs are still the same.
This is what I get when I write
Code:
head file | od -bc

(the lines are actually longer so this is adapted to my sample file)

Code:
0000000 147 156 141 141 143 137 144 141 147 137 163 165 142 155 151 164
          a   b   c   _   b   l   a   _   s   u   b   m   i   t
0000020 040 055 147 167 145 145 153 040 070 060 062 040 012 147 156 141
              -   d   e   f       8   0   0      \n   a   b   c
0000040 141 143 137 144 141 147 137 163 165 142 155 151 164 040 055 147
          _   b   l   a  etc.

Let us be clear here. The above display is not sample output from any file. The above display is not adapted output from any file. The above output is contrived output that is intended to confuse anyone who might want to help you solve your problem.

If RudiC's guess (like mine), is that the octal byte values show real data and the character values have been concocted to confuse us, then the code RudiC suggested may solve your problem. If it doesn't please show us the complete, real output from the head piped through od command I gave you so we have a chance of figuring out what is really going on.

The fake data that you did show us does not include any line that has a 0 at the end of a line (even if we ignore trailing <space>s. The only complete line shown in the seemingly real od output you concocted ends with a decimal digit 2 followed by a <space>. Guessing that all other lines in your input file (except the last one) also have trailing <space>(s) from a sample of a single concocted input line is a seemingly good guess, but it certainly isn't anything that we can rely on.

Moderator's Comments:
Mod Comment We are here to help you learn how to solve your programming problems. Posting inconsistent fake data in no way helps you reach your goal. It only shows that you intend to confuse those of us who are trying to help you.

Posting inconsistent fake data again could be interpreted as a request to be banned from this site.

Please help us help you by giving us honest answers to the questions we ask. We don't ask these questions for our amusement; we ask them because they give us data we need to be able to help you. Showing us inconsistent, fake data tells us that you do not want our help and, instead, just want to waste our time.
These 2 Users Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Remove newline character from column spread over multiple lines in a file

Hi, I came across one issue recently where output from one of the columns of the table from where i am creating input file has newline characters hence, record in the file is spread over multiple lines. Fields in the file are separated by pipe (|) delimiter. As header will never have newline... (4 Replies)
Discussion started by: Prathmesh
4 Replies

2. Shell Programming and Scripting

Text to column starting/ending with special character in each row

Hello, Here is my text data excerpted from the webpage: input My target is to get: What i tried is: sed 's/.*\(connector\)/1/' input > output but all characters coming before the word "connector" are deleted which is not good for me. My question: (9 Replies)
Discussion started by: baris35
9 Replies

3. Shell Programming and Scripting

Remove '.' from file for numbers ending in '.'

Hi, I have numerous files which have data in the following format A|B|123.|Mr.|45.66|33|zz L|16.|33.45|AC.|45. I want to remove decimal point only if it is last character in a number. O/p should be A|B|123|Mr.|45.66|33|zz L|16|33.45|AC.|45 I tried this sed -e 's/.|/|/g' Problem... (6 Replies)
Discussion started by: wahi80
6 Replies

4. Shell Programming and Scripting

Remove certain lines from file based on start of line except beginning and ending

Hi, I have multiple large files which consist of the below format: I am trying to write an awk or sed script to remove all occurrences of the 00 record except the first and remove all of the 80 records except the last one. Any help would be greatly appreciated. (10 Replies)
Discussion started by: nwalsh88
10 Replies

5. Shell Programming and Scripting

Remove ending text

Hello, I am working with a list that contains a large number of files listed by their absolute path. I am trying to determine a way to delete the file name at the end of each line, therefore leaving just the directory path. For example, I'd like to go from: /home/something/file... (2 Replies)
Discussion started by: omnivir
2 Replies

6. Shell Programming and Scripting

Execution problem ---to remove the lines which starts with one type of character

Hi, I have one file, I need to check if file exist or not and then remove the lines which starts with ? My file1.out data is some thing abcabcppp xyzxyzpqr ????????? ????????? Output should be in test.out abcabcppp xyzxyzpqr I am getting the output as below but the File does not exist... (4 Replies)
Discussion started by: Ramyajiguru1
4 Replies

7. UNIX for Dummies Questions & Answers

Remove 1st character in periodic lines

Hi, I have a file that looks like this, the unity of information is composed of four lines, and these extends for millions. My objective is to remove the highligthed "T". How to attack this? This character is always constant in type "T" and position "1st" but the rest of the line is... (7 Replies)
Discussion started by: sargotrons
7 Replies

8. UNIX for Dummies Questions & Answers

how to remove lines ending with '*'

I have a file where some lines end with '*'. I would like to remove those lines ending with '*'. inFile: a b* c d*outFile: a cThank you (7 Replies)
Discussion started by: jdhahbi
7 Replies

9. UNIX for Advanced & Expert Users

remove lines from file where > 13 occurrences of character

I have a '~' delimited file of 6 - 7 million rows. Each row should contain 13 columns delimited by 12 ~'s. Where there are 13 tildes, the row needs to be removed. Each row contains alphanumeric data and occasionally a ~ ends up in a descriptive field and therefore acts as a delimiter, resulting... (7 Replies)
Discussion started by: kpd
7 Replies

10. Shell Programming and Scripting

How do I remove lines that have more than one of a certain character in them?

I have a file with a few thousand lines and I'd like to remove all the lines that have more than 1 asterik (the * character) in it. So if it has 2 or more in a single line, I'd like the line removed (double d command in vi) (12 Replies)
Discussion started by: guitarscn
12 Replies
Login or Register to Ask a Question