Tried many options but unable to delete blank lines from text file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Tried many options but unable to delete blank lines from text file
# 1  
Old 07-24-2017
Tried many options but unable to delete blank lines from text file

Hi,

I tried the following options but was unable to delete blank lines from file

Input file = temp.hash.txt
temp.hash.txt content
Code:
90

0

89.56
0
0


57575.4544
56.89




0


0
0
0

----------------------------
Options tried
  1. sed '/^$/d' temp.hash.txt > temp.hash.out
  2. grep '[^[:blank:]]' < temp.hash.txt > temp.hash.out
  3. awk 'NF' temp.hash.txt
  4. sed -i '/^$/d' temp.hash.txt
  5. grep -v '^$' temp.hash.txt > output.txt

Nothing worked.

Any help is appreciated!

Thanks...


Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by rbatte1; 07-24-2017 at 11:20 AM.. Reason: RudiC Added CODE tags; rbatte1 Converted to formatted numeric list
# 2  
Old 07-24-2017
Any of your options tried does what you want them to do, so I guess your file has some "hidden" features like e.g. non-printing control characters. Try
Code:
od -tx1c temp.hash.txt

and post the result.
# 3  
Old 07-24-2017
This is the result

Code:
[oratest@uswclora15 ~]$ od -tx1c temp.hash.txt
0000000 39 30 0d 0a 0d 0a 30 0d 0a 0d 0a 38 39 2e 35 36
          9   0  \r  \n  \r  \n   0  \r  \n  \r  \n   8   9   .   5   6
0000020 0d 0a 30 0d 0a 30 0d 0a 0d 0a 0d 0a 35 37 35 37
         \r  \n   0  \r  \n   0  \r  \n  \r  \n  \r  \n   5   7   5   7
0000040 35 2e 34 35 34 34 0d 0a 35 36 2e 38 39 0d 0a 0d
          5   .   4   5   4   4  \r  \n   5   6   .   8   9  \r  \n  \r
0000060 0a 0d 0a 0d 0a 0d 0a 30 0d 0a 0d 0a 0d 0a 30 0d
         \n  \r  \n  \r  \n  \r  \n   0  \r  \n  \r  \n  \r  \n   0  \r
0000100 0a 30 0d 0a 30 0d 0a 0d 0a 0d 0a 35 37 35 36 35
         \n   0  \r  \n   0  \r  \n  \r  \n  \r  \n   5   7   5   6   5
0000120



Moderator's Comments:
Mod Comment
Please wrap all code, file content, input and output/errors in CODE tags.
It makes it easier to read and preserves multiple spaces for indenting or fixed-width data.

Last edited by rbatte1; 07-24-2017 at 11:18 AM.. Reason: Added CODE tags.
# 4  
Old 07-24-2017
Hello uuuunnnn,

So it is confirmed now that \r carriage characters are present in your Input_file, so could you please first remove them and then try your any of the code mentioned in POST#1 and let us know how it goes then.
Code:
tr -d '\r' < Input_file > temp_Input_file  &&  mv temp_Input_file  Input_file

Thanks,
R. Singh
# 5  
Old 07-24-2017
You can also use grep but with [[:space:]] instead of [[:blank:]] :
Code:
grep -v '^[[:space:]]*$'  file

But this will leave the Windows carriage return character on the other lines.

--
You could do it all at once like so:
Code:
awk '{sub(/\r/,x)}NF' file


Last edited by Scrutinizer; 07-24-2017 at 11:38 AM..
# 6  
Old 07-24-2017
I did that, but it completely deleted all lines

Code:
[oratest@uswclora15 ~]$ tr -d '\r' < temp.hash.txt > temp_Input_file  &&  mv temp_Input_file  temp.hash.txt
[oratest@uswclora15 ~]$ vi temp.hash.txt
[oratest@uswclora15 ~]$ sed '/^$/d' temp.hash.txt > temp.hash.txt
[oratest@uswclora15 ~]$ vi temp.hash.txt
[oratest@uswclora15 ~]$


Thanks,
Niladri
# 7  
Old 07-24-2017
Quote:
Originally Posted by uuuunnnn
I did that, but it completely deleted all lines

Code:
[oratest@uswclora15 ~]
...
...
[oratest@uswclora15 ~]$ sed '/^$/d' temp.hash.txt > temp.hash.txt
...
...

...
...
Whoa!! I hope you have a backup of your file. Smilie

You are essentially trying to read from a file, make some changes and write back to it. It doesn't work the way it looks. (*)

A few options are:
1) If your sed version allows it, try the "in-place edit" modifier; something like "sed -i". Check your sed manual "man sed".

2) Use a temporary file as you did for the "tr" command.

======
(*)
As for why it doesn't work the way it looks - you have two different processes (a) a read process and (b) a write process, both sharing the same resource - your file. When they are executed by the OS, a race condition occurs between them which renders the final output unpredictable.
If the writing is done before the reading, you get a zero-byte file. If the writing is done mid-way during the reading process, you may get a "half-baked" output file. The safest way, therefore, is to either use a temporary file explicitly or let sed do it via the "-i" modifier - which is not available in all sed versions.
You could also use other tools - Bash, awk, Perl etc. but the temporary file will be involved, either explicitly or implicitly.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Delete blank spaces and blank lines in a file

Hi Gurus, Somebody can say me how to delete blank spaces and blank lines in a file unix, please. Thank you for advanced. (10 Replies)
Discussion started by: systemoper
10 Replies

2. Shell Programming and Scripting

Delete blank lines in a file

Hi All, I have a file and I need to delete the lines that are blank and is starting with some characters below. Something like below: Regular Ascii File: Line1: AGODA1 BUSAN||SK Lord Beach 4/6/2012 4/7/2012 68060 Line2: AGODA2 BUSAN||SK Beach Hotel 4/6/2012 4/7/2012 610200 Line3: ... (4 Replies)
Discussion started by: rkumar28
4 Replies

3. Shell Programming and Scripting

Delete the last empty/blank line of the text file

Hi All, I have a file.txt which seems like having three lines. wc -l file.txt 3 file.txt In fact, once it is open in text editor, this file has four lines where the last line is empty. how can i delete this last empty line of the file.txt? I tried the codes below so far but they... (6 Replies)
Discussion started by: senayasma
6 Replies

4. Shell Programming and Scripting

Delete blank lines, if blank lines are more than one using shell

Hi, Consider a file named "testfile" The contents of file are as below first line added for test second line added for test third line added for test fourth line added for test fifth line added for test (5 Replies)
Discussion started by: anil8103
5 Replies

5. Shell Programming and Scripting

Delete blank lines from a file

Hi, I want to use diff to compare two files in a Perl file. But one of the files has some blank lines at the end. So I want to delete the blank lines from the file firstly and then use diff to compare them. But I dont know how to delete the blank lines from the files. Meanwhile, the system is... (5 Replies)
Discussion started by: Damon_Qu
5 Replies

6. UNIX for Dummies Questions & Answers

Remove blank lines and comments from text file

Hi, I am using BASH. How can I remove any lines in a text file that are either blank or begin with a # (ie. comments)? Thanks in advance. Mike (3 Replies)
Discussion started by: msb65
3 Replies

7. Shell Programming and Scripting

Delete a block of text delimited by blank lines when pattern is found

I have a file which contains blocks of text - each block is a multi-lines text delimited by blank lines eg. <blank line> several lines of text ... pattern found on this line several more lines of text ... <blank line> How do you delete the block of text (including the blank lines) when... (17 Replies)
Discussion started by: gleu
17 Replies

8. UNIX for Dummies Questions & Answers

delete blank lines from a file

can anyone show me how to delete blank lines from a file. thanks in advance (2 Replies)
Discussion started by: sachin.gangadha
2 Replies

9. Shell Programming and Scripting

Delete blank lines at the end of file

I am attempting to delete blank lines in my file and I've used this command: sed '/^$/d' $file > $file.fixed all this seems to do is copy the file and not delete the blank lines located at the end of the file. Any assistance would be greatly appreciated. (3 Replies)
Discussion started by: TL56
3 Replies

10. Shell Programming and Scripting

regex to delete multiple blank lines in a file?

can't figure out a way to delete multiple empty lines but keep single empty lines in a file, file is like this #cat file 1 2 3 4 5 6 - What I want is 1 2 (6 Replies)
Discussion started by: fedora
6 Replies
Login or Register to Ask a Question