Remove last newline character..


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove last newline character..
# 8  
Old 06-07-2013
Code:
awk NF file

works fine on an Aix system but i do not understand why.
Can someone detail it ?
Thank You
# 9  
Old 06-08-2013
Lightbulb

dude open the file using VI editor
vi filename

after that go to the line which u want to delete, and then press esc+dd to remove the line then save it using wq! command..thats it
# 10  
Old 06-08-2013
Quote:
Originally Posted by sathish38
dude open the file using VI editor
vi filename

after that go to the line which u want to delete, and then press esc+dd to remove the line then save it using wq! command..thats it
You could automate that using ed, of course, or any of the other solutions given.

I suspect Sathya83aa wants to have it automated, somehow.
# 11  
Old 06-08-2013
Quote:
Originally Posted by Sathya83aa
...
When I try wc -l the file name,the number of lines coming is 3 only, however blank space is there in the file.
...
Quote:
Originally Posted by Sathya83aa
... when I take the count of the file, it is giving me the value without the next line.
However when I open the file in vi mode the cursor goes till the line 'ijkl'. This is looking strange. ...
Maybe the last line of your file has control/non-printing character(s).
Try the following command and check the ASCII code of the character(s) in the last line.

Code:
od -bc your_file_name

# 12  
Old 06-08-2013
I am sure that if you follow Durden's suggestion, the last character contained in the file as printed by the od command will not be a newline.

From the symptoms that have been described, it is clear that this is not a text file (because the last character of a non-empty text file must be a newline character and wc -l would return 4 instead of 3). Since there are characters following the last newline character in this file, the behavior of awk, ed, sed, vi, and many other utilities is unspecified. On some implementations these utilities may add a newline to the end of the file (with or without a diagnostic or warning message), may ignore all characters following the last newline character in the file, (although probably less likely) do absolutely anything the programmer thought would be fun in this case, or (more likely) do something the programmer never considered (because (s)he never considered the possibility that the input might not be a properly formed text file).

One reliable way on any Linux or UNIX system to delete any characters following the last newline whether or not there are any is:
Code:
echo "" >> filename
printf '$d\nw\nq\n' | ed -s filename

However, even this is only guaranteed to work if there aren't any NUL bytes in the file and there aren't more than (LINE_MAX - 1) consecutive bytes in the file that do not contain a newline character. You should be able to get the value of LINE_MAX on your system by running the command:
Code:
getconf LINE_MAX

This User Gave Thanks to Don Cragun For This Post:
# 13  
Old 06-09-2013
A portable approach that can cope with any file type:
Code:
file=$1
offset=$(
    od -An -v -tu1 "$file" |
    sed 's/[[:blank:]]\{1,\}/ /g; s/^ //; s/ $//; y/ /\n/' |
    awk '$0=="10" {i=NR} END {print i+0}'
)
dd bs=1 seek=$offset if=/dev/null of="$file"

Everything after the final newline is truncated. Files without a newline are truncated to zero length. The contents of files ending in a newline are not modified.

The byte value sought could be trivially parameterized so that anything following the final occurrence of that value is discarded.

In the vast majority of cases, Don's simpler approach should suffice.

Regards,
Alister
This User Gave Thanks to alister For This Post:
# 14  
Old 06-09-2013
Quote:
Originally Posted by alister
A portable approach that can cope with any file type:
Code:
file=$1
offset=$(
    od -An -v -tu1 "$file" |
    sed 's/[[:blank:]]\{1,\}/ /g; s/^ //; s/ $//; y/ /\n/' |
    awk '$0=="10" {i=NR} END {print i+0}'
)
dd bs=1 seek=$offset if=/dev/null of="$file"

Everything after the final newline is truncated. Files without a newline are truncated to zero length. The contents of files ending in a newline are not modified.

The byte value sought could be trivially parameterized so that anything following the final occurrence of that value is discarded.

In the vast majority of cases, Don's simpler approach should suffice.

Regards,
Alister
Nicely done Alister. But even this approach has an exception. This code will remove all characters in a file after the last <newline> character as long as the file is encoded using a codeset that is a superset of ASCII.

If you want to use this on IBM's AIX on a mainframe computer (or any other OS that uses EBCDIC to encode text), you'll need to change the $0=="10" in the awk command to $0=="37".
This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Remove newline character from column spread over multiple lines in a file

Hi, I came across one issue recently where output from one of the columns of the table from where i am creating input file has newline characters hence, record in the file is spread over multiple lines. Fields in the file are separated by pipe (|) delimiter. As header will never have newline... (4 Replies)
Discussion started by: Prathmesh
4 Replies

2. Shell Programming and Scripting

How to remove newline character if it is the only character in the entire file.?

I have a file which comes every day and the file data look's as below. Vi abc.txt a|b|c|d\n a|g|h|j\n Some times we receive the file with only a new line character in the file like vi abc.txt \n (8 Replies)
Discussion started by: rak Kundra
8 Replies

3. Shell Programming and Scripting

Remove newline character between two delimiters

hi i am having delimited .dat file having content like below. test.dat(5 line of records) ====== PT2~Stag~Pt2 Stag Test. Updated~PT2 S T~Area~~UNCEF R20~~2012-05-24 ~2014-05-24~~ PT2~Stag y~Pt2 Stag Test. Updated~PT2 S T~Area~METR~~~2012-05-24~2014-05-24~~test PT2~Pt2 Stag Test~~PT2 S... (4 Replies)
Discussion started by: sushine11
4 Replies

4. Shell Programming and Scripting

Remove newline character or join the broken record

Hi, I have a very huge file, around 1GB of data. I want to remove the newline characters in the file but not preceded by the original end delimiter {} sample data will look like this 1234567 abcd{} 1234sssss as67 abcd{} 12dsad3dad 4sdad567 abcdsadd{} this should look like this... (6 Replies)
Discussion started by: ratheeshjulk
6 Replies

5. Shell Programming and Scripting

Remove \n <newline> character inside the records.

Hi, In my file, I have '\n' characters inside a single record. Because of this, a single records appears in many lines and looks like multiple records. In the below file. File 1 ==== 1,nmae,lctn,da\n t 2,ghjik,o\n ut,de\n fk Expected output after the \n removed File 2 =====... (5 Replies)
Discussion started by: machomaddy
5 Replies

6. Shell Programming and Scripting

any savant ? using AWK/SED to remove newline character between two strings : conditional removal

I'd like to remove (do a pattern or precise replacement - this I can handle in SED using Regex ) ---AFTER THE 1ST Occurrence ( i.e. on the 2nd occurrence - from the 2nd to fourth occurance ) of a specific string : type 1 -- After the 1st occurrence of 1 string1 till the 1st occurrence of... (4 Replies)
Discussion started by: sieger007
4 Replies

7. Shell Programming and Scripting

Remove newline character conditionally

Hi All, I have 5000 records like this Request_id|Type|Status|Priority|Ticket Submitted Date and Time|Actual Resolved Date and Time|Current Ticket Owner Group|Case final Ticket Owner Group|Customer Severity|Reported Symptom/Request|Component|Hot Topic|Reason for Missed SLA|Current Ticket... (2 Replies)
Discussion started by: j_53933
2 Replies

8. Shell Programming and Scripting

To remove the newline character while appending into a file

Hi All, We append the output of a file's size in a file. But a newline character is appended after the variable. Pls help how to clear this. filesize=`ls -l test.txt | awk `{print $5}'` echo File size of test.txt is $filesize bytes >> logfile.txt The output we got is, File size of... (4 Replies)
Discussion started by: amio
4 Replies

9. Shell Programming and Scripting

How to remove a newline character at the end of filename

Hi All, I have named a file with current date,time and year as follows: month=`date | awk '{print $2}'` date=`date | awk '{print $3}'` year=`date | awk '{print $6}'` time=`date +%Hh_%Mm_%Ss'` filename="test_"$month"_"$date"_"$year"_"$time".txt" > $filename The file is created with a... (2 Replies)
Discussion started by: amio
2 Replies

10. UNIX for Dummies Questions & Answers

newline character

hi, I want to print the below lines "Message from bac logistics The Confirmation File has not been received." When i give like this in the code "Message from bac logistics\n The Confirmation File has not been received." It is giving only Message from bac logistics\n The... (9 Replies)
Discussion started by: trichyselva
9 Replies
Login or Register to Ask a Question