Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Remove or truncate trailing nulls from file Post 302975361 by Tribe on Saturday 11th of June 2016 02:02:22 PM
Old 06-11-2016
Quote:
Originally Posted by RudiC
It looks like (even though we are not dealingImage with *nix text files due to missing <NL> char at the end) GNU sed could do it, not with octal constants, but with hex constants:
Code:
sed ':L;s/\000$//;tL' XX | hd

No idea what can be the cause, but with large files it doesn't remove all the trailing null characters.

A file of 3239571456 bytes resulted in 2902450247 with dtn.c
The same file resulted in 3239570892 bytes using sed on ubuntu 64 bits.
The same file resulted in 3239457516 bytes using sed on cygwin 32 bits.

I've compared the original file with the created with dtn.c using HexCmp and the result is the wanted, while using sed is not.
Quote:
Originally Posted by Don Cragun
Not only are non-empty POSIX text files required to end with a <newline> character, they are not allowed to contain any NUL bytes either.

If GNU sed works with:
Code:
sed ':L;s/\x00$//;tL' file

does it also work with:
Code:
sed 's/\x00*$//' file

???
This doesn't work. This damages completely the file.

Last edited by Tribe; 06-11-2016 at 03:17 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to remove trailing spaces

Hi, I have a file like this (ADD_MONTHS((Substr(Trim(BOTH FROM Translate(Maximum(closeDa ------------------------------------------------------------ 2007-06-30 00:00:00 I have a requirement where i need just the date. When i do: tail -1... (2 Replies)
Discussion started by: mahek_bedi
2 Replies

2. Shell Programming and Scripting

Remove trailing G

Hello, I am trying to write a script that will calculate the amount of data remaining in a storage volume. I'm running Tru64 Unix version 5.1B patch kit 6. The script is being run against an AdvFS domain. I am programming in Korn Shell version M-11/16/88f. The basic idea is that I want to run df... (3 Replies)
Discussion started by: Heathe_Kyle
3 Replies

3. UNIX for Dummies Questions & Answers

I don't want to truncate trailing spaces and ^M at the end of line

I have a script wherein I access each line of the file using a FOR loop and then perform some operations in each line. The problem is each line that gets extracted in FOR loop truncates trailing blank spaces and control characters (^M) that is present at the end of each line. I don't wan this to... (5 Replies)
Discussion started by: Shobana_s
5 Replies

4. Shell Programming and Scripting

Remove trailing spaces from file

I'm currently writing my sql results to a file and they have trailing spaces after each field. I want to get rid of these spaces and I'm using this code: TVXTEMP=$(echo $TVXTEMP|sed -e 's/\ //g') It doesn't work though. I'm not familiar with sedscript, and the other codes I've found online... (6 Replies)
Discussion started by: avillanueva
6 Replies

5. Shell Programming and Scripting

Remove trailing zeros

Hi I have a simple request but can't find the answer. I want to remove trailing zeros, and in some cases the fullstops, from the input data. Example of input file: FR002_15.000_20.000 SD475_5.000_10.500 FG5647_12.250_15.500 BH2463_30.555_32.000 Desired output file would be: ... (10 Replies)
Discussion started by: theflamingmoe
10 Replies

6. Shell Programming and Scripting

Remove trailing 0 from the field

Hi Freinds, I have file1.txt as below file1.txt 1521894~~-0.400~201207 1521794~~-0.486~201207 152494~~-0.490~201207 152154894~~-0.490~201207 1521894354~~-0.489~201207 expected output : 1521894~~-0.4~201207 1521794~~-0.486~201207 152494~~-0.49~201207... (9 Replies)
Discussion started by: i150371485
9 Replies

7. Shell Programming and Scripting

Remove trailing number

I have some strings such as ABC1 ABC2 TYFASDD12 They will only have letters and numbers. In each case I want to remove the last digit? The lengths will vary. So a hard coded substr won't work. What do I do? if it doesn't end in a number, I don't want to remove any characters. (6 Replies)
Discussion started by: guessingo
6 Replies

8. Shell Programming and Scripting

Remove trailing space from file and folder names

I have a folder that contains many sub folders and files. This tree has to be backed up to an archive system. According to the tech support, one of the archives is failing to back up due to the possibility of trailing spaces on file and folder names. Therefore, I would like to have a script... (16 Replies)
Discussion started by: vipertech
16 Replies

9. Shell Programming and Scripting

Remove the leading and trailing date from a CSV file

I'm a newbie to shell scripting. Can anyone help with the below requirement ? The leading and trailing date of a files to be removed. 2017-07-12_gmr_tag_log_20170711.csv 2017-07-12_gmr_call_log_20170711.csv 2017-07-12_gmr_outgoing_log_20170711.csv I'm looking for output like... (7 Replies)
Discussion started by: shivamayam
7 Replies

10. Shell Programming and Scripting

Remove leading and trailing spaces from a file

Hi, I am trying to remove leading and trailing spaces from a file using awk but somehow I have not been able to do it. Here is the data that I want to trim. 07/12/2017 15:55:00 |entinfdev |AD ping Time ms | .474| 1.41| .581|green |flat... (9 Replies)
Discussion started by: svajhala
9 Replies
TRS(1)								Linux User's Manual							    TRS(1)

NAME
trs - filter replacing strings SYNOPSIS
trs [-[r]e] 'REPLACE_THIS WITH_THAT [AND_THIS WITH_THAT]...' trs [-[r]f] FILE DESCRIPTION
Copy stdin to stdout replacing every occurence of given strings with other ones. This is similar to tr(1), but replaces strings, not only single chars. Rules (separated by whitespace) can be given directly after -e option, or can be read from FILE. Argument not preceded by -e or -f is guessed to be a script when it contains some whitespace, or a filename otherwise. Comments are allowed from # until the end of line. The character # in strings must be specified as #. Standard C-like escapes a  e f v \ nn are recognized. In addition, s means a space character and ! means an empty string. Sets of acceptable characters at a given position can be specified between [ and ]. ASCII ranges in sets can be shortly written as FIRST-LAST. When a set consists of only a single range, [ and ] can be omitted. When a part of the string to translate is enclosed in {...}, only that part is replaced. Any text outside {...} serves as an assertion: a string is translated only if it is preceded by the given text and followed by another one. { at the beginning or } at the end of the string can be omitted. Text outside {...} is treated as untranslated. Before the beginning of the file and after its end there are only 's. Thus, for example, {.} matches . on a line by itself, including the first line, and the last one even without the marker. A fragment of the form ?x=N, where x is a letter A-Za-z and N is a digit 0-9, contained in the target text sets the variable x to the value N when that rule succeeds. Similar fragment in the source text causes the given rule to be considered only if that variable has such value. Initially all variables have the value of 0. Several assignments or conditions can be present in one rule - they are ANDed together. OPTIONS -e Give the translation rules directly in the command line. -f Get them from the file specified. -r Reverse every rule. This affects only the next -e or -f option. Of course this doesn't have to give the reverse translation! Any rule containing any of {}[]{}- is taken in only one direction. You may force any rule to be taken in only one direction by enclosing the string to translate in {...}. --help display help and exit --version output version information and exit Multiple -e or -f options are allowed. All rules are loaded together then, and earlier ones have precedence. EXAMPLE
$ echo Leeloo |trs -e 'el n e i i aqq o} x o u' Linux DIFFERENCES FROM sed The main difference between trs and sed 's///g; ...' (excluding sed's regular expressions) is that sed takes every rule in the order speci- fied and applies it to the whole line of translated file, whereas trs examines every position and tries all rules in this place first. In sed every next rule is fed with the text produced by the previous one, whereas in trs every piece of text can be translated at most once (if more than one rule matches at a given position, the one mentioned earlier wins). That's why sed isn't well suited for translating between character sets. On the other hand, tr translates only single bytes, so it can't be used for Unicode conversions, or TeX / SGML ways for specifying extended characters. Another example: $ echo 642 |trs -e '4 7 72 66 64 4' 42 $ echo 642 |sed 's/4/7/g; s/72/66/g; s/64/4/g' 666 The string to replace can be empty; there must be something outside {} then. In this special case only one such create-from-nothing rule can success at a given position. For example, }x80-xFF @ precedes every character with high byte set with @. The rule of the form some{ thing doesn't work at the end of a file. SEE ALSO
tr(1), konwert(1) COPYRIGHT
trs is a filter replacing strings. It forms part of the konwert package. Copyright (c) 1998 Marcin 'Qrczak' Kowalczyk This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MER- CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA AUTHOR
__("< Marcin Kowalczyk * qrczak@knm.org.pl http://qrczak.home.ml.org/ \__/ GCS/M d- s+:-- a21 C+++>+++$ UL++>++++$ P+++ L++>++++$ E->++ ^^ W++ N+++ o? K? w(---) O? M- V? PS-- PE++ Y? PGP->+ t QRCZAK 5? X- R tv-- b+>++ DI D- G+ e>++++ h! r--%>++ y- Konwert 12 Jul 1998 TRS(1)
All times are GMT -4. The time now is 10:44 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy