Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Remove or truncate trailing nulls from file Post 302975331 by RudiC on Saturday 11th of June 2016 06:20:49 AM
Old 06-11-2016
It looks like (even though we are not dealing with *nix text files due to missing <NL> char at the end) GNU sed could do it, not with octal constants, but with hex constants:
Code:
sed ':L;s/\000$//;tL' XX | hd
00000000  73 64 66 65 66 65 65 72  76 30 30 09 00 00 00 00  |sdfefeerv00.....|
00000010  00 00 00 00 00 00 00 00  00 00 00                 |...........|
sed ':L;s/\x00$//;tL' XX | hd
00000000  73 64 66 65 66 65 65 72  76 30 30 09              |sdfefeerv00.|

FreeBSD's sed doesn't work like above.
These 2 Users Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to remove trailing spaces

Hi, I have a file like this (ADD_MONTHS((Substr(Trim(BOTH FROM Translate(Maximum(closeDa ------------------------------------------------------------ 2007-06-30 00:00:00 I have a requirement where i need just the date. When i do: tail -1... (2 Replies)
Discussion started by: mahek_bedi
2 Replies

2. Shell Programming and Scripting

Remove trailing G

Hello, I am trying to write a script that will calculate the amount of data remaining in a storage volume. I'm running Tru64 Unix version 5.1B patch kit 6. The script is being run against an AdvFS domain. I am programming in Korn Shell version M-11/16/88f. The basic idea is that I want to run df... (3 Replies)
Discussion started by: Heathe_Kyle
3 Replies

3. UNIX for Dummies Questions & Answers

I don't want to truncate trailing spaces and ^M at the end of line

I have a script wherein I access each line of the file using a FOR loop and then perform some operations in each line. The problem is each line that gets extracted in FOR loop truncates trailing blank spaces and control characters (^M) that is present at the end of each line. I don't wan this to... (5 Replies)
Discussion started by: Shobana_s
5 Replies

4. Shell Programming and Scripting

Remove trailing spaces from file

I'm currently writing my sql results to a file and they have trailing spaces after each field. I want to get rid of these spaces and I'm using this code: TVXTEMP=$(echo $TVXTEMP|sed -e 's/\ //g') It doesn't work though. I'm not familiar with sedscript, and the other codes I've found online... (6 Replies)
Discussion started by: avillanueva
6 Replies

5. Shell Programming and Scripting

Remove trailing zeros

Hi I have a simple request but can't find the answer. I want to remove trailing zeros, and in some cases the fullstops, from the input data. Example of input file: FR002_15.000_20.000 SD475_5.000_10.500 FG5647_12.250_15.500 BH2463_30.555_32.000 Desired output file would be: ... (10 Replies)
Discussion started by: theflamingmoe
10 Replies

6. Shell Programming and Scripting

Remove trailing 0 from the field

Hi Freinds, I have file1.txt as below file1.txt 1521894~~-0.400~201207 1521794~~-0.486~201207 152494~~-0.490~201207 152154894~~-0.490~201207 1521894354~~-0.489~201207 expected output : 1521894~~-0.4~201207 1521794~~-0.486~201207 152494~~-0.49~201207... (9 Replies)
Discussion started by: i150371485
9 Replies

7. Shell Programming and Scripting

Remove trailing number

I have some strings such as ABC1 ABC2 TYFASDD12 They will only have letters and numbers. In each case I want to remove the last digit? The lengths will vary. So a hard coded substr won't work. What do I do? if it doesn't end in a number, I don't want to remove any characters. (6 Replies)
Discussion started by: guessingo
6 Replies

8. Shell Programming and Scripting

Remove trailing space from file and folder names

I have a folder that contains many sub folders and files. This tree has to be backed up to an archive system. According to the tech support, one of the archives is failing to back up due to the possibility of trailing spaces on file and folder names. Therefore, I would like to have a script... (16 Replies)
Discussion started by: vipertech
16 Replies

9. Shell Programming and Scripting

Remove the leading and trailing date from a CSV file

I'm a newbie to shell scripting. Can anyone help with the below requirement ? The leading and trailing date of a files to be removed. 2017-07-12_gmr_tag_log_20170711.csv 2017-07-12_gmr_call_log_20170711.csv 2017-07-12_gmr_outgoing_log_20170711.csv I'm looking for output like... (7 Replies)
Discussion started by: shivamayam
7 Replies

10. Shell Programming and Scripting

Remove leading and trailing spaces from a file

Hi, I am trying to remove leading and trailing spaces from a file using awk but somehow I have not been able to do it. Here is the data that I want to trim. 07/12/2017 15:55:00 |entinfdev |AD ping Time ms | .474| 1.41| .581|green |flat... (9 Replies)
Discussion started by: svajhala
9 Replies
funhist(1)							SAORD Documentation							funhist(1)

NAME
funhist - create a 1D histogram of a column (from a FITS binary table or raw event file) or an image SYNOPSIS
funhist [-n|-w|-T] <iname> [column] [[lo:hi:]bins] OPTIONS
-n # normalize bin value by the width of each bin -w # specify bin width instead of number of bins in arg3 -T # output in rdb/starbase format (tab separators) DESCRIPTION
funhist creates a one-dimensional histogram from the specified columns of a FITS Extension binary table of a FITS file (or from a non-FITS raw event file), or from a FITS image or array, and writes that histogram as an ASCII table. Alternatively, the program can perform a 1D projection of one of the image axes. The first argument to the program is required, and specifies the Funtools file: FITS table or image, raw event file, or array. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions, and filters. For a table, the second argument also is required. It specifies the column to use in generating the histogram. If the data file is of type image (or array), the column is optional: if "x" (or "X"), "y" (or "Y") is specified, then a projection is performed over the x (dim1) or y (dim2) axes, respectively. (That is, this projection will give the same results as a histogram performed on a table containing the equiva- lent x,y event rows.) If no column name is specified or "xy" (or "XY") is specified for the image, then a histogram is performed on the values contained in the image pixels. The argument that follows is optional and specifies the number of bins to use in creating the histogram and, if desired, the range of bin values. For image and table histograms, the range should specify the min and max data values. For image histograms on the x and y axes, the range should specify the min and max image bin values. If this argument is omitted, the number of output bins for a table is calcu- lated either from the TLMIN/TLMAX headers values (if these exist in the table FITS header for the specified column) or by going through the data to calculate the min and max value. For an image, the number of output bins is calculated either from the DATAMIN/DATAMAX header val- ues, or by going through the data to calculate min and max value. (Note that this latter calculation might fail if the image cannot be fit in memory.) If the data are floating point (table or image) and the number of bins is not specified, an arbitrary default of 128 is used. For binary table processing, the -w (bin width) switch can be used to specify the width of each bin rather than the number of bins. Thus: funhist test.ev pha 1:100:5 means that 5 bins of width 20 are used in the histogram, while: funhist -w test.ev pha 1:100:5 means that 20 bins of width 5 are used in the histogram. The data are divvied up into the specified number of bins and the resulting 1D histogram (or projection) is output in ASCII table format. For a table, the output displays the low_edge (inclusive) and hi_edge (exclusive) values for the data. For example, a 15-row table con- taining a "pha" column whose values range from -7.5 to 7.5 can be processed thus: [sh] funhist test.ev pha # data file: /home/eric/data/test.ev # column: pha # min,max,bins: -7.5 7.5 15 bin value lo_edge hi_edge ------ --------- --------------------- --------------------- 1 22 -7.50000000 -6.50000000 2 21 -6.50000000 -5.50000000 3 20 -5.50000000 -4.50000000 4 19 -4.50000000 -3.50000000 5 18 -3.50000000 -2.50000000 6 17 -2.50000000 -1.50000000 7 16 -1.50000000 -0.50000000 8 30 -0.50000000 0.50000000 9 16 0.50000000 1.50000000 10 17 1.50000000 2.50000000 11 18 2.50000000 3.50000000 12 19 3.50000000 4.50000000 13 20 4.50000000 5.50000000 14 21 5.50000000 6.50000000 15 22 6.50000000 7.50000000 [sh] funhist test.ev pha 1:6 # data file: /home/eric/data/test.ev # column: pha # min,max,bins: 0.5 6.5 6 bin value lo_edge hi_edge ------ --------- --------------------- --------------------- 1 16 0.50000000 1.50000000 2 17 1.50000000 2.50000000 3 18 2.50000000 3.50000000 4 19 3.50000000 4.50000000 5 20 4.50000000 5.50000000 6 21 5.50000000 6.50000000 [sh] funhist test.ev pha 1:6:3 # data file: /home/eric/data/test.ev # column: pha # min,max,bins: 0.5 6.5 3 bin value lo_edge hi_edge ------ --------- --------------------- --------------------- 1 33 0.50000000 2.50000000 2 37 2.50000000 4.50000000 3 41 4.50000000 6.50000000 For a table histogram, the -n(normalize) switch can be used to normalize the bin value by the width of the bin (i.e., hi_edge-lo_edge): [sh] funhist -n test.ev pha 1:6:3 # data file: test.ev # column: pha # min,max,bins: 0.5 6.5 3 # width normalization (val/(hi_edge-lo_edge)) is applied bin value lo_edge hi_edge ------ --------------------- --------------------- --------------------- 1 16.50000000 0.50000000 2.50000000 2 6.16666667 2.50000000 4.50000000 3 4.10000000 4.50000000 6.50000000 This could used, for example, to produce a light curve with values having units of counts/second instead of counts. For an image histogram, the output displays the low and high image values (both inclusive) used to generate the histogram. For example, in the following example, 184 pixels had a value of 1, 31 had a value of 2, while only 2 had a value of 3,4,5,6, or 7: [sh] funhist test.fits # data file: /home/eric/data/test.fits # min,max,bins: 1 7 7 bin value lo_val hi_val ------ --------------------- --------------------- --------------------- 1 184.00000000 1.00000000 1.00000000 2 31.00000000 2.00000000 2.00000000 3 2.00000000 3.00000000 3.00000000 4 2.00000000 4.00000000 4.00000000 5 2.00000000 5.00000000 5.00000000 6 2.00000000 6.00000000 6.00000000 7 2.00000000 7.00000000 7.00000000 For the axis projection of an image, the output displays the low and high image bins (both inclusive) used to generate the projection. For example, in the following example, 21 counts had their X bin value of 2, etc.: [sh] funhist test.fits x 2:7 # data file: /home/eric/data/test.fits # column: X # min,max,bins: 2 7 6 bin value lo_bin hi_bin ------ --------------------- --------------------- --------------------- 1 21.00000000 2.00000000 2.00000000 2 20.00000000 3.00000000 3.00000000 3 19.00000000 4.00000000 4.00000000 4 18.00000000 5.00000000 5.00000000 5 17.00000000 6.00000000 6.00000000 6 16.00000000 7.00000000 7.00000000 [sh] funhist test.fits x 2:7:2 # data file: /home/eric/data/test.fits # column: X # min,max,bins: 2 7 2 bin value lo_bin hi_bin ------ --------------------- --------------------- --------------------- 1 60.00000000 2.00000000 4.00000000 2 51.00000000 5.00000000 7.00000000 You can use gnuplot or other plotting programs to graph the results, using a script such as: #!/bin/sh sed -e '1,/---- .*/d /^$/,$d' | awk ' BEGIN{print "set nokey; set title "funhist"; set xlabel "bin"; set ylabel "counts"; plot "-" with boxes"} {print $3, $2, $4-$3}' | gnuplot -persist - 1>/dev/null 2>&1 Similar plot commands are supplied in the script funhist.plot: funhist test.ev pha ... | funhist.plot gnuplot SEE ALSO
See funtools(7) for a list of Funtools help pages version 1.4.2 January 2, 2008 funhist(1)
All times are GMT -4. The time now is 10:29 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy