Perl for comparing numbers from previous lines in a file?
Hi everyone
I have a question for you, as I am trying to learn more about Perl and work with some weather data. I have an ascii file (shown below) that has 10 lines with different columns. What I would like is have Perl find an "anomalous" value by comparing a field with the values from the last 3 lines (for the same field, of course). For instance, if we look at the data below:
We see here that on the 8th line, the value in column 4 is "off" with respect to the previous lines (all of which are approx. 27.3 degrees Celsius).
Thus, how can I do to compare the values for a line in column 4 with the values from the previous 3 lines in order to detect these "bad" data values? Assume that any value that dirfts +/- than 0.5deg C is "bad data"
I am not sure how to proceed. Do I need to load the data into an array? Do I need to read the file line by line? Or maybe something else?
Any help, suggestions, and especially examples would be really helpful.
Thanks in advance
Sorry... I forgot to mention that this is what I have in my script so far:
Last edited by Scott; 11-21-2013 at 10:17 PM..
Reason: Code tags for data too
Hello, I'm searching for a quick method to read numeric values from a file or a defined variable and identifying the largest number. For instance if the following numbers are in a file or defined to a variable:
09192007 09202007 09182007 09172007 09162007
What "short" method could be used to... (7 Replies)
I have a text file that has blocks of text. Each block starts with ### and ends with End_###.
I wrote a perl script to search a string from line 2 (ignore any line starts with ###) of each block
if matched, need to print that whole block. According to the input file in below, it will print... (5 Replies)
Hi,
I am new to ksh scripting and I have a problem.
I have a file in which I have to search for a particular pattern say 'a' then from that line I need to search for another pattern say 'b' in the previous lines and thne print the file from pattern 'b' till the end of file.
For eg:
... (2 Replies)
i have 2 files and i want to compare
i currently cat the files and awk print $1, $2 and doing if file1=file2 then fail, else exit 0
what i want to do is compare values, with column 1 being a reference i want to compare line by line and then still be able to do if then statement to see if worked... (1 Reply)
Hi,
I am trying to locate the occurences of certain pattern like 'Possible network disconnect' in a text file. I can get the actual lines matching the pttern using:
grep -w 'Possible network disconnect' file_name.
But I am more interested in getting the timing of these events which are... (7 Replies)
Hi,
I am currently trying to work out how to compare one line with the last line I have read in via ksh. I have a file which has sorted output from a previous sort command so all the lines are in order already and the file would look something like show below. Each line has a name and a time... (5 Replies)
I have files called printfile.log.1121, printfile.log.1122, etc where 1121 and 1122 are the date of the file creation.
The file contents are like:
================================================================================
User = d_prod
Env = d_prod
Dir =... (3 Replies)
Hi all,
I have a text data file. My aim here is to find line called *FIELD* AV for every record and print lines after that till *FIELD* RF. But here I want first 3 to four lines for very record as well. FIELD AV is some where in between for very record. SO I am not sure how to retrieve lines in... (2 Replies)
I want to remove commands having no output. In below text file.
bash-3.2$ cat abc_do_it.txt
grpg10so>show trunk group all status
grpg11so>show trunk group all status
grpg12so>show trunk group all status
GCPKNYAIGT73IMO 1440 1345 0 0 94 0 0 INSERVICE 93% 0%... (4 Replies)
Hi All,
I have two files in the following format, with numbers being defined under columns(described by a set of headers) and rows(again defined by a set of identifiers)
2013 2013
Make200 Make201
Merc BMW
Jpur Del
... (9 Replies)
Discussion started by: dev.devil.1983
9 Replies
LEARN ABOUT NETBSD
rs
RS(1) BSD General Commands Manual RS(1)NAME
rs -- reshape a data array
SYNOPSIS
rs [-CcSs [x]] [-GgKkw N] [-EeHhjmnTty] [rows [cols]]
DESCRIPTION
rs reads the standard input, interpreting each line as a row of blank-separated entries in an array, transforms the array according to the
options, and writes it on the standard output. With no arguments it transforms stream input into a columnar format convenient for terminal
viewing.
The shape of the input array is deduced from the number of lines and the number of columns on the first line. If that shape is inconvenient,
a more useful one might be obtained by skipping some of the input with the -k option. Other options control interpretation of the input col-
umns.
The shape of the output array is influenced by the rows and cols specifications, which should be positive integers. If only one of them is a
positive integer, rs computes a value for the other which will accommodate all of the data. When necessary, missing data are supplied in a
manner specified by the options and surplus data are deleted. There are options to control presentation of the output columns, including
transposition of the rows and columns.
The options are described below.
-C [x] Output columns are delimited by the single character x. A missing x is taken to be '^I'.
-c [x] Input columns are delimited by the single character x. A missing x is taken to be '^I'.
-e Consider each line of input as an array entry.
-G N The gutter width (inter-column space) has N percent of the maximum column width added to it.
-g N The gutter width (inter-column space), normally 2, is taken to be N.
-H Like -h, but also print the length of each line.
-h Print the shape of the input array and do nothing else. The shape is just the number of lines and the number of entries on the
first line.
-j Right adjust entries within columns.
-K N Like -k, but print the ignored lines.
-k N Ignore the first N lines of input.
-m Do not trim excess delimiters from the ends of the output array.
-n On lines having fewer entries than the first line, use null entries to pad out the line. Normally, missing entries are taken
from the next line of input.
-S [x] Like -C, but padded strings of x are delimiters.
-s [x] Like -c, but maximal strings of x are delimiters.
-T Print the pure transpose of the input, ignoring any rows or cols specification.
-t Fill in the rows of the output array using the columns of the input array, that is, transpose the input while honoring any rows
and cols specifications.
-w N The width of the display, normally 80, is taken to be the positive integer N.
-y If there are too few entries to make up the output dimensions, pad the output by recycling the input from the beginning. Nor-
mally, the output is padded with blanks.
-z Adapt column widths to fit the largest entries appearing in them.
With no arguments, rs transposes its input, and assumes one array entry per input line unless the first non-ignored line is longer than the
display width. Option letters which take numerical arguments interpret a missing number as zero unless otherwise indicated.
EXAMPLES
rs can be used as a filter to convert the stream output of certain programs (e.g., spell(1), du(1), file(1), look(1), nm(1), who(1), and
wc(1)) into a convenient ``window'' format, as in
who | rs
This function has been incorporated into the ls(1) program, though for most programs with similar output rs suffices.
To convert stream input into vector output and back again, use
rs 1 0 | rs 0 1
A 10 by 10 array of random numbers from 1 to 100 and its transpose can be generated with
jot -r 100 | rs 10 10 | tee array | rs -T > tarray
In the editor vi(1), a file consisting of a multi-line vector with 9 elements per line can undergo insertions and deletions, and then be
neatly reshaped into 9 columns with
:1,$!rs 0 9
Finally, to sort a database by the first line of each 4-line field, try
rs -eC 0 4 | sort | rs -c 0 1
SEE ALSO jot(1), pr(1), sort(1), vi(1)BUGS
Handles only two dimensional arrays.
The algorithm currently reads the whole file into memory, so files that do not fit in memory will not be reshaped.
Fields cannot be defined yet on character positions.
Re-ordering of columns is not yet possible.
There are too many options.
BSD December 18, 2001 BSD