Sponsored Content
Full Discussion: Reducing file lines in awk
Top Forums Shell Programming and Scripting Reducing file lines in awk Post 302574981 by agama on Saturday 19th of November 2011 11:50:56 AM
Old 11-19-2011
Quote:
Originally Posted by vasanth.vadalur
Expected output
Code:
1 2 3 4    
1.275 3 1.925 3 
1.275 3 1.225 3.025

But output got is
Code:
1 2 3 4
1.275 3 1.325 3       
1.275 3 1.225 3.025
1.325 3 1.575 3       
1.625 3 1.575 3
1.625 3 1.925 3

Still repeats are there.

Where went wrong...
Well, actually it didn't go wrong. Your original post indicated that only sequential lines in the file need to be tested, and I inferred that the 'new line' was to be matched against the next line in the file if there was a match. The programme is doing exactly this and the output you see is expected given those parameters.

Thinking on the minimum/maximum redefinition of the problem.

---------- Post updated at 11:50 ---------- Previous update was at 11:30 ----------

I'm not as confident in this as I don't know what combinations fields 2 and 4 might take. I've made an assumption based on your example and this does work for it, but there might be other unexpected results. Have a go with this and see how it does:

Code:
awk '
    {
        idx = $2 "," $4;
        if( min[idx] == ""  ||  min[idx] > $1+0 ) 
             min[idx] = $1+0; 
        if( max[idx] == "" || max[idx] < $3+0 ) 
             max[idx] = $3+0; 
    }

    END {
        for( x in min )
        {
            split( x, a, "," );
            printf( "%.3f %.3f %.3f %.3f\n", min[x], a[1], max[x], a[2] );
        }
    }
'  input-file


Last edited by agama; 11-19-2011 at 12:30 PM.. Reason: typo
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Help with splitting lines in a file using awk

I have a file which is one big long line of text about 10Kb long. Can someone provide a way using awk to introduce carriage returns every 40 chars in this file. Any other solutions would also be welcome. Thank you in advance. (5 Replies)
Discussion started by: martinbarretto
5 Replies

2. UNIX for Dummies Questions & Answers

Reducing file names

I have a script which includes an FTP. The filename is too long for my target area. The filename is HD012_ABCD_EFGH_061004_F_300_40. I need to the filename to be HD012_ABCD_EFGH_061004_F_. Any ideas. (5 Replies)
Discussion started by: paul1s
5 Replies

3. Solaris

reducing to root file size

My root file size has reached 80% and I am looking where all i can reduce the file size . Here is the output of top directories in / . To me none of this looks useful but not sure . We use an appplication and email. Which all can be deleted . Please advise . 2016989 989445 /var 930059 ... (2 Replies)
Discussion started by: Hitesh Shah
2 Replies

4. Shell Programming and Scripting

Select some lines from a txt file and create a new file with awk

Hi there, I have a text file with several colums separated by "|;#" I need to search the file extracting all columns starting with the value of "1" or "2" saving in a separate file just the first 7 columns of each row maching the criteria, with replacement of the saparators in the nearly created... (4 Replies)
Discussion started by: capnino
4 Replies

5. Shell Programming and Scripting

Reducing text file using similar lines

Hello, I am a java programmer but want to try unix for a purpose where I need to reduce a file using its first field.. Here is the sample data: admin;2;0;; admission;8;0;; aman;1;0;; caroline;0;4;; cook;0;4;; cook;2;0;; far;0;3;; far;1;5;; I am explaining the dataset first. There... (5 Replies)
Discussion started by: shekhar2010us
5 Replies

6. Shell Programming and Scripting

Counting lines in a file using awk

I want to count lines of a file using AWK (only) and not in the END part like this awk 'END{print FNR}' because I want to use it. Does anyone know of a way? Thanks a lot. (7 Replies)
Discussion started by: guitarist684
7 Replies

7. Shell Programming and Scripting

Reducing the decimal points of numbers (3d coordinates) in a file; how to input data to e.g. Python

I have a file full of coordinates of the form: 37.68899917602539 58.07500076293945 57.79100036621094 The numbers don't always have the same number of decimal points. I need to reduce the decimal points of all the numbers (there are 128 rows of 3 numbers) to 2. I have tried to do this... (2 Replies)
Discussion started by: crunchgargoyle
2 Replies

8. Shell Programming and Scripting

awk last n lines of file

Just my second week working on awk I need a hint for the following tasks. I want to limit my logfile from the very outset to 200 lines. All I do until now is head -c 10K >> /home/uplog.txt | awk 'END{print NR " swap " NF$5; exit}' /home/uplog.txt; After being read it shall print the very... (27 Replies)
Discussion started by: 1in10
27 Replies

9. Shell Programming and Scripting

awk remove/grab lines from file with pattern from other file

Sorry for the weird title but i have the following problem. We have several files which have between 10000 and about 500000 lines in them. From these files we want to remove lines which contain a pattern which is located in another file (around 20000 lines, all EAN codes). We also want to get... (28 Replies)
Discussion started by: SDohmen
28 Replies

10. UNIX for Beginners Questions & Answers

Reducing input file size after pattern search

I have a very large file with millions of entries identified by @M. I am using the following script to "extract" entries based on specific strings/patterns: #!/bin/bash if ] then file=$1 else echo "Input_file passed as an argument $1 is NOT found." exit; fi MID=(NULL "string-1"... (10 Replies)
Discussion started by: Xterra
10 Replies
XML::SAX::ByRecord(3pm) 				User Contributed Perl Documentation				   XML::SAX::ByRecord(3pm)

NAME
XML::SAX::ByRecord - Record oriented processing of (data) documents SYNOPSIS
use XML::SAX::Machines qw( ByRecord ) ; my $m = ByRecord( "My::RecordFilter1", "My::RecordFilter2", ... { Handler => $h, ## optional } ); $m->parse_uri( "foo.xml" ); DESCRIPTION
XML::SAX::ByRecord is a SAX machine that treats a document as a series of records. Everything before and after the records is emitted as- is while the records are excerpted in to little mini-documents and run one at a time through the filter pipeline contained in ByRecord. The output is a document that has the same exact things before, after, and between the records that the input document did, but which has run each record through a filter. So if a document has 10 records in it, the per-record filter pipeline will see 10 sets of ( start_document, body of record, end_document ) events. An example is below. This has several use cases: o Big, record oriented documents Big documents can be treated a record at a time with various DOM oriented processors like XML::Filter::XSLT. o Streaming XML Small sections of an XML stream can be run through a document processor without holding up the stream. o Record oriented style sheets / processors Sometimes it's just plain easier to write a style sheet or SAX filter that applies to a single record at at time, rather than having to run through a series of records. Topology Here's how the innards look: +-----------------------------------------------------------+ | An XML:SAX::ByRecord | | Intake | | +----------+ +---------+ +--------+ Exhaust | --+-->| Splitter |--->| Stage_1 |-->...-->| Merger |----------+-----> | +----------+ +---------+ +--------+ | | ^ | | | | | +---------->---------------+ | | Events not in any records | | | +-----------------------------------------------------------+ The "Splitter" is an XML::Filter::DocSplitter by default, and the "Merger" is an XML::Filter::Merger by default. The line that bypasses the "Stage_1 ..." filter pipeline is used for all events that do not occur in a record. All events that occur in a record pass through the filter pipeline. Example Here's a quick little filter to uppercase text content: package My::Filter::Uc; use vars qw( @ISA ); @ISA = qw( XML::SAX::Base ); use XML::SAX::Base; sub characters { my $self = shift; my ( $data ) = @_; $data->{Data} = uc $data->{Data}; $self->SUPER::characters( @_ ); } And here's a little machine that uses it: $m = Pipeline( ByRecord( "My::Filter::Uc" ), $out, ); When fed a document like: <root> a <rec>b</rec> c <rec>d</rec> e <rec>f</rec> g </root> the output looks like: <root> a <rec>B</rec> c <rec>C</rec> e <rec>D</rec> g </root> and the My::Filter::Uc got three sets of events like: start_document start_element: <rec> characters: 'b' end_element: </rec> end_document start_document start_element: <rec> characters: 'd' end_element: </rec> end_document start_document start_element: <rec> characters: 'f' end_element: </rec> end_document METHODS
new my $d = XML::SAX::ByRecord->new( @channels, \%options ); Longhand for calling the ByRecord function exported by XML::SAX::Machines. CREDIT
Proposed by Matt Sergeant, with advise by Kip Hampton and Robin Berjon. Writing an aggregator. To be written. Pretty much just that "start_manifold_processing" and "end_manifold_processing" need to be provided. See XML::Filter::Merger and it's source code for a starter. perl v5.10.0 2009-06-11 XML::SAX::ByRecord(3pm)
All times are GMT -4. The time now is 04:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy