Sponsored Content
Top Forums Shell Programming and Scripting Extracting records with unique fields from a fixed width txt file Post 302165817 by KevinADC on Saturday 9th of February 2008 03:56:46 AM
Old 02-09-2008
Your requirements are a bit vague, but here is a possible perl solution:

Code:
#!/usr/bin/perl
use warnings;
use strict;
#use Data::Dumper; #uncomment for debugging
unless (scalar @ARGV == 2){
   die "Usage: perl scriptname.pl inputfile outputfile\n";
}

my $outfile = pop @ARGV;
my %names = ();
my %count = ();

while (<>){
   chomp;
   my ($first,$mi,$last,$state) = unpack("a10a2a11a2",$_);
   (s/^\s*//, s/\s*$//) for ($first,$mi,$last,$state);
   $names{"$first,$last"}={count => ++$count{"$first,$last"},
                           name => "$first $mi $last $state",
                          };
}

#print Dumper \%names; #uncomment for debugging  

open my $out , '>' , $outfile or die "$!"; 

foreach my $person (keys %names) {
    next if $names{$person}{count}>1;
    print $out $names{$person}{name},"\n";
}

close $out;

print STDOUT "finished";
exit(0);

Usage:

perl scriptname.pl path/to/inputfile path/to/outputfile

Last edited by KevinADC; 02-09-2008 at 05:08 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX Desktop Questions & Answers

Help with Fixed width File Parsing

I am trying to parse a Fixed width file with data as below. I am trying to assign column values from each record to variables. When I parse the data, the spaces in all coumns are dropped. I would like to retain the spaces as part of the dat stored in the variables. Any help is appreciated. I... (4 Replies)
Discussion started by: sate911
4 Replies

2. Shell Programming and Scripting

summing up the fields in fixed width file

Hi, I have a fixed width file with some records as given below: " 1000Nalsdjflj243324jljlj" "-0300Njfowjljl309933fsf" " 0010Njsfsjklj342344fsl" I want to sum-up first field values(i.e from 2nd character to 6th character)of each record. so for the above file i want to add (1000 - 300+... (2 Replies)
Discussion started by: srilaxmi
2 Replies

3. Shell Programming and Scripting

Fixed-Width file from Oracle

Hi All, I have created a script which generates FIXED-WIDTH file by executing Oracle query. SELECT RPAD(NVL(col1,CHR(9)),20)||NVL(col2,CHR(9))||NVL(col3,CHR(9) FROM XYZ It generates the data file with proper alignment. But if same file i transfer to windows server or Mainframe... (5 Replies)
Discussion started by: Amit.Sagpariya
5 Replies

4. Shell Programming and Scripting

Manupulating Records in a fixed width file

I am trying to determine what would be a fast and simple way to manipulate data that comes in a fixed width format. This data has 6 segments within a record. Each record needs to written out with a header and the 6 segments. Based on the value in column #6 the fields will be defined accordingly.... (4 Replies)
Discussion started by: Muga801
4 Replies

5. Shell Programming and Scripting

Comparing two fixed width file

Hi Guys I am checking the treads to get the answer but i am not able to get the answer for my question. I have two files. First file is a pattern file and the second file is the file i want to search in it. Output will be the lines from file2. File1: P2797f12af 44751228... (10 Replies)
Discussion started by: anshul_er
10 Replies

6. Shell Programming and Scripting

Apply condition on fixed width file and filter records

Dear members.. I have a fixed width file. Requirement is as below:- 1. Scan each record from this fixed width file 2. Check for value under field no "6" equals to "ABC". If yes, then filter this record into the output file Please suggest a unix command to achieve this, my guess awk might... (6 Replies)
Discussion started by: sureshg_sampat
6 Replies

7. Shell Programming and Scripting

variable fixed-width fields

Hi there, CTL Port IO Rate(IOPS) Read Rate(IOPS) Write Rate(IOPS) Read Hit(%) Write Hit(%) Trans. Rate(MB/S) Read Trans. Rate(MB/S) Write Trans. Rate(MB/S) 09:36:48 0 A 136 0 135 97 100 ... (6 Replies)
Discussion started by: gray380
6 Replies

8. Shell Programming and Scripting

How to parse fixed-width columns which may include empty fields?

I am trying to selectively display several columns from a db2 query, which gives me a fixed-width output (partial output listed here): --------- -------------------------- ------------ ------ 000 0000000000198012 702 29 000 0000000000198013 ... (9 Replies)
Discussion started by: ahsh79
9 Replies

9. UNIX for Dummies Questions & Answers

Length of a fixed width file

I have a fixed width file of length 53. when is try to get the lengh of the record of that file i get 2 different answers. awk '{print length;exit}' <File_name> The above code gives me length 50. wc -L <File_name> The above code gives me length 53. Please clarify on... (2 Replies)
Discussion started by: Amrutha24
2 Replies

10. Shell Programming and Scripting

Alter Fixed Width File

Thank u so much .Its working fine as expected. ---------- Post updated at 03:41 PM ---------- Previous update was at 01:46 PM ---------- I need one more help. I have another file(fixed length) that will get negative value (ex:-00000000003000) in postion (98 - 112) then i have to... (6 Replies)
Discussion started by: vinus
6 Replies
AnyEvent::XMPP::Ext::DataForm(3pm)			User Contributed Perl Documentation			AnyEvent::XMPP::Ext::DataForm(3pm)

NAME
AnyEvent::XMPP::Ext::DataForm - XEP-0004 DataForm SYNOPSIS
DESCRIPTION
This module represents a Data Form as specified in XEP-0004. METHODS
new (%args) append_field ($field) This method appends a field to the form. $field must have the structure as described in "FIELD STRUCTURE" below. from_node ($node) This method interprets the AnyEvent::XMPP::Node object in $node as data form XML node and reads out the fields and all associated information. ($node must be the XML node of the <x xmlns='jabber:x:data'> tag). make_answer_form ($request_form) This method initializes this form with default answers and other neccessary fields from $request_form, which must be of type AnyEvent::XMPP::Ext::DataForm or compatible. The result will be a form with a copy of all fields which are not of type "fixed". The fields will also have the default value copied over. The form type will be set to "submit". The idea is: this creates a template answer form from $request_form. To strip out the unneccessary fields later you don't need call the "clear_empty_fields" method. clear_empty_fields This method removes all fields that have no values and options. remove_field ($field_or_var) This method removes a field either by it's unique name or by reference. $field_or_var can either be the unique name or the actual field hash reference you get from "get_field" or "get_fields". set_form_type ($type) This method sets the type of the form, which must be one of: form, submit, cancel, result form_type This method returns the type of the form, which is one of the options described in "set_form_type" above or undef if no type was yet set. get_reported_fields If this is a search result this method returns more than one element here. The returned list consists of fields as described in "FIELD STRUCTURE", only that they lack values and options. See also the "get_items" method. get_items If this form is a search result this method returns the list of items of that search. An item is a array ref of fields (field structure is described in "FIELD STRUCTURE"). This method returns a list of items. get_fields This method returns a list of fields. Each field has the structure as described in "FIELD STRUCTURE". get_field ($var) Returns the field with the unique field name $var or undef if no such field is in this form. set_field_value ($var, $value) This method sets the value of the field with the unique name $var. If the field has supports multiple values all values will be removed and only $value will be added, if $value is undefined the field's value will be deleted. add_field_value ($var, $value) This method adds the $value to the field with the unique name $var. If the field doesn't support multiple values this method has the same effect as "set_field_value". to_simxml This method converts the form to a data strcuture that you can pass as "node" argument to the "simxml" function which is documented in AnyEvent::XMPP::Util. Example call might be: my $node = $form->to_simxml; simxml ($w, defns => $node->{ns}, node => $node); NOTE: The returned simxml node has the "dns" field set so that no prefixes are generated for the namespace it is in. as_debug_string This method returns a string that represents the form. Only for debugging purposes. FIELD STRUCTURE
{ label => 'field label', type => 'field type', var => '(unique) field name' required => true or false value, values => [ 'value text', ... ], options => [ ['option label', 'option text'], ... ] } For the semantics of all fields please consult XEP 0004. SEE ALSO
XEP 0004 AUTHOR
Robin Redeker, "<elmex at ta-sa.org>", JID: "<elmex at jabber.org>" COPYRIGHT &; LICENSE Copyright 2007, 2008 Robin Redeker, all rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.14.2 2012-01-23 AnyEvent::XMPP::Ext::DataForm(3pm)
All times are GMT -4. The time now is 02:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy