Sponsored Content
Top Forums Shell Programming and Scripting Conditional aggregation and print of a column in file Post 302372109 by dennis.jacob on Tuesday 17th of November 2009 05:41:36 AM
Old 11-17-2009
Quote:
Originally Posted by bittoo
Hi dennis

Brilliant command..
Suppose if the input is fixed length like
Code:
field1<1 space>field2<3 spaces>field3<5spaces>field4<10 spaces>field5

where spaces can vary..
I need output in same format as input just the field 5 will change value.
I need output in format printf "%-20s%-10s%-10s%-10s%10s"

Please help
try this. But am concerned about the speed of execution now...

Code:
awk '{ str=$1" "$2" "$3" "$4; if (arr[str]== "") arr[str]=$5; else arr[str]+=$5; } END { for (x in arr) { str=x" "arr[x]"\n"; split(str,a," ");printf "%-20s%-10s%-10s%-10s%10s\n", a[1],a[2],a[3],a[4],a[5]; }}'  file

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace a column with a value conditional on a value in col1

Hi, Perhaps a rather simple problem...? I have data that looks like this. BPC0013 ANNUL_49610 0 0 1 1 BPC0014 ANNUL_49642 0 0 2 1 BPC0015 ANNUL_49580 0 0 1 1 BPC0016 ANNUL_49596 0 0 2 1 BPC0017 VULGO_49612 0 0 1 1 BPC0018 ANNUL_49628 0 0 1 1 BPC0019 ANNUL_49692 0 0 2 1 170291_HMG... (4 Replies)
Discussion started by: genehunter
4 Replies

2. Shell Programming and Scripting

Strings from one file which exactly match to the 1st column of other file and then print lines.

Hi, I have two files. 1st file has 1 column (huge file containing ~19200000 lines) and 2nd file has 2 columns (small file containing ~6000 lines). ################################# huge_file.txt a a ab b ################################## small_file.txt a 1.5 b 2.5 ab ... (4 Replies)
Discussion started by: AshwaniSharma09
4 Replies

3. Shell Programming and Scripting

comparing column of two different files and print the column from in order of 2nd file

Hi friends, My file is like: Second file is : I need to print the rows present in file one, but in order present in second file....I used while read gh;do awk ' $1=="' $gh'" {print >> FILENAME"output"} ' cat listoffirstfile done < secondfile but the output I am... (14 Replies)
Discussion started by: CAch
14 Replies

4. Shell Programming and Scripting

How to print file without first column

Hi, unix Gurus, I hit a problem, I want print out a file without print first column. for example: 123 abc cde ... 234 cde def ... I want to get as following: abc cde .. cde def .. :wall: anybody can help me out. Thanks in advance. (8 Replies)
Discussion started by: ken002
8 Replies

5. Shell Programming and Scripting

Print a column with the name of the output file

I have n files and I am using cat to combine them in to one. Before that simply add the name of the output file to 4th column and then print the output. Is it possible ? input1 chr start end name 0 + key input2 chr start end name 0 + key inputn... (1 Reply)
Discussion started by: quincyjones
1 Replies

6. Shell Programming and Scripting

awk Print New Column For Every Two Lines and Match On Multiple Column Values to print another column

Hi, My input files is like this axis1 0 1 10 axis2 0 1 5 axis1 1 2 -4 axis2 2 3 -3 axis1 3 4 5 axis2 3 4 -1 axis1 4 5 -6 axis2 4 5 1 Now, these are my following tasks 1. Print a first column for every two rows that has the same value followed by a string. 2. Match on the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

7. Shell Programming and Scripting

Print the file into column

Hi All, Seeking for your assistance regarding on how to print multiple columns from file. Ex. nik.csv - input file 1,2,3,4,5,0054,May 1 11,12,13,14,15,0053,May 2 the 6th field is i converted it into decimal I create a variable on each field - pls see below. T1=`awk -F ","... (5 Replies)
Discussion started by: nikki1200
5 Replies

8. Shell Programming and Scripting

Filter on one column and then perform conditional calculations on another column with a Linux script

Hi, I have a file (stats.txt) with columns like in the example below. Destination IP address, timestamp, TCP packet sequence number and packet length. destIP time seqNo packetLength 1.2.3.4 0.01 123 500 1.2.3.5 0.03 44 1500 1.3.2.5 0.08 44 1500 1.2.3.4 0.44... (12 Replies)
Discussion started by: Zooma
12 Replies

9. Shell Programming and Scripting

Conditional Column Value

Hi Folks, I'm trying tog ain further experience with shell programming and have set my a small goal of writing a little filesystem monitoring script. So far my output is as follows: PACMYDB03 Filesystem Size Used Avail Use% Status /usr/local/mysql/data ... (5 Replies)
Discussion started by: Axleuk
5 Replies

10. UNIX for Beginners Questions & Answers

Compare 1st column from 2 file and if match print line from 1st file and append column 7 from 2nd

hi I have 2 file with more than 10 columns for both 1st file apple,0,0,0...... orange,1,2,3..... mango,2,4,5..... 2nd file apple,2,3,4,5,6,7... orange,2,3,4,5,6,8... watermerlon,2,3,4,5,6,abc... mango,5,6,7,4,6,def.... (1 Reply)
Discussion started by: tententen
1 Replies
EasyOBJ(3pm)						User Contributed Perl Documentation					      EasyOBJ(3pm)

NAME
XML::EasyOBJ - Easy XML object navigation VERSION
Version 1.12 SYNOPSIS
# open exisiting file my $doc = new XML::EasyOBJ('my_xml_document.xml'); my $doc = new XML::EasyOBJ(-type => 'file', -param => 'my_xml_document.xml'); # create object from XML string my $doc = new XML::EasyOBJ(-type => 'string', -param => $xml_source); # create new file my $doc = new XML::EasyOBJ(-type => 'new', -param => 'root_tag'); # read from document my $text = $doc->some_element($index)->getString; my $attr = $doc->some_element($index)->getAttr('foo'); my $element = $doc->some_element($index); my @elements = $doc->some_element; # first "some_element" element my $elements = $doc->some_element; # list of "some_element" elements my @elements = $doc->some_element; # write to document $doc->an_element->setString('some string') $doc->an_element->addString('some string') $doc->an_element->setAttr('attrname', 'val') $doc->an_element->setAttr('attr1' => 'val', 'attr2' => 'val2') # access elements with non-name chars and the underlying DOM my $element = $doc->getElement('foo-bar')->getElement('bar-none'); my $dom = $doc->foobar->getDomObj; # get elements without specifying the element name my @elements = $doc->getElement(); my $sixth_element = $doc->getElement('', 5); # remove elements/attrs $doc->remElement('tagname', $index); $doc->tag_name->remAttr($attr); # remap builtin methods $doc->remapMethod('getString', 's'); my $text = $doc->some_element->s; DESCRIPTION
I wrote XML::EasyOBJ a couple of years ago because it seemed to me that the DOM wasn't very "perlish" and the DOM is difficult for us mere mortals that don't use it on a regular basis. As I only need to process XML on an occasionally I wanted an easy way to do what I needed to do without having to refer back to DOM documentation each time. A quick fact list about XML::EasyOBJ: * Runs on top of XML::DOM * Allows access to the DOM as needed * Simple routines to reading and writing elements/attributes REQUIREMENTS
XML::EasyOBJ uses XML::DOM. XML::DOM is available from CPAN (www.cpan.org). METHODS
Below is a description of the methods avialable. new You can create a new object from an XML file, a string of XML, or a new document. The constructor takes a set of key value pairs as follows: -type The type is either "file", "string" or "new". "file" will create the object from a file source, "string" will create the object from a string of XML code, and "new" will create a new document object. -param This value depends on the -type that is passed to the constructor. If the -type is "file" this will be the filename to open and parse. If -type is "string", this is a string of XML code. If -type is "new", this is the name of the root element. Creating an object from an XML file: my $doc = new XML::EasyOBJ(-type => 'file', -param => 'my_xml_document.xml'); Creating an object from a string containing the XML source: my $doc = new XML::EasyOBJ(-type => 'string', -param => $xml_source); Creating a new XML document by passing the root tag name: my $doc = new XML::EasyOBJ(-type => 'new', -param => 'root_tag'); -expref Passing a value of 1 will force the expansion of references when grabbing string data from the XML file. The default value is 0, not to expand references. Obtionally you may also pass the filename to open as the first argument instead of passing the -type and -param parameters. This is backwards compatable with early version of XML::EasyOBJ which did not handle -type and -param parameters. my $doc = new XML::EasyOBJ('my_xml_document.xml'); makeNewNode( NEW_TAG ) Append a new element node to the current node. Takes the tag name as the parameter and returns the created node as a convienence. my $p_element = $doc->body->makeNewNode('p'); remapMethod( CUR_METHOD, NEW_METHOD ) Allows you to change the name of any of the object methods. You might want to do this for convienience or to avoid a naming collision with an element in the document. Two parameters need to be passed; the current name of the method and the new name. Returns 1 on a successful mapping and undef on failure. A failure can result if you don't pass two parameters if if the "copy from" method name does not exist. $doc->remapMethod('getString', 's'); $doc->s(); After remapping you must use the new name if you with to remap the method again. You can call the remapMethod method from any place in the XML tree and it will always change the method globally. In the following example $val1 and $val2 are equal: $doc->some_element->another_element->('getString', 's'); my $val1 = $doc->s(); $doc->remapMethod('s', 'getString'); my $val2 = $doc->getString(); getString( ) Recursively extracts text from the current node and all children element nodes. Returns the extracted text as a single scalar value. Expands entities based on if the -expref flag was supplied during object creation. extractText( ) Same as getString() but does not check the -expref flag. Included for compatability with inital version of interface. setString( STRING ) Sets the text value of the specified element. This is done by first removing all text node children of the current element and then appending the supplied text as a new child element. Take this XML fragment and code for example: <p>This elment has <b>text</b> and <i>child</i> elements</p> $doc->p->setString('This is the new text'); This will change the fragment to this: <p><b>text</b><i>child</i>This is the new text</p> Because the <b> and <i> tags are not text nodes they are left unchanged, and the new text is added at the end of the specified element. If you need more specific control on the change you should either use the getDomObj() method and use the DOM methods directly or remove all of the child nodes and rebuild the <p> element from scratch. Also see the addString() method. addString( STRING ) Adds to the the text value of the specified element. This is done by appending the supplied text as a new child element. Take this XML fragment and code for example: <p>This elment has <b>text</b></p> $doc->p->addString(' and elements'); This will change the fragment to this: <p>This elment has <b>text</b> and elements</p> getAttr( ATTR_NAME ) Returns the value of the named attribute. my $val = $doc->body->img->getAttr('src'); getTagName( ) Returns the tag name of the specified element. This method is useful when you are enumerating child elements and do not know their element names. foreach my $element ( $doc->getElement() ) { print $element->getTagName(); } setAttr( ATTR_NAME, ATTR_VALUE, [ATTR_NAME, ATTR_VALUE]... ) For each name/value pair passed the attribute name and value will be set for the specified element. remAttr( ATTR_NAME ) Removes the specified attribute from the current element. remElement( TAG_NAME, INDEX ) Removes a child element of the current element. The name of the child element and the index must be supplied. An index of 0 will remove the first occurance of the named element, 1 the second, 2 the third, etc. getElement( TAG_NAME, INDEX ) Returns the node from the tag name and index. If no index is given the first child with that name is returned. Use this method when you have element names that include characters that are not legal as a perl method name. For example: <foo> <!-- root element --> <bar> <foo-bar>test</foo-bar> </bar> </foo> # "foo-bar" is not a legal method name print $doc->bar->getElement('foo-bar')->getString(); getDomObj( ) Returns the DOM object associated with the current node. This is useful when you need fine access via the DOM to perform a specific function. BEGINNER QUICK START GUIDE
Introduction You too can write XML applications, just as long as you understand the basics of XML (elements and attributes). You can learn to write your first program that can read data from an XML file in a mere 10 minutes. Assumptions It is assumed that you are familiar with the structure of the document that you are reading. Next, you must know the basics of perl lists, loops, and how to call a function. You must also have an XML document to read. Simple eh? Loading the XML document use XML::EasyOBJ; my $doc = new XML::EasyOBJ('my_xml_document.xml') || die "Can't make object"; Replace the string "my_xml_document.xml" with the name of your XML document. If the document is in another directory you will need to specify the path to it as well. The variable $doc is an object, and represents our root XML element in the document. Reading text with getString Each element becomes an object. So lets assume that the XML page looks like this: <table> <record> <rec2 foo="bar"> <field1>field1a</field1> <field2>field2b</field2> <field3>field3c</field3> </rec2> <rec2 foo="baz"> <field1>field1d</field1> <field2>field2e</field2> <field3>field3f</field3> </rec2> </record> </table> As mentioned in he last step, the $doc object is the root element of the XML page. In this case the root element is the "table" element. To read the text of any field is as easy as navigating the XML elements. For example, lets say that we want to retrieve the text "field2e". This text is in the "field2" element of the SECOND "rec2" element, which is in the FIRST "record" element. So the code to print that value it looks like this: print $doc->record(0)->rec2(1)->field2->getString; The "getString" method returns the text within an element. We can also break it down like this: # grab the FIRST "record" element (index starts at 0) my $record = $doc->record(0); # grab the SECOND "rec2" element within $record my $rec2 = $record->rec2(1); # grab the "field2" element from $rec2 # NOTE: If you don't specify an index, the first item # is returned and in this case there is only 1. my $field2 = $rec2->field2; # print the text print $field2->getString; Reading XML attributes with getAttr Looking at the example in the previous step, can you guess what this code will print? print $doc->record(0)->rec2(0)->getAttr('foo'); print $doc->record(0)->rec2(1)->getAttr('foo'); If you couldn't guess, they will print out the value of the "foo" attribute of the first and second rec2 elements. Looping through elements Lets take our example in the previous step where we printed the attribute values and rewrite it to use a loop. This will allow it to print all of the "foo" attributes no matter how many "rec2" elements we have. foreach my $rec2 ( $doc->record(0)->rec2 ) { print $rec2->getAttr('foo'); } When we call $doc->record(0)->rec2 this way (i.e. in list context), the module will return a list of "rec2" elements. That's it! You are now an XML programmer! *start rejoicing now* PROGRAMMING NOTES
When creating a new instance of XML::EasyOBJ it will return an object reference on success, or undef on failure. Besides that, ALL methods will always return a value. This means that if you specify an element that does not exist, it will still return an object reference (and create that element automagically). This is just another way to lower the bar, and make this module easier to use. You will run into problems if you have XML tags which are named after perl's special subroutine names (i.e. "DESTROY", "AUTOLOAD"), or if they are named after subroutines used in the module ( "getString", "getAttr", etc ). You can get around this by using the getElement() method of using the remapMethod() method which can be used on every object method (except AUTOLOAD and DESTROY). AUTHOR
/COPYRIGHT Copyright (C) 2000-2002 Robert Hanson <rhanson@blast.net> This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
XML::DOM perl v5.10.0 2009-08-19 EasyOBJ(3pm)
All times are GMT -4. The time now is 01:52 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy