Sponsored Content
Top Forums Shell Programming and Scripting Extracting unique values of a column from a feed file Post 302894033 by Kingcobra on Sunday 23rd of March 2014 08:03:06 AM
Old 03-23-2014
Blade

Code:
sed '1 d' fo|cut -'d|' -f16|sort|uniq
811
831
861

easiest way to do is,in case if you want an explanations whats happening

sed 1d is selecting all but first line and rest all is self explanatory
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

return a list of unique values of a column from csv format file

Hi all, I have a huge csv file with the following format of data, Num SNPs, 549997 Total SNPs,555352 Num Samples, 157 SNP, SampleID, Allele1, Allele2 A001,AB1,A,A A002,AB1,A,A A003,AB1,A,A ... ... ... I would like to write out a list of unique SNP (column 1). Could you... (3 Replies)
Discussion started by: phoeberunner
3 Replies

2. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the... (6 Replies)
Discussion started by: amigarus
6 Replies

3. UNIX for Dummies Questions & Answers

Extracting rows from a text file based on numerical values of a column

I have a text file where the second column is a list of numbers going from small to large. I want to extract the rows where the second column is smaller than or equal to 0.0001. My input: rs10082730 9e-08 12 46002702 rs2544081 1e-07 12 46015487 rs1425136 1e-06 7 35396742 rs2712590... (1 Reply)
Discussion started by: evelibertine
1 Replies

4. UNIX for Dummies Questions & Answers

Extracting rows from a space delimited text file based on the values of a column

I have a space delimited text file. I want to extract rows where the third column has 0 as a value and write those rows into a new space delimited text file. How do I go about doing that? Thanks! (2 Replies)
Discussion started by: evelibertine
2 Replies

5. Shell Programming and Scripting

AWK, Perl or Shell? Unique strings and their maximum values from 3 column data file

I have a file containing data like so: 2012-01-02 GREEN 4 2012-01-02 GREEN 6 2012-01-02 GREEN 7 2012-01-02 BLUE 4 2012-01-02 BLUE 3 2012-01-02 GREEN 4 2012-01-02 RED 4 2012-01-02 RED 8 2012-01-02 GREEN 4 2012-01-02 YELLOW 5 2012-01-02 YELLOW 2 I can't always predict what the... (4 Replies)
Discussion started by: rich@ardz
4 Replies

6. Shell Programming and Scripting

Script for extracting data from csv file based on column values.

Hi all, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (3 Replies)
Discussion started by: Vivekit82
3 Replies

7. Linux

To get all the columns in a CSV file based on unique values of particular column

cat sample.csv ID,Name,no 1,AAA,1 2,BBB,1 3,AAA,1 4,BBB,1 cut -d',' -f2 sample.csv | sort | uniq this gives only the 2nd column values Name AAA BBB How to I get all the columns of CSV along with this? (1 Reply)
Discussion started by: sanvel
1 Replies

8. UNIX for Dummies Questions & Answers

Unique values in a row sum the next column in UNIX

Hi would like to ask you guys any advise regarding my problem I have this kind of data file.txt 111111111,20 111111111,50 222222222,70 333333333,40 444444444,10 444444444,20 I need to get this file1.txt 111111111,70 222222222,70 333333333,40 444444444,30 using this code I can... (6 Replies)
Discussion started by: reks
6 Replies

9. UNIX for Beginners Questions & Answers

Find unique values but only in column 1

Hi All, Does anyone have any suggestions/examples of how i could show only lines where the first field is not duplicated. If the first field is listed more than once it shouldnt be shown even if the other columns make it unique. Example file : 876,RIBDA,EC2 876,RIBDH,EX7 877,RIBDF,E28... (4 Replies)
Discussion started by: mutley2202
4 Replies

10. Shell Programming and Scripting

Count number of unique values in each column of array

What is an efficient way of counting the number of unique values in a 400 column by 1000 row array and outputting the counts per column, assuming the unique values in the array are: A, B, C, D In other words the output should look like: Value COL1 COL2 COL3 A 50 51 52... (16 Replies)
Discussion started by: Geneanalyst
16 Replies
XML::Feed(3pm)						User Contributed Perl Documentation					    XML::Feed(3pm)

NAME
XML::Feed - Syndication feed parser and auto-discovery SYNOPSIS
use XML::Feed; my $feed = XML::Feed->parse(URI->new('http://example.com/atom.xml')) or die XML::Feed->errstr; print $feed->title, " "; for my $entry ($feed->entries) { } ## Find all of the syndication feeds on a given page, using ## auto-discovery. my @feeds = XML::Feed->find_feeds('http://example.com/'); DESCRIPTION
XML::Feed is a syndication feed parser for both RSS and Atom feeds. It also implements feed auto-discovery for finding feeds, given a URI. XML::Feed supports the following syndication feed formats: o RSS 0.91 o RSS 1.0 o RSS 2.0 o Atom The goal of XML::Feed is to provide a unified API for parsing and using the various syndication formats. The different flavors of RSS and Atom handle data in different ways: date handling; summaries and content; escaping and quoting; etc. This module attempts to remove those differences by providing a wrapper around the formats and the classes implementing those formats (XML::RSS and XML::Atom::Feed). For example, dates are handled differently in each of the above formats. To provide a unified API for date handling, XML::Feed converts all date formats transparently into DateTime objects, which it then returns to the caller. USAGE
XML::Feed->new($format) Creates a new empty XML::Feed object using the format $format. $feed = XML::Feed->new('Atom'); $feed = XML::Feed->new('RSS'); $feed = XML::Feed->new('RSS', version => '0.91'); XML::Feed->parse($stream) XML::Feed->parse($stream, $format) Parses a syndication feed identified by $stream and returns an XML::Feed obhect. $stream can be any one of the following: o Scalar reference A reference to string containing the XML body of the feed. o Filehandle An open filehandle from which the feed XML will be read. o File name The name of a file containing the feed XML. o URI object A URI from which the feed XML will be retrieved. $format allows you to override format guessing. XML::Feed->find_feeds($uri) Given a URI $uri, use auto-discovery to find all of the feeds linked from that page (using <link> tags). Returns a list of feed URIs. XML::Feed->identify_format($xml) Given the xml of a feed return what format it is in ("Atom", or some version of "RSS"). $feed->convert($format) Converts the XML::Feed object into the $format format, and returns the new object. $feed->splice($other_feed) Splices in all of the entries from the feed $other_feed into $feed, skipping posts that are already in $feed. $feed->format Returns the format of the feed ("Atom", or some version of "RSS"). $feed->title([ $title ]) The title of the feed/channel. $feed->base([ $base ]) The url base of the feed/channel. $feed->link([ $uri ]) The permalink of the feed/channel. $feed->tagline([ $tagline ]) The description or tagline of the feed/channel. $feed->description([ $description ]) Alias for $feed->tagline. $feed->author([ $author ]) The author of the feed/channel. $feed->language([ $language ]) The language of the feed. $feed->copyright([ $copyright ]) The copyright notice of the feed. $feed->modified([ $modified ]) A DateTime object representing the last-modified date of the feed. If present, $modified should be a DateTime object. $feed->generator([ $generator ]) The generator of the feed. $feed->self_link ([ $uri ]) The Atom Self-link of the feed: <http://validator.w3.org/feed/docs/warning/MissingAtomSelfLink.html> A string. $feed->entries A list of the entries/items in the feed. Returns an array containing XML::Feed::Entry objects. $feed->items A synonym (alias) for <$feed->entries>. $feed->add_entry($entry) Adds an entry to the feed. $entry should be an XML::Feed::Entry object in the correct format for the feed. $feed->as_xml Returns an XML representation of the feed, in the format determined by the current format of the $feed object. PACKAGE VARIABLES
$XML::Feed::Format::RSS::PREFERRED_PARSER If you want to use another RSS parser class than XML::RSS (default), you can change the class by setting $PREFERRED_PARSER variable in the XML::Feed::Format::RSS package. $XML::Feed::Format::RSS::PREFERRED_PARSER = "XML::RSS::LibXML"; Note: this will only work for parsing feeds, not creating feeds. Note: Only "XML::RSS::LibXML" version 0.3004 is known to work at the moment. $XML::Feed::MULTIPLE_ENCLOSURES Although the RSS specification states that there can be at most one enclosure per item some feeds break this rule. If this variable is set then "XML::Feed" captures all of them and makes them available as a list. Otherwise it returns the last enclosure parsed. Note: "XML::RSS" version 1.44 is needed for this to work. VALID FEEDS
For reference, this cgi script will create valid, albeit nonsensical feeds (according to "http://feedvalidator.org" anyway) for Atom 1.0 and RSS 0.90, 0.91, 1.0 and 2.0. #!perl -w use strict; use CGI; use CGI::Carp qw(fatalsToBrowser); use DateTime; use XML::Feed; my $cgi = CGI->new; my @args = ( $cgi->param('format') || "Atom" ); push @args, ( version => $cgi->param('version') ) if $cgi->param('version'); my $feed = XML::Feed->new(@args); $feed->id("http://".time.rand()."/"); $feed->title('Test Feed'); $feed->link($cgi->url); $feed->self_link($cgi->url( -query => 1, -full => 1, -rewrite => 1) ); $feed->modified(DateTime->now); my $entry = XML::Feed::Entry->new(); $entry->id("http://".time.rand()."/"); $entry->link("http://example.com"); $entry->title("Test entry"); $entry->summary("Test summary"); $entry->content("Foo"); $entry->modified(DateTime->now); $entry->author('test@example.com (Testy McTesterson)'); $feed->add_entry($entry); my $mime = ("Atom" eq $feed->format) ? "application/atom+xml" : "application/rss+xml"; print $cgi->header($mime); print $feed->as_xml; LICENSE
XML::Feed is free software; you may redistribute it and/or modify it under the same terms as Perl itself. AUTHOR &; COPYRIGHT Except where otherwise noted, XML::Feed is Copyright 2004-2008 Six Apart, cpan@sixapart.com. All rights reserved. SUBVERSION
The latest version of XML::Feed can be found at http://code.sixapart.com/svn/XML-Feed/trunk/ perl v5.14.2 2012-03-21 XML::Feed(3pm)
All times are GMT -4. The time now is 08:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy