Sponsored Content
Top Forums Shell Programming and Scripting Help removing lines with duplicated columns Post 302196215 by era on Saturday 17th of May 2008 07:19:10 AM
Old 05-17-2008
The forums' own search tool stupidly treats "awk" as a stop word, so I took a detour via Google.

site:unix.com awk duplicate - Google Search

a is just the name of a variable; if the associative array already contains a value for the given key, we have already seen that key before, and suppress printing. (The default if no action is given is to print anything matching the condition.)
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

remove duplicated lines without sort

Hi Just wondering whether or not I can remove duplicated lines without sort For example, I use the command who, which shows users who are logging on. In some cases, it shows duplicated lines of users who are logging on more than one terminal. Normally, I would do who | cut -d" " -f1 |... (6 Replies)
Discussion started by: lalelle
6 Replies

2. UNIX for Dummies Questions & Answers

Removing lines that are (same in content) based on columns

I have a file which looks like AA BB CC DD EE FF GG HH KK AA BB GG HH KK FF CC DD EE AA BB CC DD EE UU VV XX ZZ AA BB VV XX ZZ UU CC DD EE .... I want the script to give me only one line based on duplicate contents: AA BB CC DD EE FF GG HH KK AA BB CC DD EE UU VV XX ZZ (7 Replies)
Discussion started by: adsforall
7 Replies

3. Shell Programming and Scripting

remove duplicated columns

hi all, i have a file contain multicolumns, this file is sorted by col2 and col3. i want to remove the duplicated columns if the col2 and col3 are the same in another line. example fileA AA BB CC DD CC XX CC DD BB CC ZZ FF DD FF HH HH the output is AA BB CC DD BB CC ZZ FF... (6 Replies)
Discussion started by: kamel.seg
6 Replies

4. Shell Programming and Scripting

using sed to get rid of duplicated columns...

I can not figure out this one, so I turn to unix.com for help, I have a file, in which there are some lines containing continuously duplicate columns, like the following adb abc abc asd adfj 123 123 123 345 234 444 444 444 444 444 23 and the output I want is adb abc asd adfj 123 345... (5 Replies)
Discussion started by: fedora
5 Replies

5. Shell Programming and Scripting

finding duplicates in columns and removing lines

I am trying to figure out how to scan a file like so: 1 ralphs office","555-555-5555","ralph@mail.com","www.ralph.com 2 margies office","555-555-5555","ralph@mail.com","www.ralph.com 3 kims office","555-555-5555","kims@mail.com","www.ralph.com 4 tims... (17 Replies)
Discussion started by: totus
17 Replies

6. UNIX for Dummies Questions & Answers

Removing duplicated lines??

Hi Guys.. I have a problem for some reason my database has copied everything 4 times. My Database looks like this: >BAC233456 rhjieaheiohjteo tjtjrj6jkk6k6 j54ju54jh54jh >ANI124365 afrhtjykulilil htrjykuk rtkjryky ukrykyrk >BAC233456 rhjieaheiohjteo tjtjrj6jkk6k6 j54ju54jh54jh... (6 Replies)
Discussion started by: Iifa
6 Replies

7. UNIX for Dummies Questions & Answers

Find duplicated values in two columns out of three

hi! could u help in the following? I have the data (long list!) that looks like (three coumns white space separated): rs3094315 0.0665173 742429 rs12562034 0.0738998 758311 rs3934834 0.396449 995669 rs9442372 0.402693 1008567 rs3737728 0.406271 1011278 rs6687776 0.435429 1020428 rs9651273... (4 Replies)
Discussion started by: kush
4 Replies

8. Shell Programming and Scripting

How to remove duplicated lines?

Hi, if i have a file like this: Query=1 a a b c c c d Query=2 b b b c c e . . . (7 Replies)
Discussion started by: the_simpsons
7 Replies

9. Shell Programming and Scripting

Removing duplicated first field rows

Hello, I am trying to eliminate rows where the first field is duplicated, leaving the row where the last field is "NET". Data file: 345234|22.34|LST 546543|55.33|LST 793929|98.23|LST 793929|64.69|NET 149593|49.22|LST Desired output: 345234|22.34|LST 546543|55.33|LST... (2 Replies)
Discussion started by: palex
2 Replies

10. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies
Finance::QuoteHist::Google(3pm) 			User Contributed Perl Documentation			   Finance::QuoteHist::Google(3pm)

NAME
Finance::QuoteHist::Google - Site-specific class for retrieving historical stock quotes. SYNOPSIS
use Finance::QuoteHist::Google; $q = Finance::QuoteHist::Google->new ( symbols => [qw(IBM UPS AMZN)], start_date => '01/01/1999', end_date => 'today', ); foreach $row ($q->quotes()) { ($symbol, $date, $open, $high, $low, $close, $volume) = @$row; ... } DESCRIPTION
Finance::QuoteHist::Google is a subclass of Finance::QuoteHist::Generic, specifically tailored to read historical quotes from the Google web site (http://finance.google.com/). Google does not currently provide information on dividends or splits. Please see Finance::QuoteHist::Generic(3) for more details on usage and available methods. If you just want to get historical quotes and are not interested in the details of how it is done, check out Finance::QuoteHist(3). METHODS
The basic user interface consists of a single method, as shown in the example above. That method is: quotes() Returns a list of rows (or a reference to an array containing those rows, if in scalar context). Each row contains the Symbol, Date, Open, High, Low, Close, and Volume for that date. Quote values are pre-adjusted for this site. REQUIRES
Finance::QuoteHist::Generic DISCLAIMER
The data returned from these modules is in no way guaranteed, nor are the developers responsible in any way for how this data (or lack thereof) is used. The interface is based on URLs and page layouts that might change at any time. Even though these modules are designed to be adaptive under these circumstances, they will at some point probably be unable to retrieve data unless fixed or provided with new parameters. Furthermore, the data from these web sites is usually not even guaranteed by the web sites themselves, and oftentimes is acquired elsewhere. Details for Googles's terms of use can be found here: http://www.google.com/accounts/TOS?loc=us If you still have concerns, then use another site-specific historical quote instance, or none at all. Above all, play nice. AUTHOR
Matthew P. Sisk, <sisk@mojotoad.com> COPYRIGHT
Copyright (c) 2007-2010 Matthew P. Sisk. All rights reserved. All wrongs revenged. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
Finance::QuoteHist::Generic(3), Finance::QuoteHist(3), perl(1). perl v5.12.4 2010-06-07 Finance::QuoteHist::Google(3pm)
All times are GMT -4. The time now is 10:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy