Sponsored Content
Top Forums Shell Programming and Scripting Find Duplicate records in first Column in File Post 302409278 by Murugesh on Wednesday 31st of March 2010 02:30:32 PM
Old 03-31-2010
Works fine. Thanks for your help.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to find Duplicate Records in a text file

Hi all pls help me by providing soln for my problem I'm having a text file which contains duplicate records . Example: abc 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452 abc 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452 tas 3420 3562 ... (1 Reply)
Discussion started by: G.Aavudai
1 Replies

2. Shell Programming and Scripting

find duplicate records... again

Hi all: Let's suppose I have a file like this (but with many more records). XX ME 342 8688 2006 7 6 3c 60.029 -38.568 2901 0001 74 4 7603 8 969.8 958.4 3.6320 34.8630 985.5 973.9 3.6130 34.8600 998.7 986.9 3.6070 34.8610 1003.6 991.7 ... (4 Replies)
Discussion started by: rleal
4 Replies

3. Shell Programming and Scripting

find out duplicate records in file?

Dear All, I have one file which looks like : account1:passwd1 account2:passwd2 account3:passwd3 account1:passwd4 account5:passwd5 account6:passwd6 you can see there're two records for account1. and is there any shell command which can find out : account1 is the duplicate record in... (3 Replies)
Discussion started by: tiger2000
3 Replies

4. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

5. Shell Programming and Scripting

Removing duplicate records in a file based on single column

Hi, I want to remove duplicate records including the first line based on column1. For example inputfile(filer.txt): ------------- 1,3000,5000 1,4000,6000 2,4000,600 2,5000,700 3,60000,4000 4,7000,7777 5,999,8888 expected output: ---------------- 3,60000,4000 4,7000,7777... (5 Replies)
Discussion started by: G.K.K
5 Replies

6. Shell Programming and Scripting

Removing duplicate records in a file based on single column explanation

I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1. https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html Can someone explain this command please? How are there no... (5 Replies)
Discussion started by: cokedude
5 Replies

7. Shell Programming and Scripting

Deleting duplicate records from file 1 if records from file 2 match

I have 2 files "File 1" is delimited by ";" and "File 2" is delimited by "|". File 1 below (3 record shown): Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies

8. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

9. Shell Programming and Scripting

Filter duplicate records from csv file with condition on one column

I have csv file with 30, 40 columns Pasting just three column for problem description I want to filter record if column 1 matches CN or DN then, check for values in column 2 if column contain 1235, 1235 then in column 3 values must be sequence of 2345, 2345 and if column 2 contains 6789, 6789... (5 Replies)
Discussion started by: as7951
5 Replies

10. Shell Programming and Scripting

CSV File:Filter duplicate records from column1 & another column having unique record

Hi Experts, I have csv file with 30, 40 columns Pasting just 2 column for problem description. Need to print error if below combination is not present in file check for column-1 (DocumentNumber) and filter columns where value in DocumentNumber field is same. For all such rows, the field... (7 Replies)
Discussion started by: as7951
7 Replies
LWP(3pm)						User Contributed Perl Documentation						  LWP(3pm)

NAME
Coro::LWP - make LWP non-blocking - as much as possible SYNOPSIS
use Coro::LWP; # afterwards LWP should not block ALTERNATIVES
Over the years, a number of less-invasive alternatives have popped up, which you might find more acceptable than this rather invasive and fragile module. All of them only support HTTP (and sometimes HTTPS). AnyEvent::HTTP Works fine without Coro. Requires using a very different API than LWP. Probably the best choice iff you can do with a completely different event-based API. LWP::Protocol::AnyEvent::http Makes LWP use AnyEvent::HTTP. Does not make LWP event-based, but allows Coro threads to schedule unimpeded through its AnyEvent integration. Let's you use the LWP API normally. LWP::Protocol::Coro::http Basically the same as above, distinction unclear. :) AnyEvent::HTTP::LWP::UserAgent A different user agent implementation, not completely transparent to users, requires Coro. DESCRIPTION
This module is an AnyEvent user, you need to make sure that you use and run a supported event loop. This module tries to make LWP non-blocking with respect to other coroutines as much as possible, and with whatever means it takes. LWP really tries very hard to be blocking (and relies on a lot of undocumented functionality in IO::Socket), so this module had to be very invasive and must be loaded very early to take the proper effect. Note that the module AnyEvent::HTTP might offer an alternative to the full LWP that is designed to be non-blocking. Here is what it currently does (future versions of LWP might require different tricks): It loads Coro::Select, overwriting the perl "select" builtin globally. This is necessary because LWP calls select quite often for timeouts and who-knows-what. Impact: everybody else uses this (slower) version of select, too. It should be quite compatible to perls builtin select, though. It overwrites Socket::inet_aton with Coro::Util::inet_aton. This is necessary because LWP might (and does) try to resolve hostnames this way. Impact: some code might not expect coroutine semantics, for example, when you fork you might prefer the blocking variant because other coroutines shouldn't actually run. It replaces the base class of Net::HTTP, Net::FTP, Net::NNTP. This is necessary because LWP does not always use select to see whether a filehandle can be read/written without blocking, so the base class "IO::Socket::INET" needs to be replaced by "Coro::Socket". Impact: Coro::Socket is not at all compatible to IO::Socket::INET. While it duplicates some undocumented functionality required by LWP, it does not have all the methods of IO::Socket::INET and might act quite differently in practise. Also, protocols other than the above mentioned will still block, at least some of the time. All this likely makes other libraries than just LWP not block, but thats just a side effect you cannot rely on. Increases parallelism is not supported by all libraries, some might cache data globally. AUTHOR
Marc Lehmann <schmorp@schmorp.de> http://home.schmorp.de/ perl v5.14.2 2012-04-13 LWP(3pm)
All times are GMT -4. The time now is 06:55 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy