Sponsored Content
Top Forums Shell Programming and Scripting Find duplicates in the first column of text file Post 302432800 by gameboy87 on Sunday 27th of June 2010 03:37:30 AM
Old 06-27-2010
Error Find duplicates in the first column of text file

Hello,

My text file has input of the form

Code:
abc dft45.xml
ert  rt653.xml
abc ert57.xml

I need to write a perl script/shell script to find duplicates in the first column and write it into a text file of the form...

Code:
abc dft45.xml
abc ert57.xml

Can some one help me plz?

Last edited by Scott; 06-27-2010 at 06:23 AM.. Reason: Code tags, please...
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to find the number of column in the text file...?

Hi, i have text file with ~ seperated columns. it is very huge size of file, in the file sompulsary supposed to has 20 columns with ~ seperated. so how can i find if the file has 20 column in the all rows...? Sample file: APA+VU~10~~~~~03~101~101~~~APA.N O 20081017 120.00... (1 Reply)
Discussion started by: psiva_arul
1 Replies

2. UNIX for Dummies Questions & Answers

Remove duplicates based on a column in fixed width file

Hi, How to output the duplicate record to another file. We say the record is duplicate based on a column whose position is from 2 and its length is 11 characters. The file is a fixed width file. ex of Record: DTYU12333567opert tjhi kkklTRG9012 The data in bold is the key on which... (1 Reply)
Discussion started by: Qwerty123
1 Replies

3. Shell Programming and Scripting

need to remove duplicates based on key in first column and pattern in last column

Given a file such as this I need to remove the duplicates. 00060011 PAUL BOWSTEIN ad_waq3_921_20100826_010517.txt 00060011 PAUL BOWSTEIN ad_waq3_921_20100827_010528.txt 0624-01 RUT CORPORATION ad_sade3_10_20100827_010528.txt 0624-01 RUT CORPORATION ... (13 Replies)
Discussion started by: script_op2a
13 Replies

4. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

5. Red Hat

How to find a garbage entry in a column wise text file in Linux?

Suppose I have a file containing :- 1 Apple $50 2 Orange $30 3 Banana $10 4 Guava $25 5 Pine@apple $12 6 Strawberry $21 7 Grapes $12 In the 5th row, @ character inserted. I want through sort command or by any other way this row should either on top or bottom. By sort command garbage... (1 Reply)
Discussion started by: Dipankar Mitra
1 Replies

6. Shell Programming and Scripting

Find duplicates in column 1 and merge their lines (awk?)

Hi, I have a file (sorted by sort) with 8 tab delimited columns. The first column contains duplicated fields and I need to merge all these identical lines. My input file: comp100002 aaa bbb ccc ddd eee fff ggg comp100003 aba aba aba aba aba aba aba comp100003 fff fff fff fff fff fff fff... (5 Replies)
Discussion started by: falcox
5 Replies

7. Shell Programming and Scripting

Find duplicates in 2 & 3rd column and their ID

with below given format, I have been trying to find out all IDs for those entries with duplicate names in 2nd and 3rd columns and their count like how many time duplication happened for any name if any, 0.237788 Aaban Aahva 0.291066 Aabheer Aahlaad 0.845814 Aabid Aahan 0.152208 Aadam... (6 Replies)
Discussion started by: busyboy
6 Replies

8. Shell Programming and Scripting

awk to Sum columns when other column has duplicates and append one column value to another with Care

Hi Experts, Please bear with me, i need help I am learning AWk and stuck up in one issue. First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique. Second point : For... (1 Reply)
Discussion started by: as7951
1 Replies

9. UNIX for Beginners Questions & Answers

Find duplicates in file with line numbers

Hello All, This is a noob question. I tried searching for the answer but the answer found did not help me . I have a file that can have duplicates. 100 200 300 400 100 150 the number 100 is duplicated twice. I want to find the duplicate along with the line number. expected... (4 Replies)
Discussion started by: vatigers
4 Replies
Test::XML::Simple(3pm)					User Contributed Perl Documentation				    Test::XML::Simple(3pm)

NAME
Test::XML::Simple - easy testing for XML SYNOPSIS
use Test::XML::Simple tests=>5; xml_valid $xml, "Is valid XML"; xml_node $xml, "/xpath/expression", "specified xpath node is present"; xml_is, $xml, '/xpath/expr', "expected value", "specified text present"; xml_like, $xml, '/xpath/expr', qr/expected/, "regex text present"; xml_is_deeply, $xml, '/xpath/expr', $xml2, "structure and contents match"; # Not yet implemented: # xml_like_deeply would be nice too... DESCRIPTION
"Test::XML::Simple" is a very basic class for testing XML. It uses the XPath syntax to locate nodes within the XML. You can also check all or part of the structure vs. an XML fragment. TEST ROUTINES
xml_valid $xml, 'test description' Pass an XML file or fragment to this test; it succeeds if the XML (fragment) is valid. xml_node $xml, $xpath, 'test description' Checks the supplied XML to see if the node described by the supplied XPath expression is present. Test fails if it is not present. xml_is_long $xml, $xpath, $value, 'test description' Finds the node corresponding to the supplied XPath expression and compares it to the supplied value. Succeeds if the two values match. Uses Test::More's "is" function to do the comparison. xml_is $xml, $xpath, $value, 'test description' Finds the node corresponding to the supplied XPath expression and compares it to the supplied value. Succeeds if the two values match. Uses Test::LongString's "is_string" function to do the test. xml_like_long $xml, $xpath, $regex, 'test description' Find the XML corresponding to the the XPath expression and check it against the supplied regular expression. Succeeds if they match. Uses Test::More's "like" function to do the comparison. xml_like $xml, $xpath, $regex, 'test description' Find the XML corresponding to the the XPath expression and check it against the supplied regular expression. Succeeds if they match. Uses Test::LongString's "like_string" function to do the test. xml_is_deeply_long $xml, $xpath, $xml2, 'test description' Find the piece of XML corresponding to the XPath expression, and compare its structure and contents to the second XML (fragment) supplied. Succeeds if they match in structure and content. Uses Test::More's "is" function to do the comparison. xml_is_deeply $xml, $xpath, $xml2, 'test description' Find the piece of XML corresponding to the XPath expression, and compare its structure and contents to the second XML (fragment) supplied. Succeeds if they match in structure and content. Uses Test::LongString's "is_string" function to do the test. AUTHOR
Joe McMahon, <mcmahon@cpan.org> LICENSE
Copyright (c) 2005 by Yahoo! and Joe McMahon This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.6.1 or, at your option, any later version of Perl 5 you may have available. SEE ALSO
XML::XPath, Test::More, Test::Builder. perl v5.14.2 2012-06-05 Test::XML::Simple(3pm)
All times are GMT -4. The time now is 06:56 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy