Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Remove Duplicate lines from File Post 302135012 by psiva_arul on Wednesday 5th of September 2007 05:21:08 AM
Old 09-05-2007
Hi All,

I have one below text file which is seperated by "|". This file has two Rows each and every row has three values(file_header,file_header and invoice_detail). like this

file_header|GEES-EU|343|2007-08-29T07:10:51|SSPF|12|2008|0|0|SSS||
invoice_header|39478-198192-2|250489|BRIDGEWATER108|2007-08-28T00:00:00|1|STANDARD|Receipt Invoic
invoice_detail|39478-198192-2|2|ITEM|1|1|1|AUG-07|5896.45820.000.00.00.000000.000|Advert|856974521|257ZA|zas78

file_header|GEAS-EU|343|2007-08-29T07:10:51|SSPF|12|2008|0|0|SSS||
invoice_header|35818-198192-2|250489|BRIDGEWATER108|2007-08-28T00:00:00|1|STANDARD|Receipt Invoic
invoice_detail|1235-198192-2|2|ITEM|1|1|1|AUG-07|17.121504.45820.000.00.00.000000.000|Advert|123456789|257ZA|zas78

Expected output:
=============

i want cut the above 11th column from the above inpu file (we need to cut the 11th column from invoice_detail row.

856974521
123456789

Thanks in advance,

Thanks,
Siva.P
Bangalore.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Duplicate Lines in File

I am doing KSH script to remove duplicate lines in a file. Let say the file has format below. FileA 1253-6856 3101-4011 1827-1356 1822-1157 1822-1157 1000-1410 1000-1410 1822-1231 1822-1231 3101-4011 1822-1157 1822-1231 and I want to simply it with no duplicate line as file... (5 Replies)
Discussion started by: Teh Tiack Ein
5 Replies

2. Shell Programming and Scripting

Command/Script to remove duplicate lines from the file?

Hello, Can anyone tell Command/Script to remove duplicate lines from the file? (2 Replies)
Discussion started by: Rahulpict
2 Replies

3. UNIX for Dummies Questions & Answers

How to delete or remove duplicate lines in a file

Hi please help me how to remove duplicate lines in any file. I have a file having huge number of lines. i want to remove selected lines in it. And also if there exists duplicate lines, I want to delete the rest & just keep one of them. Please help me with any unix commands or even fortran... (7 Replies)
Discussion started by: reva
7 Replies

4. Shell Programming and Scripting

remove duplicate lines from file linux/sh

greetings, i'm hoping there is a way to cat a file, remove duplicate lines and send that output to a new file. the file will always vary but be something similar to this: please keep in mind that the above could be eight occurrences of each hostname or it might simply have another four of an... (2 Replies)
Discussion started by: crimso
2 Replies

5. Shell Programming and Scripting

Remove duplicate lines from first file comparing second file

Hi, I have two files with below data:: file1:- 123|aaa|ppp 445|fff|yyy 999|ttt|jjj 555|hhh|hhh file2:- 445|fff|yyy 555|hhh|hhh The records present in file1, not present in file 2 should be writtent to the out put file. output:- 123|aaa|ppp 999|ttt|jjj Is there any one line... (3 Replies)
Discussion started by: gani_85
3 Replies

6. Shell Programming and Scripting

How do I remove the duplicate lines in this file?

Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file. I wrote the following script, but does not work. What is the problem? The output file should only contain five lines: Later! (5 Replies)
Discussion started by: Ernst
5 Replies

7. Shell Programming and Scripting

Remove duplicate lines from a 50 MB file size

hi, Please help me to write a command to delete duplicate lines from a file. And the size of file is 50 MB. How to remove duplicate lins from such a big file. (6 Replies)
Discussion started by: vsachan
6 Replies

8. Shell Programming and Scripting

Remove duplicate lines from a file

Hi, I have a csv file which contains some millions of lines in it. The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line). I don't want to use any pattern from the Header as I have some... (7 Replies)
Discussion started by: sudhakar T
7 Replies

9. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies

10. Shell Programming and Scripting

Remove duplicate lines, sort it and save it as file itself

Hi, all I have a csv file that I would like to remove duplicate lines based on 1st field and sort them by the 1st field. If there are more than 1 line which is same on the 1st field, I want to keep the first line of them and remove the rest. I think I have to use uniq or something, but I still... (8 Replies)
Discussion started by: refrain
8 Replies
CMP(1)								   User Commands							    CMP(1)

NAME
cmp - compare two files byte by byte SYNOPSIS
cmp [OPTION]... FILE1 [FILE2 [SKIP1 [SKIP2]]] DESCRIPTION
Compare two files byte by byte. -b --print-bytes Print differing bytes. -i SKIP --ignore-initial=SKIP Skip the first SKIP bytes of input. -i SKIP1:SKIP2 --ignore-initial=SKIP1:SKIP2 Skip the first SKIP1 bytes of FILE1 and the first SKIP2 bytes of FILE2. -l --verbose Output byte numbers and values of all differing bytes. -n LIMIT --bytes=LIMIT Compare at most LIMIT bytes. -s --quiet --silent Output nothing; yield exit status only. -v --version Output version info. --help Output this help. SKIP1 and SKIP2 are the number of bytes to skip in each file. SKIP values may be followed by the following multiplicative suffixes: kB 1000, K 1024, MB 1,000,000, M 1,048,576, GB 1,000,000,000, G 1,073,741,824, and so on for T, P, E, Z, Y. If a FILE is `-' or missing, read standard input. Exit status is 0 if inputs are the same, 1 if different, 2 if trouble. AUTHOR
Written by Torbjorn Granlund and David MacKenzie. REPORTING BUGS
Report bugs to <bug-gnu-utils@gnu.org>. COPYRIGHT
Copyright (C) 2007 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. SEE ALSO
The full documentation for cmp is maintained as a Texinfo manual. If the info and cmp programs are properly installed at your site, the command info diff should give you access to the complete manual. diffutils 2.8.7-cvs January 2008 CMP(1)
All times are GMT -4. The time now is 02:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy