Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Problem with a record

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Problem with a record
# 1  
Old 06-06-2017
Problem with a record
hi,
i have a problem with linux redhat .
I have a sequencial files and i want to delete a duplicate records .
How i can to do.
Thanks lot
Regards
# 2  
Old 06-06-2017
What is the file name ?

try
Code:
sort -u <filename> >> new_filename
rm filename
mv new_filename filename

# 3  
Old 06-06-2017
example :

Code:
100212903062|74247966|MTW|2013082712:00:00|2017052512:00:00|2017052512:00:00|1-2PKWW7I|BTS_FLAS|F||ABR|7000.0|ATM||8|35|256.0|256.0|2130|11|271|256.0||MILAITBR|0021290
3062  W40|2130|0255017800|


Code:
100212903062|74247966|MTW|2013082712:00:00|2017052512:00:00|2017052512:00:00|1-2PKWW7P|BTS_FLAS|F||ABR|7000.0|ATM||8|36|256.0|256.0|2130|11|272|256.0||MILAITBR|0021290
3062  W40|2130|0255017800

i want to delete a row beacase there is a duplicate file in the first field .
Thanks a lot


Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!
Last edited by RudiC; 06-06-2017 at 04:52 PM.. Reason: Added CODE tags.
# 4  
Old 06-06-2017
I guess - as you didn't specify a field separator - the pipe chars are, and your first field is 100212903062. Did you consider sort's -k option to define key fields?
# 5  
Old 06-07-2017
Sort my file with option
Code:
more <file> | sort -k1

how can i delete a duplicate row?
Thanks a lot
Regards
Last edited by vbe; 06-26-2017 at 05:29 AM.. Reason: code tags
# 6  
Old 06-07-2017
I don't think you want to use more file | sort ....

Did you try arunkumar_mca's proposal, and does it do what you want for identical lines? If yes, extend the sort by -t"|" -k1,1. And, read the man page...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract timestamp from first record in xml file and it checks if not it will replace first record

I have test.xml <emp><id>101</id><name>AAA</name><date>06/06/14 1811</date></emp> <Join><id>101</id><city>london</city><date>06/06/14 2011</date></join> <Join><id>101</id><city>new york</city><date>06/06/14 1811</date></join> <Join><id>101</id><city>sydney</city><date>06/06/14... (2 Replies)
Discussion started by: vsraju
2 Replies

2. Shell Programming and Scripting

Problem facing to compare different column and print out record with smallest number

Hi, Input file 1 : 37170 37196 77 51 37174 37195 73 52 37174 37194 73 53 Desired Output file 1 : 37170 37196 77 51 Input file 2 : 37174 37195 73 0 37170 37196 77 0 Desired Output file 2 : 37174 37195 73 0 (1 Reply)
Discussion started by: cpp_beginner
1 Replies

3. Shell Programming and Scripting

Problem to print out record got smallest number in specific column

Hi, Anybody know how to print out the record that shown smallest number among column 3 and column 4 Case 1 Input : 37170 37196 77 51 37174 37195 73 52 37174 37194 73 53 Case 1 Output : 37170 37196 77 51 Case 2 Input : 469613 469660 73 ... (4 Replies)
Discussion started by: cpp_beginner
4 Replies

4. Shell Programming and Scripting

Execution problem with print out record that follow specific pattern

Hi, Do anybody know how to print out only those record that column 1 is "a" , then followed by "b"? Input file : a comp92 2404242 2405172 b comp92 2405303 2406323 b comp92 2408786 2410278 a comp92 2410271 2410337 a comp87 1239833 1240418 b comp87... (3 Replies)
Discussion started by: patrick87
3 Replies

5. Shell Programming and Scripting

Help with sum total number of record and total number of record problem asking

Input file SFSQW 5192.56 HNRNPK 611.486 QEQW 1202.15 ASDR 568.627 QWET 6382.11 SFSQW 4386.3 HNRNPK 100 SFSQW 500 Desired output file SFSQW 10078.86 3 QWET 6382.11 1 QEQW 1202.15 1 HNRNPK 711.49 2 ASDR 568.63 1 The way I tried: (2 Replies)
Discussion started by: patrick87
2 Replies

6. Shell Programming and Scripting

problem in reading a record from the file

Hi Guys, I need to check whether the last column is RP, If so, then i have to second column and pass it to a select statement as sonid and fetch the value to a variable and run it. This has to be done till the last column is RW. value Fatherid sonid topid ... (8 Replies)
Discussion started by: mac4rfree
8 Replies

7. Shell Programming and Scripting

filter the uniq record problem

Anyone can help for filter the uniq record for below example? Thank you very much Input file 20090503011111|test|abc 20090503011112|tet1|abc|def 20090503011112|test1|bcd|def 20090503011131|abc|abc 20090503011131|bbc|bcd 20090503011152|bcd|abc 20090503011151|abc|abc... (8 Replies)
Discussion started by: bleach8578
8 Replies

8. UNIX for Dummies Questions & Answers

Record count problem using sed command

Hi, I have a script which removes 2 header records and 1 trailer record in a list of files. The commands doing the actions are sed '1,2d' $file > tempfile1.dat sed '$d' < tempfile1.dat > $output.txt Its working fine for all records except a file having size=1445509814 and number of... (2 Replies)
Discussion started by: ayanbiswas
2 Replies

9. Shell Programming and Scripting

awk & cut record separator problem

Hi All, I've got some strange behaviour going on when trying to manipulate a file that contains spaces. My input file looks something like this: xxxxxxxxx,yyyy,sss sss sss,bbbbbbb If I use awk: When running from the command line I get: sss sss sss But when running from a... (7 Replies)
Discussion started by: pondlife
7 Replies

10. Shell Programming and Scripting

Record Length too long -- AWK Problem

Hi All, I have a txt file which is an export of a query result from the database. The txt file contains 'Processid#sqlquery' from the database table.As the sqlquery is too long.... i am unable to get the fields seperated using the awk script as below:- cat sql.txt | awk -F'#' '{printf $2}'... (2 Replies)
Discussion started by: venkatajay_18
2 Replies
Login or Register to Ask a Question