Sponsored Content
Top Forums Shell Programming and Scripting Removing duplicate lines on first column based with pipe delimiter Post 302956184 by RudiC on Monday 28th of September 2015 06:19:07 AM
Old 09-28-2015
Please use code tags as required by forum rules!

I'm getting
Code:
02006YB|09/25/15|0.859
12593PS|09/25/15|2.803
14041NL|09/25/15|1.415
38376KZ|09/25/15|1.057

The fourth line in your desired output is a duplicate to the third input line due to the -n option.
A possible reason for the missing line is you have a <CR> char as a line terminator somewhere that causes overwriting a line.
This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

removing duplicate lines from a file

Hi, I am trying to remove duplicate lines from a file. For example the contents of example.txt is: this is a test 2342 this is a test 34343 this is a test 43434 and i want to remove the "this is a test" lines only and end up with the numbers in the file, that is, end up with: 2342... (4 Replies)
Discussion started by: ocelot
4 Replies

2. Shell Programming and Scripting

removing duplicate blank lines

Hi, how to remove the blank lines from the file only If we have more than one blank line. thanks rameez (8 Replies)
Discussion started by: rameezrajas
8 Replies

3. Shell Programming and Scripting

Removing Duplicate Lines per Section

Hello, I am in need of removing duplicate lines from within a file per section. File: ABC1 012345 header ABC2 7890-000 ABC3 012345 Header Table ABC4 ABC5 593.0000 587.4800 ABC5 593.5000 587.6580 <= dup need to remove ABC5 593.5000 ... (5 Replies)
Discussion started by: petersf
5 Replies

4. UNIX for Dummies Questions & Answers

Removing Lines based on matching first column

I have a file1 that looks like this: File 1 a b b c c e d e and a file 2 that looks like this: File 2 b c e e Note that file 2 is the right hand column from file1. I want to remove any lines from file1 that begin with the column in file2. In this case the desired output... (6 Replies)
Discussion started by: kschiltz55
6 Replies

5. Shell Programming and Scripting

Removing duplicate records in a file based on single column

Hi, I want to remove duplicate records including the first line based on column1. For example inputfile(filer.txt): ------------- 1,3000,5000 1,4000,6000 2,4000,600 2,5000,700 3,60000,4000 4,7000,7777 5,999,8888 expected output: ---------------- 3,60000,4000 4,7000,7777... (5 Replies)
Discussion started by: G.K.K
5 Replies

6. Shell Programming and Scripting

Perl: filtering lines based on duplicate values in a column

Hi I have a file like this. I need to eliminate lines with first column having the same value 10 times. 13 18 1 + chromosome 1, 122638287 AGAGTATGGTCGCGGTTG 13 18 1 + chromosome 1, 128904080 AGAGTATGGTCGCGGTTG 13 18 1 - chromosome 14, 13627938 CAACCGCGACCATACTCT 13 18 1 + chromosome 1,... (5 Replies)
Discussion started by: polsum
5 Replies

7. Shell Programming and Scripting

Removing duplicate records in a file based on single column explanation

I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1. https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html Can someone explain this command please? How are there no... (5 Replies)
Discussion started by: cokedude
5 Replies

8. UNIX for Dummies Questions & Answers

awk solution to duplicate lines based on column

Hi experts, I have a tab-delimited file with one column containing values separated by a comma. I wish to duplicate the entire line for every value in that comma-delimited field. For example: $cat file 4444 4444 4444 4444 9990 2222,7777 6666 2222 ... (3 Replies)
Discussion started by: torchij
3 Replies

9. Shell Programming and Scripting

awk to sum a column based on duplicate strings in another column and show split totals

Hi, I have a similar input format- A_1 2 B_0 4 A_1 1 B_2 5 A_4 1 and looking to print in this output format with headers. can you suggest in awk?awk because i am doing some pattern matching from parent file to print column 1 of my input using awk already.Thanks! letter number_of_letters... (5 Replies)
Discussion started by: prashob123
5 Replies

10. Shell Programming and Scripting

Replace delimiter for a particular column in a pipe delimited file

I have an input file as below Emp1|FirstName|MiddleName|LastName|Address|Pincode|PhoneNumber 1234|FirstName1|MiddleName2|LastName3| Add1 || ADD2|123|000000000 Output : 1234|FirstName1|MiddleName2|LastName3| Add1 ,, ADD2|123|000000000 OR 1234,FirstName1,MiddleName2,LastName3, Add1 ||... (2 Replies)
Discussion started by: styris
2 Replies
DIALTEST(8)						      System Manager's Manual						       DIALTEST(8)

NAME
dialtest - HylaFAX dial string processing rules test program SYNOPSIS
/usr/sbin/dialtest [ options ] dialrules DESCRIPTION
dialtest is an interactive program for the testing and development of dial string processing rules used by HylaFAX. dialtest reads the file of rules specified on the command line and then prompts for a dial string. For each string typed in, dialtest prints the result of applying the canonicalization and dial-string preparation rule sets. A rule set whose name is setname can be specified by typing ``set- name(string)''; for example, ``CanonicalName(+1.415.965.7824)''. Note that dialtest reads the rules file only when it is initially started up. This means that dialtest must be restarted for it to apply changes to a rules file. OPTIONS
-a code Use code as the value of AreaCode in the rules file. The default area code is ``415''. -c code Use code as the value of CountryCode in the rules file. The default country code is ``1''. -i prefix Use prefix as the value of InternationalPrefix in the rules file. The default international dialing prefix is ``011''. -l prefix Use prefix as the value of LongDistancePrefix in the rules file. The default long distance dialing prefix is ``1''. -q Be quiet, suppressing most all output except for the actual results. This is useful for using dialtest in an automated fashion. -v Print each input line before displaying the results of applying the dial string rules to the input string. This can be useful, for example, in shell scripts. SEE ALSO
dialrules(5) March 3, 1995 DIALTEST(8)
All times are GMT -4. The time now is 12:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy