09-13-2005
Removing duplicates
Hi, I've been trying to removed duplicates lines with similar columns in a fixed width file and it's not working.
I've search the forum but nothing comes close.
I have a sample file:
27147140631203RA CCD *
27147140631203RA PPN *
37147140631207RD AAA
47147140631203RD JNA
47147140631204DC ADK *
47147140631204DC ALK *
67147140631203DA ALM *
67147140631203DA CCD *
77147140631209QC RRP
87147140631203QA RRN
There are 3 spaces between first set of alphanumerics and the last three letter codes.
I want to remove lines that match only up to the 3 blanks and ignore the 3 letter codes or whatever else is on that line after the 3 letter codes.
Anyone know how I can do this? I want to keep at least one instance of any duplicates...doesn't matter which.
I put asteriks where I need to keep one of any two.
Thanks.
Gianni
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hello experts,
I am trying to remove all lines in a csv file where the 2nd columns is a duplicate. I am try to use sort with the key parameter
sort -u -k 2,2 File.csv > Output.csv
File.csv
File Name|Document Name|Document Title|Organization
Word Doc 1.doc|Word Document|Sample... (3 Replies)
Discussion started by: orahi001
3 Replies
2. Shell Programming and Scripting
Hi I have a file that are a list of people & their credentials i recieve frequently The issue is that whne I catnet this list that duplicat entries exists & are NOT CONSECUTIVE (i.e. uniq -1 may not weork here )
I'm trying to write a scrip that will remove duplicate entries
the script can... (5 Replies)
Discussion started by: stevie_velvet
5 Replies
3. Shell Programming and Scripting
Hi,
I have a file in the below format.,
test test (10)
to to (25)
see see (45)
and i need the output in the format of
test 10
to 25
see 45
Some one help me? (6 Replies)
Discussion started by: imdadulla
6 Replies
4. UNIX for Advanced & Expert Users
Hi All
In unix ,we have a file ,there we have to remove the duplicates by using one specific column.
Can any body tell me the command.
ex:
file1
id,name
1,ww
2,qwq
2,asas
3,asa
4,asas
4,asas
o/p:
1,ww
2,qwq
3,asa (7 Replies)
Discussion started by: raju4u
7 Replies
5. Shell Programming and Scripting
I have a test file with the following 2 columns:
Col 1 | Col 2
T1 | 1 <= remove
T5 | 1
T4 | 2
T1 | 3
T3 | 3
T4 | 1 <= remove
T1 | 2 <= remove
T3 ... (7 Replies)
Discussion started by: gctex
7 Replies
6. Emergency UNIX and Linux Support
i want to remove all the duplictaes in a file.I dont want even a single entry.
For the input data:
12345|12|34
12345|13|23
3456|12|90
15670|12|13
12345|10|14
3456|12|13
i need the below data in one file
15670|12|13
and the below data in another file (9 Replies)
Discussion started by: pandeesh
9 Replies
7. Shell Programming and Scripting
I have an input file abc.txt with info like:
abcd
rateuse
inklite
robet
rateuse
abcd
I need to remove duplicates from the file (eg: abcd,rateuse) from the file and need to place the contents in same file abc.txt if needed can be placed in another file.
can anyone help me in this :( (4 Replies)
Discussion started by: rkrish
4 Replies
8. UNIX for Dummies Questions & Answers
Hi All,
I am merging files coming from 2 different systems ,while doing that I am getting duplicates entries in the merged file
I,01,000131,764,2,4.00
I,01,000131,765,2,4.00
I,01,000131,772,2,4.00
I,01,000131,773,2,4.00
I,01,000168,762,2,2.00
I,01,000168,763,2,2.00... (5 Replies)
Discussion started by: Sri3001
5 Replies
9. Shell Programming and Scripting
Hi All,
i have a file like below,
@DB_FCTS\src\Data\Scripts\Delete_CU_OM_BIL_PRT_STMT_TYP.sql
@DB_FCTS\src\Data\Scripts\Delete_CDP_BILL_LBL_MSG.sql
@DB_FCTS\src\Data\Scripts\Delete_OM_BIDDR.sql
@DB_FCTS\src\Data\Scripts\Insert_CU_OM_LBL_MSG.sql... (11 Replies)
Discussion started by: mechvijays
11 Replies
10. Shell Programming and Scripting
i hav two files like
i want to remove/delete all the duplicate lines in file2 which are viz unix,unix2,unix3 (2 Replies)
Discussion started by: sagar_1986
2 Replies
LEARN ABOUT PHP
geoip_continent_code_by_name
GEOIP_CONTINENT_CODE_BY_NAME(3) 1 GEOIP_CONTINENT_CODE_BY_NAME(3)
geoip_continent_code_by_name - Get the two letter continent code
SYNOPSIS
string geoip_continent_code_by_name (string $hostname)
DESCRIPTION
The geoip_continent_code_by_name(3) function will return the two letter continent code corresponding to a hostname or an IP address.
PARAMETERS
o $hostname
- The hostname or IP address whose location is to be looked-up.
RETURN VALUES
Returns the two letter continent code on success, or FALSE if the address cannot be found in the database.
Continent codes
+-----+----------------+
|Code | |
| | |
| | Continent name |
| | |
+-----+----------------+
| | |
| AF | |
| | |
| | Africa |
| | |
| | |
| AN | |
| | |
| | Antarctica |
| | |
| | |
| AS | |
| | |
| | Asia |
| | |
| | |
| EU | |
| | |
| | Europe |
| | |
| | |
| NA | |
| | |
| | North america |
| | |
| | |
| OC | |
| | |
| | Oceania |
| | |
| | |
| SA | |
| | |
| | South america |
| | |
+-----+----------------+
EXAMPLES
Example #1
A geoip_continent_code_by_name(3) example
This will print where the host example.com is located.
<?php
$continent = geoip_continent_code_by_name('www.example.com');
if ($continent) {
echo 'This host is located in: ' . $continent;
}
?>
The above example will output:
This host is located in: NA
SEE ALSO
geoip_country_code_by_name(3).
PHP Documentation Group GEOIP_CONTINENT_CODE_BY_NAME(3)