10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
There are duplicate entries in file, but uniq will not see because first field is different. How will I catch all lines, which are having duplicate IPs ?
bash-2.05# cat db.file | grep 172.30.133.11
dsrq-ctrl1-prod A 172.30.133.11
e911q-db1-nxge0 A 172.30.133.11... (4 Replies)
Discussion started by: solaris_1977
4 Replies
2. Shell Programming and Scripting
I have a script that builds a database ~30 million lines, ~3.7 GB .cvs file. After multiple optimzations It takes about 62 min to bring in and parse all the files and used to take 10 min to remove duplicates until I was requested to add another column. I am using the highly optimized awk code:
awk... (34 Replies)
Discussion started by: Michael Stora
34 Replies
3. Shell Programming and Scripting
Dear all
From below mention input file I needed op file as show below. I am using below code but not worked.
I/p file
BSCBCH1 EXAL-1-4 WO* SMPS MAINS FAIL
BSCBCH1 EXAL-1-5 WO* SMPS RECTIFIER FAIL
BSCBCH1 EXAL-1-6 WO* SMPS MAJOR ALARM
BSCBCH2 EXAL-1-10 WO* ... (5 Replies)
Discussion started by: jaydeep_sadaria
5 Replies
4. UNIX for Advanced & Expert Users
I have an input file of 5GB which contains duplicate records and have to remove duplicate records by retaing first instance of that record .
Based on 5 fields the duplicates has to be removed .
Kindly request to help me in writing a Unix Script.
Thanks
Asim (11 Replies)
Discussion started by: duplicate
11 Replies
5. Shell Programming and Scripting
I'm looking to remove duplicate rows from a CSV file with a twist.
The first row is a header.
There are 31 columns. I want to remove duplicates when the first 29 rows are identical ignoring row 30 and 31 BUT the duplicate that is kept should have the shortest total character length in rows 30... (6 Replies)
Discussion started by: Michael Stora
6 Replies
6. Shell Programming and Scripting
Hi,
I have a very big (with around 1 million entries) txt file with IPv4 addresses in the standard format, i.e. a.b.c.d
The file looks like
10.1.1.1
10.1.1.1
10.1.1.1
10.1.2.4
10.1.2.4
12.1.5.6
.
.
.
.
and so on....
There are duplicate/multiple entries for some IP... (3 Replies)
Discussion started by: sajal.bhatia
3 Replies
7. Shell Programming and Scripting
Hi,
I forgot how to start a new thread. :(
Can somebody please guide me?
I have one problem related to /.rhosts file.
According to my understanding, /.rhosts file is used for "rsh". What will happen if I have duplicate entries in this file?
e.g> my .rhosts file looks like
wcars42g... (2 Replies)
Discussion started by: akash_mahakode
2 Replies
8. Shell Programming and Scripting
Hi All,
I have one problem related to /.rhosts file.
According to my understanding, /.rhosts file is used for "rsh". What will happen if I have duplicate entries in this file?
e.g> my .rhosts file looks like
Code:
wcars42g
wcars89j
wcars42g
wcars42b
wcars42b
Will duplicate entries... (1 Reply)
Discussion started by: akash_mahakode
1 Replies
9. UNIX for Dummies Questions & Answers
Hi All,
I have a problem with the sort and duplicate filter command I am using in one of my scripts. I have a '|' delimited file and want to sort and remove duplicates on the 1,2,15 fields. These fields constitute the primary key of the table I will be loading the data into. But I see that some... (4 Replies)
Discussion started by: novice1324
4 Replies
10. Shell Programming and Scripting
I need a KSH script that will check a host file for duplicate IP's and/or host names and report out the errors. Anyone out there have one they would like to share?
Something like:
Hostname blahblah appears X times
IP Address xxx.xxx.xxx.xxx appears X times
TIA (4 Replies)
Discussion started by: ThreeDot
4 Replies