10-29-2012
the duplicate entries in all the columns should be removed...and sorry for the bad post
10 More Discussions You Might Find Interesting
1. Linux
I have huge txt file having millions of trade data.
For e.g
Trade.txt (first 8 lines in the file is header info)
COB_DATE,TRADE_ID,SOURCE_SYSTEM_TRADE_ID,TRADE_GROUP_ID,
TRADE_TYPE,DEALER_NAME,EXTERNAL_COUNTERPARTY_ID,
EXTERNAL_COUNTERPARTY_NAME,DB_COUNTERPARTY_ID,... (6 Replies)
Discussion started by: nmumbarkar
6 Replies
2. Shell Programming and Scripting
Can anyone help me to removing duplicate records from 2 separate files in UNIX?
Please find the sample records for both the files
cat Monday.dat
3FAHP0JA1AR319226MOHMED ATEK 966504453742 SAU2010DE
3LNHL2GC6AR636361HEA DEUK CHOI 821057314531 KOR2010LE
3MEHM0JG7AR652083MUTLAB NAL-NAFISAH... (4 Replies)
Discussion started by: zooby
4 Replies
3. Shell Programming and Scripting
Hi,
I have a very big (with around 1 million entries) txt file with IPv4 addresses in the standard format, i.e. a.b.c.d
The file looks like
10.1.1.1
10.1.1.1
10.1.1.1
10.1.2.4
10.1.2.4
12.1.5.6
.
.
.
.
and so on....
There are duplicate/multiple entries for some IP... (3 Replies)
Discussion started by: sajal.bhatia
3 Replies
4. Shell Programming and Scripting
Input:
hello hello
hello hello
monkey
donkey
hello hello
drink
dance
drink
Output should be:
hello hello
monkey
donkey
drink
dance (9 Replies)
Discussion started by: cola
9 Replies
5. Post Here to Contact Site Administrators and Moderators
I have made some threads that were identical and were about the same question :( I've made them in 3 forums , the moderator has moved and merged one of these threads. There is one thread left and it need to be merged or deleted. Is there any way I can delete it or merge it myself ? I have delete... (1 Reply)
Discussion started by: k.a.docpp
1 Replies
6. Linux
I have log file having Traffic line
2011-05-21 15:11:50.356599 TCP (6), length: 52) 10.10.10.1.3020 > 10.10.10.254.50404:
2011-05-21 15:11:50.652739 TCP (6), length: 52) 10.10.10.254.50404 > 10.10.10.1.3020:
2011-05-21 15:11:50.652558 TCP (6), length: 89) 10.10.10.1.3020 >... (1 Reply)
Discussion started by: Rastamed
1 Replies
7. Shell Programming and Scripting
Hi,
I have the following command in place
nawk -F, '!a++' file > file.uniq
It has been working perfectly as per requirements, by removing duplicates by taking into consideration only first 3 fields. Recently it has started giving below error:
bash-3.2$ nawk -F, '!a++'... (17 Replies)
Discussion started by: makn
17 Replies
8. Shell Programming and Scripting
Hi everybody
I have a .txt file that contains some assembly code for optimizing it i need to remove some replicated parts.
for example I have:e_li r0,-1
e_li r25,-1
e_lis r25,0000
add r31, r31 ,r0
e_li r28,-1
e_lis r28,0000
add r31, r31 ,r0
e_li r28,-1 ... (3 Replies)
Discussion started by: Behrouzx77
3 Replies
9. Shell Programming and Scripting
Hello..
I am trying to remove the duplicate entries in a log files and used the the below shell script to do the same.
awk '!x++' <filename>
Can I do without using the awk command and the regex? I do not want to start the search from the beginning of the line in the log file as it contains... (9 Replies)
Discussion started by: sandeepcm
9 Replies
10. Shell Programming and Scripting
I have a file which has connections given as:
A B 0.1
B C 5.8
C B 5.8
E F 0.67
B A 0.1
A B and B A are same, so I want to remove one of them. Same with BC and CB.
Desired output:
A B 0.1
B C 5.8
E F 0.67 (2 Replies)
Discussion started by: Sanchari
2 Replies
LEARN ABOUT DEBIAN
bcfg2-reports
bcfg2-reports(8) System Manager's Manual bcfg2-reports(8)
NAME
bcfg2-reports - Query reporting system for client status
SYNOPSIS
bcfg2-reports [-v]
DESCRIPTION
bcfg2-reports allows you to retrieve data from the database about clients, and the states of their current interactions. It also allows you
to change the expired/unexpired states. The utility runs as a standalone application. It does, however, use the models from
/src/lib/Server/Reports/reports/models.py.
OPTIONS
-a
Shows all hosts, including expired hosts.
-b NAME
Single-host mode - shows bad entries from the current interaction of NAME. NAME is the name of the entry.
-c
Shows only clean hosts.
-d
Shows only dirty hosts.
-e NAME
Single host mode - shows extra entries from the current interaction of NAME. NAME is the name of the entry.
-h
Shows help and usage info about bcfg2-reports.
-m NAME
Single-host mode - shows modified entries from the current interaction of NAME. NAME is the name of the entry.
-s NAME
Single host mode - shows bad, modified, and extra entries from the current interaction of NAME. NAME is the name of the entry.
-x NAME
Toggles expired/unexpired state of NAME. NAME is the name of the entry.
--badentry=KIND,NAME
Shows only hosts whose current interaction has bad entries in of KIND kind and NAME name; if a single argument ARG1 is given, then
KIND,NAME pairs will be read from a file of name ARG1. KIND is the type of entry (Package, Path, Service, etc). NAME is the name of
the entry.
--extraentry=KIND,NAME
Shows only hosts whose current interaction has extra entries in of KIND kind and NAME name; if a single argument ARG1 is given, then
KIND,NAME pairs will be read from a file of name ARG1. KIND is the type of entry (Package, Path, Service, etc). NAME is the name of
the entry.
--fields=ARG1,ARG2,...
Only displays the fields ARG1,ARG2,... (name, time, state, total, good, bad)
--modifiedentry=KIND,NAME
Shows only hosts whose current interaction has modified entries in of KIND kind and NAME name; if a single argument ARG1 is given,
then KIND,NAME pairs will be read from a file of name ARG1. KIND is the type of entry (Package, Path, Service, etc). NAME is the
name of the entry.
--sort=ARG1,ARG2,...
Sorts output on ARG1,ARG2,... (name, time, state, total, good, bad)
--stale
Shows hosts which haven't run in the last 24 hours
SEE ALSO
bcfg2(1), bcfg2-server(8)
BUGS
None currently known
bcfg2-reports(8)