03-09-2006
Remove duplicate ???
Hi all,
I have a out.log file
CARR|02/26/2006 10:58:30.107|CDxAcct=1405157051
CARR|02/26/2006 11:11:30.107|CDxAcct=1405157051
CARR|02/26/2006 11:18:30.107|CDxAcct=7659579782
CARR|02/26/2006 11:28:30.107|CDxAcct=9534922327
CARR|02/26/2006 11:38:30.107|CDxAcct=9534922327
CARR|02/26/2006 11:49:30.107|CDxAcct=1405157067
LMWC|02/26/2006 11:59:28.307|CDxAcct=7659579780
LMWC|02/26/2006 12:01:28.307|Case Modify 10021 20024
LMWC|02/26/2006 12:11:28.307|Case Access 10021 20025
LMWC|02/26/2006 12:14:28.307|Case Modify 10022 20026
LMWC|02/26/2006 12:29:28.307|Case Modify 10023 20027
How do I write a script to get the result like this ??
CARR,02/26/2006 10:58:30.107,CDxAcct=1405157051|CDxAcct=7659579782|CDxAcct=9534922327|CDxAcct=1405157067
LMWC,02/26/2006 11:59:28.307,CDxAcct=7659579780|Case Modify 10021|Case Modify 10022|Case Modify 10023
Here is my bad code
#!/bin/ksh
cd /DAT/
cat out.log |
awk -F"|" '
function writeit()
{ printf "%s,%s\,%s\n\n",outputId,datetime,outputAction;
outputAction = "";
return
} {
id=$1;
datetime=$2;
action = $3;
lastacct = "";
if (outputId == ""){
outputId = id;
}
if (id != outputId ) {
printf "!!!! A: %s %s\n",outputId;
writeit();
outputId = id;
}
if (action ~ /CDxAcct/) {
acct = substr(action,9,10);
if (acct != lastacct) {
outputAction = outputAction "CDxAcct=" acct "|";
lastacct = acct;
} ## Remove duplicate acct like CDxAcct=1405157051|CDxAcct=1405157051|CDxAcct=7659579782
} else if (action ~ /Modify/) {
outputAction = action "|";
type = substr(action,1,index(action,"Modify")-2); #get Case
mainkey = substr(action,index(action,"Modify")+7,index(substr(action,index(action,"Modify")+7,20)," ")-1); #get number after Modify
searchspec1 = type " " mainkey;
searchspec2 = type " Modify " mainkey;
searchspec3 = type " Access " mainkey;
if ((outputAction ~ searchspec1) || (outputAction ~ searchspec2)) {
} else if ((outputAction ~ searchspec3)) {
# NEEDS FURTHER WORK To remove the Case Access XX from Case Modify XX
outputAction = outputAction action "|";
} else {
outputAction = outputAction action "|";
}
}
} END {
writeit();
}'
Thanks for your help.
Last edited by sabercats; 03-10-2006 at 05:15 PM..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
i have a text its contain many record, but its written in one line,
i want to remove from that line the duplicate record,
not record have fixed width ex: width = 4
inputfile test.txt =abc cdf abc abc cdf fgh fgh abc abc
i want the outputfile =abc cdf fgh
only those records
can any one help... (4 Replies)
Discussion started by: kazanoova2
4 Replies
2. Shell Programming and Scripting
Hi all,
I have a text file fileA.txt
DXRV|02/28/2006 11:36:49.049|SAC||||CDxAcct=2420991350
DXRV|02/28/2006 11:37:06.404|SAC||||CDxAcct=6070970034
DXRV|02/28/2006 11:37:25.740|SAC||||CDxAcct=2420991350
DXRV|02/28/2006 11:38:32.633|SAC||||CDxAcct=6070970034
DXRV|02/28/2006... (2 Replies)
Discussion started by: sabercats
2 Replies
3. UNIX for Dummies Questions & Answers
Hi,
I have a list of numbers stored in an array as below.
5 7 10 30 30 40 50
Please advise how could I remove the duplicate value in the array ?
Thanks in advance. (5 Replies)
Discussion started by: Rock
5 Replies
4. Shell Programming and Scripting
Hi,
I am tryung to use shell or perl to remove duplicate characters
for example , if I have " I love google" it will become I love ggle"
or even "I loveggle" if removing duplicate white space
Thanks
CC (6 Replies)
Discussion started by: ccp
6 Replies
5. Shell Programming and Scripting
Input file
data_1 10 US
data_1 2 US
data_1 5 UK
data_2 20 ENGLAND
data_2 12 KOREA
data_3 4 CHINA
.
.
data_60 123 US
data_60 23 UK
data_60 45 US
Desired output file
data_1 10 US
data_1 5 UK
data_2 20 ENGLAND
data_2 12 KOREA (2 Replies)
Discussion started by: perl_beginner
2 Replies
6. Shell Programming and Scripting
HI
I have file contains 1000'f of duplicate id's with (upper and lower first character) as below
i/p:
a411532A411532a508661A508661c411532C411532
Requirement: But i need to ignore lowercase id's and need only below id's
o/p:
A411532
A508661
C411532 (9 Replies)
Discussion started by: buzzme
9 Replies
7. Shell Programming and Scripting
Hi ,
I have a pipe seperated file repo.psv where i need to remove duplicates based on the 1st column only. Can anyone help with a Unix script ?
Input:
15277105||Common Stick|ESHR||Common Stock|CYRO AB
15277105||Common Stick|ESHR||Common Stock|CYRO AB
16111278||Common Stick|ESHR||Common... (12 Replies)
Discussion started by: samrat dutta
12 Replies
8. UNIX for Dummies Questions & Answers
Hi,
How can I replace || with space and then remove duplicate from following text?
T111||T222||T444||T222||T555
Thanks in advance (10 Replies)
Discussion started by: tinku981
10 Replies
9. UNIX for Dummies Questions & Answers
Hi
I need this output. Thanks.
Input:
TAZ
YET
FOO
FOO
VAK
TAZ
BAR
Output:
YET
VAK
BAR (10 Replies)
Discussion started by: tara123
10 Replies
10. Shell Programming and Scripting
Hi All,
I am storing the result in the variable result_text using the below code.
result_text=$(printf "$result_text\t\n$name") The result_text is having the below text. Which is having duplicate lines.
file and time for the interval 03:30 - 03:45
file and time for the interval 03:30 - 03:45 ... (4 Replies)
Discussion started by: nalu
4 Replies
LEARN ABOUT DEBIAN
lr_check_service
LR_CHECK_SERVICE.IN(1) LogReport's Lire Documentation LR_CHECK_SERVICE.IN(1)
NAME
lr_check_service - check existence of Lire service
SYNOPSIS
lr_check_service service
lr_check_service -l
DESCRIPTION
lr_check_service checks if service service is supported in Lire, i.e. if there is a DLF converter registered under that name.
When called with the -l (for `list') option, lr_check_service lists all registered DLF converters.
Like all Lire scripts, you might want to call this script via lr_run(1), to deal sanely with this scripts' debug- and logging output.
You might prefer the lr_log2report(1) --help options to list available services: this command creates a more human-friendly format of the
list.
EXAMPLE
Call this script in shell scripts as
LR_SERVICE="$1"
LR_SERVICE=`lr_check_service "$LR_SERVICE" || true`
if test -z "$LR_SERVICE"
then
echo >&2 "$tag err lr_check_service failed"
exit 1
fi
# now go using $LR_SERVICE
.
SEE ALSO
Lire::PluginManager(3pm), lr_log2report(1)
VERSION
$Id: lr_check_service.in,v 1.13 2006/07/23 13:16:32 vanbaal Exp $
COPYRIGHT
Copyright (C) 2002-2003 Stichting LogReport Foundation LogReport@LogReport.org
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program (see COPYING); if not, check with
http://www.gnu.org/copyleft/gpl.html.
AUTHOR
Joost van Baal <joostvb@logreport.org>
Lire 2.1.1 2006-07-23 LR_CHECK_SERVICE.IN(1)