Avoiding duplicates with some special case


 
Thread Tools Search this Thread
Operating Systems SCO Avoiding duplicates with some special case
# 1  
Old 03-24-2008
Avoiding duplicates with some special case

Hi Gurus,

I had a question regarding avoiding duplicates.i have a file abc.txt
abc.txt
-------
READER_1_1_1> HIER_28056 XML Reader: Error [UnterminatedXMLDecl] occurred while parsing:[Error at (file /home/abc.xml, line 6, char 1 ): Invalid document structure.]; line number [6]; column number [1]
READER_1_3_1> Sun Mar 23 23:52:48 2008
READER_1_3_1> HIER_28056 XML Reader: Error [UnterminatedXMLDecl] occurred while parsing:[Error at (file /home/abc.xml, line 6, char 1 ): Invalid document structure.]; line number [6]; column number [1]
READER_1_2_1> Sun Mar 23 23:52:48 2008
READER_1_2_1> HIER_28056 XML Reader: Error [UnterminatedXMLDecl] occurred while parsing:[Error at (file /home/abc.xml, line 6, char 1 ): Invalid document structure.]; line number [6]; column number [1]
READER_1_1_1> Sun Mar 23 23:52:48 2008
READER_1_1_1> HIER_28058 XML Reader Error
READER_1_2_1> Sun Mar 23 23:52:48 2008
READER_1_2_1> HIER_28058 XML Reader Error
READER_1_3_1> Sun Mar 23 23:52:48 2008
READER_1_3_1> HIER_28058 XML Reader Error.


I will get this file when i egrep the session log file for error specific messages.

here i want to remove the repeated lines in this file?
i am try to use uniq command, but here READER_1_3_1> will be varying in the above abc.txt file
i.e., in every line we have 1,2,3 in place of READER_1_**_1>.

So i am not getting the desired output.

Can any one let me know how to get the requirement done.

My output should be
READER_1_1_1> HIER_28056 XML Reader: Error [UnterminatedXMLDecl] occurred while parsing:[Error at (file /home/abc.xml, line 6, char 1 ): Invalid document structure.]; line number [6]; column number [1]
READER_1_1_1> HIER_28058 XML Reader Error

And one more question is there any way to trace out only error messages from session log file using any command?

Thanks & Regards,
San
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Special case to skip function in bash menu

In the bash menu below if the variant that is inputted is in the format NM_004004.3:c.274G>T the below works perfectly. My question is if the variant inputted isNM_004004.3:-c.274G>T or NM_004004.3:+c.274G>T then the code as is will throw an error due to a biological issue. Is it possible to to... (1 Reply)
Discussion started by: cmccabe
1 Replies

2. Shell Programming and Scripting

awk special parse case

I have a special case that awk could be used but I do not have the skill. Trying to create a final output file (indel_parse.txt) that is created from using some information from each of the two files (attached). parse rules: The header is skipped FNR>1 1. 4 zeros after the NC_ (not... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

Search avoiding special characters

Hi all, I have a list which I want to search in another file. I can do that using grep -f but the search is failing due to special characters, how do I solve this? One row in that list is amino-acid permease inda1 gb|EDU41782.1| amino-acid permease inda1 Input file to be searched... (2 Replies)
Discussion started by: gina.lizar
2 Replies

4. Shell Programming and Scripting

Looping in case of duplicates

14`~abc`~9`~11 14`~abc`~9`~10 36`~ee`~7`~9 36`~ee`~8`~9 58`~rtt`~12`~7 70`~gff`~13`~8 86`~tyu`~6`~12 86`~tyu`~6`~13 92`~mjh`~5`~6 28`~jkl`~10`~DNA 32`~mjk`~SNA`~5 82`~jkli`~11`~DNA ------------------------------------ Field Seperator: `~ The concept is to start with SNA in the 3rd... (8 Replies)
Discussion started by: barath
8 Replies

5. Shell Programming and Scripting

Looping in case of duplicates ...

complinagetest () #function name { if ;then rm complins.dat fi touch complins.dat i=0 while read line do if ; then va=`grep -w "$line" datalins1.dat | awk BEGIN'{FS="\`~"}{if ( $3=="'$line'" ) {print $4}}'` i=$(($i+1)) varits=$(echo $va|awk -v varif="$i" '{print... (1 Reply)
Discussion started by: amitpaul01
1 Replies

6. UNIX for Dummies Questions & Answers

Avoiding the history

In bash shell, how we can avoid the commands getting recorded in history file. One way i can think of is : export HISTSIZE=0 Is there any other way to achieve this? Thanks (1 Reply)
Discussion started by: pandeesh
1 Replies

7. Shell Programming and Scripting

Row to columns - special case

Hi. Let me start saying that i am kinda new to bash, and have few skills in programming. I've been advised to use bash to manipulate large .csv files. I've been able to do some data filtering using fors, grep and tail commands. That was kinda easy seeing examples. But now i need to do some hard... (1 Reply)
Discussion started by: jmarmitt
1 Replies

8. Shell Programming and Scripting

Check input for lenght, special characters and letter case

I made menu script for users so they can run other script without going in shell just from menu. But i must control their input. These are criteria: Input must have 4 signs First two signs are always lower case letters Input shall not have some special signs just letters and numbers ... (1 Reply)
Discussion started by: waso
1 Replies

9. UNIX for Dummies Questions & Answers

rename more files special case

hello all i have a big problems for me if i have more files as example test.ghrt.part01.rar test.ghrt.part02.rar test.ghrt.part03.rar test.ghrt.part04.rar test.ghrt.part05.rar test.ghrt.part06.rar test.ghrt.part07.rar test.ghrt.part08.rar test.ghrt.part09.rar test.ghrt.part10.rar... (13 Replies)
Discussion started by: ateya
13 Replies

10. UNIX for Dummies Questions & Answers

how to delete whole directory in special case

Hello, Today, as a root user, i want to copy recursively all files and diretories in a source directory to a destination directory using the following command, cp -r /home/smith/* /home/bob/ However, I carelessly missed the '*' out when I executed the command. Now, i noticed a... (1 Reply)
Discussion started by: cy163
1 Replies
Login or Register to Ask a Question