Looping in case of duplicates


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Looping in case of duplicates
# 1  
Old 10-11-2013
Hammer & Screwdriver Looping in case of duplicates

Code:
14`~abc`~9`~11
14`~abc`~9`~10
36`~ee`~7`~9
36`~ee`~8`~9
58`~rtt`~12`~7
70`~gff`~13`~8
86`~tyu`~6`~12
86`~tyu`~6`~13
92`~mjh`~5`~6
28`~jkl`~10`~DNA
32`~mjk`~SNA`~5
82`~jkli`~11`~DNA

------------------------------------

Field Seperator: `~
The concept is to start with SNA in the 3rd Field.
Fetch the 4th field, search that 4th field content in the
third field of some other line , save the 1st field, get that line's 4th field and so on till
I reach DNA in the 4th field.
I have already developed part of the script which does that. But I am not sure how to handle, when I search for 4th field content in 3rd field, if it has more than one occurance.
I am getting the below output:
Code:
32~92~86~58~36~14~82

Expected output for the above example:
Code:
32~92~86~58~36~14~82
32~92~86~58~36~14~28
32~92~86~70~36~14~82
32~92~86~70~36~14~28

Could some one plz help me with this logic approach with unix script.

Thanks in Advance

Last edited by Franklin52; 10-11-2013 at 10:20 AM.. Reason: Please use code tags
# 2  
Old 10-11-2013
Please use code tags as required by forum rules!

Why don't you post your script that solves part of the problem, and why don't you specify correctly and in detail? e.g. the 32 at the start of each solution comes from where? SNA, I know, but don't leave us guessing. Also, you should clearly state that ALL four solutions are needed, i.e. duplicates in $3 are equivalent.
# 3  
Old 10-15-2013
Looping in case of duplicates

Code:
 
complinagetest () #function name
{
if [ -f complins.dat ];then
rm complins.dat
fi
touch complins.dat
i=0
while read line
do
if [ $line == "SNA" ]; then
 
 va=`grep -w "$line" datalins1.dat | awk  BEGIN'{FS="\`~"}{if ( $3=="'$line'" ) {print $4}}'`
 i=$(($i+1))
 varits=$(echo $va|awk -v varif="$i" '{print $varif}')
 
      if [ "$varits" = "DNA" ]; then
          grep -w "$varits" datalins1.dat | awk  BEGIN'{FS="`~"}{if ( $3=="'$line'" ) {print $1}}'|sed 's/$/~>/' >> complins.dat
        else 
       grep -w "$varits" datalins1.dat | awk  BEGIN'{FS="`~"}{if ( $3=="'$line'" ) {print $1}}'| tr '\n' '~' | sed 's/~/~>/g'   >> complins.dat
 
 
 while [ "$varits" != "DNA" ]
 do
 
                while read line
                do
 
                  if [ $line == "$varits" ] ; then
                varits=`grep -w "$line" datalins1.dat | awk  BEGIN'{FS="\`~"}{if ( $3=='$line' ) {print $4}}'`
                check=`echo $varits|awk '{print NF}'`
 
                        if [ "$check" == "1" ];then 
                echo "inside if"                  
                grep -w "$varits" datalins1.dat  | awk  BEGIN'{FS="`~"}{if ( $3=='$line' ) {print $1}}'| tr '\n' '~'| sed 's/~/~>/' >> complins.dat
 
                        else
 
    echo "problem occurs here in else case when $check > 1(ie. $4 from datalins1.dat has two number for same value of $1)"
 
                         fi
 
                fi
 
                done < source.dat
 
 done
      fi
echo "" >> complins.dat
fi
done < source.dat
 
}
 
complinagetest #calling function

--------------------------------------------------------------------------
input files for above function are datalins1.dat and source.dat
output files for above function is complins.dat

Code:
#datalins1.dat
14`~abc`~9`~11
14`~abc`~9`~10
36`~ee`~7`~9
36`~ee`~8`~9
58`~rtt`~12`~7
70`~gff`~13`~8
86`~tyu`~6`~12
86`~tyu`~6`~13
92`~mjh`~5`~6
28`~jkl`~10`~DNA
32`~mjk`~SNA`~5
82`~jkli`~11`~DNA

Code:
#source.dat
9
7
8
12
13
6
5
10
SNA
11


-----------------------------------------------------------------------------
Field Seperator: `~
The concept is to start with SNA in the 3rd Field.
Fetch the 4th field, search that 4th field content in the
third field of some other line , save the 1st field, get that line's 4th field and so on till
I reach DNA in the 4th field.
I have already developed part of the script which does that. But I am not sure how to handle, when I search for 4th field content in 3rd field, if it has more than one occurance.
------------------------------------------------------------------
Expected output for the above example:
Code:
#complins.dat
32~92~86~58~36~14~82
32~92~86~58~36~14~28
32~92~86~70~36~14~82
32~92~86~70~36~14~28

-------------------------------------------------------------------
the function works perfectly if we have single number in $4 of datalins1.dat for same value in $1

Could some one plz help me with this logic approach with unix script.

Thanks in Advance..

Last edited by barath; 10-15-2013 at 06:09 AM..
# 4  
Old 10-15-2013
How many records (lines) in a typical datalins1.dat file? Are they actually as small as your sample (12 lines)? Or are they much larger?

Regards,
Alister
# 5  
Old 10-15-2013
yes the file (datalins1.dat) could be much larger......not necessary only 12 lines(for the above example only 12 lines).....but the format of the file would be always constant ( four fields separated by delimiter `~ )...
# 6  
Old 10-15-2013
What do you consider "much larger"? A hundred lines? A thousand? A million? A billion? A simple solution may scale from 12 to 1000, but perhaps not to a million and beyond.

It would be a shame for someone to waste their time crafting code that can never be used because it takes forever to complete or because it requires more memory than the system has available. So, please, be more precise than "much larger". Also, if the file can approach the size of your system's memory, you should definitely mention that.

Regards,
Alister
# 7  
Old 10-15-2013
Sorry for the inconvenience......much larger doesn't meant here that it can go to thousand or million lines...the above code is crafted on the basis of similar kind(datalins1.dat) of input scenario...

Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Change first letter of a word from lower case to upper case

Hi all, I am trying to find a way to change first letter in a word from lower case to upper case. It should be done for each first word in text or in paragraph, and also for each word after punctuation like . ; : ! ?I found the following command sed -i 's/\s*./\U&\E/g' $@ filenamebut... (7 Replies)
Discussion started by: georgi58
7 Replies

2. Shell Programming and Scripting

Looping in case of duplicates ...

complinagetest () #function name { if ;then rm complins.dat fi touch complins.dat i=0 while read line do if ; then va=`grep -w "$line" datalins1.dat | awk BEGIN'{FS="\`~"}{if ( $3=="'$line'" ) {print $4}}'` i=$(($i+1)) varits=$(echo $va|awk -v varif="$i" '{print... (1 Reply)
Discussion started by: amitpaul01
1 Replies

3. Shell Programming and Scripting

multiple looping with case and funtion showing error, Please help

Hello All, I have code as follows :- while true do {opening a case1 statement} 1) {opening another case2 statement} {closing case 2} 2) Showing error for "2)" as Syntax error at line 59 : `)' is not expected. *) {closing case 1} ... (5 Replies)
Discussion started by: Renjesh
5 Replies

4. Shell Programming and Scripting

Conversion from Upper Case to Lower Case Condition based

Hello Unix Gurus : It would be really appreciative if can find a solution for this . I have records in a file . I need to Capitalize the records based on condition . For Example i tried the following Command COMMAND --> fgrep "2000YUYU" /export/home/oracle/TST/data.dat | tr '' ''... (12 Replies)
Discussion started by: tsbiju
12 Replies

5. Shell Programming and Scripting

sed ignoring case for search but respecting case for subtitute

Hi I want to make string substitution ignoring case for search but respecting case for subtitute. Ex changing all occurences of "original" in a file to "substitute": original becomes substitute Origninal becomes Substitute ORIGINAL becomes SUBSTITUTE I know this a little special but it's not... (1 Reply)
Discussion started by: kmchen
1 Replies

6. Shell Programming and Scripting

data array needs to change upper case to lower case

Hi all, i have a data array as followes. ARRAY=DFSG345GGG ARRAY=234FDFG090 ARRAY=VDFVGBGHH so on.......... i need all english letters to be change to lower case. So i am expecting to see ARRAY=dfsg345ggg ARRAY=234fdfg090 ARRAY=vdfvgbghh so on........ If i have to copy this data in... (8 Replies)
Discussion started by: usustarr
8 Replies

7. SCO

Avoiding duplicates with some special case

Hi Gurus, I had a question regarding avoiding duplicates.i have a file abc.txt abc.txt ------- READER_1_1_1> HIER_28056 XML Reader: Error occurred while parsing:; line number ; column number READER_1_3_1> Sun Mar 23 23:52:48 2008 READER_1_3_1> HIER_28056 XML Reader: Error occurred while... (0 Replies)
Discussion started by: pssandeep
0 Replies

8. Shell Programming and Scripting

Script needed to select and delete lower case and mixed case records

HELLO ALL, URGENTLY NEEDED A SCRIPT TO SELECT AND DELETE LOWER AND MIXED CASE RECORDS FROM A COLUMN IN A TABLE. FOR EXAMPLE : Table name is EMPLOYEE and the column name is CITY and the CITY column records will be: Newyork washington ... (1 Reply)
Discussion started by: abhilash mn
1 Replies

9. UNIX for Dummies Questions & Answers

lower case to upper case string conversion in shell script

How can convert a Lower case variable value to an upper case in the kron shell script. (3 Replies)
Discussion started by: dchalavadi
3 Replies
Login or Register to Ask a Question