Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Deleting consecutive equal values in a file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Deleting consecutive equal values in a file
# 1  
Old 11-18-2014
Deleting consecutive equal values in a file
Hello everyone,

I have a requirement as shown below. I need to delete consecutive same values from the source file and give it as output file.

Code:
Source:

a,b,c,d,e,e,f,g

Code:
Target:

a,b,c,d,f,g

The repeating value "e" should be deleted from the file completely. How can I achieve this through Unix script... thanks in advance.

Note: The position of the repeating string may not be same always. It may change.
# 2  
Old 11-18-2014
Hello Vamsikrishna928,

Following may help you in same.

Code:
awk -F"," '{for(i=1;i<=NF;i++){if($i == V){$i="\b"} if(i==NF){ORS="";print $i"\n"} else {print $i;V=$i}}}' ORS="," Input_file

Output will be as follows.
Code:
a,b,c,d,e,f,g

Thanks,
R. Singh
# 3  
Old 11-18-2014
Thanks for the reply..

But the output should completely eliminate the matching string "e". It output file should be like

Code:
a,b,c,d,f,g

# 4  
Old 11-18-2014
Quote:
Originally Posted by vamsikrishna928
Thanks for the reply..

But the output should completely eliminate the matching string "e". It output file should be like

Code:
a,b,c,d,f,g

Hello Vamsikrishna928,

Following may help you, not that handy but it will provide the output as requirement.

Code:
awk -F"," '{for(i=1;i<=NF;i++){if($i == V){$i="\b\b\b"} if(i==NF){ORS="";print $i"\n"} else {print $i;V=$i}}}' ORS="," Input_file

Output will be as follows.
Code:
a,b,c,d,f,g

Thanks,
R. Singh
# 5  
Old 11-18-2014
Try

Code:
awk -F, 'NF{s="";for(i=1; i<=NF; i++){ if($i == $(i+1)){i+=1; continue } s = s (length(s)?OFS:"") $i }print s}' OFS=',' infile

Code:
$ echo 'a,b,c,d,e,e,f,g' | awk -F, 'NF{s="";for(i=1; i<=NF; i++){ if($i == $(i+1)){i+=1; continue } s = s (length(s)?OFS:"") $i }print s}' OFS=','
a,b,c,d,f,g

# 6  
Old 11-18-2014
Try:
Code:
sed 's/\([^,]\{1,\}\),\1,\{0,1\}//g' file

Or GNU -r / BSD -E:
Code:
sed -E 's/([^,]+),\1,?//g' file

Or perl
Code:
perl -pe 's/(.+?),\1,?//g' file

# 7  
Old 11-18-2014
Not to rain on anyone's parade, but does consecutive == 2 occurrences?
Code:
perl -pe 's/(.+?)(,\1)+,?//g'

(stolen, er, adapted from Scrutinizer)

---------- Post updated at 05:38 AM ---------- Previous update was at 05:21 AM ----------

And I missed the boundary errors:
Code:
echo 'a,b,c,d,e,e,e,ee,f,g,g,gg' | perl -pe 's/(.+?)(,\1)+,?//g'
a,b,c,d,e,f,g

So let's go with:
Code:
echo 'a,b,c,d,e,e,e,ee,f,g,g,gg' | perl -pe 's/$/,/; s/([^,]+)(,\1)+,//g; s/,$//;'
a,b,c,d,ee,f,gg

Last edited by derekludwig; 11-18-2014 at 06:39 AM.. Reason: missing ;
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete row if both percentage values are equal to zero

Hello, I have compiled a script but I have stucked at one point. Each line contains two pcs of % value and what I want to do is to delete any line if both % values are zero. data: expected output: ow3 should be deleted as both percentage value in related line are equal to zero. ... (2 Replies)
Discussion started by: baris35
2 Replies

2. Shell Programming and Scripting

awk print values between consecutive lines

I have a file in below format: file01.txt TERM TERM TERM ABC 12315 68.53 12042013 165144 ABC 12315 62.12 12042013 165145 ABC 12315 122.36 12052013 165146 ABC 12315 582.18 12052013 165147 ABC 12316 2.36 12052013 165141 ABC 12316 ... (8 Replies)
Discussion started by: alex2005
8 Replies

3. Shell Programming and Scripting

Adding the corresponding values for every 5th consecutive numbers

Dear All, I have a file which is as follows: Input File: 231 100.1 233 99 235 200.9 238 80.1 239 90.2 240 77.0 243 99.5 245 16.20 246 13.55 247 11.8 249 13.7 250 99.6 (1 Reply)
Discussion started by: NamS
1 Replies

4. UNIX for Dummies Questions & Answers

Replacing values in a column if they equal a certain value

Hi, I have a tab delimited text file where some lines have the string "NA" in the second column. For these lines, I want to replace NA with the value in the first column, the symbol underscore followed by the value in the fourth column. How do I go about doing that? Thanks! Input: 1 ... (3 Replies)
Discussion started by: evelibertine
3 Replies

5. Shell Programming and Scripting

How to compare the values of a column in awk in a same file and consecutive lines..

I would like to compare the values of 2nd column of consecutive lines of same file in such a way so that if the difference between first value and second value is more than 100 it should print complete line else ignore line. Input File ========== PDB 2500 RTDB 123 RTDB-EAGLE 122 VSCCP 2565... (4 Replies)
Discussion started by: manuswami
4 Replies

6. Shell Programming and Scripting

AWK: combining consecutive values in a field

Hi, Here is my sample input X 2 AAA Y 3 BBB Y 2 CCC Z 4 DDD In field 1, if the value of one line is same as that of next line, I want to concatenate the corresponding value of the second line in the third field with the value of the third field of first line. And I dont need the third... (2 Replies)
Discussion started by: polsum
2 Replies

7. Shell Programming and Scripting

compare columns for equal values and output a summary

Hi all I am trying to scan a file that has 3 columns: red blue 123351 red blue 848655 red blue 126354 red blue 023158 black white 654896 red blue 650884 I want an output that sums the rows that have matching columns 1 and 2 :wall: red blue has 5 entries black white has 1 entry ... (4 Replies)
Discussion started by: reno
4 Replies

8. Programming

FORTRAN: Getting Consecutive DIST Values

I have this code and I want to get the first two consecutive distances (obtained by calling GET_TH(IT, 'DIST', DIST) at index IT, the result stored in DIST) for which (ITFLG(IT) .NE. 1). L_FIRST get the first DIST for which ITFLG(IT) .NE. 1. Getting a bit confused on how to achieve this. ... (0 Replies)
Discussion started by: kristinu
0 Replies

9. Shell Programming and Scripting

concatenate consecutive field values

Hi, I have a file like this A Bob A Sam A John B David C Paul C Sandra If the consecutive field values in column one is same, then concatenate the corresponding strings. So, I need an output like this, A Bob_Sam_John B David C Paul_Sandra I usually work with excel but... (3 Replies)
Discussion started by: polsum
3 Replies

10. Shell Programming and Scripting

My Values are Equal but They are Not

Does anybody understand why this is not being interpreted as true. Script: #!/bin/bash errored=`grep "errored" new_update_scripts.txt` echo $errored = "errored" if ; then echo true else echo false fi Output: $ UpdateScripts errored = errored false (7 Replies)
Discussion started by: scottwmackey
7 Replies
Login or Register to Ask a Question