Sponsored Content
Top Forums Shell Programming and Scripting Comparing alternate lines of code Post 303025784 by cabled on Monday 12th of November 2018 09:56:51 AM
Old 11-12-2018
Comparing alternate lines of code

Hi gents,

Have only a passing familiarity with linux/shell at this point, so please forgive simple question.

I have text files that have lines something like the following:

Code:
a
b
c
d
d
d
e
f
e
f
e
f
a
b
c
d
e
f
etc

I'm trying to remove 2 types of duplicates while preserving line order/format.
1) consecutive duplicate lines
2) alternate lines if they are duplicate

For removing type 1 lines,
Code:
cat "$file" | uniq > ./output/"$file"

gives me an output file that looks like

Code:
a
b
c
d
e
f
e
f
e
f
a
b
c
d
e
f
etc

which is fine.

I'm kinda stumped about type 2 duplicates though...

Ideally I'd like to get:

Code:
a
b
c
d
e
f
a
b
c
d
e
f

Not entirely sure how to compare alternate lines... Any assistance is appreciated

Last edited by RudiC; 11-12-2018 at 11:06 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

alternate lines from two files

A basic request two files want to combine them but on alternate lines (1 Reply)
Discussion started by: SummitElse
1 Replies

2. Shell Programming and Scripting

alternate lines

Hi, I'm new to Unix. I want to read the all the lines from a text file and write the alternate lines into another file. Please give me a shell script solution. file1 ----- one two three four five six seven newfile(it should contain the alternate lines from the file1) ------- one... (6 Replies)
Discussion started by: pstanand
6 Replies

3. Shell Programming and Scripting

comparing lines in file

i have 2 files and i want to compare i currently cat the files and awk print $1, $2 and doing if file1=file2 then fail, else exit 0 what i want to do is compare values, with column 1 being a reference i want to compare line by line and then still be able to do if then statement to see if worked... (1 Reply)
Discussion started by: sigh2010
1 Replies

4. Shell Programming and Scripting

reading alternate lines of a file

hi, i have 2 files. file1: 1 2 3 4 5 6 file2: a b c d e f g h i (5 Replies)
Discussion started by: vidyaj
5 Replies

5. Shell Programming and Scripting

Insert string in alternate lines

Hi All, In continuation of my previous thread 'Add text at the end of line conditionally', I need to further modfiy the file after adding text at the end of the line. Now, I need to add a fixed charater string at alternate lines starting from first line using awk or sed.My file is now as below:... (10 Replies)
Discussion started by: angshuman
10 Replies

6. Shell Programming and Scripting

Comparing lines of two different files

Hello, Please help me with this problem if you have a solution. I have two files: <file1> : In each line, first word is an Id and then other words that belong to this Id piMN-1 abc pqr xyz py12 niLM y12 FY4 pqs fiRLym F12 kite red <file2> : same as file1, but can have extra lds... (3 Replies)
Discussion started by: mira
3 Replies

7. Programming

Perl : joining alternate lines

Hi, I need to join every alternate line in a file for eg:input file $ cat abc abc def ghi jkloutput abc def ghi jklcode i wrote for this $ cat add_line.pl #!/usr/bin/perl -w my $count=1; #my $line=undef; my @mem_line; my $i=0; my $x=0; (2 Replies)
Discussion started by: sam05121988
2 Replies

8. UNIX for Dummies Questions & Answers

Comparing lines of data

Total UNIX Rookie, but I'm learning. I have columns of integer data separated by spaces, and I'm using a Mac terminal. What I want to do: 1. Compare "line 1 column 2" (x) to "line 2 column 2" (y); is y-x>=100? 2. If yes, display difference and y's line number 3. If no, increment x and y by... (9 Replies)
Discussion started by: markymarkg123
9 Replies

9. Shell Programming and Scripting

Grep values on alternate lines

Hi, I have a file like 2011|ACC|.* 2013|ACC|.* 2011|ACCC|.* 2013|ACCC|.* 2013|ACCV|.* 2011|ADB|.* 2013|ADB|.* 2011|ADBC|.* 2013|ADBC|.* 2011|AIA|.* 2013|AXJ|.* 2013|NNN|.* .* represnts any alphanumeric characters after this part of the string I need a code to return only the... (3 Replies)
Discussion started by: sam05121988
3 Replies

10. Shell Programming and Scripting

Process alternate lines in awk/sed/perl

hi.. i have a fasta file with the following format >sequence1 CCGGTTTTCGATTTGGTTTGACT >sequence2 AAAGTGCCGCCAGGTTTTGAGTGT >sequence3 AGTGCCGCAGAGTTTGTAGTGT Now, i want to read alternate line and add "GGGGGGGGGGG" to end of every sequence Desired output: >sequence1... (4 Replies)
Discussion started by: empyrean
4 Replies
diff(1) 						      General Commands Manual							   diff(1)

Name
       diff - differential file comparator

Syntax
       diff [options] dir1 dir2
       diff [options] file1 file2

Description
       The command compares the contents of files or groups of files, and lists any differences it finds. When run on regular files, and when com-
       paring text files that differ during directory comparison, tells what lines must be changed in the files  to  bring  them  into	agreement.
       Except  in rare circumstances, finds a smallest sufficient set of file differences.  If neither file1 nor file2 is a directory, then either
       can be specified as `-', in which case the standard input is used.  If file1 is a directory, then a file in that directory  whose  filename
       is the same as the filename of file2 is used and likewise if file2 is a directory.

       If  both  arguments  are directories, sorts the contents of the directories by name, and then runs the regular file algorithm on text files
       that are different.  Binary files that differ, common subdirectories, and files that appear in only one directory are listed.

Options
       The following options are used when comparing directories:

       -l	 Displays the output in long format.  Each text file is piped through to paginate it; other differences are summarized	after  all
		 text file differences are reported.

       -n	 Produces a script similar to that of -e, but in reverse order and with a count of changed lines on each insert or delete command.

       -r	 Recursively checks files in common subdirectories.

       -s	 Displays names of files that are the same.

       -Sname	 Starts a directory in the middle beginning with the specified file.

       Except for the -b, i, t, and w options, which may be given with any of the others, the following formatting options are mutually exclusive:

       -b	 Ignores trailing blanks and other strings of blanks and treats such portions as equal.

       -c	 Displays three context lines with each output line.  For backwards compatibility, -cn causes n number of context lines.

       -C n	 Displays specified number of context lines with each output line.  With -c or -C the output format is modified slightly: the out-
		 put begins with identification of the files involved and their creation dates and then each change is separated by a line with  a
		 dozen asterisks (*).  The lines removed from file1 are marked with minus sign (-); those added to file2 are marked plus sign (+).
		 Lines that are changed from one file to the other are marked in both files with an exclamation point (!).

		 Changes within n context lines of each other are grouped together in the output.  This results in output  that  is  usually  much
		 easier to interpret.

       -Dstring  Causes  to  create a merged version of file1 and file2 on the standard output.  With C preprocessor controls included, a compila-
		 tion of the result without defining string is equivalent to compiling file1, while defining string will yield file2.

       -e	 Writes output to an script.  In connection with -e, the following shell program can help maintain multiple versions  of  a  file.
		 Only  an  ancestral  file ($1) and a chain of version-to-version scripts ($2,$3,...) made by need be available.  A latest version
		 message appears on the standard output.
		  (shift; cat $*; echo '1,$p') | ed - $1
		 If you specify -e when comparing directories the result is a script for converting text files that are common to the two directo-
		 ries from their state in dir1 to their state in dir2.

       -f	 Writes the output in reverse order to a script.

       -h	 Makes	a hasty comparison.  It works only when changed portions are short and well separated, but does work on files of unlimited
		 length.

       -i	 Ignores the case of letters.  For example 'A' will compare equal to `a'.

       -t	 Expand tabs in output lines.  Normal or -c output adds character(s) to the front of each line which may affect the indentation of
		 the  original	source lines and make the output listing difficult to interpret.  This option will preserves the original indenta-
		 tion.

       -w	 Causes whitespace (blanks and tabs) to be totally ignored.  For example, `if ( a == b )' will compare equal to `if(a==b)'.

       There are several options for output format; the default output format contains lines of these forms:

	    n1 a n3,n4
	    n1,n2 d n3
	    n1,n2 c n3,n4

       These lines resemble commands to convert file1 into file2.  The numbers after the letters pertain to file2.  In fact, by exchanging `a' for
       `d' and reading backward you can tell how to convert file2 into file1.  As in identical pairs where n1 = n2 or n3 = n4 are abbreviated as a
       single number.

       Following each of these lines come all the lines that are affected in the first file flagged by a left angle bracket  (<).   Then  all  the
       lines that are affected in the second file are listed, flagged by a right angle bracket (>).

Restrictions
       Editing scripts produced under the -e or -f option have trouble creating lines consisting of a single period (.).

       When comparing directories with the -b, i, t, or w options specified, first compares the files as does, and then runs the algorithm if they
       are not equal.  If the only differences are in the blank strings, may report these as differences.

Diagnostics
       Exit status is 0 for no differences, 1 for some differences,and 2 if the specified file cannot be found.

Files
       for		   -h

       See Also
	      cc(1), cmp(1), comm(1), diff3(1), ed(1)

																	   diff(1)
All times are GMT -4. The time now is 06:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy