Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Diff command on two Fastq.gz files Post 302876927 by jim mcnamara on Wednesday 27th of November 2013 08:27:27 AM
Old 11-27-2013
There are VERY, VERY likely NO errors in the merge because gzip, zcat, etc., do a checksum.

A better verification if you honestly do not understand what compression does:
Code:
(gunzip -c file1.gz; gunzip -c file2.gz)> tmp1; gunzip -c bigfile > tmp2
diff tmp1 tmp2
rm tmp1 tmp2

gunzip -c is the same as zcat....
If compression had problems everyone would be doing diffs, and probably not using gzip, gunzip for anything.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

diff 2 files; output diff's to 3rd file

Hello, I want to compare two files. All records in file 2 that are not in file 1 should be output to file 3. For example: file 1 123 1234 123456 file 2 123 2345 23456 file 3 should have 2345 23456 I have looked at diff, bdiff, cmp, comm, diff3 without any luck! (2 Replies)
Discussion started by: blt123
2 Replies

2. Shell Programming and Scripting

Find duplicates from multuple files with 2 diff types of files

I need to compare 2 diff type of files and find out the duplicate after comparing each types of files: Type 1 file name is like: file1.abc (the extension abc could any 3 characters but I can narrow it down or hardcode for 10/15 combinations). The other file is file1.bcd01abc (the extension... (2 Replies)
Discussion started by: ricky007
2 Replies

3. AIX

diff command

hello i've two files. how i get the diff between the two files to new file. thanks best regards ariec (3 Replies)
Discussion started by: ariec
3 Replies

4. Shell Programming and Scripting

diff command help

Hi all diff file1 file 2 command will give us op of diff between two file. But it aslo give its position and sign "<" or ">". I dont want position and sign in op. Only diff of content should be come as op. Kindly help me for this. Regards Jaydeep (1 Reply)
Discussion started by: jaydeep_sadaria
1 Replies

5. UNIX for Dummies Questions & Answers

Diff command of two files

Hi, I use the diff command to compare two files and append this output to a file. I would like to now not only produce the differences but be able to output the total number of changes made, the number of new files added and the number of files deleted, is there I can do this using the diff... (2 Replies)
Discussion started by: cyberfrog
2 Replies

6. Shell Programming and Scripting

Combining 3 fastq files

Hello, I am working with next-gen short-read sequence data, which we receive in 3 fastq files. These are arranged in 4-line groups for each read: line1: read identifier, beginning, e.g., "@HWI-ST1342..." line2: DNA sequence, for files 1 and 2, 101 characters, for file 3, 7 chars. line3: "+"... (2 Replies)
Discussion started by: ljk
2 Replies

7. Shell Programming and Scripting

Extract length wise sequences from fastq file

I have a fastq file from small RNA sequencing with sequence lengths between 15 - 30. I wanted to filter sequence lengths between 21-25 and write to another fastq file. how can i do that? (4 Replies)
Discussion started by: empyrean
4 Replies

8. Shell Programming and Scripting

One-way diff command?

Hello, I am trying to find the different files between multiple directories in Linux, here is a small assumption of what is inside the directories dir1 dir2 dir3 1.txt 1.txt 1.txt 2.txt 3.txt 3.txt 5.txt 4.txt 5.txt 6.txt 7.txt 8.txt I am using the following... (4 Replies)
Discussion started by: Error404
4 Replies

9. Shell Programming and Scripting

Diff 3 files, but diff only their 2nd column

Guys i have 3 files, but i want to compare and diff only the 2nd column path=`/home/whois/doms` for i in `cat domain.tx` do whois $i| sed -n '/Registry Registrant ID:/,/Registrant Email:/p' > $path/$i.registrant whois $i| sed -n '/Registry Admin ID:/,/Admin Email:/p' > $path/$i.admin... (10 Replies)
Discussion started by: kenshinhimura
10 Replies

10. UNIX for Beginners Questions & Answers

Comparing fastq files and outputting common records

I have two files: File_1: @M04961:22:000000000-B5VGJ:1:1101:9280:7106 1:N:0:86 GGCATGAAAACATACAAACCGTCTTTCCAGAAATTGTTCCAAGTATCGGCAACAGCTTTATCAATACCATGAAAAATATCAACCACACCAGAAGCAGCAT + GGGGGGGGGGGGGGGGGCCGGGGGF,EDFFGEDFG,@DGGCGGEGGG7DCGGGF68CGFFFGGGG@CGDGFFDFEFEFF:30CGAFFDFEFF8CAF;;8F ... (3 Replies)
Discussion started by: Xterra
3 Replies
Merge(3pm)						User Contributed Perl Documentation						Merge(3pm)

NAME
Algorithm::Merge - Three-way merge and diff SYNOPSIS
use Algorithm::Merge qw(merge diff3 traverse_sequences3); @merged = merge(@ancestor, @a, @b, { CONFLICT => sub { } }); @merged = merge(@ancestor, @a, @b, { CONFLICT => sub { } }, $key_generation_function); $merged = merge(@ancestor, @a, @b, { CONFLICT => sub { } }); $merged = merge(@ancestor, @a, @b, { CONFLICT => sub { } }, $key_generation_function); @diff = diff3(@ancestor, @a, @b); @diff = diff3(@ancestor, @a, @b, $key_generation_function); $diff = diff3(@ancestor, @a, @b); $diff = diff3(@ancestor, @a, @b, $key_generation_function); @trav = traverse_sequences3(@ancestor, @a, @b, { # callbacks }); @trav = traverse_sequences3(@ancestor, @a, @b, { # callbacks }, $key_generation_function); $trav = traverse_sequences3(@ancestor, @a, @b, { # callbacks }); $trav = traverse_sequences3(@ancestor, @a, @b, { # callbacks }, $key_generation_function); USAGE
This module complements Algorithm::Diff by providing three-way merge and diff functions. In this documentation, the first list to "diff3", "merge", and "traverse_sequences3" is called the `original' list. The second list is the `left' list. The third list is the `right' list. The optional key generation arguments are the same as in Algorithm::Diff. See Algorithm::Diff for more information. diff3 Given references to three lists of items, "diff3" performs a three-way difference. This function returns an array of operations describing how the left and right lists differ from the original list. In scalar context, this function returns a reference to such an array. Perhaps an example would be useful. Given the following three lists, original: a b c e f h i k left: a b d e f g i j k right: a b c d e h i j k merge: a b d e g i j k we have the following result from diff3: [ 'u', 'a', 'a', 'a' ], [ 'u', 'b', 'b', 'b' ], [ 'l', 'c', undef, 'c' ], [ 'o', undef, 'd', 'd' ], [ 'u', 'e', 'e', 'e' ], [ 'r', 'f', 'f', undef ], [ 'o', 'h', 'g', 'h' ], [ 'u', 'i', 'i', 'i' ], [ 'o', undef, 'j', 'j' ], [ 'u', 'k', 'k', 'k' ] The first element in each row is the array with the difference: c - conflict (no two are the same) l - left is different o - original is different r - right is different u - unchanged The next three elements are the lists from the original, left, and right arrays respectively that the row refers to (in the synopsis, these are @ancestor, @a, and @b, respectively). merge Given references to three lists of items, "merge" performs a three-way merge. The "merge" function uses the "diff3" function to do most of the work. The only callback currently used is "CONFLICT" which should be a reference to a subroutine that accepts two array references. The first array reference is to a list of elements from the left list. The second array reference is to a list of elements from the right list. This callback should return a list of elements to place in the merged list in place of the conflict. The default "CONFLICT" callback returns the following: q{<!-- ------ START CONFLICT ------ -->}, (@left), q{<!-- ---------------------------- -->}, (@right), q{<!-- ------ END CONFLICT ------ -->}, traverse_sequences3 This is the workhorse function that goes through the three sequences and calls the callback functions. The following callbacks are supported. NO_CHANGE This is called if all three sequences have the same element at the current position. The arguments are the current positions within each sequence, the first argument being the current position within the first sequence. A_DIFF This is called if the first sequence is different than the other two sequences at the current position. This callback will be called with one, two, or three arguments. If one argument, then only the element at the given position from the first sequence is not in either of the other two sequences. If two arguments, then there is no element in the first sequence that corresponds to the elements at the given positions in the second and third sequences. If three arguments, then the element at the given position in the first sequence is different than the corresponding element in the other two sequences, but the other two sequences have corresponding elements. B_DIFF This is called if the second sequence is different than the other two sequences at the current position. This callback will be called with one, two, or three arguments. If one argument, then only the element at the given position from the second sequence is not in either of the other two sequences. If two arguments, then there is no element in the second sequence that corresponds to the elements at the given positions in the first and third sequences. If three arguments, then the element at the given position in the second sequence is different than the corresponding element in the other two sequences, but the other two sequences have corresponding elements. C_DIFF This is called if the third sequence is different than the other two sequences at the current position. This callback will be called with one, two, or three arguments. If one argument, then only the element at the given position from the third sequence is not in either of the other two sequences. If two arguments, then there is no element in the third sequence that corresponds to the elements at the given positions in the first and second sequences. If three arguments, then the element at the given position in the third sequence is different than the corresponding element in the other two sequences, but the other two sequences have corresponding elements. CONFLICT This is called if all three sequences have different elements at the current position. The three arguments are the current positions within each sequence. BUGS
Most assuredly there are bugs. If a pattern similar to the above example does not work, send it to <jsmith@cpan.org> or report it on <http://rt.cpan.org/>, the CPAN bug tracker. Algorithm::Diff's implementation of "traverse_sequences" may not be symmetric with respect to the input sequences if the second and third sequence are of different lengths. Because of this, "traverse_sequences3" will calculate the diffs of the second and third sequences as passed and swapped. If the differences are not the same, it will issue an `Algorithm::Diff::diff is not symmetric for second and third sequences...' warning. It will try to handle this, but there may be some cases where it can't. SEE ALSO
Algorithm::Diff. AUTHOR
James G. Smith, <jsmith@cpan.org> COPYRIGHT
Copyright (C) 2003, 2007 Texas A&M University. All Rights Reserved. This module is free software; you may redistribute it and/or modify it under the same terms as Perl itself. perl v5.10.1 2010-10-15 Merge(3pm)
All times are GMT -4. The time now is 11:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy