Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Diff command on two Fastq.gz files Post 302876927 by jim mcnamara on Wednesday 27th of November 2013 08:27:27 AM
Old 11-27-2013
There are VERY, VERY likely NO errors in the merge because gzip, zcat, etc., do a checksum.

A better verification if you honestly do not understand what compression does:
Code:
(gunzip -c file1.gz; gunzip -c file2.gz)> tmp1; gunzip -c bigfile > tmp2
diff tmp1 tmp2
rm tmp1 tmp2

gunzip -c is the same as zcat....
If compression had problems everyone would be doing diffs, and probably not using gzip, gunzip for anything.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

diff 2 files; output diff's to 3rd file

Hello, I want to compare two files. All records in file 2 that are not in file 1 should be output to file 3. For example: file 1 123 1234 123456 file 2 123 2345 23456 file 3 should have 2345 23456 I have looked at diff, bdiff, cmp, comm, diff3 without any luck! (2 Replies)
Discussion started by: blt123
2 Replies

2. Shell Programming and Scripting

Find duplicates from multuple files with 2 diff types of files

I need to compare 2 diff type of files and find out the duplicate after comparing each types of files: Type 1 file name is like: file1.abc (the extension abc could any 3 characters but I can narrow it down or hardcode for 10/15 combinations). The other file is file1.bcd01abc (the extension... (2 Replies)
Discussion started by: ricky007
2 Replies

3. AIX

diff command

hello i've two files. how i get the diff between the two files to new file. thanks best regards ariec (3 Replies)
Discussion started by: ariec
3 Replies

4. Shell Programming and Scripting

diff command help

Hi all diff file1 file 2 command will give us op of diff between two file. But it aslo give its position and sign "<" or ">". I dont want position and sign in op. Only diff of content should be come as op. Kindly help me for this. Regards Jaydeep (1 Reply)
Discussion started by: jaydeep_sadaria
1 Replies

5. UNIX for Dummies Questions & Answers

Diff command of two files

Hi, I use the diff command to compare two files and append this output to a file. I would like to now not only produce the differences but be able to output the total number of changes made, the number of new files added and the number of files deleted, is there I can do this using the diff... (2 Replies)
Discussion started by: cyberfrog
2 Replies

6. Shell Programming and Scripting

Combining 3 fastq files

Hello, I am working with next-gen short-read sequence data, which we receive in 3 fastq files. These are arranged in 4-line groups for each read: line1: read identifier, beginning, e.g., "@HWI-ST1342..." line2: DNA sequence, for files 1 and 2, 101 characters, for file 3, 7 chars. line3: "+"... (2 Replies)
Discussion started by: ljk
2 Replies

7. Shell Programming and Scripting

Extract length wise sequences from fastq file

I have a fastq file from small RNA sequencing with sequence lengths between 15 - 30. I wanted to filter sequence lengths between 21-25 and write to another fastq file. how can i do that? (4 Replies)
Discussion started by: empyrean
4 Replies

8. Shell Programming and Scripting

One-way diff command?

Hello, I am trying to find the different files between multiple directories in Linux, here is a small assumption of what is inside the directories dir1 dir2 dir3 1.txt 1.txt 1.txt 2.txt 3.txt 3.txt 5.txt 4.txt 5.txt 6.txt 7.txt 8.txt I am using the following... (4 Replies)
Discussion started by: Error404
4 Replies

9. Shell Programming and Scripting

Diff 3 files, but diff only their 2nd column

Guys i have 3 files, but i want to compare and diff only the 2nd column path=`/home/whois/doms` for i in `cat domain.tx` do whois $i| sed -n '/Registry Registrant ID:/,/Registrant Email:/p' > $path/$i.registrant whois $i| sed -n '/Registry Admin ID:/,/Admin Email:/p' > $path/$i.admin... (10 Replies)
Discussion started by: kenshinhimura
10 Replies

10. UNIX for Beginners Questions & Answers

Comparing fastq files and outputting common records

I have two files: File_1: @M04961:22:000000000-B5VGJ:1:1101:9280:7106 1:N:0:86 GGCATGAAAACATACAAACCGTCTTTCCAGAAATTGTTCCAAGTATCGGCAACAGCTTTATCAATACCATGAAAAATATCAACCACACCAGAAGCAGCAT + GGGGGGGGGGGGGGGGGCCGGGGGF,EDFFGEDFG,@DGGCGGEGGG7DCGGGF68CGFFFGGGG@CGDGFFDFEFEFF:30CGAFFDFEFF8CAF;;8F ... (3 Replies)
Discussion started by: Xterra
3 Replies
Algorithm::DiffOld(3)					User Contributed Perl Documentation				     Algorithm::DiffOld(3)

NAME
Algorithm::DiffOld - Compute `intelligent' differences between two files / lists but use the old (<=0.59) interface. NOTE
This has been provided as part of the Algorithm::Diff package by Ned Konz. This particular module is ONLY for people who HAVE to have the old interface, which uses a comparison function rather than a key generating function. Because each of the lines in one array have to be compared with each of the lines in the other array, this does M*N comparisions. This can be very slow. I clocked it at taking 18 times as long as the stock version of Algorithm::Diff for a 4000-line file. It will get worse quadratically as array sizes increase. SYNOPSIS
use Algorithm::DiffOld qw(diff LCS traverse_sequences); @lcs = LCS( @seq1, @seq2, $comparison_function ); $lcsref = LCS( @seq1, @seq2, $comparison_function ); @diffs = diff( @seq1, @seq2, $comparison_function ); traverse_sequences( @seq1, @seq2, { MATCH => $callback, DISCARD_A => $callback, DISCARD_B => $callback, }, $comparison_function ); COMPARISON FUNCTIONS
Each of the main routines should be passed a comparison function. If you aren't passing one in, use Algorithm::Diff instead. These functions should return a true value when two items should compare as equal. For instance, @lcs = LCS( @seq1, @seq2, sub { my ($a, $b) = @_; $a eq $b } ); but if that is all you're doing with your comparison function, just use Algorithm::Diff and let it do this (this is its default). Or: sub someFunkyComparisonFunction { my ($a, $b) = @_; $a =~ m{$b}; } @diffs = diff( @lines, @patterns, &someFunkyComparisonFunction ); which would allow you to diff an array @lines which consists of text lines with an array @patterns which consists of regular expressions. This is actually the reason I wrote this version -- there is no way to do this with a key generation function as in the stock Algorithm::Diff. perl v5.16.3 2006-07-31 Algorithm::DiffOld(3)
All times are GMT -4. The time now is 02:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy