09-15-2009
Quote:
Originally Posted by
ripat
Linux and ksh. But in this case I don't think that the type of shell is relevant as all solutions are using external programs. I ran that test on large files as one can assume that the OP was just giving a sample and will be working on larger files.
You are right. How big were your files? Only OS and system load would make difference. I want to try it on Solaris 10. In the case of large files, there would be a threshold at which awk could gain advantage. It all depends on how these utilities are written. The multiple commands have smaller memory footprint but use multiple redirections (waiting for I/O).
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I can't find how to achive such thing, please help.
I have try with uniq and comm but those command can't compare columns just whole lines,
I think awk will be the best but awk is magic for me as of now.
file a
a1~a2~a3~a4~a6~a7~a8
file b
b1~b2~b3~b4~b6~b7~b8
output 1:
compare... (2 Replies)
Discussion started by: pp56825
2 Replies
2. Shell Programming and Scripting
I am trying to compare two file and make changes where ever its different.
for example:
Contents of file1
IP=192.165.89.11
NM=255.255.0.0
GW=192.165.89.1
Contents of file2
IP=192.165.89.11
NM=255.255.255.255
GW=192.165.89.1
NOTE HERE THAT NM IS DIFFERENT
So i want the changes... (6 Replies)
Discussion started by: pradeepreddy
6 Replies
3. Ubuntu
Hello,
I have a 1.6 GB file that I would like to modify by matching some ids in col1 with the ids in col 1 of file2.txt and save the results into a 3rd file.
For example:
File 1 has 1411 rows, I ignore how many columns it has (thousands)
File 2 has 311 rows, 1 column
Would like to... (7 Replies)
Discussion started by: sogi
7 Replies
4. Shell Programming and Scripting
Hi,
My first file has
592155 9 rs16916098 1
592156 19 rs7249604 1
592157 4 rs885156 1
592158 5 rs350067 12nd file has
9 rs16916098 0 113228129 2 4
19 rs7249604 0 58709070 4 2
2 rs17042833 0 113558750 4 2... (2 Replies)
Discussion started by: genehunter
2 Replies
5. Shell Programming and Scripting
Hi, Experts,
I have a requirement as following:
my source file:
a
a
a
b
b
c
c
c
c
I need add one more colume as following:
1 a
2 a
3 a
1 b
2 b
1 c
2 c (4 Replies)
Discussion started by: ken002
4 Replies
6. UNIX for Advanced & Expert Users
Hello everyone,
I am writing a script to process data from the ATP world tour.
I have a file which contains:
t=540 y=2011 r=1 p=N409
t=540 y=2011 r=2 p=N409
t=540 y=2011 r=3 p=N409
t=540 y=2011 r=4 p=N409
t=520 y=2011 r=1 p=N409
t=520 y=2011 r=2 p=N409
t=520 y=2011 r=3 p=N409
The... (4 Replies)
Discussion started by: imahmoud
4 Replies
7. Shell Programming and Scripting
Hello,
I have some tab delimited data and I need to move the last col. I could hard code it,
awk '{ print $1,$NF,$2,$3,$4,etc }' infile > outfile
but it would be nice to know the syntax to print a range cols.
I know in cut you can do,
cut -f 1,4-8,11-
to print fields 1,... (8 Replies)
Discussion started by: LMHmedchem
8 Replies
8. Shell Programming and Scripting
Hi Friends,
This is the only solution to my task. So, any help is highly appreciated.
I have a file
cat input1.bed
chr1 100 200 abc
chr1 120 300 def
chr1 145 226 ghi
chr2 567 600 unix
Now, I have another file by name
input2.bed (This file is a binary file not readable by the... (7 Replies)
Discussion started by: jacobs.smith
7 Replies
9. Shell Programming and Scripting
Hi,
Please help with this.
I have several excel files (with and .xlsx format) with 10-15 columns each.
They all have the same type of data but the columns are not ordered in the same way.
Here is a 3 column example. What I want to do add the alphabet
from column 2 to column 3, provided... (9 Replies)
Discussion started by: newbie83
9 Replies
10. UNIX for Beginners Questions & Answers
hi
I have 2 file with more than 10 columns for both
1st file
apple,0,0,0......
orange,1,2,3.....
mango,2,4,5.....
2nd file
apple,2,3,4,5,6,7...
orange,2,3,4,5,6,8...
watermerlon,2,3,4,5,6,abc...
mango,5,6,7,4,6,def.... (1 Reply)
Discussion started by: tententen
1 Replies
LEARN ABOUT DEBIAN
fitscheck
fitscheck(1) PyFITS 3.0.7 fitscheck(1)
NAME
fitscheck - script to detect and fix FITS standards violations
SYNOPSIS
fitscheck [OPTION]... [FILE]...
DESCRIPTION
fitscheck is a command line script based on pyfits for verifying and updating the CHECKSUM and DATASUM keywords of FITS files. itscheck
can also detect and often fix other FITS standards violations. fitscheck facilitates re-writing the non-standard checksums originally gen-
erated by pyfits with standard checksums which will interoperate with cfitsio.
fitscheck will refuse to write new checksums if the checksum keywords are missing or their values are bad. Use --force to write new check-
sums regardless of whether or not they currently exist or pass. Use --ignore-missing to tolerate missing checksum keywords without com-
ment.
OPTIONS
-h, --help
Display terse usage information (help).
-k [standard | nonstandard | either | none], --checksum=[standard | nonstandard | either | none]
Choose FITS checksum mode or none. Defaults to standard.
-w, --write
Write out file checksums and/or FITS compliance fixes.
-f, --force
Do file update even if original checksum was bad.
-c, --compliance
Do FITS compliance checking, fix if possible.
-i, --ignore-missing
Ignore missing checksums.
-v, --verbose
Generate extra output.
EXAMPLES
% fitscheck --checksum either --write *.fits
Verify and update checksums, tolerating non-standard checksums, updating to standard checksum.
% fitscheck --write --force *.fits
Write new checksums, even if existing checksums are bad or missing.
% fitscheck --compliance *.fits
Verify standard checksums and FITS compliance without changing the files.
% fitscheck --checksum nonstandard *.fits
Verify original nonstandard checksums only.
% fitscheck --checksum none --compliance --write *.fits
Only check and fix compliance problems, ignoring checksums.
% fitscheck *.fits
Verify standard interoperable checksums.
% fitscheck --checksum none --write *.fits
Delete checksum keywords.
fitscheck June 2012 fitscheck(1)