|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Filter/remove duplicate .dat file with certain criteria
Post 302504332 by mukeshguliao on Monday 14th of March 2011 10:27:08 AM
03-14-2011
it seems fine to me, but its still anit working
trying this (csh shell)
awk -F',' '{key=$1$2$3;if(key in a) next; else a[$1$2$3]=$0; print a[$1$2$3]}' UnixEg.dat
getting
awk: syntax error near line 1
awk: illegal statement near line 1
awk: illegal statement near line 1
trying this (csh shell)
awk -F',' '{key=$1$2$3;if(key in a) next; else a[$1$2$3]=$0; print a[$1$2$3]}' UnixEg.dat
getting
awk: syntax error near line 1
awk: illegal statement near line 1
awk: illegal statement near line 1
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hello to all,
I have this file
2002 1 23 0 0 2435.60 131.70 5.60 20.99 0.89 0.00 285.80 2303.90
2002 1 23 15 0 2436.60 132.90 6.45 21.19 1.03 0.00 285.80 2303.70
2002 1 23 ... (6 Replies)
Discussion started by: joggdial3000
6 Replies
2. Shell Programming and Scripting
All,
I have a file 1181CUSTOMER-L061411_003500.dat.Z having duplicate records in it.
bash-2.05$ zcat 1181CUSTOMER-L061411_003500.dat.Z|grep "90876251S"
90876251S|ABG, AN ADAYANA COMPANY|3550 DEPAUW BLVD|||US|IN|INDIANAPOLIS||DAL|46268||||||GEN|||||||USD|||ABG, AN ADAYANA... (3 Replies)
Discussion started by: Oracle_User
3 Replies
3. Shell Programming and Scripting
Heya there,
A small selection of my data is shown below.
DATE TIME FRAC_DAYS_SINCE_JAN1
2011-06-25 08:03:20.000 175.33564815
2011-06-25 08:03:25.000 175.33570602
2011-06-25 ... (4 Replies)
Discussion started by: gd9629
4 Replies
4. Shell Programming and Scripting
Hello,
Although I have found similar questions, I could not find advice that
could help with our problem.
The issue:
We have several hundreds text files containing repeated blocks of text
(I guess back at the time they were prepared like that to optmize
printing).
The block of texts... (13 Replies)
Discussion started by: samask
13 Replies
5. Shell Programming and Scripting
Hi,
The source system has created the file in the dat format and put into the linux directory as mentioned below. I want to do foloowing things.
a) Delete the Line started with <CR><LF> in the record
b)Also line
...........................................................<CR><LF>
... (1 Reply)
Discussion started by: mr_harish80
1 Replies
6. Shell Programming and Scripting
Input file:
##fileformat=tab
##reference=file:input.txt
#Line Position Score
Input_185827_2127 1071 67
Input_18213_21 1021 100
Input_9012_214 200 150
Input_935_217 124 70
Output file:
##fileformat=tab
##reference=file:input.txt
#Line Position Score
Input_18213_21 1021... (2 Replies)
Discussion started by: perl_beginner
2 Replies
7. Shell Programming and Scripting
Hello,
I am trying to extract valid data blocks from invalid ones. In the input the data blocks are separated by one or more blank rows. The criteria are
1) second column value must be 30 or more for the row to be valid and considered for calculation and output.
2) the sum of all valid... (2 Replies)
Discussion started by: sheetalk
2 Replies
8. Shell Programming and Scripting
Hello,
I have a script that is generating a tab delimited output file.
num Name PCA_A1 PCA_A2 PCA_A3
0 compound_00 -3.5054 -1.1207 -2.4372
1 compound_01 -2.2641 0.4287 -1.6120
3 compound_03 -1.3053 1.8495 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies
9. Shell Programming and Scripting
I have two files and would need to filter out records based on certain criteria, these column are of variable lengths, but the lengths are uniform throughout all the records of the file. I have shown a sample of three records below. Line 1-9 is the item number "0227546_1" in the case of the first... (15 Replies)
Discussion started by: MIA651
15 Replies
10. Shell Programming and Scripting
I have csv file with 30, 40 columns
Pasting just three column for problem description
I want to filter record if column 1 matches CN or DN then,
check for values in column 2 if column contain 1235, 1235 then in column 3 values must be sequence of 2345, 2345
and if column 2 contains 6789, 6789... (5 Replies)
Discussion started by: as7951
5 Replies
LEARN ABOUT HPUX
unknown
UNKNOWN(1) General Commands Manual UNKNOWN(1) NAME
unknown - identify possible genotypes for unknowns SYNOPSIS
A program to rapidly identify which genotypes are possible for individuals typed as unknowns in the input pedigree. unknown [ -cl ] DESCRIPTION
unknown infers possible genotypes and mating combinations for parents with unknown genotypes for ilink(1), mlink(1) and linkmap(1). OPTIONS
-c Use conditional allele frequencies. -l Choose a good set of loop breakers automatically. RETURN VALUE
0 Successful completion ERRORS
10 File not found 255 Failure EXAMPLES
Normally, unknown(1) is run immediately prior to its sister programs, ilink(1), mlink(1) and linkmap(1), like this: unknown mlink FILES
unknown(1) reads the two files pedfile.dat and datafile.dat as its own input and produces various temporary files that are used as input to the next program. These temporary files are ipedfile.dat, upedfile.dat, speedfile.dat and newspeedfile.dat. NOTES
unknown(1) is part of the FASTLINK package, which is a re-implementation of the LINKAGE suite of computer tools that help investigate genetic linkage as first proposed G.M. Lathrop, J.M. Lalouel, C. Julier, and J. Ott. AUTHORS
Dylan Cooper, Alejandro Schaffer, and Tony Schurtz based on work originally by Jurg Ott, Ph.D, et. al. This manual page was written by Elizabeth Barham <lizzy@soggytrousers.net> for the Debian GNU/Linux system (but may be used by others). WORD-WIDE-WEB http://www.ncbi.nlm.nih.gov/CBBResearch/Schaffer/fastlink.html SEE ALSO
ilink(1), linkmap(1), lodscore(1), mlink(1). April 15, 2003 UNKNOWN(1)