I have a CSV file that has around 50K records in it. I have to delete all the duplicate rows from the file except one, depending upon data in column4.
Lets say there are 3 rows for 'column4data' in the file. I have to retain only that line which has the smallest date value in column2.
The file is already sorted based on column4data.
e.g For the lines below :
Result Expected ( Since 20080101 is least among all ):
The goal is to read every row of the file & read column4data in every row. Lets say column4data is unique ( does not occur twice) , move ahead and read column4data from second row and so on. Once we get a dulicate occurrence , read column2data from all the rows and do date comparsion and subsequent deletion.
Can somebody pls help me with a solution using 'awk' or 'sed' !
TIA
Last edited by Franklin52; 02-08-2011 at 04:28 AM..
Reason: Please use code tags, thank you
Hi everybody:
Could anybody tell me how I can delete repeated rows from a file?, this is, for exemple I have a file like this:
0.490 958.73 281.85 6.67985 0.002481
0.490 954.833 283.991 8.73019 0.002471
0.590 950.504 286.241 6.61451 0.002461
0.690 939.323 286.112 6.16451 0.00246
0.790... (8 Replies)
Hi
I have a file having 1000 rows. Now I would like to remove 10 rows from it. Plz give me the script.
Eg:
input file like
4 1 4500.0 1
5 1 1.0 30
6 1 1.0 4500
7 1 4.0 730
7 2 500000.0 730
8 1 785460.0 45
8 7 94255.0 30
9 1 31800.0 30
9 4 36000.0 30
10 1 15000.0 30... (5 Replies)
I have a file content like below.
"0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","",""
"0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","",""
"0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","",""... (5 Replies)
Dear unix gurus,
I have a data file with header information about a subject and also 3 columns of n rows of data on various items he owns. The data file looks something like this:
adam peter
blah blah blah
blah blah blah
car
01 30 200
02 31 400
03 57 121
.. .. ..
.. .. ..
n y... (8 Replies)
I need to delete rows based on the number of lines in a different file, I have a piece of code with me working but when I merge with my C application, it doesnt work.
sed '1,'\"`wc -l < /tmp/fileyyyy`\"'d' /tmp/fileA > /tmp/filexxxx
Can anyone give me an alternate solution for the above (2 Replies)
Say i have a file with X rows and Y columns....i see that in some of the rows,some columns are blank (no value set)...i wish to delete such rows....how can it be done?
e.g
181766 100 2009-06-04
184443 2009-06-04
10962 151 2009-06-04
161 2009-06-04... (7 Replies)
Hi all,
I have a big file (about 6 millions rows) and I have to delete same occurrences, stored in a small file (about 9000 rews). I have tried this:
while read line
do
grep -v $line big_file > ok_file.tmp
mv ok_file.tmp big_file
done < small_file
It works, but is very slow.
How... (2 Replies)
ENVIROMENT
Linux: Fedora Core release 1 (Yarrow)
iPlanet: iPlanet-WebServer-Enterprise/6.0SP1
Log Path: /usr/iplanet/servers/https-company/logs
I have iPlanet log rotation enabled rotating files on a daily basis.
The rotated logs are NOT compressed & are taking up too much space.
I... (7 Replies)
ENVIROMENT
Linux: RHEL 6.4
Log Path: /usr/iplanet/servers/https-company/logs
Log Format: user.log.03-15-2015
I have log4j log rotation enabled rotating files on a daily basis.
The rotated logs are NOT compressed & are taking up too much space.
I need a script that will run daily that... (1 Reply)
Hi All,
Do you have any sample script,
- auto get file from SFTP remote server and delete file in remove server after downloaded.
- only download specify filename
- auto upload file from local to SFTP remote server and delete local folder file after uploaded
- only upload specify filename
... (3 Replies)
Discussion started by: weesiong
3 Replies
LEARN ABOUT CENTOS
clags2.f
clags2.f(3) LAPACK clags2.f(3)NAME
clags2.f -
SYNOPSIS
Functions/Subroutines
subroutine clags2 (UPPER, A1, A2, A3, B1, B2, B3, CSU, SNU, CSV, SNV, CSQ, SNQ)
CLAGS2
Function/Subroutine Documentation
subroutine clags2 (logicalUPPER, realA1, complexA2, realA3, realB1, complexB2, realB3, realCSU, complexSNU, realCSV, complexSNV, realCSQ,
complexSNQ)
CLAGS2
Purpose:
CLAGS2 computes 2-by-2 unitary matrices U, V and Q, such
that if ( UPPER ) then
U**H *A*Q = U**H *( A1 A2 )*Q = ( x 0 )
( 0 A3 ) ( x x )
and
V**H*B*Q = V**H *( B1 B2 )*Q = ( x 0 )
( 0 B3 ) ( x x )
or if ( .NOT.UPPER ) then
U**H *A*Q = U**H *( A1 0 )*Q = ( x x )
( A2 A3 ) ( 0 x )
and
V**H *B*Q = V**H *( B1 0 )*Q = ( x x )
( B2 B3 ) ( 0 x )
where
U = ( CSU SNU ), V = ( CSV SNV ),
( -SNU**H CSU ) ( -SNV**H CSV )
Q = ( CSQ SNQ )
( -SNQ**H CSQ )
The rows of the transformed A and B are parallel. Moreover, if the
input 2-by-2 matrix A is not zero, then the transformed (1,1) entry
of A is not zero. If the input matrices A and B are both not zero,
then the transformed (2,2) element of B is not zero, except when the
first rows of input A and B are parallel and the second rows are
zero.
Parameters:
UPPER
UPPER is LOGICAL
= .TRUE.: the input matrices A and B are upper triangular.
= .FALSE.: the input matrices A and B are lower triangular.
A1
A1 is REAL
A2
A2 is COMPLEX
A3
A3 is REAL
On entry, A1, A2 and A3 are elements of the input 2-by-2
upper (lower) triangular matrix A.
B1
B1 is REAL
B2
B2 is COMPLEX
B3
B3 is REAL
On entry, B1, B2 and B3 are elements of the input 2-by-2
upper (lower) triangular matrix B.
CSU
CSU is REAL
SNU
SNU is COMPLEX
The desired unitary matrix U.
CSV
CSV is REAL
SNV
SNV is COMPLEX
The desired unitary matrix V.
CSQ
CSQ is REAL
SNQ
SNQ is COMPLEX
The desired unitary matrix Q.
Author:
Univ. of Tennessee
Univ. of California Berkeley
Univ. of Colorado Denver
NAG Ltd.
Date:
November 2011
Definition at line 158 of file clags2.f.
Author
Generated automatically by Doxygen for LAPACK from the source code.
Version 3.4.2 Tue Sep 25 2012 clags2.f(3)