CSV File:Filter duplicate records from column1 & another column having unique record
Hi Experts,
I have csv file with 30, 40 columns
Pasting just 2 column for problem description.
Need to print error if below combination is not present in file
check for column-1 (DocumentNumber) and filter columns where value in DocumentNumber field is same.
For all such rows, the field LineNumber (column-2) should be unique for each row.
if column1 contain duplicate value(2345,2345) on row(1-2) then, column 2 must contain any random unique value like (1,2) in row(1-2)
similary for column 1 row(3-4) with duplicate value(6789,6789), then column 2 must contain uniquie value as below 5,6 If combination as explained above is not present, then logs must be printed in another file with error code and line number
Hi all,
I have a huge csv file with the following format of data,
Num SNPs, 549997
Total SNPs,555352
Num Samples, 157
SNP, SampleID, Allele1, Allele2
A001,AB1,A,A
A002,AB1,A,A
A003,AB1,A,A
...
...
...
I would like to write out a list of unique SNP (column 1). Could you... (3 Replies)
Hi,
Need to find a duplicate records on the first column,
ANU4501710430989 0000000W20389390
ANU4501710430989 0000000W67065483
ANU4501130050520 0000000W80838713
ANU4501210170685 0000000W69246611... (3 Replies)
Hi Unix gurus,
Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me.
File format: CSV file
File has four columns with no header... (8 Replies)
Hi,
I want to remove duplicate records including the first line based on column1. For example
inputfile(filer.txt):
-------------
1,3000,5000
1,4000,6000
2,4000,600
2,5000,700
3,60000,4000
4,7000,7777
5,999,8888
expected output:
----------------
3,60000,4000
4,7000,7777... (5 Replies)
I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1.
https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html
Can someone explain this command please? How are there no... (5 Replies)
I have a .CSV file with the below format:
"column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10
"12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013",""... (2 Replies)
Hi,
I have to output a new csv file from an input csv file with first unique value in the first column.
input csv file
color product id status
green 102 pass
yellow 201 hold
yellow 202 keep
green 101 ok
green 103 hold
yellow 203 ... (5 Replies)
cat sample.csv
ID,Name,no
1,AAA,1
2,BBB,1
3,AAA,1
4,BBB,1
cut -d',' -f2 sample.csv | sort | uniq
this gives only the 2nd column values
Name
AAA
BBB
How to I get all the columns of CSV along with this? (1 Reply)
Hello,
I have a script that is generating a tab delimited output file.
num Name PCA_A1 PCA_A2 PCA_A3
0 compound_00 -3.5054 -1.1207 -2.4372
1 compound_01 -2.2641 0.4287 -1.6120
3 compound_03 -1.3053 1.8495 ... (3 Replies)
I have csv file with 30, 40 columns
Pasting just three column for problem description
I want to filter record if column 1 matches CN or DN then,
check for values in column 2 if column contain 1235, 1235 then in column 3 values must be sequence of 2345, 2345
and if column 2 contains 6789, 6789... (5 Replies)
Discussion started by: as7951
5 Replies
LEARN ABOUT PHP
db2_special_columns
DB2_SPECIAL_COLUMNS(3) 1 DB2_SPECIAL_COLUMNS(3)db2_special_columns - Returns a result set listing the unique row identifier columns for a tableSYNOPSIS
resource db2_special_columns (resource $connection, string $qualifier, string $schema, string $table_name, int $scope)
DESCRIPTION
Returns a result set listing the unique row identifier columns for a table.
PARAMETERS
o $connection
- A valid connection to an IBM DB2, Cloudscape, or Apache Derby database.
o $qualifier
- A qualifier for DB2 databases running on OS/390 or z/OS servers. For other databases, pass NULL or an empty string.
o $schema
- The schema which contains the tables.
o $table_name
- The name of the table.
o $scope
- Integer value representing the minimum duration for which the unique row identifier is valid. This can be one of the following
values:
+--------------+--------------------------------------+---+
|Integer value | | |
| | | |
| | SQL constant | |
| | | |
| | Description | |
| | | |
+--------------+--------------------------------------+---+
| 0 | | |
| | | |
| | SQL_SCOPE_CURROW | |
| | | |
| | Row identifier is valid only while | |
| | the cursor is positioned on the row. | |
| | | |
| 1 | | |
| | | |
| | SQL_SCOPE_TRANSACTION | |
| | | |
| | Row identifier is valid for the | |
| | duration of the transaction. | |
| | | |
| 2 | | |
| | | |
| | SQL_SCOPE_SESSION | |
| | | |
| | Row identifier is valid for the | |
| | duration of the connection. | |
| | | |
+--------------+--------------------------------------+---+
RETURN VALUES
Returns a statement resource with a result set containing rows with unique row identifier information for a table. The rows are composed
of the following columns:
+------------+---------------------------------------------------+
|Column name | |
| | |
| | Description |
| | |
+------------+---------------------------------------------------+
| SCOPE | |
| | |
| | |
| | |
| | box, tab (|); c | c | c | . T{ Integer |
| | value |
| | |
| | SQL constant |
| | |
| | Description |
| | |
+------------+---------------------------------------------------+
| 0 | |
| | |
| | SQL_SCOPE_CURROW |
| | |
| | Row identifier is valid only while the cursor is |
| | positioned on the row. |
| | |
| 1 | |
| | |
| | SQL_SCOPE_TRANSACTION |
| | |
| | Row identifier is valid for the duration of the |
| | transaction. |
| | |
| 2 | |
| | |
| | SQL_SCOPE_SESSION |
| | |
| | Row identifier is valid for the duration of the |
| | connection. |
| | |
+------------+---------------------------------------------------+
T} T{ COLUMN_NAME
T} |T{ Name of the unique column.
T} T{ DATA_TYPE
T} |T{ SQL data type for the column.
T} T{ TYPE_NAME
T} |T{ Character string representation of the SQL data type for the column.
T} T{ COLUMN_SIZE
T} |T{ An integer value representing the size of the column.
T} T{ BUFFER_LENGTH
T} |T{
Maximum number of bytes necessary to store data from this column.
T} T{ DECIMAL_DIGITS
T} |T{
The scale of the column, or NULL where scale is not applicable.
T} T{ NUM_PREC_RADIX
T} |T{
An integer value of either 10 (representing an exact numeric data type), 2 (representing an approximate numeric data type), or NULL (rep-
resenting a data type for which radix is not applicable).
T} T{ PSEUDO_COLUMN
T} |T{ Always returns 1.
T}
SEE ALSO db2_column_privileges(3), db2_columns(3), db2_foreign_keys(3), db2_primary_keys(3), db2_procedure_columns(3), db2_procedures(3), db2_sta-
tistics(3), db2_table_privileges(3), db2_tables(3).
PHP Documentation Group DB2_SPECIAL_COLUMNS(3)