Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Output first unique record in csv file
Top Forums Shell Programming and Scripting Output first unique record in csv file Post 302882726 by kurumi on Wednesday 8th of January 2014 11:41:21 PM
Old 01-09-2014
if you have Ruby on your system

Code:
array=[]
File.open("file").each_line do |line|
  first = line.split(/\s+/).first
  if not array.include?(first) 
    puts line
    array << first 
  end       
end

Code:
# ruby test.rb 
color product id status
green 102 pass
yellow 201 hold

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to delete a record from a csv file

Hi Guys I have downloaded a table from oracle database in .csv format. it has many fields as Title, First Name, Last Name etc. I have to download distinct titles from database and now i have to check all those titles from data of First Name one by one. and then i have to delete matched record.... (1 Reply)
Discussion started by: Rajeev Agrawal
1 Replies

2. Shell Programming and Scripting

Record count of a csv file

Hello Gurus, We have a requirement to count the valid number of records in a comma delimited file with double quotes. The catch here is..few records have a new line carriage within the double quotes,,say for ex:we have a file called accounts the record count is 4827..but the actual valid count... (5 Replies)
Discussion started by: ajaykk
5 Replies

3. Shell Programming and Scripting

Split long record into csv file

Hi I receive a mainframe file which has very long records (1100 chars) with no field delimiters. I need to parse each record and output a comma delimited (csv) file. The record layout is fixed. If there weren't so many fields and records I would read the file into Excel, as a "fixed width"... (10 Replies)
Discussion started by: wvdeijk
10 Replies

4. Shell Programming and Scripting

return a list of unique values of a column from csv format file

Hi all, I have a huge csv file with the following format of data, Num SNPs, 549997 Total SNPs,555352 Num Samples, 157 SNP, SampleID, Allele1, Allele2 A001,AB1,A,A A002,AB1,A,A A003,AB1,A,A ... ... ... I would like to write out a list of unique SNP (column 1). Could you... (3 Replies)
Discussion started by: phoeberunner
3 Replies

5. Shell Programming and Scripting

csv file - adding total to a trailer record

Hi, I have a script which creates and modifies a csv file. I have managed to do everything I need to do apart from 1 thing. I need to append a trailer record to the file. I need this line to hold the total of an entire column of the csv file (skipping the 1st line which is a header). Can... (2 Replies)
Discussion started by: mcclunyboy
2 Replies

6. Shell Programming and Scripting

List unique values and count instances in .csv file

I need to take the second column of a .csv file and count the number of instances of each unique value in that same second column. I'd like the output to be value,count sorted by most instances. Thanks for any guidance! Data example: 317476,317756,0 816063,318861,0 313123,319091,0... (4 Replies)
Discussion started by: batcho
4 Replies

7. Linux

To get all the columns in a CSV file based on unique values of particular column

cat sample.csv ID,Name,no 1,AAA,1 2,BBB,1 3,AAA,1 4,BBB,1 cut -d',' -f2 sample.csv | sort | uniq this gives only the 2nd column values Name AAA BBB How to I get all the columns of CSV along with this? (1 Reply)
Discussion started by: sanvel
1 Replies

8. Shell Programming and Scripting

Save output of updated csv file as csv file itself

Hi, all I want to sort a csv file based on timestamp from oldest to newest and save the output as csv file itself. Here is an example of my csv file. test.csv SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0739.JPG,2015:02:17 11:32:21 /home/intannf/foto/IMG_0749.JPG,2015:02:17 11:37:28... (10 Replies)
Discussion started by: refrain
10 Replies

9. Shell Programming and Scripting

Save output of updated csv file as csv file itself, part 2

Hi, I have another problem. I want to sort another csv file by the first field. result.csv SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw /home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 ... (2 Replies)
Discussion started by: refrain
2 Replies

10. Shell Programming and Scripting

CSV File:Filter duplicate records from column1 & another column having unique record

Hi Experts, I have csv file with 30, 40 columns Pasting just 2 column for problem description. Need to print error if below combination is not present in file check for column-1 (DocumentNumber) and filter columns where value in DocumentNumber field is same. For all such rows, the field... (7 Replies)
Discussion started by: as7951
7 Replies

LEARN ABOUT DEBIAN

py_xls2csv

py_xls2csv(1)						      General Commands Manual						     py_xls2csv(1)

NAME

       py_xls2csv - convert an Excel xls file to a comma separated value csv file

SYNOPSIS

       py_xls2csv  input-file

DESCRIPTION

       This manual page was written for the Debian distribution because the original program does not have a manual page.

       py_xls2csv takes an Excel xls file as an argument and converts it to a comma separated value csv file.  Output is sent to stdout.

       Additional utility scripts can be found in the tools/ directory.

OPTIONS

       This program does not take any command line options.

AUTHOR

       pyexcelerator and py_xls2csv were written by Roman V. Kiseliov <roman@kiseliov.ru>.

       This manual page was written by Kevin Coyner <kcoyner@debian.org> for the Debian system (but may be used by others).  Permission is granted
       to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 any  later	version  published
       by the Free Software Foundation.

       On Debian systems, the complete text of the GNU General Public License can be found in /usr/share/common-licenses/GPL.

SEE-ALSO
       pyexcelerator(1), py_xls2txt(1), py_xls2html(1)

								 October 12, 2006						     py_xls2csv(1)
All times are GMT -4. The time now is 11:51 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy