Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Merge CSV files and create a column with the filename from the original file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merge CSV files and create a column with the filename from the original file
# 8  
Old 06-23-2011
Hi Klashxx,

CSVfix was working great, but unfortunately the data contained many special characters and it ended up with a few messed rows. I gave your script a go and it works fine, I only have a couple of issues:

1. For some reason it only opens and output one file (100_LARSSON_KRISTIAN.csv) although there are over 300 in the directory, all with the format number_surname_firstname.csv. I used
'/home/fran/Desktop/ex.pl' '/home/fran/Desktop/chalmers test.csv' /home/fran/Desktop/chalmers/*.csv

2. Path name is displayed after /n, which is great, but many fields contains embedded new lines, which complicate things. Do you know how to output the name only in the last /n? The difference between the last /n and the rest is that is not enclosed by "" (perhaps use CSV_XS? I'm trying to figure out how to use it)

I'm running ubuntu by the way.

Many thanks!
Last edited by fransanchezoria; 06-23-2011 at 07:49 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

I am trying to merge all csv files from source path into 1 file

I am trying to merge all csv files from source path into one single csv file in target. but getting error message: hadoop fs -cat /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/TCH-charge_*.csv > /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv getting error message:... (0 Replies)
Discussion started by: cplusplus1
0 Replies

2. UNIX for Dummies Questions & Answers

Merge two csv files using column name

Hi all, I have two separate csv files(comma delimited) file 1 and file 2. File 1 contains PAN,NAME,Salary AAAAA5467D,Raj,50000 AAFAC5467D,Ram,60000 BDCFA5677D,Kumar,90000 File 2 contains PAN,NAME,Dept,Salary ASDFG6756T,Karthik,ABC,450000 QWERT8765Y,JAX,CDR,780000... (5 Replies)
Discussion started by: Nivas
5 Replies

3. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

4. Shell Programming and Scripting

Merge different files into the original file

Hello Below is my requirement I have 3 files A1.txt , A2.txt and A3.txt . A2 is dynamically generating file I want the merge of A1,A2 and A3 in A2.txt Could you please help? (3 Replies)
Discussion started by: Pratik4891
3 Replies

5. UNIX for Dummies Questions & Answers

How to create a .csv file from 2 different .txt files?

Hi, I need to create a .csv file from information that i have in two different tab delimited .txt file. I just want to select some of the columns of each .txt file and paste them into a .cvs file. My files look like: File 1 transcript_id Seq. Description Seq. Length ... (2 Replies)
Discussion started by: alisrpp
2 Replies

6. Shell Programming and Scripting

create new column for filename

Hi, I created a list with 2 columns. Each line is from a different file. I am getting these with a loop in Perl. I would like to add a 3rd column with the name of the file that the line is coming from. I usually use pr to print the filename but this is not working here ... I was wondering if... (5 Replies)
Discussion started by: danieladna
5 Replies

7. Shell Programming and Scripting

How to create a CSV File by reading fields from separate files

SHELL SCRIPT Hi, I have 3 separate files within a folder. Every File contains data in a single column like File1 contains data mayank sushant dheeraj File2 contains DSA_AT MG_AT FLAT_09 File3 contains data 123123 232323 (2 Replies)
Discussion started by: mayanksargoch
2 Replies

8. Shell Programming and Scripting

Filename from splitting files to have the same filename of the original file with counter value

Hi all, I have a list of xml file. I need to split the files to a different files when see the <ko> tag. The list of filename are B20090908.1100-20090908.1200_CDMA=1,NO=2,SITE=3.xml B20090908.1200-20090908.1300_CDMA=1,NO=2,SITE=3.xml B20090908.1300-20090908.1400_CDMA=1,NO=2,SITE=3.xml ... (3 Replies)
Discussion started by: natalie23
3 Replies

9. Shell Programming and Scripting

Merging files to create CSV file

Hi, I have different files of the same type, as: Time: 100 snr: 88 perf: 10 other: 222 Each of these files are created periodically. What I need to do is to merge all of them into one but having the following form: (2 Replies)
Discussion started by: Ravendark
2 Replies

10. Shell Programming and Scripting

merge two two txt files into one file based on one column

Hi, I have file1.txt and file2.txt and would like to create file3.txt based on one column in UNIX Eg: file1.txt 17328756,0000786623.pdf,0000786623 20115537,0000793892.pdf,0000793892 file2.txt 12521_74_4.zip,0000786623.pdf 12521_15_5.zip,0000793892.pdf Desired Output ... (5 Replies)
Discussion started by: techmoris
5 Replies
Login or Register to Ask a Question

LEARN ABOUT CENTOS

processcsv.py

PROCESSCSV.PY(1)					      Virtualization Support						  PROCESSCSV.PY(1)

NAME

       processcsv.py - process virt-top CSV files

SUMMARY

	virt-top --csv data.csv
	processcsv.py < data.csv

DESCRIPTION

       virt-top is a top(1)-like utility for showing stats of virtualized domains.

       processcsv.py is a simple Python script that post-processes the output of "virt-top --csv".

       It is used like this:

	virt-top --csv data.csv
	processcsv.py < data.csv

       The second command will overwrite the following files in the current directory:

       "global.csv"
	   This contains the global (host) statistics columns from the CSV file.

       "domainNN.csv" (multiple files)
	   For each libvirt domain ID NN, a file is created containing the per-domain statistics from the CSV file.

SEE ALSO

       virt-top(1)

AUTHORS

       Richard W.M. Jones <rjones @ redhat . com>

COPYRIGHT

       (C) Copyright 2007-2012 Red Hat Inc., Richard W.M. Jones http://libvirt.org/

       This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
       the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

       This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

       You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation,
       Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

virt-top-1.0.8							    2013-12-29							  PROCESSCSV.PY(1)