Sponsored Content
Top Forums Shell Programming and Scripting Merge CSV files and create a column with the filename from the original file Post 302521044 by Klashxx on Tuesday 10th of May 2011 05:19:14 AM
Old 05-10-2011
A Perl:
Code:
#!/usr/bin/perl
my $result_file=shift;
my $pattern=shift;
my (@file_list) = glob $pattern;

open( OUT, ">$result_file") or die "Can't open '$result_file':$!";
foreach my $in_file (@file_list) { 
   open(IN, "$in_file"); 
   while (<IN>) {
      chomp;
      print OUT $_ . ',' . $in_file . "\n" ;
      }
   close (IN);
   }

close (OUT);

Usage:
Code:
ex.pl result.txt 'file*csv'

This User Gave Thanks to Klashxx For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge two two txt files into one file based on one column

Hi, I have file1.txt and file2.txt and would like to create file3.txt based on one column in UNIX Eg: file1.txt 17328756,0000786623.pdf,0000786623 20115537,0000793892.pdf,0000793892 file2.txt 12521_74_4.zip,0000786623.pdf 12521_15_5.zip,0000793892.pdf Desired Output ... (5 Replies)
Discussion started by: techmoris
5 Replies

2. Shell Programming and Scripting

Merging files to create CSV file

Hi, I have different files of the same type, as: Time: 100 snr: 88 perf: 10 other: 222 Each of these files are created periodically. What I need to do is to merge all of them into one but having the following form: (2 Replies)
Discussion started by: Ravendark
2 Replies

3. Shell Programming and Scripting

Filename from splitting files to have the same filename of the original file with counter value

Hi all, I have a list of xml file. I need to split the files to a different files when see the <ko> tag. The list of filename are B20090908.1100-20090908.1200_CDMA=1,NO=2,SITE=3.xml B20090908.1200-20090908.1300_CDMA=1,NO=2,SITE=3.xml B20090908.1300-20090908.1400_CDMA=1,NO=2,SITE=3.xml ... (3 Replies)
Discussion started by: natalie23
3 Replies

4. Shell Programming and Scripting

How to create a CSV File by reading fields from separate files

SHELL SCRIPT Hi, I have 3 separate files within a folder. Every File contains data in a single column like File1 contains data mayank sushant dheeraj File2 contains DSA_AT MG_AT FLAT_09 File3 contains data 123123 232323 (2 Replies)
Discussion started by: mayanksargoch
2 Replies

5. Shell Programming and Scripting

create new column for filename

Hi, I created a list with 2 columns. Each line is from a different file. I am getting these with a loop in Perl. I would like to add a 3rd column with the name of the file that the line is coming from. I usually use pr to print the filename but this is not working here ... I was wondering if... (5 Replies)
Discussion started by: danieladna
5 Replies

6. UNIX for Dummies Questions & Answers

How to create a .csv file from 2 different .txt files?

Hi, I need to create a .csv file from information that i have in two different tab delimited .txt file. I just want to select some of the columns of each .txt file and paste them into a .cvs file. My files look like: File 1 transcript_id Seq. Description Seq. Length ... (2 Replies)
Discussion started by: alisrpp
2 Replies

7. Shell Programming and Scripting

Merge different files into the original file

Hello Below is my requirement I have 3 files A1.txt , A2.txt and A3.txt . A2 is dynamically generating file I want the merge of A1,A2 and A3 in A2.txt Could you please help? (3 Replies)
Discussion started by: Pratik4891
3 Replies

8. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

9. UNIX for Dummies Questions & Answers

Merge two csv files using column name

Hi all, I have two separate csv files(comma delimited) file 1 and file 2. File 1 contains PAN,NAME,Salary AAAAA5467D,Raj,50000 AAFAC5467D,Ram,60000 BDCFA5677D,Kumar,90000 File 2 contains PAN,NAME,Dept,Salary ASDFG6756T,Karthik,ABC,450000 QWERT8765Y,JAX,CDR,780000... (5 Replies)
Discussion started by: Nivas
5 Replies

10. Shell Programming and Scripting

I am trying to merge all csv files from source path into 1 file

I am trying to merge all csv files from source path into one single csv file in target. but getting error message: hadoop fs -cat /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/TCH-charge_*.csv > /user/hive/warehouse/stage.db/PK_CLOUD_CHARGE/final/TCH_pb_charge.csv getting error message:... (0 Replies)
Discussion started by: cplusplus1
0 Replies
Perl::Critic::Policy::InputOutput::RequireBriefOpen(3)	User Contributed Perl Documentation Perl::Critic::Policy::InputOutput::RequireBriefOpen(3)

NAME
Perl::Critic::Policy::InputOutput::RequireBriefOpen - Close filehandles as soon as possible after opening them. AFFILIATION
This Policy is part of the core Perl::Critic distribution. DESCRIPTION
One way that production systems fail unexpectedly is by running out of filehandles. Filehandles are a finite resource on every operating system that I'm aware of, and running out of them is virtually impossible to recover from. The solution is to not run out in the first place. What causes programs to run out of filehandles? Usually, it's leaks: you open a filehandle and forget to close it, or just wait a really long time before closing it. This problem is rarely exposed by test systems, because the tests rarely run long enough or have enough load to hit the filehandle limit. So, the best way to avoid the problem is 1) always close all filehandles that you open and 2) close them as soon as is practical. This policy takes note of calls to "open()" where there is no matching "close()" call within "N" lines of code. If you really need to do a lot of processing on an open filehandle, then you can move that processing to another method like this: sub process_data_file { my ($self, $filename) = @_; open my $fh, '<', $filename or croak 'Failed to read datafile ' . $filename . '; ' . $OS_ERROR; $self->_parse_input_data($fh); close $fh; return; } sub _parse_input_data { my ($self, $fh) = @_; while (my $line = <$fh>) { ... } return; } As a special case, this policy also allows code to return the filehandle after the "open" instead of closing it. Just like the close, however, that "return" has to be within the right number of lines. From there, you're on your own to figure out whether the code is promptly closing the filehandle. The STDIN, STDOUT, and STDERR handles are exempt from this policy. CONFIGURATION
This policy allows "close()" invocations to be up to "N" lines after their corresponding "open()" calls, where "N" defaults to 9. You can override this to set it to a different number with the "lines" setting. To do this, put entries in a .perlcriticrc file like this: [InputOutput::RequireBriefOpen] lines = 5 CAVEATS
"IO::File->new" This policy only looks for explicit "open" calls. It does not detect calls to "CORE::open" or "IO::File->new" or the like. Is it the right lexical? We don't currently check for redeclared filehandles. So the following code is false negative, for example, because the outer scoped filehandle is not closed: open my $fh, '<', $file1 or croak; if (open my $fh, '<', $file2) { print <$fh>; close $fh; } This is a contrived example, but it isn't uncommon for people to use $fh for the name of the filehandle every time. Perhaps it's time to think of better variable names... CREDITS
Initial development of this policy was supported by a grant from the Perl Foundation. AUTHOR
Chris Dolan <cdolan@cpan.org> COPYRIGHT
Copyright (c) 2007-2011 Chris Dolan. Many rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of this license can be found in the LICENSE file included with this module perl v5.16.3 2014-06-09 Perl::Critic::Policy::InputOutput::RequireBriefOpen(3)
All times are GMT -4. The time now is 02:05 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy