Need to merge multiple text files vertically and place comma between fields
Hello expert friends,
I'm writing a script to capture stats using sar and stuck up at report generation.
I have around 10 files in a directory and need to merge them all vertically based on the time value of first column (output file should have only one time value) and insert comma after every field. For ex:
The auditors have nailed us for world writeable files....
Apparently in years gone by, quite a number of our kornshell scripts have had:
umask 000 put in the script.
We have been able to turn off world writeable for existing dirs & files, but as these scripts run, new files keep getting... (1 Reply)
I would like to know code that will delete multiple text from a comma delimited file. For example, how would the comma delimited file below delete the word 'PEST' in Perl language (previously an excel file that was converted to a csv and the last column was PEST):
1, 2,43,34, bosx,PEST
1,... (1 Reply)
Situation:
Our system currently executes a job (COBOL Program) that generates an interface file to be sent to one of our vendors. Because this system processes information for over 100,000 employees/retirees (and growing), we'd like to multi-thread the job into processing-groups in order to... (4 Replies)
Is it possible to place a comma in the desired places, like 10spaces after or 15 spaces after, irrespective of the contents???
Ex:File: TEST
TEST:
vimalthomaswants to place a comma
can he do it in the desired places?
as per the above file, i need to place a comma after 10th space... (4 Replies)
Hi all,
I have a file in which I have to insert commna between entries of 2 column and createa new file separated by commas not a columns
if input is
FHIT Adenosine Monotungstate Not Available
CS Trifluoroacetonyl Coenzyme A Not Available
Theo expected output is
... (5 Replies)
Hi All,
I am trying to merge all the text files into one file using below snippet
cat /home/Temp/Test/Log/*.txt >> all.txt
But it seems it is not working.
I have multiple files like Output_ServerName1.txt, Output_ServreName2.txt
I want to merge each file into one single file and... (6 Replies)
Hi,
Please excuse for often requesting queries and making R&D, I am trying to work out a possibility where i have two files field separated by pipe and another file containing only one field where there is no matching columns, Could you please advise how to merge two files.
$more... (3 Replies)
Hello,
I'm back again looking for your precious help-
This time I need to merge two text files with matching two fields, output only common records with mixed output.
Let's look at the example:
FILE1
56153;AAA0708;3;TEST1TEST1;
89014;BBB0708;3;TEST2TEST2;
89014;BBB0708;4;TEST3TEST3;
... (7 Replies)
can anyone help me!!!! How to I parse the CSV file
file name : abc.csv (csv file) The above file containing data like
abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies
LEARN ABOUT DEBIAN
tabmerge
TABMERGE(1p) User Contributed Perl Documentation TABMERGE(1p)NAME
tabmerge - unify delimited files on common fields
SYNOPSIS
tabmerge [action] [options] file1 file2 [...]
Actions:
--min Take only fields present in all files [DEFAULT]
--max Take all fields present
-f|--fields=f1[,f2] Take only the fields mentioned in the
comma-separated list
Options:
-l|--list List available fields
--fs=x Use "x" as the field separator
(default is tab " ")
--rs=x Use "x" as the record separator
(default is newline "
")
-s|--sort=f1[,f2] Sort data ASCII-betically on field(s)
--stdout Print data in original delimited format
(i.e., not in a table format)
--help Show brief help and quit
--man Show full documentation
DESCRIPTION
This program merges the fields -- not the rows -- of delimited text files. That is, if several files are almost but not quite entirely
unlike each other in their structure (in their field names, numbers or orders), this script allows you to easily unify the files into one
file with all the same fields. The output can be based on fields as determined by the three "action" flags.
For the following examples, consider three files that contain the following fields:
+------------+---------------------------------+
| File | Fields |
+------------+---------------------------------+
| merge1.tab | name, type, position |
| merge2.tab | name, type, position, lod_score |
| merge3.tab | name, position |
+------------+---------------------------------+
To list all available fields in the files and the number of times they are present:
$ tabmerge --list merge*
+-----------+-------------------+
| Field | No. Times Present |
+-----------+-------------------+
| lod_score | 1 |
| name | 3 |
| position | 3 |
| type | 2 |
+-----------+-------------------+
To merge the files on the minimum overlapping fields:
$ tabmerge merge*
+----------+----------+
| name | position |
+----------+----------+
| RM104 | 2.30 |
| RM105 | 4.5 |
| TX5509 | 10.4 |
| UU189 | 19.0 |
| Xpsm122 | 3.3 |
| Xpsr9556 | 4.5 |
| DRTL | 2.30 |
| ALTX | 4.5 |
| DWRF | 10.4 |
+----------+----------+
To merge the files and include all the fields:
$ tabmerge --max merge*
+-----------+----------+----------+--------+
| lod_score | name | position | type |
+-----------+----------+----------+--------+
| | RM104 | 2.30 | RFLP |
| | RM105 | 4.5 | RFLP |
| | TX5509 | 10.4 | AFLP |
| 2.4 | UU189 | 19.0 | SSR |
| 1.2 | Xpsm122 | 3.3 | Marker |
| 1.2 | Xpsr9556 | 4.5 | Marker |
| | DRTL | 2.30 | |
| | ALTX | 4.5 | |
| | DWRF | 10.4 | |
+-----------+----------+----------+--------+
To merge and extract just the "name" and "type" fields:
$ tabmerge -f name,type merge*
+----------+--------+
| name | type |
+----------+--------+
| RM104 | RFLP |
| RM105 | RFLP |
| TX5509 | AFLP |
| UU189 | SSR |
| Xpsm122 | Marker |
| Xpsr9556 | Marker |
| DRTL | |
| ALTX | |
| DWRF | |
+----------+--------+
To merge the files on just the "name" and "lod_score" fields and sort on the name:
$ tabmerge -f name,lod_score -s name merge*
+----------+-----------+
| name | lod_score |
+----------+-----------+
| ALTX | |
| DRTL | |
| DWRF | |
| RM104 | |
| RM105 | |
| TX5509 | |
| UU189 | 2.4 |
| Xpsm122 | 1.2 |
| Xpsr9556 | 1.2 |
+----------+-----------+
To do the same but mimic the original tab-delimited input:
$ tabmerge -f name,lod_score -s name --stdout merge*
name lod_score
ALTX
DRTL
DWRF
RM104
RM105
TX5509
UU189 2.4
Xpsm122 1.2
Xpsr9556 1.2
Why would you want to do this? Suppose you have several delimited text files with nearly the same structure and want to create just one
file from them, but the fields may be in a different order in each file and/or some files may contain more or fewer fields than others.
(As far-fetched as it may seem, it happens to the author more than he'd like.)
SEE ALSO
o Text::RecordParser
o Text::TabularDisplay
AUTHOR
Ken Youens-Clark <kclark@cpan.org>.
LICENSE AND COPYRIGHT
Copyright (C) 2006-10 Ken Youens-Clark. All rights reserved.
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
perl v5.10.1 2010-07-26 TABMERGE(1p)