Sponsored Content
Top Forums Shell Programming and Scripting Consolidate files based on priority Post 302886118 by Chubler_XL on Wednesday 29th of January 2014 05:48:09 PM
Old 01-29-2014
Another awk solution (keeping order same as it appears in input files)

Code:
awk '!($1 in F) {F[$1]; R[++l]=$0 OFS FILENAME} END{for(i=1;i<=l;i++) print R[i]}' file*

Edit: I think bartus11's solution can be simplified to:
Code:
awk '!a[$1]{a[$1]=$0" "FILENAME}END{for (i in a) print a[i]}' file1 file2 file3 file4


Last edited by Chubler_XL; 01-29-2014 at 06:54 PM..
 

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

consolidate file in unix

hi, i am trying to consolodate the files in the unix with the '>>' i have some 50 or 60 files.is there any another way of consolidating the alll 50 or 60 files in to one file. actually the way i m doing creaating the problem while loading the file with teradat tpump and fasload. so if there is... (3 Replies)
Discussion started by: narang.mohit
3 Replies

2. Shell Programming and Scripting

How to consolidate values in one column from different rows into one?

Hi Guys, Thank you all for helping me with my different queries and I continue to get better at scripting because of help from all of you! I have a file that would look something like - ID SUB ID VALUE 1 10 5 2 18 7 1 ... (1 Reply)
Discussion started by: sncoupons
1 Replies

3. Shell Programming and Scripting

Script to consolidate files as mails,groupid

Hi, I'm newbie in Bash scripting. Here is scenario I've a file which containslist of 100 mailgroup and in a directory i have 100 files with each group's respective mail addresses. I'd like to make one CSV file with this information as follows. consolidated files with mailid,groupname. ... (7 Replies)
Discussion started by: manish123456
7 Replies

4. Shell Programming and Scripting

Multiple lines consolidate

This post is start for me ... I stumped at something that I not sure as to how start on ... I tried so of your script that i honestly lost mind looking and looking here ... please help COL1 COl2 COL3 12222 AUH FLUEH 12222 SSC OPERA 12222 SSC ... (8 Replies)
Discussion started by: Sebastian.Thoma
8 Replies

5. Shell Programming and Scripting

How to copy files from one location to another based on a priority?

Hi Gurus, I am a newbie to shell scripting and I am facing a problem right now.I have to automate the copy of files based on a priority.The scenario is as below: 1) There will be files from Mon-Fri with Mon file being named as abc_def_01_YYYYMMDD and Tue file being abc_def_02_YYYYMMDD and so... (4 Replies)
Discussion started by: vikramgk9
4 Replies

6. Shell Programming and Scripting

Need help for automated pickup of file based on a priority

Hi experts, I am facing a problem right now.I have to automate the pickup of files based on a priority.The scenario is as below: 1) There will be files from Mon-Fri with Mon file being named as abc_def_01_YYYYMMDD and Tue file being abc_def_02_YYYYMMDD and so forth till Friday's file which... (1 Reply)
Discussion started by: vikramgk9
1 Replies

7. UNIX for Beginners Questions & Answers

Providing virtual machine priority in kvm based virtual machines

Hi All, Is there any way I can prioritize my VMs when there is resource crunch in host machine so that some VMs will be allocated more vcpu, more memory than other VMs in kvm/qemu hypervisor based virtual machines? Lets say in my cloud environment my Ubuntu 16 compute hosts are running some... (0 Replies)
Discussion started by: SanjayK
0 Replies
TABMERGE(1p)						User Contributed Perl Documentation					      TABMERGE(1p)

NAME
tabmerge - unify delimited files on common fields SYNOPSIS
tabmerge [action] [options] file1 file2 [...] Actions: --min Take only fields present in all files [DEFAULT] --max Take all fields present -f|--fields=f1[,f2] Take only the fields mentioned in the comma-separated list Options: -l|--list List available fields --fs=x Use "x" as the field separator (default is tab " ") --rs=x Use "x" as the record separator (default is newline " ") -s|--sort=f1[,f2] Sort data ASCII-betically on field(s) --stdout Print data in original delimited format (i.e., not in a table format) --help Show brief help and quit --man Show full documentation DESCRIPTION
This program merges the fields -- not the rows -- of delimited text files. That is, if several files are almost but not quite entirely unlike each other in their structure (in their field names, numbers or orders), this script allows you to easily unify the files into one file with all the same fields. The output can be based on fields as determined by the three "action" flags. For the following examples, consider three files that contain the following fields: +------------+---------------------------------+ | File | Fields | +------------+---------------------------------+ | merge1.tab | name, type, position | | merge2.tab | name, type, position, lod_score | | merge3.tab | name, position | +------------+---------------------------------+ To list all available fields in the files and the number of times they are present: $ tabmerge --list merge* +-----------+-------------------+ | Field | No. Times Present | +-----------+-------------------+ | lod_score | 1 | | name | 3 | | position | 3 | | type | 2 | +-----------+-------------------+ To merge the files on the minimum overlapping fields: $ tabmerge merge* +----------+----------+ | name | position | +----------+----------+ | RM104 | 2.30 | | RM105 | 4.5 | | TX5509 | 10.4 | | UU189 | 19.0 | | Xpsm122 | 3.3 | | Xpsr9556 | 4.5 | | DRTL | 2.30 | | ALTX | 4.5 | | DWRF | 10.4 | +----------+----------+ To merge the files and include all the fields: $ tabmerge --max merge* +-----------+----------+----------+--------+ | lod_score | name | position | type | +-----------+----------+----------+--------+ | | RM104 | 2.30 | RFLP | | | RM105 | 4.5 | RFLP | | | TX5509 | 10.4 | AFLP | | 2.4 | UU189 | 19.0 | SSR | | 1.2 | Xpsm122 | 3.3 | Marker | | 1.2 | Xpsr9556 | 4.5 | Marker | | | DRTL | 2.30 | | | | ALTX | 4.5 | | | | DWRF | 10.4 | | +-----------+----------+----------+--------+ To merge and extract just the "name" and "type" fields: $ tabmerge -f name,type merge* +----------+--------+ | name | type | +----------+--------+ | RM104 | RFLP | | RM105 | RFLP | | TX5509 | AFLP | | UU189 | SSR | | Xpsm122 | Marker | | Xpsr9556 | Marker | | DRTL | | | ALTX | | | DWRF | | +----------+--------+ To merge the files on just the "name" and "lod_score" fields and sort on the name: $ tabmerge -f name,lod_score -s name merge* +----------+-----------+ | name | lod_score | +----------+-----------+ | ALTX | | | DRTL | | | DWRF | | | RM104 | | | RM105 | | | TX5509 | | | UU189 | 2.4 | | Xpsm122 | 1.2 | | Xpsr9556 | 1.2 | +----------+-----------+ To do the same but mimic the original tab-delimited input: $ tabmerge -f name,lod_score -s name --stdout merge* name lod_score ALTX DRTL DWRF RM104 RM105 TX5509 UU189 2.4 Xpsm122 1.2 Xpsr9556 1.2 Why would you want to do this? Suppose you have several delimited text files with nearly the same structure and want to create just one file from them, but the fields may be in a different order in each file and/or some files may contain more or fewer fields than others. (As far-fetched as it may seem, it happens to the author more than he'd like.) SEE ALSO
o Text::RecordParser o Text::TabularDisplay AUTHOR
Ken Youens-Clark <kclark@cpan.org>. LICENSE AND COPYRIGHT
Copyright (C) 2006-10 Ken Youens-Clark. All rights reserved. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. perl v5.10.1 2010-07-26 TABMERGE(1p)
All times are GMT -4. The time now is 05:57 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy