Sponsored Content
Top Forums Shell Programming and Scripting Concatenate select lines from multiple files Post 302897340 by Scrutinizer on Sunday 13th of April 2014 11:25:42 AM
Old 04-13-2014
If there are too many files and you get : "Too many arguments", you could try:

Code:
for f in file*
do
   [ -f "$f" ] || continue
   cat "$f"
done | 

awk -F, '
  /header/ {
    n=c
    c=0
  } 
  !(++c in A) {
    A[c]=$0
    next
  }
  /Loc/ {
    A[c]=A[c]", "$0
  }
  $1~/^gt/ {
    A[c]=A[c] $2
  }
  END {
    for(i=1; i<=n; i++)print A[i]
  }
'

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find multiple patterns on multiple lines and concatenate output

I'm trying to parse COBOL code to combine variables into one string. I have two variable names that get literals moved into them and I'd like to use sed, awk, or similar to find these lines and combine the variables into the final component. These variable names are always VAR1 and VAR2. For... (8 Replies)
Discussion started by: wilg0005
8 Replies

2. Shell Programming and Scripting

Need script to select multiple files from archive directory based on the date range

hi all, here is the description to my problem. input parameters: $date1 & $date2 based on the range i need to select the archived files from the archived directory and moved them in to working directory. can u please help me in writing the code to select the multiple files based on the... (3 Replies)
Discussion started by: bbc17484
3 Replies

3. Shell Programming and Scripting

Concatenate multiple lines based.

Hello, I have been searching the forum for concatenation based on condition. I have been close enough but not got th exact one. infile: -----DB_Name ABC (X, Y,Z). DB_Name DEF (T). DB_Name GHI (U ,V,W). Desired Output file should be: ---------------------------DB_Name ABC... (8 Replies)
Discussion started by: indrajit_u
8 Replies

4. Shell Programming and Scripting

want to concatenate multiple files based on the rest of ls -lrt

uadm@4132> ls -lrt -rw------- 1 uadm uadm 3811819 Jun 6 04:08 data_log-2010.05.30-10:04:08.txt -rw------- 1 uadm uadm 716246 Jun 13 01:38 data_log-2010.06.06-10:04:08.txt -rw------- 1 uadm uadm 996 Jun 13 04:00 data_log-2010.06.06-10:04:22.txt -rw------- 1 uadm uadm 7471 Jun 20 02:03... (5 Replies)
Discussion started by: mail2sant
5 Replies

5. Shell Programming and Scripting

Concatenate columns from multiple files

Hi all, I want the 2nd column of every file in the directory in a single file with the file name as column header. $cat file1.txt a b c d e f $cat file2.txt f g h g h j $cat file3.txt a b d f g h (2 Replies)
Discussion started by: newbie83
2 Replies

6. Shell Programming and Scripting

[Solved] Script to concatenate 2 files with the same number of lines

Hi everyone, I have two files, namely: file1: file1Col1Row1;file1Col2Row1;file1Col3Row1 file1Col1Row2;file1Col2Row2;file1Col3Row2 file1Col1Row3;file1Col2Row3;file1Col3Row3file2: file2Col1Row1;file2Col2Row1;file2Col3Row1 file2Col1Row2;file2Col2Row2;file2Col3Row2... (0 Replies)
Discussion started by: gacanepa
0 Replies

7. Shell Programming and Scripting

Select multiple column from multiple files

Hi Friends, $ cat test1.txt emeka:1438 shelley:1439 dmeyer:1440 kurtarn:1441 abdul:1442 $ cat test2.txt 1:a 2:b 3:c 4:d $ cat test3.txt cat:dog:bat man:hot:cold (5 Replies)
Discussion started by: Jewel
5 Replies

8. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

9. UNIX for Beginners Questions & Answers

Concatenate column values when header is Matching from multiple files

there can be n number of columns but the number of columns and header name will remain same in all 3 files. Files are tab Delimited. a.txt Name 9/1 9/2 X 1 7 y 2 8 z 3 9 a 4 10 b 5 11 c 6 12 b.xt Name 9/1 9/2 X 13 19 y 14 20 z 15 21 a 16 22 b 17 23 c 18 24 c.txt Name 9/1 9/2... (14 Replies)
Discussion started by: Nina2910
14 Replies

10. Shell Programming and Scripting

Shell Scripting - Select multiple files from numbered list

I am trying to have the user select two files from a numbered list which will eventually be turned into a variable then combined. This is probably something simple and stupid that I am doing. clear echo "Please Select the Show interface status file" select FILE1 in *; echo "Please Select the... (3 Replies)
Discussion started by: dis0wned
3 Replies
COMM(1) 						    BSD General Commands Manual 						   COMM(1)

NAME
comm -- select or reject lines common to two files SYNOPSIS
comm [-123f] file1 file2 DESCRIPTION
The comm utility reads file1 and file2, which should be sorted lexically, and produces three text columns as output: lines only in file1; lines only in file2; and lines in both files. The filename ``-'' means the standard input. The following options are available: -1 Suppress printing of column 1. -2 Suppress printing of column 2. -3 Suppress printing of column 3. -f Fold case in line comparisons. Each column will have a number of tab characters prepended to it equal to the number of lower numbered columns that are being printed. For example, if column number two is being suppressed, lines printed in column number one will not have any tabs preceding them, and lines printed in column number three will have one. comm assumes that the files are lexically sorted; all characters participate in line comparisons. EXIT STATUS
comm exits 0 on success, >0 if an error occurred. SEE ALSO
cmp(1), diff(1), sort(1), uniq(1) STANDARDS
The comm utility conforms to IEEE Std 1003.2-1992 (``POSIX.2''). BSD
June 6, 1993 BSD
All times are GMT -4. The time now is 09:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy