Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

concatenating selected lines of multiple files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting concatenating selected lines of multiple files
# 1  
Old 10-28-2009
concatenating selected lines of multiple files
Hi,

I would like a shell script that reads all files in a directory and concatenate them. It is not a simple concatenation. The first few lines of the files should not be included. The lines to be included are the lines from where 'START HERE' appears up to the end of the file. For example, I have 3 files:

File 1:
Code:
    abc
    def
    START HERE
    line1
    line2

File 2:
Code:
    xyz
    efg
    START HERE
    line111
    line222

File 3:
Code:
    jkl
    mno
    START HERE
    line11
    line22

The resulting file should be:

Code:
    START HERE
    line1
    line2
    START HERE
    line111
    line222
    START HERE
    line11
    line22

Thanks in advance...
Last edited by Franklin52; 10-28-2009 at 11:11 AM.. Reason: Please use code tags!
# 2  
Old 10-28-2009
Try this:

Code:
awk 'FNR==1{f=0}/START HERE/{f=1}f' file1 file2 file3

# 3  
Old 10-29-2009
concatenating selected lines of multiple files
Thanks Franklin, but the number of files and the filenames may vary. It should be dynamic that it will process all files in the directory... Please modify..
# 4  
Old 10-29-2009
Quote:
Originally Posted by laiko
Thanks Franklin, but the number of files and the filenames may vary. It should be dynamic that it will process all files in the directory... Please modify..
It's up to you to make it more dynamic...
For example:
Code:
awk 'FNR==1{f=0}/START HERE/{f=1}f' *

* will expand to all files in the current directory
# 5  
Old 10-30-2009
concatenating selected lines of multiple files
Thanks much! That works! I got a new requirement today in addition to that. I have to attach a header to it. I have a file header whose contents (which may vary) I need to prefix to the concatenation. For example:

FileHeader:

Code:
FILES AS OF TODAY:
HEADER LINE1
HEADER LINE2

And the result should be:

Code:
FILES AS OF TODAY:
HEADER LINE1
HEADER LINE2
    START HERE
    line1
    line2
    START HERE
    line111
    line222
    START HERE
    line11
    line22

I can only think of doing this:

Code:
awk 'FNR==1{f=0}/START HERE/{f=1}f' * > concat.txt
cat concat.txt >> FileHeader

Any better way to do this????
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

2. UNIX for Dummies Questions & Answers

Read in Multiple log files and output selected variables and values to cvs file

I have several problems with my problems: I hope you can help me. 1) the If else statement I am getting an error message. My syntax must be incorrect because the entire statement is throwing an error. For example in filew.log if these items don't exist Memsize, SASFoundation and also if... (0 Replies)
Discussion started by: dellanicholson
0 Replies

3. Shell Programming and Scripting

Concatenating 2 lines from 2 files having matching strings

Hello All Unix Users, I am still new to Unix, however I am eager to learn it.. I have 2 files, some lines have some matching substrings, I would like to concatenate these lines into one lines, leaving other untouched. Here below is an example for that.. File 1 (fasta file): >292183... (6 Replies)
Discussion started by: Mohamed EL Hadi
6 Replies

4. Shell Programming and Scripting

Error in concatenating multiple files from subdirectories

I want to concatenate multiple files recursively from sub-directories intoone file in Linux. I saved the following script as script.sh in $HOME/testing1 where I have several subdirectories and .txt files into them. I ran script.sh from the command prompt of $HOME/testing1 as ./script.sh. But it... (3 Replies)
Discussion started by: raj284
3 Replies

5. Shell Programming and Scripting

Problem in concatenating multiple files from subdirectories

I want to concatenate multiple files recursively from sub-directories intoone file in Linux. I saved the following script as script.sh in $HOME/testing1 where I have several subdirectories and .txt files into them. I ran script.sh from the command prompt of $HOME/testing1 as ./script.sh. But it... (3 Replies)
Discussion started by: raj284
3 Replies

6. Shell Programming and Scripting

trying to print selected fields of selected lines by AWK

I am trying to print 1st, 2nd, 13th and 14th fields of a file of line numbers from 29 to 10029. I dont know how to put this in one code. Currently I am removing the selected lines by awk 'NR==29,NR==10029' File1 > File2 and then doing awk '{print $1, $2, $13, $14}' File2 > File3 Can... (3 Replies)
Discussion started by: ananyob
3 Replies

7. Shell Programming and Scripting

awk: switching lines and concatenating lines?

Hello, I have only recently begun with awk and need to write this: I have an input consisting of a couple of letters, a space and a number followed by various other characters: fiRcQ 9( ) klsRo 9( ) pause fiRcQ 9( ) pause klsRo continue 1 aPLnJ 62( ) fiRcQ continue 5 ... and so on I... (7 Replies)
Discussion started by: Borghal
7 Replies

8. Shell Programming and Scripting

Concatenating lines of separate files using awk or sed

For example: File 1: abc def ghi jkl mno pqr File 2: stu vwx yza bcd efg hij klm nop qrs I want the reult to be: abc def ghistu vwx yza jkl mno pqrbcd efg hij klm nop qrs (4 Replies)
Discussion started by: tamahomekarasu
4 Replies

9. Shell Programming and Scripting

Running a command on multiple selected files in nautilus script

I am trying to make a script to convert drg files to wav and so far i have this #!/bin/bash drg2sbg "$*" -o "$*".sbg sbagen -Wo "/home/nick/Desktop/I-Doser Wave Files/"$*"" "$*".sbg rm "$*".sbg cd "/home/nick/Desktop/I-Doser Wave Files" rename 's/\.drg$/\.wav/' *.drg exit the drg2sbg and... (2 Replies)
Discussion started by: Nickbowlingdude
2 Replies

10. Shell Programming and Scripting

Concatenating multiple lines to one line if match pattern

Hi all, I've been working on a script which I have hit a road block now. I have written a script using sed to extract the below data and pumped into another file: Severity............: MAJORWARNING Summary: System temperature is out of normal range. Severity............: MAJORWARNING... (13 Replies)
Discussion started by: phixsius
13 Replies
Login or Register to Ask a Question