Concatenation of files with same naming patterns dynamically Post: 302980029

Sponsored Content

Top Forums Shell Programming and Scripting Concatenation of files with same naming patterns dynamically Post 302980029 by Don Cragun on Tuesday 23rd of August 2016 03:47:26 AM

08-23-2016

Registered User

Quote:

Originally Posted by Jesshelle David

Since my last threads were closed on account of spamming, keeping just this one opened!

Hi,

I have the following reports that get generated every 1 hour and this is my requirement:

1. 5 reports get generated every hour with the names

Code:

"Report.Dddmmyy.Thhmiss.CTLR"
"Report.Dddmmyy.Thhmiss.ACCD"
"Report.Dddmmyy.Thhmiss.BCCD"
"Report.Dddmmyy.Thhmiss.CCCD"
"Report.Dddmmyy.Thhmiss.DDDD"

2. At the end of the day I need to concatenate all reports with similar names into a single file.
For example: 24 files for pattern "Report.Dddmmyy.Thhmiss.CTLR" into a single file called "Test.dat" and so on.

3.Please note that the last four letters of the file name will be a constant and the reports have to be concatenated using this.

4. I do not want to parameterize just these last four letters or list the last four letter as ls -l *CTLR* as this will be hard coding so could there be a better way of handling this!?

Kindly advise.

Thanks,
Jess

I don't understand what you're trying to do. I get that at sometime shortly before midnight on August 23, 2016 you will have 24 files matching the pattern Report.D230816.T??????.CTLR, 24 files matching the pattern Report.D230816.T??????.ACCD, 24 files matching the pattern Report.D230816.T??????.BCCD, 24 files matching the pattern Report.D230816.T??????.CCCD, and 24 files matching the pattern Report.D230816.T??????.DDDD.

And I get that you want to copy the contents of the 24 files matching the pattern Report.D230816.T??????.CTLR into a file named Test.dat. I don't know what "and so on" means? Are you saying that you want to overwrite that file four times so at that at midnight on the morning of August 24, 2016, the file Test.dat will contain the contents of the 24 files matching the pattern Report.D230816.T??????.DDDD? This doesn't make any sense to me, but that seems to be what you are requesting.

And, you say that the script you are writing can't have the strings CTLR, ACCD, BCCD, CCCD, and DDDD built in. Is this because some days may have additional filename extensions? Is it because the set of filename extensions changes from day to day? Are all of the extensions you want to process four characters long? If not, how do we determine which files are supposed to be processed? (And, if the *.CTLR files are supposed to be processed before the *.ACCD files (i.e., not in alphabetic order), if the extensions aren't built into the script, how is the script supposed to know which extension should be processed first?

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Faster Concatenation of files

Dear All I am using cat command for concatenating multiple files. Some time i also use append command when there are few files. Is there faster way of concatenating multiple files(60 to 70 files) each of 156 MB or less/more.:) Thanx

2. Shell Programming and Scripting

rowwise concatenation of files

Hi , I am having a file 1n.txt - cat 1n.txt gives get_next_sp_fixing_mod_ref_no, get_next_sp_tran_ref_no, cat 2n.txt - boxer1.cpp boxer2.cpp I want a file resn.txt which has - get_next_sp_fixing_mod_ref_no,boxer1.cpp get_next_sp_tran_ref_no,boxer2.cpp How can i do that ? Its...

3. UNIX for Dummies Questions & Answers

naming files that csplit creates

Hi, This is my first time on this forum.. I searched the previous answers, but didn't find the answer I was looking for at first glance. csplit works beautifully for me, except for one thing. My file looks like this: ad|name1|asdf...(several pages)..asdf ... ad|name2|asdf...(several...

4. Shell Programming and Scripting

Removing Header from files before Concatenation

I need to concatenate the files of same type but with different names that conatin header. Before conactenating its mandatory to remove the header from the files .. and after concatenation the output file should contain header too. how to do it... thanks in advance.

5. UNIX for Dummies Questions & Answers

naming files in awk

I have a whole directory and I need each lines of each file to be separated to a new file but I am facing problem naming them :( some of the files even might be empty the output files should be names original file name + the number of the line or any incremental number FILES="data/*" for X in...

6. Shell Programming and Scripting

Concatenation of two patterns

Read the lines from the file and concatenate the lines into single line then search the string "INPUT=" from the another file and if "INPUT" is present in that file assign the concatenate patterns into "INPUT=" pattern. example: file1.txt <some lines> INPUT = <some lines> file2.txt...

7. UNIX Desktop Questions & Answers

Combining files with specific patterns of naming in a directory

Greetings Unix exports, I am facing some problems in combining files with different name patterns with a directory and I would appreciate if you can help me I have more than 1000 files but they follow a specific pattern of naming. e.g. 64Xtest01.txt They are divided into two sets of test and...

8. Shell Programming and Scripting

Issue in Concatenation/Joining of lines in a dynamically generated file

Hi, I have a file containing many records delimited by pipe (|). Each record should contain 17 columnns/fields. there are some fields having fields less than 17.So i am extracting those records to a file using the below command awk 'BEGIN {FS="|"} NF !=17 {print}' feedfile.txt...

9. UNIX and Linux Applications

Concatenation of different reports dynamically

Hi, I have the following reports that get generated every 1 hour and this is my requirement: 1. 5 reports get generated every hour with the names "Report.Dddmmyy.Thhmiss.CTLR" "Report.Dddmmyy.Thhmiss.ACCD" "Report.Dddmmyy.Thhmiss.BCCD" "Report.Dddmmyy.Thhmiss.CCCD"...

10. Shell Programming and Scripting

Bash - Find files excluding file patterns and subfolder patterns

Hello. For a given folder, I want to select any files find $PATH1 -f $ -name "*" but omit any files like pattern name ! -iname "*.jpg" ! -iname "*.xsession*" ..... $ and also omit any subfolder like pattern name -type d \( -name "/etc/gconf/gconf.*" -o -name "*cache*" -o -name "*Cache*" -o...