Sponsored Content
Top Forums UNIX for Advanced & Expert Users Concatenation of multiple files based on file pattern Post 302979991 by Jesshelle David on Monday 22nd of August 2016 05:24:58 PM
Old 08-22-2016
Concatenation of multiple files based on file pattern

Hi,

I have the following reports that get generated every 1 hour and this is my requirement:

1. 5 reports get generated every hour with the names "Report.Dddmmyy.Thhmiss.CTLR"
"Report.Dddmmyy.Thhmiss.ACCD"
"Report.Dddmmyy.Thhmiss.BCCD"
"Report.Dddmmyy.Thhmiss.CCCD"
"Report.Dddmmyy.Thhmiss.DDDD"

2. At the end of the day I need to concatenate all reports with similar names into a single file.
For example: 24 files for pattern "Report.Dddmmyy.Thhmiss.CTLR" into a single file called "Test.dat" and so on.

3.Please note that the last four letters of the file name will be a constant and the reports have to be concatenated using this.

4. I do not want to parameterize just these last four letters or list the last four letter as ls -l *CTLR* as this will be hard coding so could there be a better way of handling this!?

Kindly advise.

Thanks,
Jess
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Concatenation of multiple files with wildcard

I have the following snippet to concatenate about a hundred csv-files: for file in *csv; do cat $file >> newfile; done This line works, but before I was experimenting with the following line, which is more intuitive and is a tad more robust: for file in *.csv; do cat $file >> newfile; done Can... (2 Replies)
Discussion started by: figaro
2 Replies

2. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

3. Shell Programming and Scripting

Splitting large file into multiple files in unix based on pattern

I need to write a shell script for below scenario My input file has data in format: qwerty0101TWE 12345 01022005 01022005 datainala alanfernanded 26 qwerty0101mXZ 12349 01022005 06022008 datainalb johngalilo 28 qwerty0101TWE 12342 01022005 07022009 datainalc hitalbert 43 qwerty0101CFG 12345... (19 Replies)
Discussion started by: jimmy12
19 Replies

4. UNIX for Dummies Questions & Answers

print multiple lines from text file based on pattern list

I have a text file with a list of items/patterns: ConsensusfromCGX_alldays_trimmedcollapsedfilteredreadscontiglist(229095contigs)contig12238 ConsensusfromCGX_alldays_trimmedcollapsedfilteredreadscontiglist(229095contigs)contig34624... (1 Reply)
Discussion started by: Oyster
1 Replies

5. Shell Programming and Scripting

split XML file into multiple files based on pattern

Hello, I am using awk to split a file into multiple files using command: nawk '{ if ( $1 == "<process" ) { n=split($2, arr, "\""); file=arr } print > file }' processes.xml <process name="Process1.process"> ... (3 Replies)
Discussion started by: chiru_h
3 Replies

6. UNIX for Advanced & Expert Users

Moving multiple files based on the pattern

I want to search for a particular file name patterns and move them to a specific folder, is it possible to do it with awk or sed? (1 Reply)
Discussion started by: rudoraj
1 Replies

7. Shell Programming and Scripting

Multiple File Rename based on pattern - one line

All, I wanted to copy the files From: Daily_XYZ_TEST_1.csv Daily_XYZ_TEST_2.csv Daily_XYZ_TEST_3.csv Daily_XYZ_TEST_4.csv To: Daily_ABC_TEST_1.csv Daily_ABC_TEST_2.csv Daily_ABC_TEST_3.csv Daily_ABC_TEST_4.csv I have tried the rename command but it is not working (5 Replies)
Discussion started by: alfredo123
5 Replies

8. Shell Programming and Scripting

Split a text file into multiple pages based on pattern

Hi, I have a text file (attached the sample). I have also, attached the way the way the files need to be split. We get this file, that will either have 24 Jurisdictions, or will miss some and retain some. Like in the attached sample file, there are only Jurisdictions 03,11,14,15, 20 and 30.... (3 Replies)
Discussion started by: ebsus
3 Replies

9. UNIX for Beginners Questions & Answers

Consternation of multiple file names based on naming pattern

Hi, I have the following reports that get generated every 1 hour and this is my requirement: 1. 5 reports get generated every hour with the names "Report.Dddmmyy.Thhmiss.CTLR" "Report.Dddmmyy.Thhmiss.ACCD" "Report.Dddmmyy.Thhmiss.BCCD" "Report.Dddmmyy.Thhmiss.CCCD"... (1 Reply)
Discussion started by: Jesshelle David
1 Replies

10. UNIX for Beginners Questions & Answers

UNIX script to append multiple text files into one file based on pattern present in filaname

Hi All-I am new to Unix , I need to write a script. Can someone help me with a requirement where I have list of files in a directory, I want to Merge the files if a pattern of string matches in filenames? AAAL_555A_ORANGE1_F190404.TXT AAAL_555A_ORANGE2_F190404.TXT AAAL_555A_ORANGE3_F190404.TXT... (6 Replies)
Discussion started by: Shankar455
6 Replies
analog(1)						      General Commands Manual							 analog(1)

NAME
analog - web server logfile analyser SYNOPSIS
analog --help analog [(-|+)1] [(-|+)4] [(-|+)5] [(-|+)6] [(-|+)7] [(-|+)A] [(-|+)a] [-B|+B[options]] [-b|+b[options]] [(-|+)C'configuration-line'] [-c|+c[options]] [(-|+)D] [(-|+)d] [-E|+E[options]] [-F|+Fdate] [-f|+f[options]] [-G] [+gfilename] [(-|+)H] [(-|+)h] [-I|+I[options]] [-i|+i[options]] [-J|+J[options]] [-j|+j[options]] [-K|+K[options]] [-k|+k[options]] [-L|+L[options]] [-l|+l[options]] [-M|+M[options]] [(-|+)m] [-N|+N[options]] [-n|+n[options]] [+Ooutput-file] [-o|+o[options]] [(-|+)P] [-p|+p[options]] [(-|+)Q] [-q|+q[[-|+]types]] [-R|+R[options]] [-r|+r[options]] [-S|+S[options]] [(-|+)S] [-T|+Tdate] [-t|+t[options]] [(-|+)Ucache] [-u|+u[options]] [-V|+V[[-|+]types]] [-v|+v[options]] [(-|+)W] [(-|+)w] [(-|+)X] [(-|+)x] [-Y|+Y[options]] [-y|+y[options]] [-Z|+Z[options]] [(-|+)z] [--settings] [logfile...] but normally just analog [-G] [+gfilename] DESCRIPTION
Analog is a program for analyzing log files, and is designed to be fast and to produce attractive statistics. The usual (and easier) way of configuring analog is through a configuration file, not command line options. For more information about writing a configuration file or about the command line options see Readme.html in the analog documentation. A description of what options are available for the non-time reports is available in othreps.html#othclarg in the analog documentation. OPTIONS
--help Display version and help information and exit. (-|+)1 Toggle the Yearly Report. (-|+)4 Toggle the Quarter-Hour Report. (-|+)5 Toggle the Five-Minute Report. (-|+)6 Toggle the Quarter-Hour Summary. (-|+)7 Toggle the Five-Minute Summary. (-|+)A Turn all the reports (except the General Summary) off or on. (-|+)a Toggle output style between HTML and PLAIN formats, respectively. -B|+B[options] Turn the Browser Report off or on, possibly setting options. -b|+b[options] Turn the Browser Summary off or on, possibly setting options. (-|+)C'configuration-line' Include configuration-line in the configuration commands. -c|+c[options] Turn the Status Code Report off or on, possibly setting options. (-|+)D Toggle the Daily Report. (-|+)d Toggle the Daily Summary. -E|+E[options] Turn the Redirection Report off or on, possibly setting options. -F|+Fdate Toggle or set the FROM date. Use -F to turn the FROM date off, and use +Fdate to set the FROM date. -f|+f[options] Turn the Referrer Report off or on, possibly setting options. -G Do not use the default configuration file. +gfilename Add filename to the list of configuration files. (-|+)H Turn the Hourly Report off or on, possibly setting options. (-|+)h Toggle the Hourly Summary. -I|+I[options] Turn the Failure Report off or on, possibly setting options. -i|+i[options] Turn the Directory Report off or on, possibly setting options. -J|+J[options] Turn the User Failure Report off or on, possibly setting options. -j|+j[options] Turn the User Redirection Report off or on, possibly setting options. -K|+K[options] Turn the Failed Referrer Report off or on, possibly setting options. -k|+k[options] Turn the Redirected Referrer Report off or on, possibly setting options. -L|+L[options] Turn the Host Failure Report off or on, possibly setting options. -l|+l[options] Turn the Host Redirection Report off or on, possibly setting options. -M|+M[options] Turn the Virtual Host Failure Report off or on, possibly setting options. (-|+)m Toggle the Monthly Report. -N|+N[options] Turn the Search Query Report off or on, possibly setting options. -n|+n[options] Turn the Search Word Report off or on, possibly setting options. (-|+)Ooutput-file Write output to output-file. Use - for stdout. -o|+o[options] Turn the Domain Report off or on, possibly setting options. (-|+)P Toggle the Processing Time Report. (-|+)p Toggle the Operating System Report. (-|+)Q Toggle the Quarterly Report. -q|+q[[-|+]types] Turn all warnings off/on, or remove/add/use exactly specified types of warnings. -R|+R[options] Turn the Virtual Host Redirection Report off or on, possibly setting options. -r|+r[options] Turn the Request Report off or on, possibly setting options. -S|+S[options] Turn the Host Report off or on, possibly setting options. -s|+s[options] Turn the Referring Site Report off or on, possibly setting options. -T|+Tdate Toggle or set the TO date. Use -T to turn the TO date off, and use +Tdate to set the TO date. -t|+t[options] Turn the File Type Report off or on, possibly setting options. (-|+)Ucache Use file cache for cache output. -u|+u[options] Turn the User Report off or on, possibly setting options. -V|+V[[-|+]types] Turn all debugging off/on, or remove/add/use exactly specified types of debugging messages. -v|+v[options] Turn the Virtual Host Report off or on, possibly setting options. (-|+)W Toggle the Weekly Report. (-|+)w Toggle the Hour of the Week Report. (-|+)X Toggle GOTOS (on and off only). (-|+)x Toggle the General Summary. -Y|+Y[options] Turn the Internal Search Query Report off or on, possibly setting options. -y|+y[options] Turn the Internal Search Word Report off or on, possibly setting options. -Z|+Z[options] Turn the Organization Report off or on, possibly setting options. (-|+)z Toggle the File Size Report. --settings Instead of processing any files, display in English the settings analog would use if it ran normally. This is useful for trou- bleshooting. logfile... The logfiles to be analysed. Use - for stdin. SEE ALSO
The analog home page at http://www.analog.cx/ The analog documentation on your local machine, or from the analog home page. BUGS
An up to date list of known bugs and bug fixes can be found at http://www.analog.cx/bugs.html AUTHORS
Analog and related documentation was written by Stephen Turner <analog-author@lists.meer.net>, with contributions from other authors. (See acknow.html in the analog documentation.) This man page was written by Alexander Toth <alex@purificator.net>. Analog Version 6.0 19 December 2004 analog(1)
All times are GMT -4. The time now is 05:16 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy