Concatenate all the files in folder on timestamps


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Concatenate all the files in folder on timestamps
# 8  
Old 08-10-2007
Quote:
Originally Posted by blowtorch
vgersh99, can you please explain the nawk? Didn't get it...
the objective is to get '20070809141159 Ack_to_MDS_20070809141159.xml' out of 'Ack_to_MDS_20070809141159.xml'; sort the file names; remove '20070809141159' (in our case) the prefix leaving the 'suffix' (the file name itself) - this will give us the sorted list of files based on the embedded timestamp.

What's the 'embedded timestamp' in the file name? The 'embedded timestamp' if the next to the last field in the '_' OR '.' filename.
'nawk' takes the next to the last field of all the wild-carded filenames and makes the timestamp from the filename a 'prefix' followed by the value of 'OFS' (space) followed by the $0 (filename).
we sort numerically '-n' - having the timestamp appearing as a prefix, make the sort work as expect.
once sorted, we remove the 'prefix' (timestamp) with the 'cut' command.
The end result is a list of file names sorted by the embedded timestamp.

And the rest is our editting/grepping body of the loop.
# 9  
Old 08-10-2007
Hi Guys

Thanks for all the pointers and help.
Got a simple and quick one liner answer.

Code:
 ls -1rt Ack_to_MDS_* | xargs cat > new_file.xml

Regards
Srikanth GR
# 10  
Old 08-10-2007
Quote:
Originally Posted by blowtorch
vgersh99, can you please explain the nawk? Didn't get it...
Quote:
Originally Posted by srikanthgr1
Hi Guys

Thanks for all the pointers and help.
Got a simple and quick one liner answer.

Code:
 ls -1rt Ack_to_MDS_* | xargs cat > new_file.xml

Regards
Srikanth GR
this assumes that your OS file timestamps are the same as the timestamps embedded in your file names - that was not your initial requirement.
# 11  
Old 08-10-2007
Code:
#!/bin/sh

awk 'BEGIN{i=0}
{   filename=FILENAME
    gsub("Ack_to_MDS_|.xml","",filename); 
    year=substr(filename,1,4)
    mth=substr(filename,5,2)
    day=substr(filename,7,2)
    hr=substr(filename,9,2)
    min=substr(filename,11,2)
    sec=substr(filename,13,2)
    t = mktime(year" "mth" "day" "hr" "min" "sec )
    array[t]=FILENAME
    nextfile
}
END{
    n=asort(array)
    print "<ACKNOWLEDGEMENTS>" > "newfile"
    for(i=1;i<=n;i++) {
	file=array[i]
	while(  (getline line < file ) >0 ) {
	    if ( line ~ /<ACKNOWLEDGEMENTS>/ || line ~ /<\/ACKNOWLEDGEMENTS>/ ) {
		continue
	    }
	    else {
	        print line > "newfile"
	    }	
	}
		
    }
    print "</ACKNOWLEDGEMENTS>" > "newfile"
}' Ack_to_MDS_*.xml

# 12  
Old 08-10-2007
ghostdog74,
this is GNU gawk specific - no need for 'mktime' - you already have a 'sortable' timestamp presentation/format.

Last edited by vgersh99; 08-10-2007 at 05:09 AM..
# 13  
Old 08-10-2007
Quote:
Originally Posted by vgersh99
ghostdog74,
this is GNU gawk specific.
my bad. i didn't read the posts properly. thanks
Anyway, the requirement doesn't seem right. The one liner doesn't solve the part where he doesn't need the <ACKNOWLEDGEMENTS> and </ACKNOWLEDGEMENTS> (or does it?)
# 14  
Old 08-10-2007
the OP seems to be gone: no OP - no problem! Smilie

BTW, no need for 'mktime' - you already have a 'sortable' timestamp presentation/format.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Request for Shell script to move files from Subfolder to Parent folder and delete sub folder

Hi Team, I am new to shell script and there is a requirement where files should be moved from Subfolder to parent folder. Eg: parent folder --> /Interface/data/test/IN Sub folder -->/Interface/data/test/IN/Invoice20180607233338 Subfolder will be always with timestamp... (6 Replies)
Discussion started by: srivarun15
6 Replies

2. Shell Programming and Scripting

Shell scripting for moving folder specific files into target directory of that country folder.

I need help to write shell script to copy files from one server to another server. Source Directory UAE(inside i have another folder Misc with files inside UAE folder).I have to copy this to another server UAE folder( Files should be copied to UAE folder and Misc files should be copied in target... (3 Replies)
Discussion started by: naresh2389
3 Replies

3. Shell Programming and Scripting

Find matching timestamps in two files.

OK. if the first file is : 3184 2014-07-28 04:15 global.Remote-Access 10.111.8.25 81.245.6.25 tcp 3268 3035 2014-07-28 04:16 global.Remote-Access 10.111.8.12 81.245.6.25 tcp 3268If the second file is: 1 Jul 28 04:12 2014-07-28 id967254(BGC-TLW-Cert) Tunneling: User with IP... (8 Replies)
Discussion started by: fxsme
8 Replies

4. Shell Programming and Scripting

Issues with comparing 2 files timestamps

Hi, Im trying to write a script to first check a directory and if the filename has "ACK" in it do nothing and exit but if it has "ORD" in the filename then compare it with a dummy file created 2 minutes previous and see which one is newer Im getting a few errors which im unsure how to rectofy... (5 Replies)
Discussion started by: 02JayJay02
5 Replies

5. Shell Programming and Scripting

Sorting the files on their timestamps

Hi, The file names are like XXXX_20111101.gz for 01 November 2011, etc. There might be several files for 01 November 2011 and I can find them using find DIR -name *_20111101.gz -exec ls -lt {} \; But I want to sort the listing returned by the above command on the timestamps of the files... (6 Replies)
Discussion started by: sagarparadkar
6 Replies

6. Shell Programming and Scripting

Bash script to copy timestamps of multiple files

Hi, I have a bunch of media files in a directory that have been converted (from MTS to MOV format), so my directory contains something like this: clip1.mts clip1.mov clip2.mts clip2.mov The problem is that the .mov files that have been created have the timestamps of the conversion task,... (2 Replies)
Discussion started by: Krakus
2 Replies

7. UNIX for Advanced & Expert Users

Recursively concatenate files in subdirectories with the same folder name

I'm trying to concatenate files in subdirectories with the same folder name. Say concatenate all the files in the 'current' subdirectories in 'Literature' parent directory. Literature/USA/current/ Literature/Europe/current/ Can anyone help with it? Thanks a lot! (2 Replies)
Discussion started by: joyce007
2 Replies

8. UNIX for Dummies Questions & Answers

recursively concatenate files in subdirectories with same folder name

I'm trying to concatenate files in subdirectories with the same folder name. Say concatenate all the files in the 'current' subdirectories in 'Literature' parent directory. Literature/USA/current/ Literature/Europe/current/ Can anyone help with it? Thanks a lot! (2 Replies)
Discussion started by: joyce007
2 Replies

9. Shell Programming and Scripting

Getting all the .gz files between Two Date/TimeStamps

Hi, I am trying to write a script to ftp and get all the files between two date/time stamps from a archive directory. I have sent an attatchment of my archive directory. With the script I intend to get files for ex: between request.log.2008-08-22-03-53-49.gz &... (3 Replies)
Discussion started by: openspark
3 Replies

10. UNIX for Advanced & Expert Users

Auto copy for files from folder to folder upon instant writing

Hello all, I'm trying to accomplish that if a file gets written to folder /path/to/a/ it gets automatically copied into /path/to/b/ the moment its get written. I thought of writing a shell script and cron it that every X amount of minutes it copies these files over but this will not help me... (2 Replies)
Discussion started by: Bashar
2 Replies
Login or Register to Ask a Question