Checking in a directory how many files are present and basing on that merge all the files
Hi,
My requirement is,there is a directory location like:
:camp/current/
In this location there can be different flat files that are generated in a single day with same header and the data will be different, differentiated by timestamp, so i need to verify how many files are generated in a single day and merge them all into a single file.
Pls be more specific, post an example of your constellation, where to find the header, and rules how to merge the files (e.g. specially ordered, by contents, no order).
in which there is a chance of generating a two or three flat files per day,
they will be with name as flatfile1_v_timestamp,flatfile2_v_timestamp.
I need to check for this pattern in this location for these files as my job runs once in a day i need to pick all these flat files combine them into a single file, header will be same for all these files.
Note: records should come in ordered way and the values should accurately go into particular columns, even if we merge the files into a single file, field separator is '|'
Example:
File1
File2
mergefile:
Header:
Header will be the same but values may come or may not come for some of the columns.
Please let me know if anything needed on this
Moderator's Comments:
Please use code tags next time for your code and data.
Last edited by Franklin52; 12-27-2012 at 08:57 AM..
Pls use code tags as advised.
So - the headers are identical across all files, and all except first should be removed. Data should be ordered according to their files' creation time sequence. Try this:
Remove the -r option to ls if you want newest files first.
Pls note: the field separator | that you mention I can see only in the headers given, not in the data lines.
when i use this command for merging the flat files into a single file header is coming two times in a single file, but i wanted it only one what i need to do here....
Example:
Flat file1:
Flatfile2:
Mergedfile:
In the merged file as well particular column value should go into particular column.
Please reply to me at the earliest..
Last edited by Franklin52; 01-02-2013 at 03:24 AM..
Reason: Please use code tags for data and code samples
Hello Folks,
Looking for a script which can keep doing ls to the directories and once file landed to the directory then ,read the files do further calculation and exit.
Kindly guide.
Regards,
Sadique (3 Replies)
Hello,
I have 40 data files where the first three columns are the same (in theory) and the 4th column is different. Here is an example of three files,
file 2: A_f0_r179_pred.txt
Id Group Name E0
1 V N(,)'1 0.2904
2 V N(,)'2 0.3180
3 V N(,)'3 0.3277
4 V N(,)'4 0.3675
5 V N(,)'5 0.3456
... (8 Replies)
Hey Guys,
I want to merge all files (Apache Tomcat Access Logs) for a particular date say "Aug 24" to be merged into a single file.
Is there any quick hack for that ?
$ ls -alrth access_log2016-08-*|grep "Aug 24"
-rw-rw-r--. 1 tomcat tomcat 16M Aug 24 00:00... (8 Replies)
Hi frnds,
My requirement is I have a zip file with name say eg: test_ABC_UH_ccde2a_awdeaea_20150422.zip
within that there are subdirectories on each directory we again have .zip files and in that we have files like mama20150422.gz and so on.
Iam in need of a bash script so that it unzips... (0 Replies)
Hi
I am unable to find files, those are present anywhere in the same directory tree, based on the creation date. I need to find the files with their path, as I need to create them in another location and move them. I need some help with a script that may do the job.
Please help (2 Replies)
Hi all,
I am looking for a script which renames all the files from the present directory.
Eg.:
In unix directory contains the below files
linux001.txt
linux002.txt
linux003.txt
......
.......
Now the files should be renamed to
unix001.txt
unix002.txt
unix003.txt
Could anyone... (8 Replies)
# ls -l
total 10
-rw-r--r-- 1 dummy2 other 140 Jun 19 21:37 local.cshrc
-rw-r--r-- 1 dummy2 other 136 Jun 19 21:37 local.cshrc~
-rw-r--r-- 1 dummy2 other 157 Jun 19 21:37 local.login
-rw-r--r-- 1 dummy2 other 178 Jun 19 21:37 local.profile... (6 Replies)
I had mount from server A to server B. I am able to access the files present under server B. I can create new files on server B, but i am not able to edit the files which are already present. When i saw the permissions on those files it is 777.
can some one tell me why i am not able to edit... (2 Replies)
Hi Gurus,
I am new to unix and need your help to make a shell script. I have a requirement, would appreciate if you could please help me on it:
Requirement:
-------------
I will pass 2 parameters in shell script 1). Directory name say errors 2). file extension say .log
First of all this... (4 Replies)