Sort log files based on numeric value in the filename
Hi,
I have a list of log files as follows:
I need to generate numbers from 1 to 30 and match them with their equivalent extracted from the filename (if duplicates are found, than multiply the rows with the numbers of duplicates) and then search for a pattern inside the log file.
With help from internet I managed to complete each operation, but I do not know to how to merge them in a single command.
Generate numbers from 1 to 30 using :
Now I need to match
Having those numbers generated first will help me to spot if a file is missing (like name_date_0003_ID0.log)
To extract the numeric value and see how many duplicates are I have used this command:
with the following results:
The last command that I'm using it is to search for a pattern in all the log files, print the 2nd line after matching, and count number of “%”
The final result should look like:
Also if possible I would like to check for the length of ID0, ID1 .... IDn and if less then 3 characters then display a warning after “number of “%” signs”
Please help me merging all the codes to achieve the above results.
Alex
Last edited by Scrutinizer; 12-22-2012 at 02:28 AM..
Reason: cleaned out all tags, reintroduced code tags...
Hi, I have a list of log files as follows: I need to generate numbers from 1 to 30 and match them with their equivalent extracted from the filename (if duplicates are found, than multiply the rows with the numbers of duplicates) and then search for a pattern inside the log file. With help from internet I managed to complete each operation, but I do not know to how to merge them in a single command. Generate numbers from 1 to 30 using :
Now I need to match
Having those numbers generated first will help me to spot if a file is missing (like name_date_0003_ID0.log)
To extract the numeric value and see how many duplicates are I have used this command:
with the following results:
The last command that I'm using it is to search for a pattern in all the log files, print the 2nd line after matching, and count number of “%”
The final result should look like:
Also if possible I would like to check for the length of ID0, ID1 .... IDn and if less then 3 characters then display a warning after “number of “%” signs”
Please help me merging all the codes to achieve the above results. Alex
I'm not sure what you want to get out of this.
You show a printf format string of "%06d\n", but all of the data you show in your examples show a zero-filled four decimal digit string; not six decimal digits. Do you want four digits or six digits?
The last printf format that you show:
does not match the output you say you want to see:
because the leading four decimal digit sequence and the following " - " are not in the printf format string. Do you want the output to match the format string you gave, or do you want the output to match the example shown?
You code has an underlying assumption that the pattern you're looking for will appear on the same line in every file. (The assumption is hidden by the fact that the array c is not cleared when you start reading a new file.) Do you always want to display a specific line number in each file, or do you want to display the 2nd line after a line that matches a given pattern?
If the pattern appears more than one line in a file, do you want the second line after each match and the count of %s on that line to be printed for each match, or only for the first match in each file?
Will you guarantee that each filename matched by *.log contains exactly three underscore characters? Is the error message that you want to be printed supposed to be a check that the characters between the 3rd underscore and the .log at the end of the filename is ID followed by a single decimal digit, or is it intended to verify that there are exactly three underscores, exactly four (or six) decimal digits between the 2nd and 3rd underscore and ID followed by a single decimal digit followed by .log after the 3rd underscore?
Please give us a sample of the start of the content of one of your log files (in CODE tags) and show us the pattern you want to match.
Thank for your reply, Sorry for the
, you are right should be
The last printf was designed to output the following (before thinking to add the numbers from 1 to 30 and match them with the filename, since here the lack of “-“ signs):
If possible I would like the output to match the example shown. Displayig the 2nd line after matching a pattern would be great. Never happened so far to find more than one pattern in a file, and if does I would like to get the last match. Yes each log file has 3 underscore characters. I would the error message to count the number of characters in “ID0”, because the length can very sometimes. For example if I get ID005 I need the script to print the warning message, but not if I have ID5. The pattern that I want to match is:
If I understood your requirements correctly, I believe the following script does everything you want. I tried it with a bunch of log files containing zero to four lines matching your pattern, with filenames matching and not matching your naming conventions. For files that have more than one line matching your pattern, it reports the contents of the line two lines after the last line matching your pattern:
On Solaris systems, use /usr/xpg4/bin/awk or nawk instead of awk.
Last edited by Don Cragun; 12-23-2012 at 03:22 AM..
Reason: Warn about which awk to use on Solaris systems.
This User Gave Thanks to Don Cragun For This Post:
Input file:
100%ABC2 3.44E-12 USA
A2M%H02579 0E0 UK
100%ABC2 5.34E-8 UK
100%ABC2 3.25E-12 USA
A2M%H02579 5E-45 UK
Output file:
100%ABC2 3.44E-12 USA
100%ABC2 3.25E-12 USA
100%ABC2 5.34E-8 UK
A2M%H02579 0E0 UK
A2M%H02579 5E-45 UK
Code try:
sort -k1,1 -g -k2 -r input.txt... (2 Replies)
Hey guys,
Sorry for the basic question but I have a lot of files that I want to separate into groups based on filenames which I can then cat together. Eg I have:
(a_b_c.txt)
WB34_2_SLA8.txt
WB34_1_SLA8.txt
WB34_1_DB10.txt
WB34_2_DB10.txt
WB34_1_SLA8.txt
WB34_2_SLA8.txt
77_1_SLA8.txt... (1 Reply)
Hi all.
I am very new to linux scripting and i have a task i can only solve with a script.
I need to sort files base on the date string in their filenames and create a folder using the same date string then move the files to their respective folders.
Scenario:
Folder Path:... (1 Reply)
Hi all.
I am very new to linux scripting and i have a task i can only solve with a script.
I need to sort files base on the date string in their filenames and create a folder using the same date string then move the files to their respective folders.
Scenario:
Folder Path:... (1 Reply)
Hi, I am a newbie to shell programming and I need some help in sorting a list of files in ascending order of date in the filenames.
The file format is always : IGL01_AC_D_<YYYYMMDD>_N01_01
For example, in a directory MyDirectory I have the following files:
IGL01_AC_D_20110712_N01_01.dat... (11 Replies)
Hi
I have a requirement like below
I need to sort the files based on the timestamp in the file name and run them in sorted order and then archive all the files which are one day old to temp directory
My files looks like this
PGABOLTXML1D_201108121235.xml... (1 Reply)
dear all,
i have .dat files named as:
34.dat
2.dat
16.dat
107.dat
i would like to sort them by their filenames as:
2.dat
16.dat
34.dat
107.dat
i have tried numerous combinations of sort and ls command (in vain) to obtain :
107.dat
16.dat
2.dat
34.dat (1 Reply)
I have a directory of about 30,000 image files. The file names are all yearmonthday.jpg however some of the files have yearmonthday-snapshot.jpg
i would like to move all files that contain the phrase -snapshot to their own directory.
Any assistance with the proper commands would be much... (1 Reply)
Hi,
I am new to Unix shell scripting. Can you please help me with this immediate requirement to code.. The requirement is as given below.
In a directory say Y, I have files like this.
PP_100000_28062006_122731_746.dat
PP_100000_28062006_122731_745.dat
PP_100000_28062006_122734_745.dat... (4 Replies)