I need to generate numbers from 1 to 30 and match them with their equivalent extracted from the filename (if duplicates are found, than multiply the rows with the numbers of duplicates) and then search for a pattern inside the log file.
With help from internet I managed to complete each operation, but I do not know to how to merge them in a single command.
Generate numbers from 1 to 30 using :
Code:
awk 'BEGIN { for (i = 1; i <= 30; i++) printf "%06d\n", i }'
0001 - name_date_0001_ID0.log “2nd line after the pattern is found” “number of “%” signs”
0001 - name_date_0001_ID2.log “2nd line after the pattern is found” “number of “%” signs”
0002 - name_date_0001_ID2.log “2nd line after the pattern is found” “number of “%” signs”
0003 -
......
0030 - name_date_0030_ID0.log “2nd line after the pattern is found” “number of “%” signs”
Also if possible I would like to check for the length of ID0, ID1 .... IDn and if less then 3 characters then display a warning after “number of “%” signs”
Please help me merging all the codes to achieve the above results.
Alex
Last edited by Scrutinizer; 12-22-2012 at 02:28 AM..
Reason: cleaned out all tags, reintroduced code tags...
I need to generate numbers from 1 to 30 and match them with their equivalent extracted from the filename (if duplicates are found, than multiply the rows with the numbers of duplicates) and then search for a pattern inside the log file. With help from internet I managed to complete each operation, but I do not know to how to merge them in a single command. Generate numbers from 1 to 30 using :
Code:
awk 'BEGIN { for (i = 1; i <= 30; i++) printf "%06d\n", i }'
0001 - name_date_0001_ID0.log “2nd line after the pattern is found” “number of “%” signs” 0001 - name_date_0001_ID2.log “2nd line after the pattern is found” “number of “%” signs” 0002 - name_date_0001_ID2.log “2nd line after the pattern is found” “number of “%” signs”0003 - ......0030 - name_date_0030_ID0.log “2nd line after the pattern is found” “number of “%” signs”
Also if possible I would like to check for the length of ID0, ID1 .... IDn and if less then 3 characters then display a warning after “number of “%” signs”
Please help me merging all the codes to achieve the above results. Alex
I'm not sure what you want to get out of this.
You show a printf format string of "%06d\n", but all of the data you show in your examples show a zero-filled four decimal digit string; not six decimal digits. Do you want four digits or six digits?
does not match the output you say you want to see:
Code:
0001 - name_date_0001_ID0.log “2nd line after the pattern is found” “number of “%” signs”
because the leading four decimal digit sequence and the following " - " are not in the printf format string. Do you want the output to match the format string you gave, or do you want the output to match the example shown?
You code has an underlying assumption that the pattern you're looking for will appear on the same line in every file. (The assumption is hidden by the fact that the array c is not cleared when you start reading a new file.) Do you always want to display a specific line number in each file, or do you want to display the 2nd line after a line that matches a given pattern?
If the pattern appears more than one line in a file, do you want the second line after each match and the count of %s on that line to be printed for each match, or only for the first match in each file?
Will you guarantee that each filename matched by *.log contains exactly three underscore characters? Is the error message that you want to be printed supposed to be a check that the characters between the 3rd underscore and the .log at the end of the filename is ID followed by a single decimal digit, or is it intended to verify that there are exactly three underscores, exactly four (or six) decimal digits between the 2nd and 3rd underscore and ID followed by a single decimal digit followed by .log after the 3rd underscore?
Please give us a sample of the start of the content of one of your log files (in CODE tags) and show us the pattern you want to match.
The last printf was designed to output the following (before thinking to add the numbers from 1 to 30 and match them with the filename, since here the lack of “-“ signs):
Code:
FILENAME (even the FILENAME is varying between 22 characters and 24 I put 40 only to add some space, at the that time I didn’t knew the numbers of characters of the log file)“2nd line after the pattern is found” shows %%%%%%%% and could go up to 80 signs“number of “%” signs” count how many “%” were found (number between 1 to 80)name_date_0001_ID0.log “2nd line after the pattern is found” “number of “%” signs”
If possible I would like the output to match the example shown. Displayig the 2nd line after matching a pattern would be great. Never happened so far to find more than one pattern in a file, and if does I would like to get the last match. Yes each log file has 3 underscore characters. I would the error message to count the number of characters in “ID0”, because the length can very sometimes. For example if I get ID005 I need the script to print the warning message, but not if I have ID5. The pattern that I want to match is:
Code:
“Percentage Completed”“empty row”“%%%%%%%%%%%%%%%%%%%%%%%%%” – this number can vary and is between 1 and 80.
If I understood your requirements correctly, I believe the following script does everything you want. I tried it with a bunch of log files containing zero to four lines matching your pattern, with filenames matching and not matching your naming conventions. For files that have more than one line matching your pattern, it reports the contents of the line two lines after the last line matching your pattern:
Code:
#!/bin/ksh
awk -F% '
# Variables used:
# d1: Diagnostic message 1 (! 3 _ in lf || np[4] fails to match ID[0-9].log)
# d2: Diagnostic message 2 (np[3] > 0030 || np[3] not four decimal digits)
# d3: Diagnostic message 3 (pattern not matched or matched more than once)
# fl: Found Line (Contents of line 2 lines after last line matching
# pattern.)
# flpc: Found Line % Count
# lf: Last FILENAME (We cannot know that we have the last match in a
# file until we hit the 1st line in the next file. We save the
# last value of FILENAME until we can print the results for this
# file.)
# ln: Line Number of line 2 lines after last line matching pattern.
# mc: Match Count (Number of times we have found a line 2 lines after
# a line matching pattern.)
# np[]: lf split by _ (There should always be 4 elements in this array.)
# ps: Printable seq (np[3] if np[3] is four decimal digits; otherwise
# "????".)
# seq: Sequence number (Used to print lines noting missing sequence
# numbers in the values of np[3] while 1 <= seq <= 30.)
function pr() {
# Usage: pr()
# Verify the last FILENAME, print missing sequence numbers, and print
# results of processing the last file.
# Skip 1st call when we have not processed a file yet...
if(lf == "") return
# Verify filename format...
if(split(lf, np, /_/) != 4)
d1 = " Filename should contain 3 underscores."
else if(match(np[4], /^ID[0-9].log$/) != 1)
d1 = " " np[4] " does not match \"IDx.log\"."
ps = (match(np[3], /^[0-9][0-9][0-9][0-9]$/) != 1) ? "????" : np[3]
if(RSTART == 1) {
# Print headers sequence numbers with no matching log files...`
while(seq < np[3] && seq <= 30) printf("%04d -\n", seq++)
seq = np[3] + 1
if(np[3] > 30) d2 = " " np[3] " out of range."
} else d2 = " " np[3] " not four decimal digits."
if(mc != 1) d3 = " Pattern matched " mc " times."
printf("%s - %-40s %-80s %4d%s%s%s\n", ps, lf, fl, flpc, d1, d2, d3)
# Clear diagnostic messages, found line, and % count...
d1 = d2 = d3 = fl = ""
flpc = 0
}
BEGIN { seq = 1 } # Set starting sequence #.
FNR == 1 { # We have the first line of a new file.
pr() # Print results from previous file.
lf = FILENAME # Save FILENAME of current file.
mc = 0 # Clear match count for this file.
}
/Percentage Completed/ {# We have a match.
ln = FNR + 2 # Set line # for 2 lines later.
}
FNR == ln { # We have a line 2 lines after a match.
mc++ # Increment match count.
fl = $0 # Save the line.
flpc = NF - 1 # Save the number of % characters on this line.
ln = 0 # Clear the line number.
}
END { pr() # Print results from last file processed.
}' $(ls *.log | sort -t_ -k3n -k4) # Sort the log files to be processed.
On Solaris systems, use /usr/xpg4/bin/awk or nawk instead of awk.
Last edited by Don Cragun; 12-23-2012 at 03:22 AM..
Reason: Warn about which awk to use on Solaris systems.
This User Gave Thanks to Don Cragun For This Post:
Input file:
100%ABC2 3.44E-12 USA
A2M%H02579 0E0 UK
100%ABC2 5.34E-8 UK
100%ABC2 3.25E-12 USA
A2M%H02579 5E-45 UK
Output file:
100%ABC2 3.44E-12 USA
100%ABC2 3.25E-12 USA
100%ABC2 5.34E-8 UK
A2M%H02579 0E0 UK
A2M%H02579 5E-45 UK
Code try:
sort -k1,1 -g -k2 -r input.txt... (2 Replies)
Hey guys,
Sorry for the basic question but I have a lot of files that I want to separate into groups based on filenames which I can then cat together. Eg I have:
(a_b_c.txt)
WB34_2_SLA8.txt
WB34_1_SLA8.txt
WB34_1_DB10.txt
WB34_2_DB10.txt
WB34_1_SLA8.txt
WB34_2_SLA8.txt
77_1_SLA8.txt... (1 Reply)
Hi all.
I am very new to linux scripting and i have a task i can only solve with a script.
I need to sort files base on the date string in their filenames and create a folder using the same date string then move the files to their respective folders.
Scenario:
Folder Path:... (1 Reply)
Hi all.
I am very new to linux scripting and i have a task i can only solve with a script.
I need to sort files base on the date string in their filenames and create a folder using the same date string then move the files to their respective folders.
Scenario:
Folder Path:... (1 Reply)
Hi, I am a newbie to shell programming and I need some help in sorting a list of files in ascending order of date in the filenames.
The file format is always : IGL01_AC_D_<YYYYMMDD>_N01_01
For example, in a directory MyDirectory I have the following files:
IGL01_AC_D_20110712_N01_01.dat... (11 Replies)
Hi
I have a requirement like below
I need to sort the files based on the timestamp in the file name and run them in sorted order and then archive all the files which are one day old to temp directory
My files looks like this
PGABOLTXML1D_201108121235.xml... (1 Reply)
dear all,
i have .dat files named as:
34.dat
2.dat
16.dat
107.dat
i would like to sort them by their filenames as:
2.dat
16.dat
34.dat
107.dat
i have tried numerous combinations of sort and ls command (in vain) to obtain :
107.dat
16.dat
2.dat
34.dat (1 Reply)
I have a directory of about 30,000 image files. The file names are all yearmonthday.jpg however some of the files have yearmonthday-snapshot.jpg
i would like to move all files that contain the phrase -snapshot to their own directory.
Any assistance with the proper commands would be much... (1 Reply)
Hi,
I am new to Unix shell scripting. Can you please help me with this immediate requirement to code.. The requirement is as given below.
In a directory say Y, I have files like this.
PP_100000_28062006_122731_746.dat
PP_100000_28062006_122731_745.dat
PP_100000_28062006_122734_745.dat... (4 Replies)