Use of awk and printf - help needed


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Use of awk and printf - help needed
# 1  
Old 12-28-2012
Use of awk and printf - help needed

I have a very large file with more than 500,000 lines of dated events.

The first field contains the date/time in the following format:
20120727-files.files:20120727090044

where the first 8 numbers represent yyyymmdd. The last set of numbers represent yyyy/mm/dd/hh:mm:ss

I would like to change the value of the first field to:
YYYY-MM-DD HH:MM:SS

I have no clue how to do this. I started with displaying the first field only to simplify things.

Code:
awk -F"|" '{ print $1 }' infile

I've tried to find a cheatsheet for printf but couldn't find anything good. Is printf universal to many languages? Or is printf in bash unique to all....?

Thanks for reading..

-David
# 2  
Old 12-28-2012
The printf utility is not unique to bash; it is a standard utility present on any system that supports the common set of utilities defined by the POSIX standards and the Single UNIX Specification. But, if you're using awk to grab the 1st field out of your input file, you might as well just do all of the work in awk. If you just want to change the format of the first field in your file and print the updated line, you can use:
Code:
awk '
BEGIN { FS = OFS = "|"}
{       l=length($1)
        $1 = sprintf("%s-%s-%s %s:%s:%s", substr($1, l - 13, 4),
                substr($1, l - 9, 2), substr($1, l - 7, 2),
                substr($1, l - 5, 2), substr($1, l - 3, 2), substr($1, l - 1))
        print
}' infile 
exit

If you just want to print the dates and ignore the rest of the fields in in file, you can shorten this to:
Code:
awk -F '|' '{
        l=length($1)
        printf("%s-%s-%s %s:%s:%s", substr($1, l - 13, 4),
                substr($1, l - 9, 2), substr($1, l - 7, 2),
                substr($1, l - 5, 2), substr($1, l - 3, 2), substr($1, l - 1))
}' infile 
exit

This User Gave Thanks to Don Cragun For This Post:
# 3  
Old 12-28-2012
That's amazing.

It worked the first time around. I really appreciate your help on this.

If you could answer a qusetion... I'm having trouble finding a tutorial or information on the "l - 13" or "l - 9" portion of the code.

I read the man on awk for substr (which is vague and confusing) and I'm just curious how I can make myself fluent in this language without having to pull my hair out and ask a billion questions.... If you could give me some insight on that, I'd really appreciate it.

-David

Happy New Year!

---------- Post updated at 05:27 PM ---------- Previous update was at 05:12 PM ----------

I KIND OF understand what it's saying but not quite.

substr($1, l - 13, 4)

The above looks like it's starting the count, 4 from the end, 13 to the left, which goes over 2012.

substr($1, l - 9, 2)
The above starts the count 2 from the end, 9 to the left, which is right between the 2012 and 12, which seems perfect. Am I missing something here? (obviously I am.)
# 4  
Old 12-28-2012
Quote:
Originally Posted by LDHB2012
That's amazing.

It worked the first time around. I really appreciate your help on this.

If you could answer a qusetion... I'm having trouble finding a tutorial or information on the "l - 13" or "l - 9" portion of the code.

I read the man on awk for substr (which is vague and confusing) and I'm just curious how I can make myself fluent in this language without having to pull my hair out and ask a billion questions.... If you could give me some insight on that, I'd really appreciate it.

-David

Happy New Year!

---------- Post updated at 05:27 PM ---------- Previous update was at 05:12 PM ----------

I KIND OF understand what it's saying but not quite.

substr($1, l - 13, 4)

The above looks like it's starting the count, 4 from the end, 13 to the left, which goes over 2012.

substr($1, l - 9, 2)
The above starts the count 2 from the end, 9 to the left, which is right between the 2012 and 12, which seems perfect. Am I missing something here? (obviously I am.)
Hi David,
The awk command l=length($1) sets l to the number of characters in the first field. In your example, this sets l to 35 because there are 35 characters in "20120727-files.files:20120727090044" (not counting the terminating null byte). The substr(x, s, c) returns c characters from the string x starting with the sth character in the string. So, substr($1, l - 13, 4) returns 4 characters from 20120727-files.files:20120727090044 starting with the 22nd character (i.e., "2012"). When a count isn't given (as in substr($1,l - 1)), substr() will return the remainder of the string from the given starting point. This is all done because I assume that the files.files portion of your input lines may vary in length, but we know that the timestamp is always the last 14 characters in the string.
This User Gave Thanks to Don Cragun For This Post:
# 5  
Old 12-29-2012
Code:
awk -F":" '{print substr($2,0,4)"-"substr($2,5,2)"-"substr($2,7,2)" "substr($2,9,2)":"substr($2,11,2)":"substr($2,13,2)}' your_file

Regards,
Vijay

Last edited by Scott; 12-30-2012 at 01:41 AM.. Reason: Removed link
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk printf dillemma

Please help me format this file: Source file looks like this, there are three columns, separated by space. First column has varrying width: 1 248105240 W25_2013 10 248103710 W06_2013 100 248103710 W06_2013 1000 248103710 W06_2013 I need to transform the file into a fixed width per column.... (1 Reply)
Discussion started by: tamahomekarasu
1 Replies

2. Shell Programming and Scripting

printf format with awk

Hello Here is an easy one Data file 12345 (tab) Some text (tab) 53.432 23456 (tab) Some longer text (tab) 933.422 34567 (tab) Some different text (tab) 29.309 I need to awk these three tab-delimited columns so that the first two are unchanged (unformatted) and the third shows two decimal... (1 Reply)
Discussion started by: palex
1 Replies

3. Shell Programming and Scripting

awk with printf

Hi, I am using the following code to assign a count value to a variable. But I get nothing. Do you see anything wrong here. I am new to all this. $CTR=`remsh $m -l $MACHINES{$m} -n cat $output | grep -v sent | grep \"$input\" | sort -u | awk '{print $5}'`; Upto sort - u it's... (2 Replies)
Discussion started by: nurani
2 Replies

4. Shell Programming and Scripting

AWK printf help

Target file contains short text (never more than 1 line) and filenames. The format is, e.g.,: TEXT1 filename1 TEXT2 TEXT3 filename3dddd filename3dddd TEXT4 filename4 TEXT5 filename5dddd filename5dddd filename5 where dddd is a random 4-digit whole number. Desired output: (4 Replies)
Discussion started by: uiop44
4 Replies

5. Shell Programming and Scripting

IF and awk/printf

Hi Friends, Scripting newb here. So I'm trying to create a geektool script that uses awk and printf to output certain fields from top (namely command, cpu%, rsize, pid and time, in that order). After much trial and error, I've pretty much succeeded, with one exception. Any process whose name... (3 Replies)
Discussion started by: thom.mattson
3 Replies

6. Shell Programming and Scripting

awk and printf

echo $bbsize 1.5 echo $fillpercent .95 echo $bbsize | awk '{printf "%.2f\n",$0*$fillpercent}' 2.25 echo $bbsize | awk '{printf "%.2f\n",$0*.95}' 1.42 1.42 is what I'm expecting... echo $blocksize 4096 echo $bbsize | awk '{printf "%.2f\n",$0*$blocksize}' 2.25 echo $bbsize |... (3 Replies)
Discussion started by: xgringo
3 Replies

7. Shell Programming and Scripting

printf in awk

Hi friends.. I am confused about awk printf option.. I have a comma separated file 88562848,21-JAN-08,2741079, -1188,-7433,TESTING 88558314,21-JAN-08,2741189, -1273,-7976,TESTING and there is a line in my script ( written by someone else) What is the use of command? I guess... (10 Replies)
Discussion started by: clx
10 Replies

8. Shell Programming and Scripting

Awk printf problem

Hi, I've got a basic problem using printf statement in awk. I want to write float values with always 8 characters width. Examples : 1.345678 12.45678 123.4567 1234.678 -23.5678 -2.45678 -23456.8 ..... I cannot find the right printf format %8.1f, %7.5f.... Can anyone help ?... (4 Replies)
Discussion started by: cazhot
4 Replies

9. UNIX for Dummies Questions & Answers

AwK printf question

Hi, Does anyone know a easy way to printf $3,$4, ... all the way to the last field in the file? I will need to modify $1 and $2 and then printf modified $1 and $2 and the rest of the fields(which are not changed). I know I can use NF as the total number of field. Do I use a for next statement to... (4 Replies)
Discussion started by: whatisthis
4 Replies

10. UNIX for Advanced & Expert Users

awk printf problem

Hi Friends, Can anyone guide me how to compute sum of column4 from the below file x using awk command? when i do using awk I'm getting sum 7482350198352648.000000 which is not accurate. $ cat x 56,232,dfgjkhdfj,,56,anand 56,22,dfgjkhdfj,7482347823453123.97834 ,56,Khan 56,23,dfgjkhdfj, ... (6 Replies)
Discussion started by: krishna
6 Replies
Login or Register to Ask a Question