create csv in ksh from tricky log file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting create csv in ksh from tricky log file
# 1  
Old 11-16-2010
create csv in ksh from tricky log file

hi guys, trying to create a csv from a tricky log file in ksh,

using 'awk '{print $1" "$14" "$15" "$16" "$17" "$18" "$19}' >> $TMP_FILE' on another set of files I have an output file with hundreds of lines in which looks like so:

ABC_DEFGHI_16_JKLMNP11.20101115_095412_374.log:09:54:29.579 cars amount 29, total cars 70
ABC_DEFGHI_17_JKLMNP11.20101115_095412_374.log:09:54:29.585 cars amount 34, total cars 43
ABC_DEFGHI_11_JKLMNP11.20101115_095412_374.log:09:54:29.601 cars amount 22, total cars 44

but from the above log file i'm trying to create 2 csv files which looks like so:

date,time,cars amount
15/11/2010,09:54:29,29
15/11/2010,09:54:29,34
15/11/2010,09:54:29,22

and the other one...

date,time,cars amount
15/11/2010,09:54:29.579,29
15/11/2010,09:54:29.585,34
15/11/2010,09:54:29.601,22

could anyone provide an example awk or cut to achieve the above? it's proving tricky to achieve with cut! thanks
# 2  
Old 11-16-2010
Try:
Code:
awk -F'[ \t,_:.]*' '{print $5","$9":"$10":"$11"."$15}' infile

Code:
awk -F'[ \t,_:.]*' '{print $5","$9":"$10":"$11"."$12","$15}' infile


Last edited by Scrutinizer; 11-16-2010 at 06:03 PM.. Reason: $12 changed to $15
This User Gave Thanks to Scrutinizer For This Post:
# 3  
Old 11-16-2010
Code:
awk -F'[ \t,_:.]*' '{print $5","$9":"$10":"$11"."$12}' infile

that one returns:

Code:
richard@opensolaris:~/Shell$ awk -F'[ \t,_:.]*' '{print $5","$9":"$10":"$11"."$12}' junk.log
20101115,09:54:29.579
20101115,09:54:29.585
20101115,09:54:29.601

2nd one works though! Smilie :

Code:
richard@opensolaris:~/Shell$ awk -F'[ \t,_:.]*' '{print $5","$9":"$10":"$11"."$12","$15}' junk.log
20101115,09:54:29.579,29
20101115,09:54:29.585,34
20101115,09:54:29.601,22

is it possible to get the date in the format 15/11/2010?

could you take me through what this part '-F'[ \t,_:.]' is doing, I understand the rest but not that part much appreciated!
# 4  
Old 11-16-2010
Based from scrutinizer command:
Code:
$ awk -F'[ \t,_:.]*' '{print substr($5,7,2)"/"substr($5,5,2)"/"substr($5,1,4)","$9":"$10":"$11"."$12}' a
15/11/2010,09:54:29.579
15/11/2010,09:54:29.585
15/11/2010,09:54:29.601

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Create csv from text file

Gents, I am trying to create a csv file using the file attached. I have a problem to get all information required because the rows are not continues. Here is my code till now. awk ' /"ffid"/{if(s){print s;s=$NF}else{s=$NF}} /"LineNumber"/{s=s $NF} /"PointNumber"/{s=s $NF}... (4 Replies)
Discussion started by: jiam912
4 Replies

2. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

3. Shell Programming and Scripting

Need help. How to create csv file?

Hi I'm a beginner and I have some problem. I have multiple files in the same directory which has one column but rows following the format. File: directory/Disk.txt Content: a b c d e File: directory/Memory.txt a b c d e File: directory/CPU.txt (3 Replies)
Discussion started by: thenuie
3 Replies

4. Shell Programming and Scripting

Script to create a CSV file

I created a script that will go out and so a "/sbin/chkconfig --list | egrep XXX" against a server list that would create an output file like the following example: ---------------------------------------------------------------------------------- SERVER1 RC_Script_1 0:off 1:off 2:off... (4 Replies)
Discussion started by: asnatlas
4 Replies

5. Shell Programming and Scripting

Compare 2 csv files in ksh and o/p the difference in a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 The... (7 Replies)
Discussion started by: Naresh101
7 Replies

6. Shell Programming and Scripting

Create Log File in ksh itself

Hi, I want to create a log file for a running ksh , and the log file absolute path I want to give in ksh itself. To elaborate this - Say I have a ksh - timer.ksh and I want to create a log timer_log.log when I run, to trace this. I am aware of the fact that this can be done using redirection... (4 Replies)
Discussion started by: vinay4889
4 Replies

7. Shell Programming and Scripting

csv file editing using KSH

I'm trying to write a shell script to extract useful fields in a csv file and copy them to a new file: the input file is as below when opened using notepad++: //////////////////////////////////////////////////////////// ZZZZZZZZZZZZZZZZZZZZZZ ,"A", , , ,24,18,0,0,42,0 , ,B, ,... (1 Reply)
Discussion started by: zekruss
1 Replies

8. Shell Programming and Scripting

ksh script to create a generic csv file from different source formats

Hi all, I have a requirement to create a "superset" file out of a number of different sources with some different and some same columns. We intend to have a manually updateable SuperSetCols.csv which would look like "ColA","ColB","ColC","ColD","ColE","ColF","ColG" so someday we may add... (3 Replies)
Discussion started by: Leedor
3 Replies

9. Shell Programming and Scripting

Command to create and update csv file

Hi, I need to create a csv file to store oracle query output. This report need to be created on hourly basis. The csv file report format as "Report_22_Sep_09_13IST.csv". I have the oracle query. Now i need to create and move the oracle query output to the report row by row starting from 3rd row.... (6 Replies)
Discussion started by: Sekar1
6 Replies

10. Shell Programming and Scripting

Eval Tricky Manipulation of Arry in KSH - Help

Hi, Could any one share the intelligence to track this problem. I have any array BT_META_36 and it prints properly with contents of array. # print "BT_META_36=${BT_META_36}" # BT_META_36=cab3,cab4:HDS:052,07A cab3,cab4:HDS:052,07A Now I have a BT_META_36 assigned to a variable.... (0 Replies)
Discussion started by: ajilesh
0 Replies
Login or Register to Ask a Question