Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Arrange log files with AWK

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Arrange log files with AWK
# 1  
Old 05-05-2010
Arrange log files with AWK
Hello friends,

I have too many log files to arrange. I use a simple script to create log files with below format and i forgot to create daily directory for them at the beginning. Because of this i should move all daily logs into a directory that i need to create.

a part of "ls -l" output:
Code:
-rw-r--r--   1 root     root        2104 Apr  8 11:55 inap_host1_08042010_1152.txt
-rw-r--r--   1 root     root       53970 Apr  8 11:58 inap_host1_08042010_1157.txt
-rw-r--r--   1 root     root       46706 Apr  8 12:57 inap_host1_08042010_1246.txt
-rw-r--r--   1 root     root        1959 Apr  8 12:58 inap_host1_08042010_1258.txt
-rw-r--r--   1 root     root        1959 Apr  8 13:10 inap_host1_08042010_1310.txt
-rw-r--r--   1 root     root       15793 Apr  9 13:23 inap_host1_09042010_1317.txt
-rw-r--r--   1 root     root       22575 Apr  9 13:26 inap_host1_09042010_1324.txt
-rw-r--r--   1 root     root        1764 Apr  9 14:21 inap_host1_09042010_1421.txt
-rw-r--r--   1 root     root       57644 Apr  9 16:43 inap_host1_09042010_1638.txt
-rw-r--r--   1 root     root       49617 Apr  9 18:38 inap_host1_09042010_1745.txt
-rw-r--r--   1 root     root        1975 Apr  9 19:27 inap_host1_09042010_1839.txt
-rw-r--r--   1 root     root        1764 Apr 10 13:27 inap_host1_10042010_1327.txt
-rw-r--r--   1 root     root       53652 Apr 10 13:36 inap_host1_10042010_1334.txt
-rw-r--r--   1 root     root        1611 Apr 10 13:44 inap_host1_10042010_1344.txt
-rw-r--r--   1 root     root       51486 Apr 10 13:53 inap_host1_10042010_1352.txt
-rw-r--r--   1 root     root       54121 Apr 10 13:57 inap_host1_10042010_1355.txt
-rw-r--r--   1 root     root        1611 Apr 11 14:02 inap_host1_11042010_1402.txt
-rw-r--r--   1 root     root       93449 Apr 11 14:20 inap_host1_11042010_1409.txt
-rw-r--r--   1 root     root        1882 Apr 11 14:22 inap_host1_11042010_1422.txt
-rw-r--r--   1 root     root        1807 Apr 11 15:11 inap_host1_11042010_1511.txt
-rw-r--r--   1 root     root       33223 Apr 11 15:29 inap_host1_11042010_1526.txt
-rw-r--r--   1 root     root       57644 Apr  8 16:43 inap_host1_08042010_1638.txt
-rw-r--r--   1 root     root       23345 Apr  9 18:38 tcap_host1_09042010_1745.txt
-rw-r--r--   1 root     root        1975 Apr  9 19:27 tcap_host1_09042010_1839.txt
-rw-r--r--   1 root     root        9877 Apr 10 13:27 tcap_host1_10042010_1414.txt
-rw-r--r--   1 root     root       14414 Apr 10 13:36 tcap_host1_10042010_1334.txt
-rw-r--r--   1 root     root        1234 Apr 11 13:44 tcap_host1_11042010_1344.txt


i need to gather all logs in daily basis, i couldnt manage to get a result after some try:

Code:
awk 'split($NF,arr,"[_]");
    {if[arr[3]!= arr[3] in next record ] && if [ ! -e /path/to/files/`$"arr[3]` ]}
	then mkdir /path/to/files/`$"arr[3]`;
	mv $NF /path/to/files/`$"arr[3]`'
........
.......


I appreciate any help,

regards
# 2  
Old 05-06-2010
Try...
Code:
ls *.txt | cut -d _ -f 3 | sort -u | while read d
do 
   mkdir -p /path/to/$d/ && mv *$d*.txt /path/to/$d/
done

Last edited by Ygor; 05-06-2010 at 04:18 AM.. Reason: Forgot the -p flag on mkdir
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[awk] combine and convert time from log files

dear all, an awk newbie need your help.... i have log files with this format: mylog1a.log: "08/10/2012","5:05 PM" "Hostname","Device Address","Count" "","10.10.10.18","10234" mylog2a.log: "08/11/2012","5:05 PM" "Hostname","Device Address","Count" "","10.10.10.18","12543" ... (18 Replies)
Discussion started by: makan
18 Replies

2. Shell Programming and Scripting

awk to search similar strings and arrange in a specified pattern

Hi, I'm running a DB query which returns names of people and writes it in a text file as shown below: Carey, Jim; Cena, John Cena, John Sen, Tim; Burt, Terrence Lock, Jessey; Carey, Jim Norris, Chuck; Lee, Bruce Rock, Dwayne; Lee, Bruce I want to use awk and get all the names... (9 Replies)
Discussion started by: prashu_g
9 Replies

3. UNIX for Dummies Questions & Answers

awk and log files

Hello, I have a series of logs that I need to analyse. each looks something like:234.10.72.175 Mon Mar 02 20:25:00 GMT 2009 226.91.87.86 Thu Mar 05 03:50:26 GMT 2009 226.91.87.86 Thu Mar 05 04:06:07 GMT 2009 Using awk, so far I have been able to count the lines in a... (5 Replies)
Discussion started by: Freaky
5 Replies

4. Shell Programming and Scripting

arrange files into columns

Hi may i ask how to arrange this file in into columns like example below Problem: #cat data.txt 1 2 3 4 5 a b c d 9 8 7 6 desired output (9 Replies)
Discussion started by: jao_madn
9 Replies

5. Shell Programming and Scripting

Arrange / format data using awk

Input 217:fngadi4osa:fngadi4osa:M 217:415744:N/A 227:fngadi4osa:fngadi4osa: M 227:51200:N/A 228:fngadi4osa:fngadi4osa: M 228:102400:N/A 65:sapgt04:sapgt04: M 65:104448:N/A 228:fngadi4osa:fngadi4oma: M 228:102400:N/A Output 217:fngadi4osa:fngadi4osa:M 217:415744:N/A... (3 Replies)
Discussion started by: greycells
3 Replies

6. UNIX for Dummies Questions & Answers

How to arrange the files according to time

hi friends, m new to unix environment.. i want to know that "how could we arrange the various files in our unix system according to time at which they are accessed, and are there are ways that we can arrange them in ascending or descending manner..." please help me out with full details...... (4 Replies)
Discussion started by: adityamitra
4 Replies

7. Shell Programming and Scripting

Shell script to arrange files into several folders

Hello this is the script Im working on I have a picture collection that I rescued from a hard drive and there are thousands of pictures saved in one folder. What I need is to create several folders and put lets say around 200 pictures in each folder. At the end instead of having one huge... (8 Replies)
Discussion started by: kizofilax
8 Replies

8. Shell Programming and Scripting

gawk - reading two files & re arrange the columns

Hi, I am trying to read 2 files and writing to the 3rd file if I find the same elements in 2 files. my first file is 1 0 kb12124819 766409 1.586e-01 1 0 kb17160939 773886 8.674e-01 1 0 kb4475691 836671 8.142e-01 1 0 ... (2 Replies)
Discussion started by: ezhil01
2 Replies

9. Shell Programming and Scripting

how to arrange 3 file to one using awk...?

I have 3 file, I want to re-arrange so all data easy to read from one file only: file1 : userA 10 20 userB 30 40 userC 50 60 .... file2 : userA var1 1000 userA var2 2000 userB var2 3000 userB var3 4000 userB var4 5000 userC var1 6000 ... file3 : userA var3 7000 userA var4 8000 (9 Replies)
Discussion started by: penyu
9 Replies

10. UNIX for Dummies Questions & Answers

Constantly updating log files (tail -f? grep? awk?)

I have a log file which is continuously added to, called log.file. I'd like to monitor this file, and when certain lines are found, update some totals in another file. I've played around with tail -f, grep, and awk, but can't seem to hit the right note, so to speak. The lines I'm... (0 Replies)
Discussion started by: nortonloaf
0 Replies
Login or Register to Ask a Question