Create file every year and log data to it


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Create file every year and log data to it
# 1  
Old 05-12-2016
Create file every year and log data to it

Hi there,
Here is what I currently have and trying to come up with a logic to automatically append logs to a respective year file. The log files might get generated everyday or any other day, irrespective of which I want to copy all of the logs for that year into one file /tmp/<year>.log and then copy the logs to new location /tmp/<newyear>.log when a newyear comes and so on.
Thanks for looking.

Code:
#!/bin/ksh
cat /tmp/*.log.old >> /tmp/2016.log

rm /tmp/*.log.old > /dev/null 2>&1

ls /tmp/*.log > /tmp/system_logs

for i in `cat /tmp/system_logs`
do
  echo "New log file moving to old is $i \n"
  cp $i $i.old
  if [ $? -eq 0 ]
  then
     rm $i
  fi
done


Last edited by mbak; 05-12-2016 at 09:34 PM.. Reason: typo
# 2  
Old 05-12-2016
Would something like this get you in the direction?

Code:
year=$(date "+%Y")
for f in /tmp/*.log; do
  cat "$f" >> /tmp/${year}.log
  rm "$f"
done

# 3  
Old 05-23-2016
Aia - A belated thanks as I was away for sometime and didn't had a chance to look at the code, I'll post back if there are any issues.
# 4  
Old 05-25-2016
Code:
$ ./test.sh
++ date +%Y
+ year=2016
+ for f in '/tmp/*.log'
+ cat /tmp/2016.log
cat: /tmp/2016.log: input file is output file
+ rm /tmp/2016.log
+ for f in '/tmp/*.log'
+ cat /tmp/aaa.log
+ rm /tmp/aaa.log
+ for f in '/tmp/*.log'
+ cat /tmp/bbb.log
+ rm /tmp/bbb.log
+ for f in '/tmp/*.log'
+ cat /tmp/ccc.log
+ rm /tmp/ccc.log

This doesn't seem to work as expected, it is removing the dated file 2016.log in this case after giving message "input file is output file'whereas I'm looking to append to it. I'm trying to figure out how to append to multiple dated log files for eg...all aaa.log files should append to one aaa.2016.log, bbb.log files in bbb.2016.log file and so on.

---------- Post updated at 11:30 PM ---------- Previous update was at 06:42 PM ----------

Code:
#!/bin/ksh
YEAR=$(date "+%Y")
cd /tmp
if [ ! -d "${YEAR}_LOGS" ]; then
  echo " ${YEAR}_LOGS doesn't exist. Creating dir..." > /dev/null 2>&1
  mkdir /tmp/${YEAR}_LOGS > /dev/null 2>&1
else
  echo " ${YEAR}_LOGS directory exists" > /dev/null 2>&1
fi

for i in $(ls /tmp/ |grep old | awk -F "." '/aaa|bbb|cccc|dddd/ {print $1}'); do
  cat /tmp/${i}.log.old >> /tmp/${YEAR}_LOGS/${i}_${YEAR}.log
  rm /tmp/${i}.log.old > /dev/null 2>&1
done

for i in /tmp/*.log; do
  echo "Log being moved to old is $i ... \n"
  cp $i $i.old
  if [ $? -eq 0 ]; then
     rm $i
  fi
done

This is what I came up with and wondering if this could be made simpler or enhanced further, pls advise.

Last edited by mbak; 05-25-2016 at 03:31 AM..
# 5  
Old 05-25-2016
Although untested, this would seem to be a more efficient version of your latest script, producing the same outputs:
Code:
#!/bin/ksh
YEAR=$(date "+%Y")
cd /tmp

mkdir -p "${YEAR}_LOGS"

for i in aaa bbb cccc dddd
do
  if [ -f "$i.log.old" ]
  then
    cat "$i.log.old" >> "${YEAR}_LOGS/${i}_$YEAR.log" && rm -f "$i.log.old"
  fi
done

for i in *.log
do
  printf 'Log being moved to old is %s ... \n\n' "$PWD/$i"
  mv "$i" "$i.old"
done

Note, however, that, by definition, /tmp is a directory intended to hold temporary files. You haven't said what operating system you're using, but on many systems, ALL files under /tmp are removed every time the system is rebooted. So, attempting to archive data from multiple years in files under /tmp is extremely dangerous. I strongly suggest that you choose a different directory as the base of your active and archived log files.
These 2 Users Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Create file based on data from two other files

I have looked through several threads regarding merging files with awk and attempted using join however have been unsuccessful likely as I do not fully understand awk. What I am attempting is to take a csv file which could be between 1 and 15,000 lines with 5 colums and another csv file that will... (4 Replies)
Discussion started by: cdubu2
4 Replies

2. Shell Programming and Scripting

From 2 files create 3rd file with uncommon data

Hi All, I have two files. File1 and File2. Want to create another file with all the records of File1 those are not present in File2. Please guide. Thanks in advanced. Anupam (3 Replies)
Discussion started by: Anupam_Halder
3 Replies

3. Shell Programming and Scripting

create txt file form data file

File A.txt LL07 LL07_B_1 20 LL85 LL85_A_1 40 LL85 LL85_B_1 40 LL85 LL85_C_1 30 LL37 LL37_A_1 60 LL37 LL37_B_1 20 LL37 LL37_C_1 50 I want cretae diffrent tex file base of above file Should be threee text file LL07.txt LL85.txt LL37.txt Eaach text file have below data... (2 Replies)
Discussion started by: asavaliya
2 Replies

4. Shell Programming and Scripting

create txt file form data file and add some line on it

Hi Guys, I have file A.txt File A Data AK1521 AK2536 AK3164 I want create text file of all data above and write some data on each file. want Output on below folder /home/kka/out AK1521.txt Hi Welocme (3 Replies)
Discussion started by: asavaliya
3 Replies

5. Shell Programming and Scripting

Copy data form File A and Create File B

File A I have list of : ABCND1 ABCND2 ABCnd3 ABCnd4 I want file B like below Start+ S Pate=ABCND1 AAlo1 S Pate=ABCND1 Q1234 S Pate=ABCND1,P12345 (7 Replies)
Discussion started by: asavaliya
7 Replies

6. Solaris

Create file for group of data:

Hi folks, I have the following data.Any help is greatly appreciated. order File_name 7222245 7222245.pdf 7222245 7222245a.pdf 7222245 7222245b.pdf 7222245 7222245c.pdf 7222245 7222245d.pdf 7222250 ... (1 Reply)
Discussion started by: kumar444
1 Replies

7. UNIX for Advanced & Expert Users

Create a file and enter data in columns

Hello!!!!!!!! I have an issue regarding inserting records in a file columnwise.Is it possible using awk/nawk script? Example: File1: 1|AAA|25|2 5|qqe|20|7 4|wer|31|81 I need to create a second file in which data can be inserted in a columnwise manner i.e. File2: AAA|25|1|2... (1 Reply)
Discussion started by: abhijeet1409
1 Replies

8. UNIX for Dummies Questions & Answers

Create Year directory, date subdirectory and archive the file

Hi, After checking all the UNIX threads, I am able to come up with a solution so far. I am working on a shell script where it moves the files to a certain directory. The conditions to check are 1) Check if the file exists in the current directory. 2) Check if the destination directory... (2 Replies)
Discussion started by: madhunk
2 Replies

9. UNIX for Dummies Questions & Answers

how to know year create/modified files?

Dear all, how to know created file time, include the year created/last modified? if using #ls -al only month date and time Thank you. (5 Replies)
Discussion started by: blesets
5 Replies
Login or Register to Ask a Question

Featured Tech Videos