Sponsored Content
Top Forums Shell Programming and Scripting Writing to a log file incorrectly Post 302096790 by heprox on Saturday 18th of November 2006 04:37:25 PM
Old 11-18-2006
I got it to log correctly by having it validate the existance of the input file again before sending it to the "is_file_arrived" function:

Code:
#!/bin/ksh
######### Environment Setup #########
PATH=/gers/genret/menu/pub/sbin:/gers/genret/menu/pub/bin:/gers/genret/menu/pub/
mac:/gers/genret/menu/adm/sbin:/gers/genret/menu/adm/bin:/gers/genret/menu/adm/m
ac:/gers/genret/custom:/gers/genret/fix:/gers/genret/src_rev/fix:/gers/genret/op
t/path:/gers/genret/bin:/g/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/sbin:.
 
ORACLE_HOME=/gers/genret
ORACLE_SID=genret
export PATH
export ORACLE_HOME
export ORACLE_SID
 
########## Global Variables ##########
DATE=$(date +%m%d%y)
TIME=$(date +%H%M)
DatafilesDir="/gers/genret/datafiles"
PrintDir="/gers/genret/print"
TempDir="/gers/genret/tmp"
Logfile="/tmp/str_process_$DATE.log"
 
########## Function to Verify File is Completely Uploaded ###########
# Function : is_file_arrived file
# Arg(s)   : file = file to verify
# Output   : None
# Status   : 0 = yes file arrived, 1 = no
# Env.     : IFA_WAIT : interval (secs) for file size check (def=5)
#
 
is_file_arrived() {
   [ -z "$1" ] && return 1
   local file=$1
   arrived=1
   local size1 size2
   if [ -f "$file" -a -z "$(fuser $file 2> /dev/null)" ] ; then
      size1=$(ls -l $file 2>/dev/null | awk '{print $5}')
      sleep ${IFA_WAIT:-5}
      size2=$(ls -l $file 2>/dev/null | awk '{print $5}')
      [ ${size1:-1} -eq ${size2:-2} ] && arrived=0
   else
      log_it "Store file $file not done uploading at $TIME on $DATE"
   fi
   log_it "Arrived is $arrived"
   return $arrived
 
}
 
######### GERS Processing of File ###########
 
processFile ()
{
   local fileName=$1
   local fileExtension=$2
   local fileNewName="$DatafilesDir/str${fileExtension}.asc"
   local fileODIName="str${fileExtension}.pos"
   mv -Eignore $fileName $fileNewName
  #prepup $fileNewName $fileExtension
  #mv -Eignore  $PrintDir/$fileODIName $TempDir/$fileODIName
  #save2tmp $fileExtension
  #call_siu $fileExtension
   log_it "Store file $fileName processed at $TIME on $DATE"
}
 
########## Log File Function ###########
log_it()
{
  printf "%s\n" "$*" >> "$Logfile"
 
}
 
 
########## Main Processing ###########
 
nsec=1
while [[ "$(date +%H%M)" -lt 2340 ]]
do
   for fileName in $DatafilesDir/[Uu][Pp][Ll][Oo][Aa][Dd].[0-9][0-9][0-9][0-9]
   do
      fileExtension=${fileName#*.}
      if [ -f $fileName ]; then
         is_file_arrived "$fileName"
         if [ $arrived = 0 ]; then
            if [ ! -f "$TempDir/poll_$fileExtension.txt" ]; then
               nsec=1
               processFile $fileName $fileExtension
            else
               log_it "Store $fileExtension is already in process"
            fi
         fi
      fi
   done
   sleep $nsec
   case $nsec in
      1)   nsec=15;;
      15)  nsec=45;;
      45)  nsec=90;;
      90)  nsec=300;;
      300) nsec=600;;
      *) nsec=900;;
   esac
done

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Which Process is writing this Log file!!

Hello , Well I have some /tmp files which are growing very quickly..Can anyone suggest me a way to find which process is logging into this file :confused: ? Thanks very much in Advance!! Mohammed (2 Replies)
Discussion started by: Mohammed
2 Replies

2. UNIX for Advanced & Expert Users

writing the log file into syslog

Hi, we are trying to enable auditing for few oracle 9i database. and right now it writies into adump directory. As adump can be read/write by oracle user so could it be possible to write into syslog while oracle keeps writing to adump . thanks in advance. Pk (2 Replies)
Discussion started by: p4cldba
2 Replies

3. UNIX for Dummies Questions & Answers

Log File Writing and Reading

Hi all, I have the following shell script code which tries to sftp and writes the log into the log file. TestConnection () { echo 'Connection to ' $DESTUSERNAME@$DESTHOSTNAME $SETDEBUG if ]; then rm $SCRIPT ; fi touch $SCRIPT echo "cd" $REMOTEDIR >> $SCRIPT echo "quit" >>... (10 Replies)
Discussion started by: valluvan
10 Replies

4. Shell Programming and Scripting

Searching for Log / Bad file and Reading and writing to a flat file

Need to develop a unix shell script for the below requirement and I need your assistance: 1) search for file.log and file.bad file in a directory and read them 2) pull out "Load_Start_Time", "Data_File_Name", "Error_Type" from log file 4) concatinate each row from bad file as... (3 Replies)
Discussion started by: mlpathir
3 Replies

5. Shell Programming and Scripting

Format the output from sqlplus while writing to log file.

Hi I have developed bash script to connect to database and execute .sql files. I am logging some statements in to log file using echo. While logging I am adding the date in front of the log statements which makes sense. I am unable to add date in front of output from the sqlplus and sqlldr,... (8 Replies)
Discussion started by: murtymvvs
8 Replies

6. Shell Programming and Scripting

Totals in a file - incorrectly displaying

Afternoon, I have a script which creates/modifies data into a formatted csv. The trailer record should display 2 columns, the first is a static entry of "T" to identify it as a trailer record. The 2nd is a total of amounts in a column throughout the entire file. My total isn't displaying... (8 Replies)
Discussion started by: mcclunyboy
8 Replies

7. Shell Programming and Scripting

Need help in writing a script that do internal grep on a log file..and extract acct no's from it..

I need to write a script, which takes the input a log file and create output file with acct no's line by line from selected records with text like (in red) : 18:51:18 | 217863|Acct 0110855565|RC 17608| 16 Subs| 1596 UsgRecs| 2 Secs| 430 CPUms| prmis2:26213 <MoveUsage d aemon needs to run... (7 Replies)
Discussion started by: rkrish
7 Replies

8. UNIX for Dummies Questions & Answers

Syslog Messages from Remote Server are not writing to Log File Anymore

Hello All, Server: SUSE Linux Enterprise Server 11.3 (x86_64) Syslog-ng Version: syslog-ng 2.0.9 We have configured a Cisco router to send it's log messages to this server listed above. This has been working just perfectly for the last couple months, but we had never setup the log... (9 Replies)
Discussion started by: mrm5102
9 Replies

9. UNIX for Beginners Questions & Answers

Ls incorrectly says file not found in sftp.

Hi all When I sftp to an Oracle cloud server, to a folder where there are more than 10k files, and list a small subset of files, it works OK. When I try list more than 10k files, it says file not found. Example output below. (FYI ls/mget with 6k files works OK on this server) Has anyone... (3 Replies)
Discussion started by: pdinsdale
3 Replies

10. Shell Programming and Scripting

Large File masking incorrectly happening Ç delimeter issue

The OS version is Red Hat Enterprise Linux Server release 6.10 I have a script to mask some columns with **** in a data file which is delimeted with Ç , I am using awk for the masking , when I try to mask a small file the awk works fine and masks the required column , but when the file is... (6 Replies)
Discussion started by: LinuxUser8092
6 Replies
SYSTEMD.ENVIRONMENT-GENERATOR(7)			   systemd.environment-generator			  SYSTEMD.ENVIRONMENT-GENERATOR(7)

NAME
systemd.environment-generator - systemd environment file generators SYNOPSIS
/lib/systemd/system-environment-generators/some-generator /usr/lib/systemd/user-environment-generators/some-generator /run/systemd/system-environment-generators/* /etc/systemd/system-environment-generators/* /usr/local/lib/systemd/system-environment-generators/* /lib/systemd/system-environment-generators/* /run/systemd/user-environment-generators/* /etc/systemd/user-environment-generators/* /usr/local/lib/systemd/user-environment-generators/* /usr/lib/systemd/user-environment-generators/* DESCRIPTION
Generators are small executables that live in /lib/systemd/system-environment-generators/ and other directories listed above. systemd(1) will execute those binaries very early at the startup of each manager and at configuration reload time, before running the generators described in systemd.generator(7) and before starting any units. Environment generators can override the environment that the manager exports to services and other processes. Generators are loaded from a set of paths determined during compilation, as listed above. System and user environment generators are loaded from directories with names ending in system-environment-generators/ and user-environment-generators/, respectively. Generators found in directories listed earlier override the ones with the same name in directories lower in the list. A symlink to /dev/null or an empty file can be used to mask a generator, thereby preventing it from running. Please note that the order of the two directories with the highest priority is reversed with respect to the unit load path, and generators in /run overwrite those in /etc. After installing new generators or updating the configuration, systemctl daemon-reload may be executed. This will re-run all generators, updating environment configuration. It will be used for any services that are started subsequently. Environment file generators are executed similarly to unit file generators described in systemd.generator(7), with the following differences: o Generators are executed sequentially in the alphanumerical order of the final component of their name. The output of each generator output is immediately parsed and used to update the environment for generators that run after that. Thus, later generators can use and/or modify the output of earlier generators. o Generators are run by every manager instance, their output can be different for each user. It is recommended to use numerical prefixes for generator names to simplify ordering. EXAMPLES
Example 1. A simple generator that extends an environment variable if a directory exists in the file system # 50-xdg-data-dirs.sh #!/bin/bash # set the default value XDG_DATA_DIRS="${XDG_DATA_DIRS:-/usr/local/share/:/usr/share}" # add a directory if it exists if [[ -d /opt/foo/share ]]; then XDG_DATA_DIRS=/opt/foo/share:${XDG_DATA_DIRS} fi # write our output echo XDG_DATA_DIRS=$XDG_DATA_DIRS Example 2. A more complicated generator which reads existing configuration and mutates one variable # 90-rearrange-path.py #!/usr/bin/env python3 """ Proof-of-concept systemd environment generator that makes sure that bin dirs are always after matching sbin dirs in the path. (Changes /sbin:/bin:/foo/bar to /bin:/sbin:/foo/bar.) This generator shows how to override the configuration possibly created by earlier generators. It would be easier to write in bash, but let's have it in Python just to prove that we can, and to serve as a template for more interesting generators. """ import os import pathlib def rearrange_bin_sbin(path): """Make sure any pair of .../bin, .../sbin directories is in this order >>> rearrange_bin_sbin('/bin:/sbin:/usr/sbin:/usr/bin') '/bin:/sbin:/usr/bin:/usr/sbin' """ items = [pathlib.Path(p) for p in path.split(':')] for i in range(len(items)): if 'sbin' in items[i].parts: ind = items[i].parts.index('sbin') bin = pathlib.Path(*items[i].parts[:ind], 'bin', *items[i].parts[ind+1:]) if bin in items[i+1:]: j = i + 1 + items[i+1:].index(bin) items[i], items[j] = items[j], items[i] return ':'.join(p.as_posix() for p in items) if __name__ == '__main__': path = os.environ['PATH'] # This should be always set. # If it's not, we'll just crash, we is OK too. new = rearrange_bin_sbin(path) if new != path: print('PATH={}'.format(new)) Example 3. Debugging a generator SYSTEMD_LOG_LEVEL=debug VAR_A=something VAR_B="something else" /lib/systemd/system-environment-generators/path-to-generator SEE ALSO
systemd-environment-d-generator(8), systemd.generator(7), systemd(1), systemctl(1) systemd 237 SYSTEMD.ENVIRONMENT-GENERATOR(7)
All times are GMT -4. The time now is 01:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy