Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days
# 8  
Old 07-11-2016
Are you sure the output posted comes from that script? It nowhere prints the username, a comma, a space, the date.
For that error message, is it possible that either variable name has an unprinting control char in it?

BTW, TODAYS_DATE is undefined in that script (which doesn't hurt, there's just an empty string printed)
And, it's not too wise to mv a temp file to the actual input file, overwriting it, as you may pull the rug out from under your own feet...
# 9  
Old 07-11-2016
Anyone can provide any hint as to what could be wrong? Thanks

---------- Post updated at 02:45 PM ---------- Previous update was at 02:37 PM ----------

Sorry here's the actual script, yes I still get the same error.
Code:
#!/usr/bin/ksh
while IFS="_ " read LUSER LDATE
do
  if [ $(( $(date +%s) - $(date +%s -d"${LDATE:2:2}/${LDATE:0:2}/${LDATE: -4}") )) -ge 4320000 ]
  then
    echo "$LUSER,  $LDATE"
    grep -v "$LUSER" test > test.new && mv test.new test
  fi
done < test

Code:
 /dbashare/ToolBox/UserTermination/MonthlyDeletionoracle@11204_home>./remove.ksh
./remove.ksh[9]: : bad substitution
./remove.ksh[9]:  1468262786 -  : unexpected `end of expression'
 /dbashare/ToolBox/UserTermination/MonthlyDeletionoracle@11204_home>

---------- Post updated at 03:25 PM ---------- Previous update was at 02:45 PM ----------

I can confirm this one more time that it is caused by adding
Code:
#!/usr/bin/ksh

on top and works fine when I remove this line. However I use this as a function for a lrger ksh script and get the same error when I integrate it with my larger script.

Last edited by RudiC; 07-11-2016 at 05:14 PM..
# 10  
Old 07-11-2016
  • You are removing lines from test inside the loop when it is also the input file for the loop. That is not right.
  • What version of ksh are you using ? If it is ksh88 then the substitutions will not work.
  • and what is your OS?
This User Gave Thanks to Scrutinizer For This Post:
# 11  
Old 07-11-2016
Hi everyone, thanks for your help. A collegue helped me and it works with this code

Code:
#!/bin/ksh
echo
cat test |
while IFS="_ " read LUSER LDATE
do
        Day=`echo $LDATE | cut -c1,2`
        Mon=`echo $LDATE | cut -c3,4`
        Year=`echo $LDATE | cut -c5-8`
        Date=`echo $Mon/$Day/$Year`
        if [[ $(( $(date +%s) - $(date +%s -d${Date}) )) -ge 4320000 ]] ; then
        #echo "$LUSER, $LDATE"i
        echo "$LUSER" >> userRemovalList
        grep -v "$LUSER" test > temp && mv temp test
        fi
done
chmod 755 userRemovalList

# 12  
Old 07-12-2016
You should avoid changing the "test" file when cat could still be reading it, this will become evident if your input file is large and cat hasn't buffered the whole thing before the script starts changing it.

Also, if you have a large file to process this should run a bit quicker (not calling external cut command 3 times per line):

Code:
#!/bin/ksh

TSEC=$(date +%s)
while read LINE
do
   LDATE=${LINE#*_}
   YEAR=${LDATE#????}
   MDAY=${LDATE%????}
   DAY=${MDAY%??}
   MON=${MDAY#??}
   [[ $(( TSEC - $(date -d $MON/$DAY/$YEAR +%s) )) -lt 4320000 ]] &&
      echo "$LINE"
done < test > test.$$
mv test.$$ test

This User Gave Thanks to Chubler_XL For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to read the output of a command line by line and pass it as a variable?

Hi, I have some 2000 names in a table like below. Java Oracle/SQL ANSI SQL SQL,DWH,DB DB&Java And by using for loop in my code i am able to get a single word but if there is any special character or space then it is considering as a next line. I have to execute the below queries in... (10 Replies)
Discussion started by: Samah
10 Replies

2. UNIX for Beginners Questions & Answers

Subscribers with Date 90 days older than current date

I have to display only those subscribers which are in "unconnected state" and the date is 90 days older than today's date. Below command is used for this purpose: cat vfsubscriber_20170817.csv | sed -e 's/^"//' -e '1d' | nawk -F '",' '{if ( (substr($11,2,4) == 2017) && ( substr($11,2,8) -lt... (1 Reply)
Discussion started by: dia
1 Replies

3. Shell Programming and Scripting

Remove lines older than 30 days

Hi Experts/Gurus, Is there a way to remove lines in a file that are older than x days (i.e. 30 days) based on the date stamp in the first column? Example. $ date Sat Jan 11 14:12:06 EDT 2014 $cat sample.txt 10-10-2013 09:00:01 AM|Line test 1234567 16-10-2013 08:30:00 AM|Line test... (6 Replies)
Discussion started by: brichigo
6 Replies

4. Shell Programming and Scripting

How to read a two files, line by line in UNIX script and how to assign shell variable to awk ..?

Input are file and file1 file contains store.bal product.bal category.bal admin.bal file1 contains flip.store.bal ::FFFF:BADC:CD28,::FFFF:558E:11C5,6,8,2,1,::FFFF:81C8:CA8B,::FFFF:BADC:CD28,1,0,0,0,::FFFF:81C8:11C5,2,1,0,0,::FFFF:81DC:3111,1,0,1,0 store.bal.... (2 Replies)
Discussion started by: veeruasu
2 Replies

5. Shell Programming and Scripting

Remove files older than 2 days.

Hi All, I am new to the scripting and using solaris 10 OS. Please suggest me from the below script which modifications need to be done to delete the files more that 2days older. Current script is deleting existing file. # Remove old explorer runs if needed DIR=`dirname ${EXP_TARGET}` if ... (2 Replies)
Discussion started by: Navkreddy
2 Replies

6. UNIX for Dummies Questions & Answers

Parsing file, reading each line to variable, evaluating date/time stamp of each line

So, the beginning of my script will cat & grep a file with the output directed to a new file. The data I have in this file needs to be parsed, read and evaluated. Basically, I need to identify the latest date/time stamp and then calculate whether or not it is within 15 minutes of the current... (1 Reply)
Discussion started by: hynesward
1 Replies

7. Shell Programming and Scripting

How to read a file line by line and store it in a variable to execute a program ?

Hello, I am quite new in shell scripting and I would like to write a little scritp to run a program on some parameters files. all my parameters files are in the same directory, so pick them up with ls *.para >>dirafter that I have a dir file like that: param1.para param2.para etc... I... (2 Replies)
Discussion started by: shadok
2 Replies

8. Shell Programming and Scripting

read a file line by line in ksh

Hi, In ksh we use 'while read line' statement to read a file line by line. In my input file I have 5 spaces appended at the end of each line. When I use while read line statement it chops off the spaces at the end of each line Inp.txt aaaa<five spaces> bbbb<five spaces> cccc<five spaces> ... (3 Replies)
Discussion started by: chella
3 Replies

9. Shell Programming and Scripting

sed remove last 10 characters of a line start from 3rd line

hello experts, I need a sed command that remove last 10 characters of a line start from 3rd line. any suggestions? Thanks you (7 Replies)
Discussion started by: minifish
7 Replies

10. Shell Programming and Scripting

SED help (remove line::parse again::add line)

Aloha! I have just over 1k of users that have permissions that they shouldn't under our system. I need to parse a provided list of usernames, check their permissions file, and strip the permissions that they are not allowed to have. If upon the permissions strip they are left with no permissions,... (6 Replies)
Discussion started by: Malumake
6 Replies
Login or Register to Ask a Question