Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Processing 1 match string from grep at a time

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Processing 1 match string from grep at a time
# 1  
Old 06-12-2012
Processing 1 match string from grep at a time
hi all,

i have a few log files with dates that are incorrrect (please don't ask me why). i need to add 2852 days, 16 hours, and 21 minutes (246471660 seconds) to each of these dates to correct them. i need to write to new "test2.txt" file that corrects the dates found by grep and yet have it still maintain all other info from the original "test.txt".

right now date_configurator.sh seems to find all instances at once.
it also prints the old $date as well as the $fixed_date to test2.txt.
can someone school me on what i can do to fix?

info below. thanks!!!!
Code:
$ cat test.txt 
# the actual log files hold lines that DO NOT include dates
# as i said above the lines that DO NOT include dates also must be 
# maintained across to test2.txt 
2004-08-14 18:49:43 [2e0-00000fe0] LOG asconfigurator is exiting normally
2003-08-14 18:49:43 [2e0-00000fe0] LOG asconfigurator is exiting normally

$ cat date_configurator.sh 
#!/bin/bash -x

# get $date
grep -o "200[3-4]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]" test.txt | while read date
do
      # format $date into epoch time
      bad_date=`date -j -f "%Y-%m-%d %H:%M:%S" "${date}" "+%s"`
      # add 246471660 seconds to $bad_date
      adj_date=`expr ${bad_date} + 246471660`
      # re-format $adj_date back to original format of $date
      fixed_date=`date -j -f "%s" "${adj_date}" "+%Y-%m-%d %H:%M:%S"`
      # replace $date with $fixed_date
      sed "s/${date}/${fixed_date}/" <test.txt >>test2.txt
done

$ ./date_configurator.sh 
+ grep -o '200[3-4]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]' test.txt
+ read date
++ date -j -f '%Y-%m-%d %H:%M:%S' '2004-08-14 18:49:43' +%s
+ bad_date=1092523783
++ expr 1092523783 + 246471660
+ adj_date=1338995443
++ date -j -f %s 1338995443 '+%Y-%m-%d %H:%M:%S'
+ fixed_date='2012-06-06 11:10:43'
+ sed 's/2004-08-14 18:49:43/2012-06-06 11:10:43/'
+ read date
++ date -j -f '%Y-%m-%d %H:%M:%S' '2003-08-14 18:49:43' +%s
+ bad_date=1060901383
++ expr 1060901383 + 246471660
+ adj_date=1307373043
++ date -j -f %s 1307373043 '+%Y-%m-%d %H:%M:%S'
+ fixed_date='2011-06-06 11:10:43'
+ sed 's/2003-08-14 18:49:43/2011-06-06 11:10:43/'
+ read date

$ cat test2.txt 
2012-06-06 11:10:43 [2e0-00000fe0] LOG asconfigurator is exiting normally
2003-08-14 18:49:43 [2e0-00000fe0] LOG asconfigurator is exiting normally
2004-08-14 18:49:43 [2e0-00000fe0] LOG asconfigurator is exiting normally
2011-06-06 11:10:43 [2e0-00000fe0] LOG asconfigurator is exiting normally

Last edited by zaxxon; 06-12-2012 at 11:55 AM.. Reason: Code tags & Indention
# 2  
Old 06-12-2012
What's your system? Date math is easy on some systems and hideous on others.

You can't grep for it if you want to keep all lines.
# 3  
Old 06-12-2012
While I wait, here's how to read all lines, not just ones you want to fix, and one way to match them later without grep:
Code:
while read DATE TIME REST
do
        case "$DATE $TIME" in
        200[3-4]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9])
                # Date transformation things, should alter DATE and TIME
                ;;
        *)     # Do nothing
                ;;
        esac

        echo "$DATE $TIME $REST"
done < input > output

# 4  
Old 06-12-2012
running Mac OS X version 10.6.8
using case makes sense. i'll try it and let you know what happens.
thank you!

---------- Post updated at 11:31 AM ---------- Previous update was at 11:08 AM ----------

Code:
$ cat date_configurator.sh 
#!/bin/bash -x

while read DATE TIME REST
do
	case "$DATE $TIME" in
	200[3-4]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9])
		# format $DATE $TIME into epoch time
		BAD_DATE=`date -j -f "%Y-%m-%d %H:%M:%S" "${DATE} ${TIME}" "+%s"`
		# add 246471660 seconds to $BAD_DATE
		ADJ_DATE=`expr ${BAD_DATE} + 246471660`
		# re-format $ADJ_DATE back to original format of $DATE $TIME
		FIXED_DATE=`date -j -f "%s" "${ADJ_DATE}" "+%Y-%m-%d %H:%M:%S"`
		# replace $DATE $TIME with $FIXED_DATE
		sed "s/${DATE}/${FIXED_DATE}/"
		;;
	*)	# Do nothing
		;;
	esac

	echo "$DATE $TIME $REST"
done < test.txt > test2.txt




$ ./date_configurator.sh 
./date_configurator.sh: line 6: syntax error near unexpected token `[0-9][0-9]:[0-9][0-9]:[0-9][0-9]'
./date_configurator.sh: line 6: `	200[3-4]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9])'

---------- Post updated at 03:41 PM ---------- Previous update was at 11:31 AM ----------

after much research and toil and with Corona688's insight, i have a working script.


Code:
#!/bin/bash

echo "Enter file name that contains corrupt dates and press [ENTER]"
read file_name
output=`echo "$file_name"_edited`

while read DATE TIME REST
do
if [[ "$DATE" =~ 200[3-4]-[0-9][0-9]-[0-9][0-9] ]]
then
  BAD_DATE=`date -j -f "%Y-%m-%d %H:%M:%S" "${DATE} ${TIME}" "+%s"`
  ADJUSTED_DATE=`expr ${BAD_DATE} + 246471660`
  FIXED_DATE=`date -j -f "%s" "${ADJUSTED_DATE}" "+%Y-%m-%d %H:%M:%S"`
  echo "$FIXED_DATE $REST" >> $output
else
  echo "$DATE $TIME $REST" >> $output
fi
done < $file_name

thanks again!
# 5  
Old 06-12-2012
That looks pretty good actually, though you have some useless use of backticks in there. I'm guessing you did this to get around the problem of telling the shell where a variable name ends -- better to use { } for that.

Code:
output="${file_name}_edited"

This User Gave Thanks to Corona688 For This Post:
cilantrocado
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep command to search a regular expression in a line an only print the string after the match

Hello, one step in a shell script i am writing, involves Grep command to search a regular expression in a line an only print the string after the match an example line is below /logs/GRAS/LGT/applogs/lgt-2016-08-24/2016-08-24.8.log.zip:2016-08-24 19:12:48,602 ERROR... (9 Replies)
Discussion started by: Ramneekgupta91
9 Replies

2. Shell Programming and Scripting

Need to print the next word from the same line based on grep string condtion match.

I need to fetch particular string from log file based on grep condition match. Actual requirement is need to print the next word from the same line based on grep string condtion match. File :Java.lanag.xyz......File copied completed : abc.txt Ouput :abc.txt I have used below... (5 Replies)
Discussion started by: siva83
5 Replies

3. UNIX for Dummies Questions & Answers

Real time processing

Hi Not sure if this can be achieved by unix , but still would like to know if there is any way by which I can do the below given logic cat sam1 > out1 cat sam2 > out2 when either one of this finished the the next file shd be written in that file, meaning cat sam3 >> out1/out2... (2 Replies)
Discussion started by: Sri3001
2 Replies

4. IP Networking

Processing time

I want to know the processing time taken by a node.example suppose a node ges a rreq...then it searched through it's table to see if it has a fresh route or not.I want to know this search time...is their any function available for doing this in ns2 or in glomosim.Any help is highly appreciated ... (1 Reply)
Discussion started by: prashantgolu
1 Replies

5. Shell Programming and Scripting

grep regex, match exact string which includes "/" anywhere on line.

I have a file that contains the 2 following lines (from /proc/mounts) /dev/sdc1 /mnt/backup2 xfs rw,relatime,attr2,noquota 0 0 /dev/sdb1 /mnt/backup xfs rw,relatime,attr2,noquota 0 0 I need to match the string in the second column exactly so that only one result is returned, e.g. > grep... (2 Replies)
Discussion started by: jelloir
2 Replies

6. Shell Programming and Scripting

exact string match ; search and print match

I am trying to match a pattern exactly in a shell script. I have tried two methods awk '/\<mpath${CURR_MP}\>/{print $1 $2}' multipath perl -ne '/\bmpath${CURR_MP}\b/ and print' /var/tmp/multipath Both these methods require that I use the escape character. I am guessing that is why... (8 Replies)
Discussion started by: bash_in_my_head
8 Replies

7. Shell Programming and Scripting

Need help to grep for a title match and then make some queries after the match

Here is the sample of my file address.txt Address 1 1234 Drive way New Orleans, LA Zipcode :- 12345 Address 2 4567 Spring way Chicago, IL Zipcode :- 67890 I would like to grep for an Address title (Ex :- Address 2) , then get its zipcode and echo both in a single line. Ex :- ... (3 Replies)
Discussion started by: leo.maveriick
3 Replies

8. UNIX for Dummies Questions & Answers

Regex to match when input is not a certain string (can't use grep -v)

Hey everyone, Basically, all I'm looking for is a way to regex for not a certain string. The regex I'm looking to avoid matching is: D222 i.e. an equivalent of: awk '!/D222/' The problem is that I use this in the following command in a Bash script: ls ${source_directory} | awk... (1 Reply)
Discussion started by: kdelok
1 Replies

9. Shell Programming and Scripting

Appending string to match pattern (data processing)

Hello i have go the following result from performing 2 testing using the same file. I have used unix script to extract the result because the files are many as shown below. 01_gravity.f.tcov 7 3 42.86 02_gravity.f.tcov 9 4 80.86... (4 Replies)
Discussion started by: ganiel24
4 Replies

10. Shell Programming and Scripting

grep to show date/time of file the string was found in.

I've seen several examples of grep showing the filename the string was found in, but what I really need is grep to show the file details in long format (like ls -l would). scenario is: grep mobile_number todays_files This will show me the string I'm after & which files they turn up in, but... (2 Replies)
Discussion started by: woodstock
2 Replies
Login or Register to Ask a Question