Scripting the process to edit a large file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Scripting the process to edit a large file
# 1  
Old 12-26-2008
Scripting the process to edit a large file

Hi,

I need to make a script to edit a file. File is a large file in below format

Version: 2008120101
;$INCLUDE ./abc/xyz/Delhi
;$INCLUDE ./abc/xyz/London
$INCLUDE ./abc/xyz/New York

First line in the file is version number which is in year,month,date and serial number format. Each time when file is changed current month,date and serial number is changed.

Other 3 lines with INCLUDE statements are point of focus, where only one line will be active at a time without comment. I need to have a script which can change the version number each time when any change is being made and if input parameter is London then line with reference to London should be uncommented and other two should be commented. Same way if input parameter is New York then line with new york should be uncommented and other two should be commented and version number should change. I hope I could clarify the situation and my requirement. I am not a core Unix guy, so thought to take genious people's help. Thanks a lot in advance.

Regards.
# 2  
Old 12-26-2008

Code:
city=$1
date=$( date +%Y%m%d )

awk -v date=$date -v city="$city" '
/^Version:/ {
  serial = substr( $2, 9 )
  olddate = substr( $2, 1, 8 )
  if ( date == olddate ) ++serial
  else serial = 1
  printf "Version: %d%02d\n", date, serial
  next
}
{ gsub(/^;/,"") }
index($0, city) { print; next }
{ print ";" $0 }
' "$FILE"

# 3  
Old 12-28-2008
Thanks Cfajohnson for your reply..

Only thing that is not happening is Version.. Version is not incrementing each time when I am executing the script. it is only changing when date is changing, but serial number should also increment each time script is executed and also please suggest one more thing, I need to do the change the original file. What do you suggest, what is the best way to make the changes in original file apart from moving file with mv.


Please help me
Regards.

Last edited by makkar4u; 12-28-2008 at 06:51 AM..
# 4  
Old 12-28-2008
Quote:
Originally Posted by makkar4u
Only thing that is not happening is Version.. Version is not incrementing each time when I am executing the script. it is only changing when date is changing, but serial number should also increment each time script is executed and also

Did you copy the script exactly as I posted it? It works for me, including incrementing the version.

Do you get any error messages?
Quote:
please suggest one more thing, I need to do the change the original file. What do you suggest, what is the best way to make the changes in original file apart from moving file with mv.

That is the best way to do it: send the output to a temporary file and mv or cp it to the original file.
# 5  
Old 01-16-2009
Quote:
Originally Posted by cfajohnson

Did you copy the script exactly as I posted it? It works for me, including incrementing the version.



Do you get any error messages?

That is the best way to do it: send the output to a temporary file and mv or cp it to the original file.


Thanks Johnson for your earlier replies, but stucked again at one point. Script is running fine. Problem is if my db file is like:



version: yyyymmddrr
aaa
bbb
ccc
Include city London
ddd
eee
fff
Include city Sydney
.
.
more lines
.
..


now in this kind of db file, script is putting comment on all lines. but I need to put comment only on lines with "Include" but at the same time uncomment line matching city variable...

One last thing, I need to use this script in kind of a failover process in live production environment on large db files with around 1000 entries.

Thanks a lot in advance.

Last edited by makkar4u; 01-16-2009 at 10:10 PM..
# 6  
Old 01-19-2009
Code:
dt=`date '+%Y%m%d'`
t=`ls -l a.txt | awk '{print substr($8,0,index($8,":")-1)}'`
res=`echo $dt$t`
echo $res
echo "Pls input the city name"
read city
sed -n "
1 {
	s/.*/Version: $res/
	p
}
2,$ {
/$city/ !{
 s/^/Comment: /
 p
}
/$city/ {
	p
}
}" a.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

TCL scripting: errorInfo=integer value too large to represent

The username is of the format : 123456789110000-1234@something.com With this below TCL procedure, I am trying add first and Sec Id and get third Id. I checked in online compiler and it seems to work and add. However, when I am running this in my lab, I get error as "integer value too large to... (3 Replies)
Discussion started by: Shaibal_bp
3 Replies

2. Shell Programming and Scripting

TCL scripting: errorInfo=integer value too large to represent

The username is of the format : 123456789110000-1234@something.com With this below TCL procedure, I am trying add first and Sec Id and get third Id. I checked in online compiler and it seems to work and add. However, when I am running this in my lab, I get error as "integer value too large to... (1 Reply)
Discussion started by: Shaibal_bp
1 Replies

3. Shell Programming and Scripting

Process multiple large files with awk

Hi there, I'm camor and I'm trying to process huge files with bash scripting and awk. I've got a dataset folder with 10 files (16 millions of row each one - 600MB), and I've got a sorted file with all keys inside. For example: a sample_1 200 a.b sample_2 10 a sample_3 10 a sample_1 10 a... (4 Replies)
Discussion started by: camor
4 Replies

4. Shell Programming and Scripting

How to edit a large file

Whenever I am trying to edit a file in unix with vi editor, I am getting the following error: <data> :Tmp file too large Is there any way that I can edit the file other than vi. Any help is really appreciated. Thanks (10 Replies)
Discussion started by: bobby1015
10 Replies

5. Red Hat

Fork wait in background process - large delay

hi all, We are trying to run a process in the background and in the process we call fork ;and wait for the child process to finish .We find that the died = wait(&status); happens after 10 seconds randomly and sometimes completes in time (within 1 sec) This behavior is seen only when the... (1 Reply)
Discussion started by: vishnu.priya
1 Replies

6. Shell Programming and Scripting

Edit a large file in place

:confused:Folks, I have a file with 50 million records having 2 columns. I have to do the below: 1. Generate some random numbers of a fixed length. 2. Replace the second column of randomly chosen rows with the random numbers. I tried using a little bit of perl to generate random numbers... (6 Replies)
Discussion started by: mvijayv
6 Replies

7. Shell Programming and Scripting

how to edit large file in unix

hi All, Plz let me know how to edit a file with 2000000 records. each record contains with 40 field seperated by |. i want modify 455487 record, but i am uable to edit this large file using vi editor in unix. plz let me know how to modify this file. Thanks in advance. -Bali Reddy (3 Replies)
Discussion started by: balireddy_77
3 Replies

8. Shell Programming and Scripting

what is the most effective way to process a large logfile?

I am dealing with a very large firewall logfile (more than 10G), the logfile like this *snip* Nov 9 10:12:01 testfirewall root: Nov 9 10:12:01 testfirewall root: 0:00:11 accept testfw01-hme0 >hme0 proto: icmp; src: test001.example.net; dst: abc.dst.net; rule: 1; icmp-type: 8;... (8 Replies)
Discussion started by: fedora
8 Replies

9. Shell Programming and Scripting

Help to edit a large file

I am trying to edit a file that has 33k+ records. In this file I need to edit each record that has a 'Y' in the 107th position and change the 10 fields before the 'Y' to blanks. Not all records have a 'Y' in the 107th field. ex: ... (8 Replies)
Discussion started by: jxh461
8 Replies

10. UNIX for Dummies Questions & Answers

how to edit large files using vi

How to edit large file using vi where you can't increase /usr/var/tmp anymore? (3 Replies)
Discussion started by: nazri
3 Replies
Login or Register to Ask a Question