Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help to edit a large file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help to edit a large file
# 1  
Old 05-01-2003
Question Help to edit a large file
I am trying to edit a file that has 33k+ records. In this file I need to edit each record that has a 'Y' in the 107th position and change the 10 fields before the 'Y' to blanks. Not all records have a 'Y' in the 107th field.

ex:

.....................................2222222222Y..................

to

..................................... Y..................


Thanks in advance
# 2  
Old 05-01-2003
What kind of file is this?
Code:
sed 's/..........Y/          Y/g' someFile > TMP_00
mv TMP_00 someFile

will do it except that it doesn't check for the 107th position.. could there be a Y at other places on each line too, or just in the 107th position?
# 3  
Old 05-02-2003
I figured out a solution..

Code:
while read LINE
do
  theTest=`echo $LINE | awk '{print substr ($0, 107, 1)}'`
  if [[ $theTest == "Y" ]]; then
    echo $LINE | awk '{print substr ($0, 1, 96) "          Y" substr ($0, 108, length ($0) - 107)}' >> newFile
  else
    echo $LINE >> newFile
  fi
done < someFile

mv newFile someFile

# 4  
Old 05-02-2003
Perl's extensions to the RE vocabulary are helpful here ([b]WARNING: untested code[/b):
Code:
perl -pe '/^(.{96}).{10}(Y.*)/ && do { $_ = $1 . " " x 10 . $2 }'

This takes adavntage of the -p option, which prints $_ after processing each record.

Or you can combine oombera's separate invocations of awk into one:
Code:
awk 'substr($0,107,1) == "Y" { $0 = substr($0,1,96) "          " substr($0,107,length($0) - 107) }
{print}' someFile > newFile
mv newFile someFile

Awk is still record length limited AFIAK, so if your records are very long (> 8K ?), you may not be able to read entire lines (I was bitten by this earlier this week).
# 5  
Old 05-02-2003
In oombera's solution, each place you see $LINE (lines 3, 5, 7) you should probably change it to "$LINE" since otherwise, consecutive blanks in your records will change the records in an unwanted way.
# 6  
Old 05-02-2003
criglerj, that's cool - i can't believe the code can be that shortened! And I should've remembered the quotes.. they protect spaces, but I suppose potentially other special characters too..

Just a couple typos..
Code:
"          " should be "          Y"
substr($0,107,len... should be substr($0,108,len...

# 7  
Old 05-08-2003
I think I got it right the first time. When the OP wrote "edit each record that has a 'Y' in the 107th position", I guessed the positions started from one, which is the way awk does it. But I stand by my blank string: You want to leave the Y where it is, i.e., you keep characters 1..96, change 97..106 to blanks, then keep 107..end. Char 107 is the "Y" which is left intact. But if the OP's count starts from zero, then all the positions get adjusted by one.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Edit file

Hi All, I have file with 200K Records and each line with 400 character. I need to edit the some part of the file. For example, i need to edit character from 115 to 125, 135to 145 and 344 to 361 Can you please anyone help me to do this? Regards, (1 Reply)
Discussion started by: balasubramani04
1 Replies

2. Shell Programming and Scripting

How to edit a large file

Whenever I am trying to edit a file in unix with vi editor, I am getting the following error: <data> :Tmp file too large Is there any way that I can edit the file other than vi. Any help is really appreciated. Thanks (10 Replies)
Discussion started by: bobby1015
10 Replies

3. Shell Programming and Scripting

edit file

I have a file containing dates like below 2010 1 02 2010 2 01 2010 3 05 i want the dates to be like below 20100102 20100201 20100305 i tired using awk '{printf "%s%02s%02s",$1,$2,$3}' But it does not work,it puts all the dates in one line,i want them in seperate lines like the... (6 Replies)
Discussion started by: tomjones
6 Replies

4. Shell Programming and Scripting

Script to Edit the file content and create new file

I have a requirement, which is as follows *. Folder contains list of xmls. Script has to create new xml files by copying the existing one and renaming it by appending "_pre.xml" at the end. *. Each file has multiple <Name>fileName</Name> entry. The script has to find the first occurance of... (1 Reply)
Discussion started by: sudesh.ach
1 Replies

5. Shell Programming and Scripting

Scripting the process to edit a large file

Hi, I need to make a script to edit a file. File is a large file in below format Version: 2008120101 ;$INCLUDE ./abc/xyz/Delhi ;$INCLUDE ./abc/xyz/London $INCLUDE ./abc/xyz/New York First line in the file is version number which is in year,month,date and serial number format. Each... (5 Replies)
Discussion started by: makkar4u
5 Replies

6. Shell Programming and Scripting

Edit value in File

I have a file oratab with entry like this SCADAG:/esitst1/oracle/product/9.2.0.8:Y I am trying to discover a way to change the 9.2.0.8 part of this to something like 10.2.0.4 as part of an upgrade script. I have tried cat /etc/oratab >>/tmp/oratab... (1 Reply)
Discussion started by: sewood
1 Replies

7. Shell Programming and Scripting

Edit a large file in place

:confused:Folks, I have a file with 50 million records having 2 columns. I have to do the below: 1. Generate some random numbers of a fixed length. 2. Replace the second column of randomly chosen rows with the random numbers. I tried using a little bit of perl to generate random numbers... (6 Replies)
Discussion started by: mvijayv
6 Replies

8. Shell Programming and Scripting

how to edit large file in unix

hi All, Plz let me know how to edit a file with 2000000 records. each record contains with 40 field seperated by |. i want modify 455487 record, but i am uable to edit this large file using vi editor in unix. plz let me know how to modify this file. Thanks in advance. -Bali Reddy (3 Replies)
Discussion started by: balireddy_77
3 Replies

9. UNIX for Dummies Questions & Answers

Edit the File

Hello Everyone I am new to this forum. I am having a requirement to edit the file(the file is having some sql code). And this file is in my colleagues login. This is readonly Now I would like to edit this file. In which way can I do this? (1 Reply)
Discussion started by: pradkumar
1 Replies

10. UNIX for Dummies Questions & Answers

how to edit large files using vi

How to edit large file using vi where you can't increase /usr/var/tmp anymore? (3 Replies)
Discussion started by: nazri
3 Replies
Login or Register to Ask a Question