Editing big file in UNIX


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Editing big file in UNIX
# 8  
Old 08-17-2014
It does not work. I mean it does not change the character as requried

Code:
ISA*00* *00* *08*92 *08*123*030627*1304*U*00*00*0*P*>~ has nothing valid to change


Last edited by Don Cragun; 08-17-2014 at 04:22 AM.. Reason: Add CODE tags.
# 9  
Old 08-17-2014
It works fine when I try it. Please show us the output from the command:
Code:
head -n 1 hugefile|od -c

The sample you have shown us has the character to be changed as the 51st character on the 1st line of hugefile. Your response to my earlier questions said it was the 103rd character on the line. (Of course you also said that your 3GB file has a total length of 106 bytes, so we don't know what to believe.)

Something doesn't add up here, and your refusal to use CODE tags when displaying your sample input and output may be a contributing factor.

I have another script that might work, but it doesn't make any sense posting it if I can't get a sample to use to test it that bears some resemblance to the first few lines of your huge file.
# 10  
Old 08-23-2014
Thanks Don...In my sample file it 51 character and in my original data it 103 character.
Bottom line, it has to be changed from P to T or from T to P. Also as I said file size in Giga bytes. could you please post your other script ? I will give a try.
I will post the current script in my next post
# 11  
Old 08-23-2014
I don't write a lot of interactive awk scripts, but this seems to do what you want (as long as your version of awk supports the nextfile command. If awk on your system doesn't support nextfile and the character you want to change is in a fixed position, Corona688's suggestion (modified to loop across a list of files and to use whatever character position is in your real data (instead of 51 that was in your sample data)) would be a better approach.

The script below uses field 16 (which you refer to as ISA segment 15), no matter what character number that is within a line. If it finds a "P" or a "T", it tells you what it found and asks if you want to change it, otherwise it tells you what it found but does not change anything. In either case, it then moves on to the next file until all files you provided as operands are processed:

Code:
#!/bin/ksh
IAm=${0##*/}
if [ $# -lt 1 ]
then	printf 'Usage: %s pathname...\n' "$IAm"
	exit 1
fi
awk -F'*' '
BEGIN {	OFS = "*" }
{	old = $16
	new = (old == "P") ? "T" : (old == "T") ? "P" : old
	if(old != new) {
		printf("Change \"%s\" to \"%s\" in %s: (Y/N) ", old, new,
			FILENAME)
		getline resp < "/dev/tty"
		if(toupper(resp) ~ /^Y/) {
			$16 = new
			cmd = sprintf("dd of=\"%s\" conv=notrunc", FILENAME)
			printf("Will run commmand: %s\n", cmd)
			print | cmd
			close(cmd)
			# Clear resp here.  If user enters EOF on next round,
			# we do not want this response to be interpreted as a
			# blanket approval for future files...
			resp = ""
		} else	new = old
	}
	if(old == new) printf("\"%s\" in %s not changed.\n", old, FILENAME)
	nextfile
}' "$@"

It was written and tested using the Korn shell, but will work with any shell that handles POSIX-required variable expansions (such as bash).

If you are trying this on a Solaris system, I don't think it will work. I don't think /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk support nextfile; and /bin/awk or /usr/bin/awk on Solaris systems can't handle this script.
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. HP-UX

Editing a UNIX file in Hexadecimal format

Hi, I am a newbie to the UNIX world. I am asked to edit the file in hexadecimal format and save it. Later I should be able to print the file in char mode. please anyone tell me how to do that mostly using VI editor. (1 Reply)
Discussion started by: vkudire
1 Replies

2. UNIX for Advanced & Expert Users

best method to compare 2 big files in unix

Hi , I have a requirement to compare 2 files which can contain 40 million or more records and more than 20 fields to compare . Currently I am using awk scripting , and since awk has a memory issue, I am not able to process file more than 10 million records. Any suggestions or pointers to... (7 Replies)
Discussion started by: rashmisb
7 Replies

3. UNIX for Dummies Questions & Answers

How big is too big a config.log file?

I have a 5000 line config.log file with several "maybe" errors. Any reccomendations on finding solvable problems? (2 Replies)
Discussion started by: NeedLotsofHelp
2 Replies

4. Shell Programming and Scripting

Editing Binary Files in Unix

Hi, Is there a way to edit BINARY files in Unix. Or even are there any commands (shellscript/perl) through which I can replace all the occurences of a string inside a BINARY file with another string ?? (1 Reply)
Discussion started by: cool.aquarian
1 Replies

5. UNIX for Dummies Questions & Answers

problem editing big file in vi

I have a big file, which vi opens it with message not sufficient space with file system. I am not adding any data in the file but changing some values within. To make these changes effective, it asks for forced write (w!), even after doing this, I see this particular record, change is not... (4 Replies)
Discussion started by: videsh77
4 Replies

6. SCO

File Editing and Printing in Unix

Hi, Can some one provide me with, some good links containing help for file editing and printing in unix. Regards, Muhammad Tayyab Shereen Motor Co. Kuwait (1 Reply)
Discussion started by: tayyabq8
1 Replies

7. UNIX for Dummies Questions & Answers

How to view a big file(143M big)

1 . Thanks everyone who read the post first. 2 . I have a log file which size is 143M , I can not use vi open it .I can not use xedit open it too. How to view it ? If I want to view 200-300 ,how can I implement it 3 . Thanks (3 Replies)
Discussion started by: chenhao_no1
3 Replies

8. UNIX for Dummies Questions & Answers

editing unix files on NT

i currently am using a unix server and NT pc. i have downloaded a ziped file that should explode into 3 seperate unix based files, however when i unzip it using Alading Expander it displays only 1. This exploded version contains all 3 files ( you can scroll down when viewing the file and see the... (1 Reply)
Discussion started by: pixelmonkey
1 Replies
Login or Register to Ask a Question