The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
Google UNIX.COM


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Remove first N bytes and last N bytes from a binary file on AIX. naveendronavall Shell Programming and Scripting 1 05-24-2008 08:06 AM
How to delete initial N bytes from the starting of the file using sed? jockey007 Shell Programming and Scripting 3 11-19-2007 05:26 PM
How to delete N bytes from the starting of the file from a C function??? jockey007 High Level Programming 2 11-15-2007 07:25 PM
Rearrange bytes within a txt file yankee428 UNIX for Dummies Questions & Answers 4 06-16-2005 10:55 AM
Take a file from the system and put on tape and reset the file to 0 bytes JackieRyan26 UNIX for Dummies Questions & Answers 1 09-06-2001 02:08 PM

Reply
 
Submit Tools LinkBack Thread Tools Search this Thread Display Modes
  #1  
Old 04-12-2006
Registered User
 

Join Date: Jun 2003
Posts: 20
delete bytes from file

I'm doing a bit of hex editing with dd and I can replace values fairly simply. However, I've run across a situation where I need to delete bytes in the file and I'm not sure how to do that. For example:

Input file has:
1234567890

Output needs to be:
123abc90

I tried this:
printf 1234567890 > /tmp/test
printf abc | dd bs=1 seek=0x03 count=5 conv=notrunc of=/tmp/test

The output is:
123abc7890

Other things I've tried:
1) setting ibs and obs to different sizes
2) not using the notrunc option - that truncates everything after the info I'm inserting

The actual file I'm editing is a Mac OS X binary, so I can't actually use text manipulation - it has to be hex editing, which is why I'm using dd. I'm not stuck on using dd if something else will get the job done, but it's what I know. I only use Unix for a few scripting projects to make my life easier, so I'm probably just missing a common command that most people would know. Any suggestions?
Reply With Quote
Forum Sponsor
  #2  
Old 04-12-2006
Perderabo's Avatar
Unix Daemon
 

Join Date: Aug 2001
Location: Washington DC Area
Posts: 8,656
Code:
$ printf 1234567890 > test
$ dd if=test bs=1 count=3 of=test2
3+0 records in
3+0 records out
3 bytes transferred in 1 secs (3 bytes/sec)
$ printf abc >> test2
$ dd if=test bs=1 skip=8 seek=7 of=test2
2+0 records in
2+0 records out
2 bytes transferred in 1 secs (2 bytes/sec)
$ cat test2 ; echo
123abc90
$
Reply With Quote
  #3  
Old 04-12-2006
Registered User
 

Join Date: Jun 2003
Posts: 20
Thanks for the quick reply! That almost works for me. I had to change the seek to 6, however, because it leaves a null character after 123abc. It doesn't show up with cat, but a hexdump will show it. Anyways, thanks for the help!
Reply With Quote
  #4  
Old 04-12-2006
Registered User
 

Join Date: Jun 2003
Posts: 20
It's me again - I should have tested on the binary file. While this solution works, it takes a long time if the file is bigger than a few kb in size. The app I'm trying to patch is 11MB, so it's still going an hour later. I have a few of these patches to do and I don't have a week, unfortunately. Is there another faster way to destructively replace x bytes with y bytes if y<x?
Reply With Quote
  #5  
Old 04-12-2006
Perderabo's Avatar
Unix Daemon
 

Join Date: Aug 2001
Location: Washington DC Area
Posts: 8,656
Show the exact commands that you used.
Reply With Quote
  #6  
Old 04-13-2006
Registered User
 

Join Date: Jun 2003
Posts: 20
The actual code has 17 offsets to process, but for my test run, I just used two. Also, keep in mind it never made it through the first cycle. I know the cycle was working (the apptest2 file kept growing, kb-by-slow-kb), but it's just too slow considering the total app size is 11 MB.

Code:
offsets=(0x5766ee 0xb020ae)
length1=(8 8)
length2=(5 5)
replace=(HOCUS POKUS)

for ((i=0;$i<${#offsets[@]};i++)); do
# Enter in the correct value
    printf ${replace[$i]} | dd bs=1 count="${length2[$i]}" seek="${offsets[$i]}" conv=notrunc of=apptest
# Create the infile
    dd if=apptest bs=1 count="$((${offsets[$i]}+${length2[$i]}))" of=apptest2
# Splice infile into original
    dd bs=1 skip="$((${offsets[$i]}+${length1[$i]}))" seek="$((${offsets[$i]}+${length2[$i]}))" of=apptest2
# Replace original so we can cycle
    mv apptest2 apptest
done
I tried using the printf approach to appending the infile, but that changes the filetype of the end result, which ruins the application. I didn't know if I could use dd to append to the end, so I simply changed the order to enter the correct value first, then cut up the file. That part of the loop works in a fraction of a second. It's the infile (and I suspect the splicing, as well) that takes so long.
Reply With Quote
  #7  
Old 04-13-2006
Perderabo's Avatar
Unix Daemon
 

Join Date: Aug 2001
Location: Washington DC Area
Posts: 8,656
I don't understand this filetype ruined the application stuff. Nor do I understand why that code you show runs at all. My dd wil not accept hex input.

0x5766ee = 5727982 in decimal and I will use decimal throughout. I tried this stuff on a 7000000 byte file and it ran fairly quick. First we need a copy of our input file truncated to byte 5727981:
dd if=big1 bs=5727981 count=1 of=big1a

Now we want a copy of the input file with the first 5727989 bytes removed:
dd if=big1 bs=5727989 skip=1 of=big1c

You already know how to create the file big1b which contains the bytes of your patch. You should be able to cat these together:
cat big1a big1b big1c > big2

But this filetype business may bite you. If so try:
(dd if=big1a bs=1000000 ; dd if=big1b bs=1 ; dd if=big1c bs=1000000) | dd bs=1000000 of=big2
Reply With Quote
Google The UNIX and Linux Forums
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes




All times are GMT -7. The time now is 10:54 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008. All Rights Reserved.Ad Management by RedTyger Visit The Complex Event Processing Blog

Content Relevant URLs by vBSEO 3.2.0