delete bytes from file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting delete bytes from file
# 1  
Old 04-12-2006
delete bytes from file

I'm doing a bit of hex editing with dd and I can replace values fairly simply. However, I've run across a situation where I need to delete bytes in the file and I'm not sure how to do that. For example:

Input file has:
1234567890

Output needs to be:
123abc90

I tried this:
printf 1234567890 > /tmp/test
printf abc | dd bs=1 seek=0x03 count=5 conv=notrunc of=/tmp/test

The output is:
123abc7890

Other things I've tried:
1) setting ibs and obs to different sizes
2) not using the notrunc option - that truncates everything after the info I'm inserting

The actual file I'm editing is a Mac OS X binary, so I can't actually use text manipulation - it has to be hex editing, which is why I'm using dd. I'm not stuck on using dd if something else will get the job done, but it's what I know. I only use Unix for a few scripting projects to make my life easier, so I'm probably just missing a common command that most people would know. Any suggestions?
# 2  
Old 04-12-2006
Code:
$ printf 1234567890 > test
$ dd if=test bs=1 count=3 of=test2
3+0 records in
3+0 records out
3 bytes transferred in 1 secs (3 bytes/sec)
$ printf abc >> test2
$ dd if=test bs=1 skip=8 seek=7 of=test2
2+0 records in
2+0 records out
2 bytes transferred in 1 secs (2 bytes/sec)
$ cat test2 ; echo
123abc90
$

# 3  
Old 04-12-2006
Thanks for the quick reply! That almost works for me. I had to change the seek to 6, however, because it leaves a null character after 123abc. It doesn't show up with cat, but a hexdump will show it. Anyways, thanks for the help!
# 4  
Old 04-12-2006
It's me again - I should have tested on the binary file. While this solution works, it takes a long time if the file is bigger than a few kb in size. The app I'm trying to patch is 11MB, so it's still going an hour later. I have a few of these patches to do and I don't have a week, unfortunately. Smilie Is there another faster way to destructively replace x bytes with y bytes if y<x?
# 5  
Old 04-12-2006
Show the exact commands that you used.
# 6  
Old 04-13-2006
The actual code has 17 offsets to process, but for my test run, I just used two. Also, keep in mind it never made it through the first cycle. I know the cycle was working (the apptest2 file kept growing, kb-by-slow-kb), but it's just too slow considering the total app size is 11 MB.

Code:
offsets=(0x5766ee 0xb020ae)
length1=(8 8)
length2=(5 5)
replace=(HOCUS POKUS)

for ((i=0;$i<${#offsets[@]};i++)); do
# Enter in the correct value
    printf ${replace[$i]} | dd bs=1 count="${length2[$i]}" seek="${offsets[$i]}" conv=notrunc of=apptest
# Create the infile
    dd if=apptest bs=1 count="$((${offsets[$i]}+${length2[$i]}))" of=apptest2
# Splice infile into original
    dd bs=1 skip="$((${offsets[$i]}+${length1[$i]}))" seek="$((${offsets[$i]}+${length2[$i]}))" of=apptest2
# Replace original so we can cycle
    mv apptest2 apptest
done

I tried using the printf approach to appending the infile, but that changes the filetype of the end result, which ruins the application. I didn't know if I could use dd to append to the end, so I simply changed the order to enter the correct value first, then cut up the file. That part of the loop works in a fraction of a second. It's the infile (and I suspect the splicing, as well) that takes so long.
# 7  
Old 04-13-2006
I don't understand this filetype ruined the application stuff. Nor do I understand why that code you show runs at all. My dd wil not accept hex input.

0x5766ee = 5727982 in decimal and I will use decimal throughout. I tried this stuff on a 7000000 byte file and it ran fairly quick. First we need a copy of our input file truncated to byte 5727981:
dd if=big1 bs=5727981 count=1 of=big1a

Now we want a copy of the input file with the first 5727989 bytes removed:
dd if=big1 bs=5727989 skip=1 of=big1c

You already know how to create the file big1b which contains the bytes of your patch. You should be able to cat these together:
cat big1a big1b big1c > big2

But this filetype business may bite you. If so try:
(dd if=big1a bs=1000000 ; dd if=big1b bs=1 ; dd if=big1c bs=1000000) | dd bs=1000000 of=big2
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Get file's first x bytes

is there a better way to do this: head -c 10000k /var/dump.log | head -c 6000k unfortunately, the "-c" option is not available on sun solaris. so i'm looking at "dd". but i dont know how to use it to achieve the same exact goal as the above head command. this needs to work on both solaris... (5 Replies)
Discussion started by: SkySmart
5 Replies

2. Shell Programming and Scripting

Shell script - entered input(1-40 bytes) needs to be converted exactly 40 bytes

hello, suppose, entered input is of 1-40 bytes, i need it to be converted to 40 bytes exactly. example: if i have entered my name anywhere between 1-40 i want it to be stored with 40 bytes exactly. enter your name: donald duck (this is of 11 bytes) expected is as below - display 11... (3 Replies)
Discussion started by: shravan.300
3 Replies

3. UNIX for Dummies Questions & Answers

X bytes of 0, Y bytes of random data, Z bytes of 5, T bytes of 1. ??

Hello guys. I really hope someone will help me with this one.. So, I have to write this script who: - creates a file home/student/vmdisk of 10 mb - formats that file to ext3 - mounts that partition to /mnt/partition - creates a file /mnt/partition/data. In this file, there will... (1 Reply)
Discussion started by: razolo13
1 Replies

4. Programming

Copying 1024 bytes data in 3-bytes chunk

Hi, If I want to copy a 1024 byte data stream in to the target location in 3-bytes chunk, I guess I can use the following script. dd bs=1024 count=3 if=/src of=/dest But, I would like to know, how to do it via a C program. I have tried this with memcpy(), that did not help. (3 Replies)
Discussion started by: royalibrahim
3 Replies

5. Shell Programming and Scripting

Error PHP Fatal error: Allowed memory size of 67108864 bytes exhausted(tried to allocate 401 bytes)

While running script I am getting an error like Few lines in data are not being processed. After googling it I came to know that adding such line would give some memory to it ini_set("memory_limit","64M"); my input file size is 1 GB. Is that memory limit is based on RAM we have on... (1 Reply)
Discussion started by: elamurugu
1 Replies

6. UNIX for Dummies Questions & Answers

Bytes of character in file

Hi, How do I check for the total bytes of character used by a file? Can I used a od command to check? Thanks. (1 Reply)
Discussion started by: user50210
1 Replies

7. Shell Programming and Scripting

Remove first N bytes and last N bytes from a binary file on AIX.

Hi all, Does anybody know or guide me on how to remove the first N bytes and the last N bytes from a binary file? Is there any AWK or SED or any command that I can use to achieve this? Your help is greatly appreciated!! Best Regards, Naveen. (1 Reply)
Discussion started by: naveendronavall
1 Replies

8. Shell Programming and Scripting

How to delete initial N bytes from the starting of the file using sed?

I want to delete initial N bytes of a file using sed. How can I do that? (3 Replies)
Discussion started by: jockey007
3 Replies

9. Programming

How to delete N bytes from the starting of the file from a C function???

Hi, I want to delete the initial few lines (or even bytes) from a file. I want to do this from a C function & this is in Linux platform. The truncate & ftruncate is allowing me to delete bytes from the end only. Any linux C function calls or ideas or any suggestions?? I'm in a dead... (2 Replies)
Discussion started by: jockey007
2 Replies

10. UNIX for Dummies Questions & Answers

Take a file from the system and put on tape and reset the file to 0 bytes

:mad: I did this the other day but one of my support personnel removed my history so i could call it back up to remeber the exact command since i am air-headed at times. I am trying to take a 30 MEG file off the system and drop it to tape then i want to make the file go back to being 0 bytes so... (1 Reply)
Discussion started by: JackieRyan26
1 Replies
Login or Register to Ask a Question