Visit Our UNIX and Linux User Community


Maximum size of sed file...


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Maximum size of sed file...
# 8  
Old 03-20-2002
try one more for loop to cat large file one line at a time

Is this where you get hung up?

for u in *.ABC; do for t in $u.raw.sedfile*; do sed -f $t $u > temp; mv temp $u; done; done &&

No matter, try this:


Regarding your large sed file that you call with "sed -f", can you instead do something like this instead.

for name in `cat large.sed.file`
do
echo changing $name
sed $name
done > output.file

You would need to have the sed command there to read in each line one at a time from the "for" line.

Hope this helps.

Smilie
# 9  
Old 03-20-2002
That would work - however with sed requiring that the output be directed to a new file - each line of the sedfile that is applied would require the result to be > output.file then renamed back to the original file to continue.

I think that would become very inefficient - as I'd be doing a mv for each line in the sed file (1,000's of them.) With sed -f I can do 199 at a time before I need to do an mv.

So a file with 10,000 commands would, with your suggestion, make 10,000 sed commands and 10,000 mv commands. The way I have it it makes 50 sed -f commands (arguably still is doing 10,000) - but only 50 mv commands. That's the difference I think.

I can see the benefit - but think that the cost in processing speed outweighs the change in favour of the current script.

The script does work. There's no problem with it. What I really wanted was the ability to have sed -f process more than 199 commands at a time. I figure from the responses to date that this is not possible (built into sed I assume). So might just leave it as it is.
# 10  
Old 03-20-2002
after banging away all i found is that it is probably version specific, i used a file with 300+ sed commands and it ran fine, i'm not sure how far you want to go but you could try a different version of sed, unfortunately i'm not sure what version i'm using, sed -V is supposed to give it but not on my aix 4.3

later
# 11  
Old 03-20-2002
Thanks everyone for looking at it.

Its not really worth my effort to locate another version of sed - considering the script works - and to be of any benefit - I'd really need a sed that could read unlimited lines.

No point moving to 500 lines or even 1,000 as I'd still have the use the script I have.

Smilie
# 12  
Old 03-21-2002
One more thing.

You could use an append for the last line of my suggestion.

for name in `cat large.sed.file`
do
echo changing $name
sed $name
done >> output.file

This would allow you to append unlimited lines to this output file before you decide to do a mv back to the original file.

Give it a look. Even if you don't change your script.


Smilie
# 13  
Old 03-21-2002
I might be mistaken - but won't that append the whole file each time?

The output of the sed command is the whole file with it's changes.

If I then append the output to a file - and follow up with with the next loop - won't it just add the entire file with a few lines changed again - onto the end ot the file....hence a file that is 'n' times the length of the sed file?

I see where you're coming from - I have considered a number of options - and so far as I can see what I have is the most efficient method.

The only way I could improve it is to have my sed file with multiple substitutions on one line..... -e s/old/new/g -e s/old/new/g ...

Then I may be able to call upto 199 lines (each line however would execute maybe 20 commands.....then I could do it with one file........

That however would entail me creating a file with 20 columns rather than one - which is a whole lot of manipulation - to acheive something that already works..... might have a play to see if it works anyway!
# 14  
Old 03-22-2002
Theoretically, I believe that the for loop allows sed to process one line at a time so it would loop thru your large file one line at a time and append one line at a time. Also, I believe that sed does process the whole file at once but I am not sure what your script would do being run thru a for loop.

Let me know what happens.
Smilie
 

Previous Thread | Next Thread
Test Your Knowledge in Computers #974
Difficulty: Medium
Version 1.0 of the Linux kernel was released on 14 March 1994.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Maximum size of attachment in mail

Hi Friends, My requirment is to Query the oracle database , generate the file change the extension to .csv and send to clients automatically everyday. However i am able to perform the task. But sometimes when the file size is getting increased more than 1 MB then the mail is... (5 Replies)
Discussion started by: Showdown
5 Replies

2. Programming

Maximum buffer size for read()

Hi friends, Hope everybody is fine. First have a look at my code, then we will talk about it. $ cat copy.c #include <stdio.h> #define PERMS 0644 /* RW for owner, R for group, others */ #define BUFSIZE 1 char *progname; int main(int argc,char * argv) { int f1, f2, n; ... (4 Replies)
Discussion started by: gabam
4 Replies

3. UNIX for Advanced & Expert Users

How to identify maximum stack size?

Hi All, I have set max stack size as 4KB for my thread, but it always using very less. So I like to know what is the maximum stack size is used by my thread. I tried with gcc -fstack-usage command line option, but its not supported by mips. Kindly suggest me the way to find the max stack... (6 Replies)
Discussion started by: rajamohan
6 Replies

4. AIX

maximum size of a ramdisk on AIX 5.3

Hi, Do you know what is the maximum size I can use to create a ramdisk on AIX 5.3? I m pretty sure i've seen somewhere i can use more than 2 Gb but I can't remember where. I need to do some recommandations for one of my customer and they'll need to create a ramdisk of 20 Gb. Can this be done? ... (1 Reply)
Discussion started by: cedric hanquez
1 Replies

5. UNIX for Dummies Questions & Answers

Maximum file size ????

Hi All, - block size of 512KB & every address requires 4 bits - The inode structure contains 10 direct pointers, 1 single indirect, 1 double indirect & 1 triple indirect pointer What could be the possible maximum file size for this system Any guess? I am unable to understand the question... (0 Replies)
Discussion started by: preethgideon
0 Replies

6. UNIX for Dummies Questions & Answers

maximum tar file size?

Is there a reasonable maximum limit for tar file sizes? I want to transfer a pile of files from one server to another but have restricted means, so tarring them first will probably be best... but how big can I go - both for the file format itself and for the operating system (linux) to handle? ... (7 Replies)
Discussion started by: Bobby
7 Replies

7. UNIX for Dummies Questions & Answers

Maximum size of a file in unix

What's the maximum file size supported by unix. (3 Replies)
Discussion started by: nagalenoj
3 Replies

8. HP-UX

Mailx Maximum attachment size

Hi friends, what is the maximum allowable attachment size, we can send using mailx command. what is an alternate option if my file is > than that size? thanks sreeji (0 Replies)
Discussion started by: sreejithau
0 Replies

9. UNIX for Dummies Questions & Answers

Maximum input file size in "Diff" Command

Hello, Can anyone let me know what is the maximum file size that can be given as input for the "Diff" Command in Unix? I have a file size as large as 28MB and which can also increase. Will I face any issues with such a file size. If yes, What is the other alternative. Thanks in advance for... (1 Reply)
Discussion started by: Neeraja
1 Replies

10. Programming

Maximum File Size

Hi, When i checked for the maximum file size on solaris 5.9 the max file size obtained was only 2147483647 and all the further writes to the file which had reached that max size is not added to that file. even i had registered the signal SIGXFSZ, but the signal was not delivered to the... (5 Replies)
Discussion started by: matrixmadhan
5 Replies

Featured Tech Videos