Shell Programming and Scripting

Can anyone tell me how to remove a portion of a large file to smaller ones? What I have is a large file that was created becasue several similar files were joined together. Each individual file starts with MSG_HEAD. I want to take everything from MSG_HEAD up to were it says MSG_HEAD again and output it to a new file, so that I have one file for everytime MSG_HEAD appears, plus all the line between occurances. Thanks in advance for any help.

Tries to use the command "csplit" (see man csplit). If it will not be possible, probably you will have that to make one script using the commands grep, tail, head, etc...

I hope help you

Witt

If they are at irregular segments, you can use vi.

1st figure out each line occurence of your Keyword. From line x to line XX for each section that you want to copy out.

Then use the vi command from within the file while you are vi'ing it.

Once you vi a file, type this line " :x,XXw myfile.out "

This will copy out from line #x to line #XX to the file myfile.out.

It may be a little cumbersome, but it is a useful tool.

I believe there is even a way to write all of these vi commands in a file and execute them while you are vi'ing the file. VI is a very powerful command, I am only scratching the surface of it's capabilities.

If this is an ongoing issue you would like to automate, I'd write an awk script.

Basically

^MSG_HEAD{
if (NR <> 1) close( filename )
iterate the filename
}
{
print $0 > filename >
}
END {
close(filename)
}

I am really not familiar with awk. Thanks for you suggestions. With the awk and ^MSG_HEAD, I understand that it will write the entire line, but will it write all the lines that follow until it hits MSG_HEAD again?

Thanks

Well you'll probably need to read some documentation.
but, if you put the following code in a file code.awk
and run it with
awk -f code.awk input_file
you should get the desired results.
Awk works by reading each line of the input file and if it matches the criteria on the left it executes the code with the {}
BEGIN matches once at begining, END once at end. ^MSG_HEAD matches a line begining with MSG_HEAD, and no criteria matches for everline.
So the program below gets a new filename for every MSG_HEAD line, and writes out every line to that filename.


BEGIN{ i=1}

^MSG_HEAD {
if (NR <> 1) close( filename )
filename="out_file" i
}
{
print $0 > filename
}
END {
close(filename)
}

Well, as Harrison Ford would say, "All good suggestions".

csplit, as suggested by witt, seems the best solution here, assuming generic file names are OK:

csplit -f small. -n 4 bigfile '/^MSG_HDR/' '{*}'

Above will split bigfile into:
small.0000
small.0001
etc

The first file will have anything prior to the first MSG_HDR line, so it could be empty. To discard that first section (empty or not), you can use:

csplit -f small. -n 4 bigfile '%^MSG_HDR%' '/^MSG_HDR/' '{*}'

And awk would be the way to go if you wanted to control the new filenames, such as picking something out of the header line.