remove portion of file

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting remove portion of file
# 1  
Old 04-04-2002
Data remove portion of file

Can anyone tell me how to remove a portion of a large file to smaller ones? What I have is a large file that was created becasue several similar files were joined together. Each individual file starts with MSG_HEAD. I want to take everything from MSG_HEAD up to were it says MSG_HEAD again and output it to a new file, so that I have one file for everytime MSG_HEAD appears, plus all the line between occurances. Thanks in advance for any help.
# 2  
Old 04-05-2002
Tries to use the command "csplit" (see man csplit). If it will not be possible, probably you will have that to make one script using the commands grep, tail, head, etc...

I hope help you

# 3  
Old 04-05-2002
If they are at irregular segments, you can use vi.

1st figure out each line occurence of your Keyword. From line x to line XX for each section that you want to copy out.

Then use the vi command from within the file while you are vi'ing it.

Once you vi a file, type this line " :x,XXw myfile.out "

This will copy out from line #x to line #XX to the file myfile.out.

It may be a little cumbersome, but it is a useful tool.

I believe there is even a way to write all of these vi commands in a file and execute them while you are vi'ing the file. VI is a very powerful command, I am only scratching the surface of it's capabilities.

# 4  
Old 04-05-2002
If this is an ongoing issue you would like to automate, I'd write an awk script.


if (NR <> 1) close( filename )
iterate the filename
print $0 > filename >
# 5  
Old 04-05-2002
I am really not familiar with awk. Thanks for you suggestions. With the awk and ^MSG_HEAD, I understand that it will write the entire line, but will it write all the lines that follow until it hits MSG_HEAD again?

# 6  
Old 04-05-2002
Well you'll probably need to read some documentation.
but, if you put the following code in a file code.awk
and run it with
awk -f code.awk input_file
you should get the desired results.
Awk works by reading each line of the input file and if it matches the criteria on the left it executes the code with the {}
BEGIN matches once at begining, END once at end. ^MSG_HEAD matches a line begining with MSG_HEAD, and no criteria matches for everline.
So the program below gets a new filename for every MSG_HEAD line, and writes out every line to that filename.

BEGIN{ i=1}

if (NR <> 1) close( filename )
filename="out_file" i
print $0 > filename
# 7  
Old 04-06-2002
Well, as Harrison Ford would say, "All good suggestions".

csplit, as suggested by witt, seems the best solution here, assuming generic file names are OK:

csplit -f small. -n 4 bigfile '/^MSG_HDR/' '{*}'

Above will split bigfile into:

The first file will have anything prior to the first MSG_HDR line, so it could be empty. To discard that first section (empty or not), you can use:

csplit -f small. -n 4 bigfile '%^MSG_HDR%' '/^MSG_HDR/' '{*}'

And awk would be the way to go if you wanted to control the new filenames, such as picking something out of the header line.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to append portion of a file content to another file when a certain pattern is matching?

Hi ladies and gentleman.. I have two text file with me. I need to replace one of the file content to another file if one both files have a matching pattern. Example: text1.txt: ABCD 1234567,HELLO_WORLDA,HELLO_WORLDB DCBA 3456789,HELLO_WORLDE,HELLO_WORLDF text2.txt: XXXX,ABCD... (25 Replies)
Discussion started by: bananamen
25 Replies

2. Shell Programming and Scripting

Unix Scripting : Sort a Portion of a File and not the complete file

Need to sort a portion of a file in a Alphabetical Order. Example : The user adam is not sorted and the user should get sorted. I don't want the complete file to get sorted. Currently All_users.txt contains the following lines. ############## # ARS USERS ############## mike, Mike... (6 Replies)
Discussion started by: evrurs
6 Replies

3. UNIX for Advanced & Expert Users

Removing portion of file name

Hi , I am getting file name like ABC_DATA_CUSTIOMERS_20120617.dat ABC_DATA_PRODUCTS_20120617.dat Need to convert CUSTIOMERS.dat PRODUCTS.dat Help me how to do this. (7 Replies)
Discussion started by: reach_malu
7 Replies

4. Shell Programming and Scripting

remove large portion of web page code between two tags

Hi everybody, I am trying to remove bunch of lines from web pages between two tags: one is <h1> and the other is <table it looks like <h1>Anniversary cards roses</h1> many lines here <table summary="Free anniversary greeting cards." cellspacing="8" cellpadding="8" width="70%">my goal... (5 Replies)
Discussion started by: georgi58
5 Replies

5. UNIX for Dummies Questions & Answers

Remove the date portion from file name

hi, I am trying to remove the last field before the period (.) from a list of file names in a directory in a shell script. Below is the list of file names. ENVID_archival_20120214092258.log ENVID_Get_Source_Files_20120214091828.log ENVID_Get_Source_Files_20120214092523.log... (6 Replies)
Discussion started by: Vijay81
6 Replies

6. Shell Programming and Scripting

Remove first portion of string

I have a script which currently uses a file containing a list of directories as an argument. The file is read in to an array, and then the array is iterated in a for loop. What I would like to do is cut off the first few directories of the directory path (they won't exist on the server where the... (5 Replies)
Discussion started by: msarro
5 Replies

7. UNIX for Dummies Questions & Answers

Portion of a file in a new files

Hi, I need to devide one file into 3 files based on column numbers and put a string (FILE1, FILE2, FILE3) in the last..... Input file: Column1,Column2,Column3,Column4,Column5,Column6,Column7,Column8,Column9,Column10 Output1: Column1,Column3,Column6,Column4,Column5,FILE1 Output2:... (6 Replies)
Discussion started by: yale_work
6 Replies

8. Shell Programming and Scripting

Grep certain portion from the file

Dear Friends, Here I am with another difficulty. I have a flat file from which I wanna grep following pattern. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Statement Date : Blah blah Blah blah Blah blah Blah blah... (1 Reply)
Discussion started by: anushree.a
1 Replies

9. UNIX for Dummies Questions & Answers

Print a portion of file

Hi, I have a little problem. I am having a file with pattern like : asdf;ffgg;dfjfj;djdfjf;nnjj;djd;ssj; I just want to print the portion from last ";" upto the immediate previous ";". There are several ";" in my line. Please help me out... Thnx in advance (8 Replies)
Discussion started by: vanand420
8 Replies

10. Programming

Delete Portion of a file

hi i would like to know whether i can delete a part of a file in C for eg. if my file contained 1234567890 and i want to delete 456 so that it becomes 1237890 is there a way i can do this. well, one way i can achieve this is by creating a new file, copy whatever i want, then delete the... (2 Replies)
Discussion started by: sameersbn
2 Replies
Login or Register to Ask a Question