Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Split: File into multiple and keeping the same 3 lines from input into all output files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split: File into multiple and keeping the same 3 lines from input into all output files
# 1  
Old 12-10-2012
Wrench Split: File into multiple and keeping the same 3 lines from input into all output files
The following code will split the infile into multiple files. However, I need it to insert the same first 3 lines from the original input file into each splitted file. How do I modify my script below to do so:

Code:
print -n "Enter file name to split? " ; read infile
if [ -z "$infile" ]
then
echo "Invalid file name entered ..."
exit 255
fi
print -n "Enter Number of files to split it into? " ; read numf
if echo $numf | egrep -q '^[2-9]{1}$'; then
echo
else
echo "Invalid number <$numf> entered. Entry must be numeric and 1 digits in length and between 2 and 9 ..."
exit 255
fi
echo "Splitting file $infile into $numf files ....."
sleep 2
awk '{a[NR]=$0}END{for(i=1;i<=NR;i++)print a[i] > "'split.$infile.'" 1+int(n*((i-1)/NR))}' n=$numf $infile
echo "Done...."

# 2  
Old 12-10-2012
Show the input you have and the output you want please.
# 3  
Old 12-10-2012
Replace
Code:
awk '{a[NR]=$0}END{for(i=1;i<=NR;i++)print a[i] > "'split.$infile.'" 1+int(n*((i-1)/NR))}' n=$numf $infile

with
Code:
awk 'NR<=3{a=(a"")?a RS $0:$0;next}
NR==4{for(i=1;i<=n;i++) print a > "split." FILENAME i}
{print > "split." FILENAME ((d=(NR-3)%n)?d:n)}' n=$numf "$infile"

# 4  
Old 12-10-2012
Quote:
Originally Posted by elixir_sinari
Replace
Code:
awk '{a[NR]=$0}END{for(i=1;i<=NR;i++)print a[i] > "'split.$infile.'" 1+int(n*((i-1)/NR))}' n=$numf $infile

with
Code:
awk 'NR<=3{a=(a"")?a RS $0:$0;next}
NR==4{for(i=1;i<=n;i++) print a > "split." FILENAME i}
{print > "split." FILENAME ((d=(NR-3)%n)?d:n)}' n=$numf "$infile"



Thank you. I ran it using your new awk. Then I combined them using cat file1 ..etc > combined-file to see if missed any entries after removing the first 3 lines from file 2 ..etc. Then i did a diff combined-file <original input combine file before the split>, I get a discrapancy I think due to some hidden format characters changes in the new split file. When I used my version of the awk i get none. How can fix it?

Here is a sample of discrapancies due to some formatting issue of the new AWK:

3255a3096,3097 <- Not in the files, but shows when doing a diff
> |9787771000|
> |9787772000|
3256a3099,3100 <-- causing a diff !!
> |9787804000|

Thanks

---------- Post updated at 04:05 PM ---------- Previous update was at 03:03 PM ----------

Quote:
Originally Posted by Corona688
Show the input you have and the output you want please.

Input file:

|cast_me_query|Continue|
|name_list|0002|queue_status=active|
|begin|
1235456677777777
2169058906897090
4893859489485958
0977256252716167


Basically i will need to keep the first 3 lines in all split files. for ex:

file1:
=================
|cast_me_query|Continue|
|name_list|0002|queue_status=active|
|begin|
1235456677777777
2169058906897090

file2:
===================
|cast_me_query|Continue|
|name_list|0002|queue_status=active|
|begin|
4893859489485958
0977256252716167




etc ...........
# 5  
Old 12-10-2012
no output after run the command, but you will have several new files generated : file1 file2 etc.

Code:
awk 'NR<4{s=s==""?$0:s RS $0;next} {a=s RS $0;getline;print a RS $0 > "file" ++i ;a=""}' infile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split file into multiple files based on empty lines

I am using below code to split files based on blank lines but it does not work. awk 'BEGIN{i=0}{RS="";}{x="F"++i;}{print > x;}' Your help would be highly appreciated find attachment of sample.txt file (2 Replies)
Discussion started by: imranrasheedamu
2 Replies

2. Shell Programming and Scripting

Removing multiple lines from input file, if multiple lines match a pattern.

GM, I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed. I am assuming that sed, awk or even perl could do what I need. I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies

3. Shell Programming and Scripting

[Solved] Multiple input files and output files

Hi, I have many test*.ft1 files to which I want to read as input for a script called pipe2txt.tcl and print the output in each separate file. For example, pipe2txt.tcl < test001.ft1 > test001.txt How can I read many files in this maner? thank you very much, Best, Pahuja (5 Replies)
Discussion started by: Pahuja
5 Replies

4. Shell Programming and Scripting

awk, multiple files input and multiple files output

Hi! I'm new in awk and I need some help. I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files. Thanks in advance for help! :-) ciao (5 Replies)
Discussion started by: gabrysfe
5 Replies

5. Shell Programming and Scripting

split input data file and put into same output file

Hi All, I have two input file and need to generate a CSV file. The existing report just "GREP" the records with the Header and Tailer records with the count of records. Now i need to split the data into 25 records each in the same CSV file. id_file (Input file ) 227050994 232510151... (4 Replies)
Discussion started by: rasmith
4 Replies

6. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

7. Shell Programming and Scripting

Split the single file lines into multiple files

Let's assume that I have a file name called ‘A' and it has 100 lines in it and would like to split these 100 lines into 4 files as specified bellow. INPUT: Input file name A 1 2 3 4 5 6 7 8 9 ........100 Output: 4 output files (x,y,z,w) File x should contains (Skip 4 lines)... (15 Replies)
Discussion started by: subbarao25
15 Replies

8. Shell Programming and Scripting

Split a file into multiple files

Hi, i have a file like this: 1|2|3|4|5| 1|2|8|4|6| Trailer1||||| 1|2|3| Trailer2||| 3|4|5|6| 3|4|5|7| 3|4|5|8| Trailer2||| I want to generate 3 files out of this based on the trailer record. Trailer record string can be different for each file or it may be same for one or two. No... (24 Replies)
Discussion started by: pparthji
24 Replies

9. Shell Programming and Scripting

building output file from multiple input files

Hi there, I am trying to figure out a way to combine multiple sources with different data on a single file, and I am trying to find the best way to do it. I have multiple files, let's say A, B, C and D. A has a field in common with B, B has a field in common with C, and C has a field in... (2 Replies)
Discussion started by: ppucci
2 Replies

10. UNIX for Dummies Questions & Answers

split a file into multiple files

Hi All, I have a file ABC.txt and I need to split this file on every 250 rows. And the file name should be ABC1.txt , ABC2.txt and so on. I tried with split command split -l 250 <filename> '<filename>' but the file name returned was ABC.txtaa ABC.txtab. Please... (8 Replies)
Discussion started by: kumar66
8 Replies
Login or Register to Ask a Question