Splitting a file based on first 8 chars


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Splitting a file based on first 8 chars
# 1  
Old 03-01-2010
Splitting a file based on first 8 chars

I have an input file of this format
<Date><other data>

For example,
Code:
20081213aaaaaaaaa
20081213bbbbbbbbb
20081220ccccccccc
20081220ddddddddd
20081220eeeeeeeee
20081227ffffffffffffff

The first 8 chars are date in YYYYMMDD formT. I need to split this file into n files where n is the distinct number of dates in the input file and each output file should have the respective records based on the date.

In the above example, the file should split as

Output_file_20081213
Code:
20081213aaaaaaaaa
20081213bbbbbbbbb

Output_file_20081220
Code:
20081220ccccccccc
20081220ddddddddd
20081220eeeeeeeee

Output_file_20081227
Code:
20081227ffffffffffffff

Can anyone suggest a solution? The file is very huge. So, I do not wish to just read it multiple times and pass it through an if and redirect the output.

I am looking for something efficient with a single read and multiple redirects.

Last edited by zaxxon; 03-01-2010 at 08:22 AM.. Reason: use code tags please, ty
# 2  
Old 03-01-2010
You use the following script.

Code:
while read line
do
file=`echo Output_file_``echo $line | cut -b 1-8`
string=`echo $line | cut -b 9-`
echo $string >> $file
done < input

# 3  
Old 03-01-2010
Try this
Code:
for line in `cat files`
do
        file=`echo output_file``echo $line | cut -b 0-8`
        data=`echo $line | cut -b 9-`
        echo $data | cat >> $file
done

Here the files file will contain your input data.
i.e.
20081213aaaaaaaaa
20081213bbbbbbbbb
20081220ccccccccc
20081220ddddddddd
20081220eeeeeeeee
20081227ffffffffffffff

So now your requirement will be achieved

Last edited by thillai_selvan; 03-09-2010 at 12:57 AM..
# 4  
Old 03-01-2010
With awk:

Code:
awk '{print > "Output_file_" substr($0,1,8)}' file

# 5  
Old 03-01-2010
MySQL another way

Try this also,

This code handles the line like "20100102ddddd ccccc 12fsdsdsfg dfhhgf"

Code:
while read line
do
        content=`sed -rn "s/^[0-9]{8}(.+)$/\1/p" <<<$line`
        date=`sed -rn "s/^([0-9]{8}).+/\1/p" <<<$line`
        temp=$?
        filename="Output_file_"$date
        if [ $temp -eq 0 ];then
                echo "$content">>$filename
        fi
done <"input_file.txt"
echo "Files created "

Thanks.

Last edited by karthigayan; 03-01-2010 at 09:10 AM..
# 6  
Old 03-01-2010
Really appreciate all the quick responses. I am going to go with the awk solution suggested by Franklin. works like a charm. Thanks to everyone for your suggestions.
# 7  
Old 03-01-2010
There are probably a lot of ways to do this, but I would use sed.

Get the first three lines of a file:
Code:
sed -n -e "1,3p" file > file1

Get the next three lines:
Code:
sed -n -e "4,6p" file > file2

The -n is used to suppress standard output, because by default, sed prints the entire file with the specified modifications, in addition to anything you tell it to print with the p command.

I would definitely recommend learning sed, and maybe awk too. sed has a simple syntax and only a few commands, but it is still very useful. Both sed and awk can be run either as simple commands or by reading multiple instructions from a script.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Splitting a file based on a pattern

Hi All, I am having a problem. I tried to extract the chunk of data and tried to fix I am not able to. Any help please Basically I need to remove the for , values after K, this is how it is now A,, B, C,C, D,D, 12/04/10,12/04/10, K,1,1,1,1,0,3.0, K,1,1,1,2,0,4.0,... (2 Replies)
Discussion started by: arunkumar_mca
2 Replies

2. Shell Programming and Scripting

Splitting a file based on line number

Hi I have a file with over a million lines (rows) and I want to split everything from 500,000 to a million into another file (to make the file smaller). Is there a simple command for this? Thank you Phil (4 Replies)
Discussion started by: phil_heath
4 Replies

3. Shell Programming and Scripting

AWK splitting a string of equal parts of 500 chars

Hi , can someone help me how to make an AWK code for splitting a string of equal parts of 500 chars in while loop? Thank you! (4 Replies)
Discussion started by: sanantonio7777
4 Replies

4. Shell Programming and Scripting

Splitting file based on line numbers

Hello friends, Is there any way to split file from n to n+6 into 1 file and (n+7) to (n+16) into other file etc. f.e I have source pipe delimated file with 20 lines and i need to split 1-6 in file1 and 7-16 in file2 and 17-20 in file 3 I need to split into fixed number of file like 4 files... (2 Replies)
Discussion started by: Rizzu155
2 Replies

5. Shell Programming and Scripting

Splitting file based on column values

Hi all, I have a file (say file.txt) which contains comma-separated rows. Each row has seven columns. Only column 4 or 5 (not both) can have empty values like "" in each line. Sample lines So, now i want all the rows that have column 4 as "" go in file1.txt and all the rows that have column... (8 Replies)
Discussion started by: jakSun8
8 Replies

6. Shell Programming and Scripting

Splitting a file based on context.

I have file as shown below. Would like to split the file based on the context of data. Like, split the content between "---- XXX Info ----" and " ---- YYY Info ----" to a file. When I try using below command, 2nd file contains all the info starting after first "---- YYYY Info ----" instance.... (8 Replies)
Discussion started by: webkid
8 Replies

7. Shell Programming and Scripting

Splitting a file based on two patterns

Hi there, I've an input file as follows: *START 1001 a1 1002 a2 1003 a3 1004 a4 *END *START 1001 b1 1002 b2 1004 b4 *END *START 1001 c1 1004 c4 *END (6 Replies)
Discussion started by: kbirde
6 Replies

8. Shell Programming and Scripting

Splitting the file based on logic

Hello I have a requirement where i need to split the Input fixed width file which contains multiple invoices into multiple files with 2 invoices per file. Each invoice can be identified by its first line's second character which is "H" and sixth character is " " space and the invoice would... (10 Replies)
Discussion started by: dsdev_123
10 Replies

9. Shell Programming and Scripting

splitting files based on text in the file

I need to split a file based on certain context inside the file. Is there a unix command that can do this? I have looked into split and csplit but it does not seem like those would work because I need to split this file based on certain text. The file has multiple records and I need to split this... (1 Reply)
Discussion started by: matrix1067
1 Replies

10. Shell Programming and Scripting

Splitting a file based on some condition and naming them

I have a file given below. I want to split the file where ever I came across ***(instead you can put ### symbols in the file) . Also I need to name the file by extracting the report name from the first line which is in bold(eg:RPT507A) concatinated with DD(day on which the file runs). Can someone... (1 Reply)
Discussion started by: srivsn
1 Replies
Login or Register to Ask a Question