Shell Programming and Scripting

Hi,

I have a requirement in which I am going to receive one file and should be splitted to 9 different files based on one distinguisher called TYPE.

I heard that this can be done using awk or sed.

Can any one advise regardint the logic and simpler way other than using awk or sed is also fine.

The volume of data expecting is 5GB daily file.

Thank you in advance.

Regards,
Sagar.

Code:

cat file1
Type    value
A       1
A       2
C       3
D       5
A       9
D       8
Z       5

Code:

 awk 'NR>1{print $0 >$1".txt"}'  file1

Hi Pravin,

Thanks for quick response. What is NR in awk command. Could you please bit more clear on the command.

Also, is there any performance issue as daily we are going to get 5GB of data.

Thank you in advance.

Regards,
Sagar

Hi, if you use SPLIT command ,it will be easy.

Code:

Awk NR

gives you the total number of records being processed or line number.

You can check this link to understand it more clearly

8 Powerful Awk Built-in Variables – FS, OFS, RS, ORS, NR, NF, FILENAME, FNR (link removed)

Hi bmk,

Any examples please for split command.

Hi Vikram,

Thanks for the link and it is very much useful. But, the said variables at field level.

Regards,
Sagar.

Try like...

Code:

 ----Split the file into 2 files
maxline=17 ---File count/2
split -l $maxline -d test1.txt L
for i in L??
do
echo "1"
mv "$i" "$i"test
done