Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Splitting files into 100 files with field value

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Splitting files into 100 files with field value
# 1  
Old 02-26-2013
Splitting files into 100 files with field value
I want a script to split my file upon the last field (15)
As file
Code:
A,b,c,.......,01
C,v,n,.......,02
C,r,v,........,01
F,s,a,........,03
X,y,d,........,99

To make output
Code:
01.txt
A,b,c,.......,01
C,r,v,........,01

02.txt
C,v,n,.......,02

.
.
.
99.txt

I've used awk
Code:
Awk -F"," '{print > $12}' file name.  
And using    Print >$12 ; close($12)

But not working error: too many outputs
It saves max 10 so how can I do this task in script to perform it so fast as it depends on time
Thnx a lot
# 2  
Old 02-26-2013
Hi
if I understand correctly this is what you want

Code:
awk -F, '{print $NF}' file.to.split.txt| sort -u| while read FILE
do 
grep ${FILE}$ file.to.split.txt >> ${FILE}.txt
done

# 3  
Old 02-26-2013
Try

Code:
 awk -F, '{ print > $NF".txt" }' file

# 4  
Old 02-26-2013
You say you used print>$12 and close($12), could we have a look at how you used it in your script?
# 5  
Old 02-26-2013
For
Code:
Awk -F, '{print >$12 ; close($12) }'
Awk -F, '{print >$12}' 
Same error
Awk 2 many output files 10

---------- Post updated at 06:38 AM ---------- Previous update was at 06:38 AM ----------

For
Code:
Awk -F, '{print >$12 ; close($12) }'
Awk -F, '{print >$12}' 
Same error
Awk 2 many output files 10

# 6  
Old 02-26-2013
if its something which you are aware the limit then u can try something like this....

Code:
nawk '{for(x=1;x<100;x++) {if($NF~x) print > x}}' input.txt

Thanks
Sha
# 7  
Old 02-26-2013
@franzo
Thnx man but script get me error usage : file [-h] [-m file ] and no output I hope when we can fix it , it can process in short time
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

2. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies

3. Shell Programming and Scripting

Plz Help. Compare 2 files field by field and get the output in another file.

Hi Freinds, I have 2 files . one is source.txt and second one is target.txt. I want to keep source.txt as baseline and compare target.txt. please find the data in 2 files and Expected output. Source.txt 1|HYD|NAG|TRA|34.5|1234 2|CHE|ESW|DES|36.5|134 3|BAN|MEH|TRA|33.5|234... (5 Replies)
Discussion started by: i150371485
5 Replies

4. Shell Programming and Scripting

Compare two files Field by field and output the result in another file

Hi Friends, Need Help. I have file1.txt as File1.txt |123|A|7267|Hyder|Cross|Sell|7801 |995|A|7051|2008|Lunar|New|Year|Promotion|7801 |996|A|7022|Q108|Targ|Prospect|&|SSCC|Savings|Promo|7801 |997|A|7182|Q1|Feb-Apr|08|Credit|ITA|PA|SBA|Campaign|7801 File2.txt... (7 Replies)
Discussion started by: i150371485
7 Replies

5. Shell Programming and Scripting

Splitting files

Hello all I have a file which has around 80 million records, I want to split it to 12 equal files, I tried using the split command but it is allowing me to split according to number of lines or by size. Is there a way i can split the file into 12 files without worrying about the number of lines... (7 Replies)
Discussion started by: Sri3001
7 Replies

6. Shell Programming and Scripting

AWK: Pattern match between 2 files, then compare a field in file1 as > or < field in file2

First, thanks for the help in previous posts... couldn't have gotten where I am now without it! So here is what I have, I use AWK to match $1 and $2 as 1 string in file1 to $1 and $2 as 1 string in file2. Now I'm wondering if I can extend this AWK command to incorporate the following: If $1... (4 Replies)
Discussion started by: right_coaster
4 Replies

7. UNIX for Dummies Questions & Answers

Splitting Files Help

Hi Gurus, Lets say i have a file with some 30 records... How can i split that file into 3 files Also it shud be dynamic in the sense.. I wouldnt mind if file 1 has 15, file 2 has 10 and file 3 has 5.... Please help.. Thanks (6 Replies)
Discussion started by: saggiboy10
6 Replies

8. Shell Programming and Scripting

Comparing the matches in two files using awk when both files have their own field separators

I've two files with data like below: file1.txt: AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 file2.txt: Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 So,the field separator in file1.txt is a comma and in file2.txt,it is | Now,the output should be... (2 Replies)
Discussion started by: asyed
2 Replies

9. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

10. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50... (3 Replies)
Discussion started by: arund_01
3 Replies
Login or Register to Ask a Question