Split a file in more files based on score content


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split a file in more files based on score content
# 1  
Old 11-19-2013
Split a file in more files based on score content

Dear All,
I have the following file tabulated:
Code:
ID	distanceTSS	score
8434	571269	10
10122	393912	9
7652	6	10
4863	1451	9
8419	39	2
9363	564	21
9333	7714	22
9638	8334	9
1638	1231	11
10701	918	1000
6587	32056	111

What I would like to do is the following, create 100 new files based on content of the second column,
The first file should contain all the lines with a distance between 0 and 1000,
the second between 1000 and 2000, and so on untile 99000 and 10000,

Finally for each new file I would like to calculate the median of the third column(score)

Is there a rapid way to do so? I tried witha perl script but it seems really slow,

thanks for your help,
Paolo

Last edited by paolo.kunder; 11-19-2013 at 09:03 AM..
# 2  
Old 11-19-2013
Could this help you ?
Code:
 awk '{ print > int($2/1000)+1"_distance.txt"}'  filename

This User Gave Thanks to pravin27 For This Post:
# 3  
Old 11-19-2013
Thanks,
That's really help me!
Another issue concerning the output file:
let's imagine I have the following output for 1_distance:

Code:
ID	distanceTSS	score
8434	8	10
8434	999	10
7652	6	9
7652	451	9

As you can notice there are ID present more than once, I would like to retain only one ID, for example:

Code:
ID	distanceTSS	score
8434	8	10
7652	6	9

And finally for each new file created (distance_*) I would like to calculate the median of the third column,

Thanks again,
Paolo
# 4  
Old 11-19-2013
Which of the duplicate values do you want to retain? The first? The smallest?
Does the output order matter, esp. reg. median calculation?
# 5  
Old 11-20-2013
Hi,
Is the same because they have the same score
# 6  
Old 11-20-2013
Hi Pravin,

Is it possible for you to explain the awk commadn which u have provided...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split content based on keywords

I need to split the file contents with multiple rows based on patterns Sample: Input: ABC101testXYZ102UKMNO1092testing ABC999testKMNValid Output: ABC101test XYZ102U KMN1092testing ABC999test KMNValid In this ABC , XYZ and KMN are patterns (6 Replies)
Discussion started by: Jairaj
6 Replies

2. Shell Programming and Scripting

Split content based on keywords

I need to split the file contents with multiple rows based on patterns Sample: Input: ABC101testXYZ102UKMNO1092testing ABC999testKMNValid Output: ABC101test XYZ102U KMN1092testing ABC999test KMNValid In this ABC , XYZ and KMN are patterns Continue here./mod] Please read forum... (1 Reply)
Discussion started by: Jairaj
1 Replies

3. Shell Programming and Scripting

List the files after sorting based on file content

Hi, I have two pipe separated files as below: head -3 file1.txt "HD"|"Nov 11 2016 4:08AM"|"0000000018" "DT"|"240350264"|"56432" "DT"|"240350264"|"56432" head -3 file2.txt "HD"|"Nov 15 2016 2:18AM"|"0000000019" "DT"|"240350264"|"56432" "DT"|"240350264"|"56432" I want to list the... (6 Replies)
Discussion started by: Prasannag87
6 Replies

4. Shell Programming and Scripting

Split a single file into multiple files based on a value.

Hi All, I have the sales_data.csv file in the directory as below. SDDCCR; SOM ; MD6546474777 ;05-JAN-16 ABC ; KIRAN ; CB789 ;04-JAN-16 ABC ; RAMANA; KS566767477747 ;06-JAN-16 ABC ; KAMESH; A33535335 ;04-JAN-16 SDDCCR; DINESH; GD6674474747 ;08-JAN-16... (4 Replies)
Discussion started by: ROCK_PLSQL
4 Replies

5. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

6. Shell Programming and Scripting

How to split file into multiple files using awk based on 1 field in the file?

Good day all I need some helps, say that I have data like below, each field separated by a tab DATE NAME ADDRESS 15/7/2012 LX a.b.c 15/7/2012 LX1 a.b.c 16/7/2012 AB a.b.c 16/7/2012 AB2 a.b.c 15/7/2012 LX2 a.b.c... (2 Replies)
Discussion started by: alexyyw
2 Replies

7. Shell Programming and Scripting

Split a file into multiple files based on field value

Hi, I've one requirement. I have to split one comma delimited file into multiple files based on one of the column values. How can I achieve this Unix Here is the sample data. In this case I have split the files based on date column(c4) Input file c1,c2,c3,c4,c5... (1 Reply)
Discussion started by: manasvi24
1 Replies

8. Shell Programming and Scripting

Split the file based on the content

Arun kumar something somehting Enterting in to the line . . . . Some text text Finshing the sentence Some other text . . . . Again something somehting Enterting in to the line . . . . . . Again text text Finshing the sentence (6 Replies)
Discussion started by: arukuku
6 Replies

9. Shell Programming and Scripting

split XML file into multiple files based on pattern

Hello, I am using awk to split a file into multiple files using command: nawk '{ if ( $1 == "<process" ) { n=split($2, arr, "\""); file=arr } print > file }' processes.xml <process name="Process1.process"> ... (3 Replies)
Discussion started by: chiru_h
3 Replies

10. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies
Login or Register to Ask a Question