[Solved] Split one file into more than one file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers [Solved] Split one file into more than one file
# 8  
Old 02-13-2014
For:-
Quote:
Code:
while IFS="|" read junk junk junk junk VALUE ; do print - "$VALUE" ; done</path/to/input.file | uniq

..... surely this achieves the same much neater:-
Code:
cut -f5 -d "|" inputfile | sort -u

or if you prefer:-
Code:
cut -f5 -d "|" inputfile | uniq



Robin
This User Gave Thanks to rbatte1 For This Post:
# 9  
Old 02-13-2014
Quote:
Originally Posted by rbatte1
..... surely this achieves the same much neater:-
Code:
cut -f5 -d "|" inputfile | sort -u

or if you prefer:-
Code:
cut -f5 -d "|" inputfile | uniq

Yes - and no. I gave two variants, the second of which is basically what you suggest here. The first one was meant as a blueprint in case there has to be something done while (instead of after) processing the input. One could add arbitrary statements to the while-loop as long as its output is preserved for the following pipeline. This is what was meant by "depending on what else you want to do".

bakunin
These 2 Users Gave Thanks to bakunin For This Post:
# 10  
Old 02-13-2014
Code:
$ awk -F\| '{ F = $5".txt"; if(F in A)print $0 >>F; else print >F; A[F]; close(F)}' file

Can you explain more about this code?
# 11  
Old 02-13-2014
Quote:
Originally Posted by radius
Code:
$ awk -F\| '{ F = $5".txt"; if(F in A)print $0 >>F; else print >F; A[F]; close(F)}' file

Can you explain more about this code?

-F\| --> Field Separator is set to pipe

F = $5".txt" --> Variable F is column5 record + suffix ".txt"

if(F in A) --> If variable F is already in Array A, meaning we seen in earlier line, then redirect and append to file thats is print $0 >>F.

else just redirect to file print $0 >F, this condition will be true when awk, reads 1st line of file, as Array A is not defined before if statement, and whenever, Array A seen new key that is F

A[F] --> Array A with index being F

close(F) --> Close file F
These 2 Users Gave Thanks to Akshay Hegde For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

sed awk: split a large file to unique file names

Dear Users, Appreciate your help if you could help me with splitting a large file > 1 million lines with sed or awk. below is the text in the file input file.txt scaffold1 928 929 C/T + scaffold1 942 943 G/C + scaffold1 959 960 C/T +... (6 Replies)
Discussion started by: kapr0001
6 Replies

2. UNIX for Dummies Questions & Answers

[Solved] Perl Question - split function with csv file

Hi all, I have a csv file that appears as follows: ,2013/03/26,2012/12/26,4,1,"2017/09/26,5.75%","2017/09/26,1,2018/09/26,1,2019/09/26,1,2020/09/26,1,2021/09/26,1",,,2012/12/26,now when i use the split function like this: my @f = split/,/; the split function will split the data that is... (2 Replies)
Discussion started by: WongSifu
2 Replies

3. Shell Programming and Scripting

How to split file into multiple files using awk based on 1 field in the file?

Good day all I need some helps, say that I have data like below, each field separated by a tab DATE NAME ADDRESS 15/7/2012 LX a.b.c 15/7/2012 LX1 a.b.c 16/7/2012 AB a.b.c 16/7/2012 AB2 a.b.c 15/7/2012 LX2 a.b.c... (2 Replies)
Discussion started by: alexyyw
2 Replies

4. Shell Programming and Scripting

[Solved] Split file using awk

hlow all, need your advice i have sample.txt 1252468812,yahoo,3.5 1252468812,hotmail,2.4 1252468819,yahoo,1.2 1252468812,msn,8.9 1252468923,gmail,12 1232468819,live,3.4 1252368929,yahoo,9.0 1252468929,msn,1.2now i want filtering with awk so output will like this 12524_log.txt... (2 Replies)
Discussion started by: zvtral
2 Replies

5. Shell Programming and Scripting

Split File by Pattern with File Names in Source File... Awk?

Hi all, I'm pretty new to Shell scripting and I need some help to split a source text file into multiple files. The source has a row with pattern where the file needs to be split, and the pattern row also contains the file name of the destination for that specific piece. Here is an example: ... (2 Replies)
Discussion started by: cul8er
2 Replies

6. Shell Programming and Scripting

How to split a data file into separate files with the file names depending upon a column's value?

Hi, I have a data file xyz.dat similar to the one given below, 2345|98|809||x|969|0 2345|98|809||y|0|537 2345|97|809||x|544|0 2345|97|809||y|0|651 9685|98|809||x|321|0 9685|98|809||y|0|357 9685|98|709||x|687|0 9685|98|709||y|0|234 2315|98|809||x|564|0 2315|98|809||y|0|537... (2 Replies)
Discussion started by: nithins007
2 Replies

7. Shell Programming and Scripting

[Solved] Script to split a file into two

Hi i have a file like a 12 b 13 c 14 d 15 I want to split it based on a blank line like in first file I should have a 12 b 13 and in the second file I have c 14 d 15 How can i do this? Any help will be greatly appreciated (5 Replies)
Discussion started by: prarat
5 Replies

8. Shell Programming and Scripting

Split one file to Multiple file with report basis in unix

Hi, Please help on this. i want split the below file(11020111.CLT) to more files with some condition. :b: 1) %s stating of the report 2) %e ending of the report example starting of the report: %sAEGONCA| |MUMBAI | :EXPC|N|D ending of the report %eAEGONCA| |MUMBAI | :EXPC 3)so the... (10 Replies)
Discussion started by: krbala1985
10 Replies

9. Shell Programming and Scripting

[solved] merging two files and writing to another file- solved

i have two files as file1: 1 2 3 file2: a b c and the output should be: file3: 1~a 2~b 3~c (1 Reply)
Discussion started by: mlpathir
1 Replies

10. UNIX for Dummies Questions & Answers

Split a file with no pattern -- Split, Csplit, Awk

I have gone through all the threads in the forum and tested out different things. I am trying to split a 3GB file into multiple files. Some files are even larger than this. For example: split -l 3000000 filename.txt This is very slow and it splits the file with 3 million records in each... (10 Replies)
Discussion started by: madhunk
10 Replies
Login or Register to Ask a Question