05-16-2010
Split large file based on last digit from a column
Hello,
What's the best way to split a large into multiple files based on the last digit in the first column.
input file:
f
2738483300000x0y03772748378831x1y13478378358383x2y23743878383802x3y33787828282820x4y43748838383881x5y5
Desired Output:
f0
3738483300000x0y03787828282820x4y4
f1
3772748378831x14y143748838383881x53y51
f2
3743878383802x28y73
f3
3478378358383x56y66
the file is about 60Million records, and im using grep to do the splitting but i guess there must be a faster way. grep ^3.........."$i" ( where value of i is from 0 to 9 )
Appreciate your ideas .
Alain
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a file containing date/time sorted data of the form
...
2009/06/10,20:59:59.950,XAG/USD,Q,1,1115, 14.3025,100,1,1
2009/06/10,20:59:59.950,XAG/USD,Q,1,1116, 14.3026,125,1,1
2009/06/10,20:59:59.950,XAG/USD,R,0,0, , 0,0,0
2009/06/10,20:59:59.950,XAG/USD,R,1,0, 14.1910,100,1,1... (6 Replies)
Discussion started by: asriva
6 Replies
2. Shell Programming and Scripting
Dear All,
I would like to split a file of the following format into multiple files based on the number in the 6th column (numbers 1, 2, 3...):
ATOM 1 N GLY A 1 -3.198 27.537 -5.958 1.00 0.00 N
ATOM 2 CA GLY A 1 -2.199 28.399 -6.617 1.00 0.00 ... (3 Replies)
Discussion started by: tomasl
3 Replies
3. Shell Programming and Scripting
I am unable to spit the file based on the 2nd column passing as a parameter with awk command.
Source file:
“100”,”customer information”,”10000”
“200”,”customer information”,”50000”
“300”,”product information”,”40000”
script: the command is not allowing to pass the parameters with the awk... (7 Replies)
Discussion started by: number10
7 Replies
4. Shell Programming and Scripting
Hi,
I have a fixed width text file without any header row. One of the columns contains a date in YYYYMMDD format.
If the original file contains 3 dates, I want my shell script to split the file into 3 small files with data for each date.
I am a newbie and need help doing this. (14 Replies)
Discussion started by: bhanja_trinanja
14 Replies
5. Shell Programming and Scripting
Hi All
I have one query,say i have a requirement like the below code should be
move to diffent files whose maximum lines can be of 10 lines.Say in the below example,it consist of 14 lines.
This should be moved logically using the data in the fisrt coloumn to file1 and file 2.The data of first... (2 Replies)
Discussion started by: sarav.shan
2 Replies
6. Shell Programming and Scripting
Hi,
I have a file sample_1.txt (300k rows) which has data like below:
* Also each record is around 64k bytes
11|1|abc|102553|125589|64k bytes of data
10|2|def|123452|123356|......
13|2|geh|144351|121123|...
25|4|fgh|165250|118890|..
14|1|abc|186149|116657|......... (6 Replies)
Discussion started by: sol_nov
6 Replies
7. Shell Programming and Scripting
Help needed urgently please.
I have a large file - a few hundred thousand lines.
Sample
CP START ACCOUNT
1234556
name 1
CP END ACCOUNT
CP START ACCOUNT
2224444
name 1
CP END ACCOUNT
CP START ACCOUNT
333344444
name 1
CP END ACCOUNT
I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies
8. UNIX for Dummies Questions & Answers
i have file1.txt
asdas|csada|130310|0423|A1|canberra
sdasd|sfdsf|130426|2328|A1|sydney
Expected output : on eaceh third and fourth colum, split into each two characters
asdas|csada|13|03|10|04|23|A1|canberra
sdasd|sfdsf|13|04|26|23|28|A1|sydney (10 Replies)
Discussion started by: radius
10 Replies
9. Shell Programming and Scripting
Hi All,
I have a requirement to split file into 2 sets of file. Below is a sample data of the file
AU;PTN;24EX;25-AUG-14;AU;123;SE;123;Test NN;;;;ASD;
AU;PTN;24EX;25-AUG-14;AU;456;SE;456;Test NN;;;;ASD;
AU;PTN;24EX;25-AUG-14;AU;147;SE;147;Test NN;;;;ASD;... (6 Replies)
Discussion started by: galaxy_rocky
6 Replies
10. Shell Programming and Scripting
Hi Team,
I have a requirement in such a way that need to split the file into two based on which column particular value appears.Please find my sample file below.
Lets consider the delimiter of this file as either comma or two colons.(:: and ,). So I need to split the file in such a way that all... (2 Replies)
Discussion started by: ginrkf
2 Replies
LEARN ABOUT DEBIAN
mfontset_lookup
mfontset_lookup(3m17n) The m17n Library mfontset_lookup(3m17n)
NAME
mfontset_lookup - Lookup a fontset.
SYNOPSIS
MPlist * mfontset_lookup (MFontset * fontset, MSymbol script, MSymbol language, MSymbol charset)
DESCRIPTION
Lookup a fontset. The mfontset_lookup() function lookups fontset and returns a plist that describes the contents of fontset corresponding
to the specified script, language, and charset.
If script is Mt, keys of the returned plist are script name symbols for which some fonts are specified and values are NULL.
If script is a script name symbol, the returned plist is decided by language.
o If language is Mt, keys of the plist are language name symbols for which some fonts are specified and values are NULL. A key may be Mt
which means some fallback fonts are specified for the script.
o If language is a language name symbol, the plist is a FONT-GROUP for the specified script and language. FONT-GROUP is a plist whose keys
are FLT (FontLayoutTable) name symbols (Mt if no FLT is associated with the font) and values are pointers to MFont.
o If language is Mnil, the plist is fallback FONT-GROUP for the script.
If script is Mnil, the returned plist is decided as below.
o If charset is Mt, keys of the returned plist are charset name symbols for which some fonts are specified and values are NULL.
o If charset is a charset name symbol, the plist is a FONT-GROUP for the charset.
o If charset is Mnil, the plist is a fallback FONT-GROUP.
RETURN VALUE
It returns a plist describing the contents of a fontset. The plist should be freed by m17n_object_unref().
COPYRIGHT
Copyright (C) 2001 Information-technology Promotion Agency (IPA)
Copyright (C) 2001-2011 National Institute of Advanced Industrial Science and Technology (AIST)
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License
<http://www.gnu.org/licenses/fdl.html>.
Version 1.6.2 12 Jan 2011 mfontset_lookup(3m17n)