05-18-2006
Fast way to split a tab delimited file
I have searched the forum and tried different options. One of the options work but is very slow. The file has millions and millions of records.
It is a TAB delimited file which contains two types of records. Metadata and Detail records.
M PARTNER 8 LAST_BOOKED_DATE D YYYYMMDD
M PARTNER 8 TRIPS_YTD A 11 TRIPS_TOTAL
D NAME FIRST LAST 209 N SANBORN AVE
D NAME FIRST LAST 6997 COUNTY ROAD D
I need to split the file into two files by looking at the first character. All records that start with 'M' go into one file and all records that start with 'D' go into another file.
The following code works but it is too slow...Is there any fast way of accomplishing it?
#!/usr/bin/ksh
while read line
do
char=`echo "$line" | cut -c1`
if [ "$char" = "M" ]; then
echo "$line" >> M.txt
else
echo "$line" >> D.txt
fi
done < head10000.out
exit 0
Any help would be appreciated.
Thank You,
Madhu
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi there
Just wondered if someone could help me out
I have a file that has been delimited by tabs, ie
field1<tab>fiield2<tab>field3
Does anybody know a command that will convert tabs to commas throughout the entire file?
Note: there are a number of unpopulated fields in the file so... (6 Replies)
Discussion started by: hcclnoodles
6 Replies
2. Shell Programming and Scripting
Hi,
Can anyone let me know on how to convert a Tab delimited file to Comma delimited file in Unix
Thanks!! (22 Replies)
Discussion started by: charan81
22 Replies
3. UNIX for Dummies Questions & Answers
Hi all,
I have a file with single white space delimited values, I want to convert them to a tab delimited file.
I tried sed, tr ... but nothing is working.
Thanks,
Rajeevan D (16 Replies)
Discussion started by: jeevs81
16 Replies
4. UNIX for Dummies Questions & Answers
Hi
I have 100 files each with only one column of 10 numbers that I wish to add to a new file so that I get 100 columns collected in one tab delimited file.
I tried something like:
foreach num (1 2 3)
foreach? gawk -F '\t' '{$num=$1}1' OFS='\t' Eu9_10.2patienter/pospep_10.2patient$num >>... (5 Replies)
Discussion started by: Banni
5 Replies
5. UNIX for Dummies Questions & Answers
Hi Forum
I have a tab delimited file that opens well in Openoffice calc (excel). But when I perform any operation in command line, it reads the file incorrectly. When I 'save As' the same file in office as tab delimited then it works fine.
The file that I think is tab delimited is actually... (8 Replies)
Discussion started by: imlearning
8 Replies
6. Shell Programming and Scripting
I have a file which was pipe delimited, I need to make it tab delimited. I tried with sed but no use
cat file | sed 's/|//t/g'
The above command substituted "/t" not tab in the place of pipe.
Sample file:
abc|123|2012-01-30|2012-04-28|xyz
have to convert to:
abc 123... (6 Replies)
Discussion started by: karumudi7
6 Replies
7. Shell Programming and Scripting
Hi How to make tab delimited file to space delimited?
in put file:
ABC kgy
jkh ghj
ash kjl
o/p file:
ABC kgy
jkh ghj
ash kjl
Use code tags, thanks. (1 Reply)
Discussion started by: jagdishrout
1 Replies
8. UNIX for Dummies Questions & Answers
Hi, I have a rquirement in unix as below .
I have a text file with me seperated by | symbol and i need to generate a excel file through unix commands/script so that each value will go to each column.
ex:
Input Text file:
1|A|apple
2|B|bottle
excel file to be generated as output as... (9 Replies)
Discussion started by: raja kakitapall
9 Replies
9. Shell Programming and Scripting
Hi,
I have requirement to create tab delimited file with values coming from variables.
File will contain only two columns separated by tab.
Header will be added once.
Values will be keep adding upon the script run.
If values already exists then values will be replaced.
I have done so... (1 Reply)
Discussion started by: sukhdip
1 Replies
10. UNIX for Beginners Questions & Answers
Hello Everyone..
I want to replace the retail col from FileI with cstp1 col from FileP if the strpno matches in both files
FileP.txt
... (2 Replies)
Discussion started by: YogeshG
2 Replies
LEARN ABOUT PHP
pg_get_result
PG_GET_RESULT(3) PG_GET_RESULT(3)
pg_get_result - Get asynchronous query result
SYNOPSIS
resource pg_get_result ([resource $connection])
DESCRIPTION
pg_get_result(3) gets the result resource from an asynchronous query executed by pg_send_query(3), pg_send_query_params(3) or pg_send_exe-
cute(3).
pg_send_query(3) and the other asynchronous query functions can send multiple queries to a PostgreSQL server and pg_get_result(3) is used
to get each query's results, one by one.
PARAMETERS
o $connection
- PostgreSQL database connection resource.
RETURN VALUES
The result resource, or FALSE if no more results are available.
EXAMPLES
Example #1
pg_get_result(3) example
<?php
$dbconn = pg_connect("dbname=publisher") or die("Could not connect");
if (!pg_connection_busy($dbconn)) {
pg_send_query($dbconn, "select * from authors; select count(*) from authors;");
}
$res1 = pg_get_result($dbconn);
echo "First call to pg_get_result(): $res1
";
$rows1 = pg_num_rows($res1);
echo "$res1 has $rows1 records
";
$res2 = pg_get_result($dbconn);
echo "Second call to pg_get_result(): $res2
";
$rows2 = pg_num_rows($res2);
echo "$res2 has $rows2 records
";
?>
The above example will output:
First call to pg_get_result(): Resource id #3
Resource id #3 has 3 records
Second call to pg_get_result(): Resource id #4
Resource id #4 has 1 records
SEE ALSO
pg_send_query(3).
PHP Documentation Group PG_GET_RESULT(3)