Programming

is there any unix tools that can read the text files like through SQL queries? (ie in Hadoop, Impala DB support flat file query)

Not last time I asked. It doesn't make much sense to do SQL on something without keys, data types, or any kind of constraints.

Figuring how to manage without was the impetus for me to learn awk. awk can do simple filtering as a trivial one-liner:

Code:

awk -F"\t" '($1 == "MYTYPE") && (($2+0) > 37)' tab-separated-flatfile

All lines where column one is MYTYPE and column 2 is an integer or floating point greater than 37 will be printed. (The +0 forces it to interpret $2 as a number, not a string.)

There's also the join utility which can join flatfiles on a common field.

Please explain in more detail what you need.