Shell Programming and Scripting

Hi All,

Is there any method we can use to find what is the delimiter used in a text file, asuming the files has fixed number of colomns.

Thanks in advance.
Js

Delimiters are chosen because they have a property: they are not usually part of the data.
Spaces can delimit columns of numbers, but not columns of addresses because addresses may have spaces occur in them and you would get more columns than really exist in the data. This is kind of a hard problem the way you asked it.

So your question needs some help. Generally you get files and as a programmer you are given the file format, or for example a file.csv uses commas to separate fields, with " " around text that may contain a comma.

What problem are you trying to solve? Not how you think it should be solved.

This code tells you if certain commonly used delimters exist. BUT it does not mean that they are really the delimiter. This assumes your unix supports grep -q

Code:

tmp=$(head -1 filename)
echo "$tmp" | grep -q '[:space:]' && echo 'has white space: tab or spaces' 
echo "$tmp" | grep -q ',' && echo 'has commas'
echo "$tmp" | grep -q '|' && echo 'has pipe symbols'

Try:

Code:

perl -F"//" -nlae 'for (@F) {$a{$_}++};for (keys %a){print $_ if ($a{$_}+1==5 && !($_=~/\w/))};undef %a' file

Change red number for the number of columns in file. It will print non-word characters that are possible delimiters for every row. When you see the same character in every line, then it will probably be your delimiter.

Code:

### Open the file
open (FIN, "INPUT_FILE_PATH") || die "Cannot Open the input file : $!";
@file=<FIN>;
close (FIN);
### Take the first line only.
### which string found maximum time
### that considered the delimit
for ($file[0] =~ m{[^\w\n\.]+}g) # ignore the word character, '.' and enter mark
{
 # Get the maximum occurrence
 if ($max<$delim{$&}++)
 {
  $destr=$&;    # assign the delimiter string to a variable
  $max=$delim{$&}; # reset the maximum number
 }
}
# print the delimiter string
print "\n--$destr--";

I am trying to develop a scriptwhich takes 2 arguments and one option
Example

./1.pl <File1> <outputfile> -s ","

Here file1 is a text file withsome data (Say extracted from DB).
I do know know what this file contains or waht is the delimiter it has. But i am assuming the delimiter is coma or pipe(|) or single space or tab (4 spaces).

-s is the option indicating that "," is the delimiter of the file <File1>
-s can be "," or "|" or " " or "\t" for coma, pipe, space, tab respectively.

While reading File1 each line, I am using a split function passing the -s option.

My Problem here is
./1.pl <File1> <outputfile> -s "," ==> Works fine if File1 is coma separated

But what if File1 is pipe separated and i am specifying the -s option as coma.
Then too it works but i dont get teh expected result.

I hope the question is clear or am i confusing

Apply the above logic in your program. The '$destr' gives the delimiter string...

The code satisfies my requirement .. Thank you ..

Could you please explain me what the below code do?

if ($max<$delim{$&}++)
{
$destr=$&; # assign the delimiter string to a variable
$max=$delim{$&}; # reset the maximum number
}

I havenot used $& so far. And in the for loop you have given only $file[0] which means it reads only the first line of the file. Please correct me If i am wrong.

---------- Post updated at 12:24 PM ---------- Previous update was at 12:09 PM ----------


Thank you so much
This gives me a long list and takes the "." also. I shoud tune it ..