Finding distinct characters from flat file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Finding distinct characters from flat file
# 8  
Old 04-08-2013
Yoda's awk solution works for me. What do you mean you are not getting the expected result?

Code:
[root@Imperfecto_1 ~]# cat infile
a
b
c c
d d d g
e
f
[root@Imperfecto_1 ~]# awk '{ for(i=1;i<=NF;i++){a[$i]} } END{for (i in a) {print i}}' infile
a
b
c
d
e
f
g

And for typeset error, try typeset -a ARR
Code:
man typeset
-a     Each name is an array variable (see Arrays above).

I used bash though, not sure about ksh

--ahamed
# 9  
Old 04-08-2013
Please note that -A option used to define associative arrays is available only on modern KSH version. The option -a is used to define indexed arrays.

From KSH manual:
Code:
-A     Declares vname to be an associative array.
-a     Declares vname to be an indexed array.

This User Gave Thanks to Yoda For This Post:
# 10  
Old 04-08-2013
Hello All....the nawk is now working for me also. Thank you guys.

But do not know why the typeset option is not working for me. I have been trying with all options even tried with declare option.

Thank you Guys....
# 11  
Old 04-08-2013
Quote:
Originally Posted by Krishanu Saha
But do not know why the typeset option is not working for me. I have been trying with all options even tried with declare option.
You probably have an old version of ksh that lacks this feature.
# 12  
Old 04-08-2013
Hello All....1 quick question. If the file contains the character list in following way then the suggested awk command is working fine. Now if there is no space bewteen 2 characters in single line (for eg. CC instead of C C) then this command is not working. Any help if there is no space between 2 chars then how do I get the same result from this string?

Character List:

a
b
c c
d d d g
e
f
# 13  
Old 04-09-2013
Code:
awk ' {
        v = $0
        gsub (/[ \t]*/, x, v)
        for ( i = 1; i <= length(v); i++ )
        {
                d = substr ( v, i, 1 )
                A[d]
        }
} END {
        for ( k in A )
                print k
} ' file

# 14  
Old 04-09-2013
Code:
$ cat input.txt
a
b hi
cc
d d d g
 e
f

Code:
$ sed "s/\(.\)/\1\n/g" input.txt | sort | uniq | grep -v "^ *$"
a
b
c
d
e
f
g
h
i

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Finding total distinct count from multiple csv files through UNIX script

Hi All , I have multiple pipe delimited csv files are present in a directory.I need to find out distinct count on a column on those files and need the total distinct count on all files. We can't merge all the files here as file size are huge in millions.I have tried in below way for each... (9 Replies)
Discussion started by: STCET22
9 Replies

2. Shell Programming and Scripting

Finding first 6 characters

Hi , I'm using KSH88 I tried the following example to get the last 6 characters from a string echo 'abcdefghids' | sed 's/.*\(.\{6\}\)$/\1/' What chages i need to do to get the first 6 characters from the string my desired output should be abcdef Thank you (6 Replies)
Discussion started by: smile689
6 Replies

3. Shell Programming and Scripting

Finding Strings between 2 characters in a file

Hi All, Assuming i have got a file test.dat which has contains as follows: Unix = abc def fgt jug 111 2222 3333 Linux = gggg pppp qqq C# = ccc ffff llll I would like to traverse through the file, get the 1st occurance of "=" and then need to get the sting... (22 Replies)
Discussion started by: rtagarra
22 Replies

4. UNIX for Dummies Questions & Answers

How to remove numeric characters in the flat file

HI, can any one help me please .. i have flat file like qwer123rt ass3242ccf jjk654 kjh838ppp nhdg453ok hdkk34 i want remove numeric characters in the flat file i want output like this qwerrt assccf jjk kjhppp nhdgok hdkk help me... (4 Replies)
Discussion started by: rafimd1985
4 Replies

5. Shell Programming and Scripting

Finding File Names Ending In 3 Random Numerical Characters

Hi, I have a series of files (upwards of 500) the filename format is as follows CC10-1234P1999.WGS84.p190 each of this files is in a directory named for the file but excluding the extension. Now the last three numeric characters, in this case 999, can be anything from 001 to 999, I need to... (3 Replies)
Discussion started by: roche.j.mike
3 Replies

6. Shell Programming and Scripting

Select distinct values from a flat file

Hi , I have a similar problem. Please can anyone help me with a shell script or a perl. I have a flat file like this fruit country apple germany apple india banana pakistan banana saudi mango india I want to get a output like fruit country apple ... (7 Replies)
Discussion started by: smalya
7 Replies

7. AIX

How to cut a flat file according to a certain number of characters?

hello everybody i am looking for a shell to cut a flat file (with a long unique line) according to a certain number of characters and redirect every result to an output file. here is an example MyFile : 12 3 456 12 3 456 12 3 456 ..... and i took every 9-characters including BLANKS... (6 Replies)
Discussion started by: fastlane3000
6 Replies

8. Shell Programming and Scripting

Merge lines in Flat file based on first 5 characters

Hi I have the fixed width flat file having the following data 12345aaaaaaaaaabbbbbbbbbb 12365sssssssssscccccccccc 12365sssss 12367ddddddddddvvvvvvvvvv 12367 vvvvv Here the first column is length 5 second is length 10 third is length 10 if the second or third column exceeds... (3 Replies)
Discussion started by: Brado
3 Replies

9. UNIX for Advanced & Expert Users

foreign characters in flat file

Hey, Is there anyway I anks, Pocha (12 Replies)
Discussion started by: pochaman
12 Replies

10. Shell Programming and Scripting

Help Replacing Characters in Flat File

I was wondering if somebody could help me with something on UNIX. I have a file that looks like this - "nelson,bill","bill","123 Main St","Mpls","MN",55444,8877,william I want to replace all comma with pipes (|), except if the comma is within double quotes. (The first field is an example of... (8 Replies)
Discussion started by: nelson553011
8 Replies
Login or Register to Ask a Question