Finding distinct characters from flat file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Finding distinct characters from flat file
# 15  
Old 04-09-2013
Thanks a lot Yoda....the awk code is working fine and showing me the exact result what I'm expecting. Thanks a lot.
# 16  
Old 04-09-2013
Quote:
Originally Posted by hanson44
Code:
$ sed "s/\(.\)/\1\n/g" input.txt | sort | uniq | grep -v "^ *$"

A few thoughts regarding your solution ...

Most sed implementations do not recognize the "\n" escape sequence in replacement text. Typically, a backslash followed by a newline is required.

The \1 backreference isn't necessary, since the entirety of the matching text is all that's needed and sed already makes that available via &.

You don't need to use uniq. sort -u will do the job.

grep isn't necessary. Since you are already using sed, you can use it to delete blanks at the start.

An untested variation of your approach:
Code:
sed 's/ //g; /./!d; s/./&\
/g; s/.$//' file | sort -u

Equally untested code:
Code:
tr -d ' \b\t\r' < file | fold -w1 | sort -u

Regards,
Alister
# 17  
Old 04-09-2013
Thank you for the thoughtful suggestions.
# 18  
Old 04-09-2013
Quote:
Originally Posted by alister
Most sed implementations do not recognize the "\n" escape sequence in replacement text. Typically, a backslash followed by a newline is required.
The backslash would prevent a literal newline from being fed into sed. Leave it off.

Code:
$ echo "a\
b"

ab

$ echo "a
b"

a
b

$

# 19  
Old 04-09-2013
Quote:
Originally Posted by Corona688
The backslash would prevent a literal newline from being fed into sed. Leave it off.
You are mistaken. To portably insert a newline in sed replacement text, a backslash-newline pair is required.

From POSIX sed:
Quote:
A line can be split by substituting a <newline> into it. The application shall escape the <newline> in the replacement by preceding it by a <backslash>.
Quote:
Originally Posted by Corona688
Code:
$ echo "a\
b"

ab

$ echo "a
b"

a
b

$

What you are demonstrating is sh line continuation. This is unrelated to the requirements of a sed script.

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Finding total distinct count from multiple csv files through UNIX script

Hi All , I have multiple pipe delimited csv files are present in a directory.I need to find out distinct count on a column on those files and need the total distinct count on all files. We can't merge all the files here as file size are huge in millions.I have tried in below way for each... (9 Replies)
Discussion started by: STCET22
9 Replies

2. Shell Programming and Scripting

Finding first 6 characters

Hi , I'm using KSH88 I tried the following example to get the last 6 characters from a string echo 'abcdefghids' | sed 's/.*\(.\{6\}\)$/\1/' What chages i need to do to get the first 6 characters from the string my desired output should be abcdef Thank you (6 Replies)
Discussion started by: smile689
6 Replies

3. Shell Programming and Scripting

Finding Strings between 2 characters in a file

Hi All, Assuming i have got a file test.dat which has contains as follows: Unix = abc def fgt jug 111 2222 3333 Linux = gggg pppp qqq C# = ccc ffff llll I would like to traverse through the file, get the 1st occurance of "=" and then need to get the sting... (22 Replies)
Discussion started by: rtagarra
22 Replies

4. UNIX for Dummies Questions & Answers

How to remove numeric characters in the flat file

HI, can any one help me please .. i have flat file like qwer123rt ass3242ccf jjk654 kjh838ppp nhdg453ok hdkk34 i want remove numeric characters in the flat file i want output like this qwerrt assccf jjk kjhppp nhdgok hdkk help me... (4 Replies)
Discussion started by: rafimd1985
4 Replies

5. Shell Programming and Scripting

Finding File Names Ending In 3 Random Numerical Characters

Hi, I have a series of files (upwards of 500) the filename format is as follows CC10-1234P1999.WGS84.p190 each of this files is in a directory named for the file but excluding the extension. Now the last three numeric characters, in this case 999, can be anything from 001 to 999, I need to... (3 Replies)
Discussion started by: roche.j.mike
3 Replies

6. Shell Programming and Scripting

Select distinct values from a flat file

Hi , I have a similar problem. Please can anyone help me with a shell script or a perl. I have a flat file like this fruit country apple germany apple india banana pakistan banana saudi mango india I want to get a output like fruit country apple ... (7 Replies)
Discussion started by: smalya
7 Replies

7. AIX

How to cut a flat file according to a certain number of characters?

hello everybody i am looking for a shell to cut a flat file (with a long unique line) according to a certain number of characters and redirect every result to an output file. here is an example MyFile : 12 3 456 12 3 456 12 3 456 ..... and i took every 9-characters including BLANKS... (6 Replies)
Discussion started by: fastlane3000
6 Replies

8. Shell Programming and Scripting

Merge lines in Flat file based on first 5 characters

Hi I have the fixed width flat file having the following data 12345aaaaaaaaaabbbbbbbbbb 12365sssssssssscccccccccc 12365sssss 12367ddddddddddvvvvvvvvvv 12367 vvvvv Here the first column is length 5 second is length 10 third is length 10 if the second or third column exceeds... (3 Replies)
Discussion started by: Brado
3 Replies

9. UNIX for Advanced & Expert Users

foreign characters in flat file

Hey, Is there anyway I anks, Pocha (12 Replies)
Discussion started by: pochaman
12 Replies

10. Shell Programming and Scripting

Help Replacing Characters in Flat File

I was wondering if somebody could help me with something on UNIX. I have a file that looks like this - "nelson,bill","bill","123 Main St","Mpls","MN",55444,8877,william I want to replace all comma with pipes (|), except if the comma is within double quotes. (The first field is an example of... (8 Replies)
Discussion started by: nelson553011
8 Replies
Login or Register to Ask a Question