Sponsored Content
Top Forums UNIX for Dummies Questions & Answers sort and find duplicates for files with no white space Post 302442459 by mmarshall on Wednesday 4th of August 2010 09:38:19 AM
Old 08-04-2010
sort and find duplicates for files with no white space

example data

5666700842511TAfmoham03151008075205999900000001000001000++
5666700843130MAfmoham03151008142606056667008390315100005001
6666666663130MAfmoham03151008142606056667008390315100005001


I'd like to sort on position 10-14 where the characters are eq "130MA".
Then based on positions 37 -60 if its a duplicate write it to a file.

I've seen a lot of examples where u can use delimeters but none where u can use column positions like the example above. Thx for your help in advance.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

stripping white space...

Hi All; Having a problem with a file.. the file contains the following data... (a snapshot) 1331F9E9DB7C2BB80EAEDE3A8F043B94,AL7 1DZ,M,50 186FDF93E1303DBA217279EC3671EA91,NG5 1JU,M,24 3783FFAF602015056A8CD21104B1AAAF,CH42 4NQ,M,17 It has 3 columns sepreated by a , the second column... (7 Replies)
Discussion started by: Zak
7 Replies

2. Shell Programming and Scripting

Find duplicates from multuple files with 2 diff types of files

I need to compare 2 diff type of files and find out the duplicate after comparing each types of files: Type 1 file name is like: file1.abc (the extension abc could any 3 characters but I can narrow it down or hardcode for 10/15 combinations). The other file is file1.bcd01abc (the extension... (2 Replies)
Discussion started by: ricky007
2 Replies

3. UNIX for Dummies Questions & Answers

SED with White Space

Dear Members, Suppose i have a variable test which stores a string as below: test='John drives+++++++++a+++++car' now i want to use sed on the above variable and replace + with a white space, so that i get echo $test should give me 'john drives a car' Between... (1 Reply)
Discussion started by: sandeep_1105
1 Replies

4. Shell Programming and Scripting

Delete trailing white space

I have a string "disk0 with a trailing white space after it" but I want to get rid of this white space from right to left so that I am left with "disk0" only. Using sed 's/ $//g' doesn't seem to work Any ideas ? Thanks (5 Replies)
Discussion started by: cillmor
5 Replies

5. Shell Programming and Scripting

sed + white space

Hi, What sed command (if sed is the right command) can remove ALL white space from my file. I have a csv, except I want to remove all white space between commas and characters. My idea (without testing) sed 's/ //g' Is there a better way? (18 Replies)
Discussion started by: mcclunyboy
18 Replies

6. UNIX for Dummies Questions & Answers

filename with white space

our user creates a text file with a white space on the filename. this same file is transfered to unix via automation tool. i have a korn shell script that reads these files on a input directory and connects to oracle database to run the oracle procedures which will load the data from each of the... (2 Replies)
Discussion started by: wtolentino
2 Replies

7. Shell Programming and Scripting

Remove unwanted white space

Hi, I have a very big file 25GB with information present in it like $ head ind_stats update index statistics pfirm001.dbo.Office using 200 values go ... (11 Replies)
Discussion started by: sam05121988
11 Replies

8. UNIX for Advanced & Expert Users

How to find duplicates contents in a files by comparing other files?

Hi Guys , we have one directory ...in that directory all files will be set on each day.. files must have header ,contents ,footer.. i wants to compare the header,contents,footer ..if its same means display an error message as 'files contents same' (7 Replies)
Discussion started by: Venkatesh1
7 Replies

9. Shell Programming and Scripting

Add white space

hi guys how can i add spacein file name with sed if strings have no space around dash input 19-20 ( 18-19 ) ABC-EFG output after add white space 19 - 20 (18 - 19 ) ABC - EFG thx in advance (2 Replies)
Discussion started by: mhs
2 Replies

10. UNIX for Dummies Questions & Answers

Find a list of files in directory, move to new, allow duplicates

Greetings. I know enough Unix to be dangerous (!) and know that there is a clever way to do the following and it will save me about a day of agony (this time) and I will use it forever after! (many days of agony saved in the future)! Basically I need to find any image files (JPGs, PSDs etc)... (5 Replies)
Discussion started by: Clyde Lovett
5 Replies
SORT(1) 						      General Commands Manual							   SORT(1)

NAME
sort - sort a file of ASCII lines SYNOPSIS
sort [-bcdfimnru] [-tc] [-o name] [+pos1] [-pos2] file ... OPTIONS
-b Skip leading blanks when making comparisons -c Check to see if a file is sorted -d Dictionary order: ignore punctuation -f Fold upper case onto lower case -i Ignore nonASCII characters -m Merge presorted files -n Numeric sort order -o Next argument is output file -r Reverse the sort order -t Following character is field separator -u Unique mode (delete duplicate lines) EXAMPLES
sort -nr file # Sort keys numerically, reversed sort +2 -4 file # Sort using fields 2 and 3 as key sort +2 -t: -o out # Field separator is : sort +.3 -.6 # Characters 3 through 5 form the key DESCRIPTION
Sort sorts one or more files. If no files are specified, stdin is sorted. Output is written on standard output, unless -o is specified. The options +pos1 -pos2 use only fields pos1 up to but not including pos2 as the sort key, where a field is a string of characters delim- ited by spaces and tabs, unless a different field delimiter is specified with -t. Both pos1 and pos2 have the form m.n where m tells the number of fields and n tells the number of characters. Either m or n may be omitted. SEE ALSO
comm(1), grep(1), uniq(1). SORT(1)
All times are GMT -4. The time now is 03:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy