Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Find duplicates in file with line numbers Post 303020210 by RudiC on Sunday 15th of July 2018 02:42:24 AM
Old 07-15-2018
That seems an indicator your file has non-*nix DOS line terminators (<carriage return>, <CR>, \r, ^M, 0x0D). Remove before continuing, e.g. with dos2unix.


There's no colon in the file - no reason to set it as FS. Try
Code:
awk 'a[$1]++ {print $0, NR}' file
100 5

This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

find 2 line numbers, grab text in between

hi, i have a large text file that I just want to extract the important information from. It will be a random number of lines but between two specific line numbers/markers. I was thinking I could get the line number for the first marker: Tablespace Percent Total Free Then get the line... (11 Replies)
Discussion started by: Da_Duck
11 Replies

2. UNIX for Dummies Questions & Answers

Question About Getting Line Numbers in a File

Ok, this is a unique question. Say I have a word like "jamamamama" in a file called foo.bar. Now, how do you get the line number that the word "jamammama" exist in the file foo.bar without having to go into the foo.bar file to edit it? is there a command i can run on the foo.bar... (4 Replies)
Discussion started by: SkySmart
4 Replies

3. UNIX for Dummies Questions & Answers

Adding Numbers in a line from a file

Hello everyone ! Ive searched everywhere and still havnt found enough information to help me overcome (what seems like) a small problem I have created a temporary file in which i store numbers which a seperated by a space, eg) 5 10 46 23 866 392 i wish to take the numbers for each line... (2 Replies)
Discussion started by: healwithchaos
2 Replies

4. Shell Programming and Scripting

Unix help to find blank lines in a file and print numbers on that line

Hi, I would like to know how to solve one of my problems using expert unix commands. I have a file with occasional blank lines; for example; dertu frthu fghtu frtty frtgy frgtui frgtu ghrye frhutp frjuf I need to edit the file so that the file looks like this; (10 Replies)
Discussion started by: Lucky Ali
10 Replies

5. Shell Programming and Scripting

Find duplicates in the first column of text file

Hello, My text file has input of the form abc dft45.xml ert rt653.xml abc ert57.xml I need to write a perl script/shell script to find duplicates in the first column and write it into a text file of the form... abc dft45.xml abc ert57.xml Can some one help me plz? (5 Replies)
Discussion started by: gameboy87
5 Replies

6. Shell Programming and Scripting

Assign Line Numbers to each line of the file

Hi! I'm trying to assign line numbers to each line of the file for example consider the following.. The contents of the input file are hello how are you? I'm fine. How about you? I'm trying to get the following output.. 1 hello how are you? 2 I'm fine. 3 How about you? ... (8 Replies)
Discussion started by: abk07
8 Replies

7. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

8. Shell Programming and Scripting

find all numbers > x and replace with y within a file

How would I do this? How could i use <> symbols for numbers in the find/replace code below? perl -pi -e 's/test/tst/' OR is there a better way? 100 5000 2 432 4 2 33 4 5 6 65 300 301 needs to be: 100 300 2 300 4 2 33 4 5 6 65 300 300 also it might not always need spaces... i... (12 Replies)
Discussion started by: herot
12 Replies

9. Shell Programming and Scripting

How to find numbers in text file?

Hi I have a text file with rows like this: 7 Herman ASI-40 Jungle (L) Blueprint (L) Weapon Herman ASI-40 Jungle (L) 215.00 57 65.21 114.41 and 9 Herman CAP-505 (L) Blueprint (L) Weapon Herman CAP-505 (L) 220.00 46.84 49.1 104.82 and 2 ClericDagger 1C blueprint Melee - Shortblade... (2 Replies)
Discussion started by: pesa
2 Replies

10. Shell Programming and Scripting

Find Special/Null/Control Chars and Print Line Numbers

Hi All, This might be a basic question... I need to write a script to find all/any Speacial/Null/Control Chars and Print Line Numbers from an input file. Output something like Null Characters in File Name at : Line Numbers Line = Print the line Control Characters in File Name at : Line... (2 Replies)
Discussion started by: Kevin Tivoli
2 Replies
dos2unix(1)							   User Commands						       dos2unix(1)

NAME
dos2unix - convert text file from DOS format to ISO format SYNOPSIS
dos2unix [-ascii] [-iso] [-7] [-437 | -850 | -860 | -863 | -865] originalfile convertedfile DESCRIPTION
The dos2unix utility converts characters in the DOS extended character set to the corresponding ISO standard characters. This command can be invoked from either DOS or SunOS. However, the filenames must conform to the conventions of the environment in which the command is invoked. If the original file and the converted file are the same, dos2unix will rewrite the original file after converting it. OPTIONS
The following options are supported: -ascii Removes extra carriage returns and converts end of file characters in DOS format text files to conform to SunOS require- ments. -iso This is the default. It converts characters in the DOS extended character set to the corresponding ISO standard charac- ters. -7 Converts 8 bit DOS graphics characters to 7 bit space characters so that SunOS can read the file. On non-i386 systems, dos2unix will attempt to obtain the keyboard type to determine which code page to use. Otherwise, the default is US. The user may override the code page with one of the following options: -437 Use US code page -850 Use multilingual code page -860 Use Portuguese code page -863 Use French Canadian code page -865 Use Danish code page OPERANDS
The following operands are required: originalfile The original file in DOS format that is being converted to ISO format. convertedfile The new file in ISO format that has been converted from the original DOS file format. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWesu | +-----------------------------+-----------------------------+ SEE ALSO
unix2dos(1), ls(1), attributes(5) DIAGNOSTICS
File filename not found, or no read permission The input file you specified does not exist, or you do not have read permission. Check with the SunOS command, ls -l (see ls(1)). Bad output filename filename, or no write permission The output file you specified is either invalid, or you do not have write permission for that file or the directory that contains it. Check also that the drive or diskette is not write-protected. Error while writing to temporary file An error occurred while converting your file, possibly because there is not enough space on the current drive. Check the amount of space on the current drive using the DIR command. Also be certain that the default diskette or drive is write-enabled (not write-pro- tected). Notice that when this error occurs, the original file remains intact. Translated temporary file name = filename. Could not rename temporary file to filename. The program could not perform the final step in converting your file. Your converted file is stored under the name indicated on the second line of this message. SunOS 5.10 14 Sep 2000 dos2unix(1)
All times are GMT -4. The time now is 06:51 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy