Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Find duplicates in file with line numbers Post 303020210 by RudiC on Sunday 15th of July 2018 02:42:24 AM
Old 07-15-2018
That seems an indicator your file has non-*nix DOS line terminators (<carriage return>, <CR>, \r, ^M, 0x0D). Remove before continuing, e.g. with dos2unix.


There's no colon in the file - no reason to set it as FS. Try
Code:
awk 'a[$1]++ {print $0, NR}' file
100 5

This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

find 2 line numbers, grab text in between

hi, i have a large text file that I just want to extract the important information from. It will be a random number of lines but between two specific line numbers/markers. I was thinking I could get the line number for the first marker: Tablespace Percent Total Free Then get the line... (11 Replies)
Discussion started by: Da_Duck
11 Replies

2. UNIX for Dummies Questions & Answers

Question About Getting Line Numbers in a File

Ok, this is a unique question. Say I have a word like "jamamamama" in a file called foo.bar. Now, how do you get the line number that the word "jamammama" exist in the file foo.bar without having to go into the foo.bar file to edit it? is there a command i can run on the foo.bar... (4 Replies)
Discussion started by: SkySmart
4 Replies

3. UNIX for Dummies Questions & Answers

Adding Numbers in a line from a file

Hello everyone ! Ive searched everywhere and still havnt found enough information to help me overcome (what seems like) a small problem I have created a temporary file in which i store numbers which a seperated by a space, eg) 5 10 46 23 866 392 i wish to take the numbers for each line... (2 Replies)
Discussion started by: healwithchaos
2 Replies

4. Shell Programming and Scripting

Unix help to find blank lines in a file and print numbers on that line

Hi, I would like to know how to solve one of my problems using expert unix commands. I have a file with occasional blank lines; for example; dertu frthu fghtu frtty frtgy frgtui frgtu ghrye frhutp frjuf I need to edit the file so that the file looks like this; (10 Replies)
Discussion started by: Lucky Ali
10 Replies

5. Shell Programming and Scripting

Find duplicates in the first column of text file

Hello, My text file has input of the form abc dft45.xml ert rt653.xml abc ert57.xml I need to write a perl script/shell script to find duplicates in the first column and write it into a text file of the form... abc dft45.xml abc ert57.xml Can some one help me plz? (5 Replies)
Discussion started by: gameboy87
5 Replies

6. Shell Programming and Scripting

Assign Line Numbers to each line of the file

Hi! I'm trying to assign line numbers to each line of the file for example consider the following.. The contents of the input file are hello how are you? I'm fine. How about you? I'm trying to get the following output.. 1 hello how are you? 2 I'm fine. 3 How about you? ... (8 Replies)
Discussion started by: abk07
8 Replies

7. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

8. Shell Programming and Scripting

find all numbers > x and replace with y within a file

How would I do this? How could i use <> symbols for numbers in the find/replace code below? perl -pi -e 's/test/tst/' OR is there a better way? 100 5000 2 432 4 2 33 4 5 6 65 300 301 needs to be: 100 300 2 300 4 2 33 4 5 6 65 300 300 also it might not always need spaces... i... (12 Replies)
Discussion started by: herot
12 Replies

9. Shell Programming and Scripting

How to find numbers in text file?

Hi I have a text file with rows like this: 7 Herman ASI-40 Jungle (L) Blueprint (L) Weapon Herman ASI-40 Jungle (L) 215.00 57 65.21 114.41 and 9 Herman CAP-505 (L) Blueprint (L) Weapon Herman CAP-505 (L) 220.00 46.84 49.1 104.82 and 2 ClericDagger 1C blueprint Melee - Shortblade... (2 Replies)
Discussion started by: pesa
2 Replies

10. Shell Programming and Scripting

Find Special/Null/Control Chars and Print Line Numbers

Hi All, This might be a basic question... I need to write a script to find all/any Speacial/Null/Control Chars and Print Line Numbers from an input file. Output something like Null Characters in File Name at : Line Numbers Line = Print the line Control Characters in File Name at : Line... (2 Replies)
Discussion started by: Kevin Tivoli
2 Replies
dos2unix(1)						      General Commands Manual						       dos2unix(1)

NAME
dos2unix - mtools utility to convert a DOS file to UNIX SYNOPSIS
dos2unix msdosfile OPTIONS
None OPERANDS
Name of the file to convert. DESCRIPTION
The dos2unix command converts a file from DOS format to UNIX format. The converted file replaces the original file. Subdirectory names that contain the '/' or '' separator are supported. If you use the '' separator or wildcards, you must enclose file names in quotes to protect them from the shell. The mcd command can be used to establish the device and the current working directory (relative to DOS), otherwise the default is A:. EXIT STATUS
The following exit values are returned: Success. Failure. ENVIRONMENT VARIABLES
The following environment variables affect the execution of dos2unix: If set, this variable names the file that contains the name of the current mtools working directory as established by the mcd command. If this variable is not set, the file $HOME/.mcwd is used. FILES
Contains the name of the current mtools working directory as established by the mcd command. If this file does not exist, the default mtools working directory is A:. Executable file SEE ALSO
Commands: mcopy(1), mread(1), mtools(1) mtype(1), mwrite(1) dos2unix(1)
All times are GMT -4. The time now is 12:18 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy