Sponsored Content
Top Forums Shell Programming and Scripting Find duplicates in the first column of text file Post 302432811 by guruprasadpr on Sunday 27th of June 2010 05:23:48 AM
Old 06-27-2010
Hi
First time, when the file is processed, it takes the count of 1st column duplicates. Second time, when it is processed, it starts printing those lines which has count more than 1.

btw, did it work?

Guru.
This User Gave Thanks to guruprasadpr For This Post:
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to find the number of column in the text file...?

Hi, i have text file with ~ seperated columns. it is very huge size of file, in the file sompulsary supposed to has 20 columns with ~ seperated. so how can i find if the file has 20 column in the all rows...? Sample file: APA+VU~10~~~~~03~101~101~~~APA.N O 20081017 120.00... (1 Reply)
Discussion started by: psiva_arul
1 Replies

2. UNIX for Dummies Questions & Answers

Remove duplicates based on a column in fixed width file

Hi, How to output the duplicate record to another file. We say the record is duplicate based on a column whose position is from 2 and its length is 11 characters. The file is a fixed width file. ex of Record: DTYU12333567opert tjhi kkklTRG9012 The data in bold is the key on which... (1 Reply)
Discussion started by: Qwerty123
1 Replies

3. Shell Programming and Scripting

need to remove duplicates based on key in first column and pattern in last column

Given a file such as this I need to remove the duplicates. 00060011 PAUL BOWSTEIN ad_waq3_921_20100826_010517.txt 00060011 PAUL BOWSTEIN ad_waq3_921_20100827_010528.txt 0624-01 RUT CORPORATION ad_sade3_10_20100827_010528.txt 0624-01 RUT CORPORATION ... (13 Replies)
Discussion started by: script_op2a
13 Replies

4. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

5. Red Hat

How to find a garbage entry in a column wise text file in Linux?

Suppose I have a file containing :- 1 Apple $50 2 Orange $30 3 Banana $10 4 Guava $25 5 Pine@apple $12 6 Strawberry $21 7 Grapes $12 In the 5th row, @ character inserted. I want through sort command or by any other way this row should either on top or bottom. By sort command garbage... (1 Reply)
Discussion started by: Dipankar Mitra
1 Replies

6. Shell Programming and Scripting

Find duplicates in column 1 and merge their lines (awk?)

Hi, I have a file (sorted by sort) with 8 tab delimited columns. The first column contains duplicated fields and I need to merge all these identical lines. My input file: comp100002 aaa bbb ccc ddd eee fff ggg comp100003 aba aba aba aba aba aba aba comp100003 fff fff fff fff fff fff fff... (5 Replies)
Discussion started by: falcox
5 Replies

7. Shell Programming and Scripting

Find duplicates in 2 & 3rd column and their ID

with below given format, I have been trying to find out all IDs for those entries with duplicate names in 2nd and 3rd columns and their count like how many time duplication happened for any name if any, 0.237788 Aaban Aahva 0.291066 Aabheer Aahlaad 0.845814 Aabid Aahan 0.152208 Aadam... (6 Replies)
Discussion started by: busyboy
6 Replies

8. Shell Programming and Scripting

awk to Sum columns when other column has duplicates and append one column value to another with Care

Hi Experts, Please bear with me, i need help I am learning AWk and stuck up in one issue. First point : I want to sum up column value for column 7, 9, 11,13 and column15 if rows in column 5 are duplicates.No action to be taken for rows where value in column 5 is unique. Second point : For... (1 Reply)
Discussion started by: as7951
1 Replies

9. UNIX for Beginners Questions & Answers

Find duplicates in file with line numbers

Hello All, This is a noob question. I tried searching for the answer but the answer found did not help me . I have a file that can have duplicates. 100 200 300 400 100 150 the number 100 is duplicated twice. I want to find the duplicate along with the line number. expected... (4 Replies)
Discussion started by: vatigers
4 Replies
fold(1) 							   User Commands							   fold(1)

NAME
fold - filter for folding lines SYNOPSIS
fold [-bs] [-w width | -width] [file]... DESCRIPTION
The fold utility is a filter that will fold lines from its input files, breaking the lines to have a maximum of width column positions (or bytes, if the -b option is specified). Lines will be broken by the insertion of a NEWLINE character such that each output line (referred to later in this section as a segment) is the maximum width possible that does not exceed the specified number of column positions (or bytes). A line will not be broken in the middle of a character. The behavior is undefined if width is less than the number of columns any single character in the input would occupy. If the CARRIAGE-RETURN, BACKSPACE, or TAB characters are encountered in the input, and the -b option is not specified, they will be treated specially: BACKSPACE The current count of line width will be decremented by one, although the count never will become negative. fold will not insert a NEWLINE character immediately before or after any BACKSPACE character. CARRIAGE-RETURN The current count of line width will be set to 0. fold will not insert a NEWLINE character immediately before or after any CARRIAGE-RETURN character. TAB Each TAB character encountered will advance the column position pointer to the next tab stop. Tab stops will be at each column position n such that n modulo 8 equals 1. OPTIONS
The following options are supported: -b Counts width in bytes rather than column positions. -s If a segment of a line contains a blank character within the first width column positions (or bytes), breaks the line after the last such blank character meeting the width constraints. If there is no blank character meeting the require- ments, the -s option will have no effect for that output segment of the input line. -w width|-width Specifies the maximum line length, in column positions (or bytes if -b is specified). If width is not a positive deci- mal number, an error is returned. The default value is 80. OPERANDS
The following operand is supported: file A path name of a text file to be folded. If no file operands are specified, the standard input will be used. EXAMPLES
Example 1 Submitting a file of possibly long lines to the line printer An example invocation that submits a file of possibly long lines to the line printer (under the assumption that the user knows the line width of the printer to be assigned by lp(1)): example% fold -w 132 bigfile | lp ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of fold: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 All input files were processed successfully. >0 An error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |CSI |enabled | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
cut(1), pr(1), attributes(5), environ(5), standards(5) NOTES
fold and cut(1) can be used to create text files out of files with arbitrary line lengths. fold should be used when the contents of long lines need to be kept contiguous. cut should be used when the number of lines (or records) needs to remain constant. fold is frequently used to send text files to line printers that truncate, rather than fold, lines wider than the printer is able to print (usually 80 or 132 column positions). fold may not work correctly if underlining is present. SunOS 5.11 1 Feb 1995 fold(1)
All times are GMT -4. The time now is 09:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy