|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Finding duplicate lines and deleting folders based on them
Post 302253164 by paresh n doshi on Friday 31st of October 2008 05:37:53 AM
10-31-2008
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hey all, a relative bash/script newbie trying solve a problem.
I've got a text file with lots of lines that I've been able to clean up and format with awk/sed/cut, but now I'd like to remove the lines with duplicate usernames based on time stamp. Here's what the data looks like
2007-11-03... (3 Replies)
Discussion started by: mattv
3 Replies
2. UNIX for Dummies Questions & Answers
hello all,
I have an input file with four columns like this with a lot of lines
and for example, line 1 and line 5 match because the first 4 characters match and the fourth column matches too. I want to keep the line that has the lowest number in the third column. So I discard line 5.... (5 Replies)
Discussion started by: TheTransporter
5 Replies
3. UNIX for Dummies Questions & Answers
HI,
My input file contains below data:
DFHDR
12345110
1,200
2,-100
1,100
2,123
12345110
1,300
2,200
DFTLR
In the above data, the first line and last lines should be remove as well as the lines in which contains 110 as position(6,7,8 position) should also be removed,
How we... (0 Replies)
Discussion started by: pandeesh
0 Replies
4. Shell Programming and Scripting
I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Discussion started by: cokedude
8 Replies
5. Shell Programming and Scripting
Greeting all! I could use some assistance please. :)
I've been searching for the best way to duplicate a line based on a variable in the next line.
Sample Data:
Nov 22 00:00:19 10.10.10.1 "%ASA-4-313005: No matching connection for ICMP error message: icmp src Outside:1.2.3.4 dst... (3 Replies)
Discussion started by: sjrupp
3 Replies
6. Shell Programming and Scripting
hi
i have a set of similar files. i want to delete lines until certain pattern appears in those files. for a single file the following command can be used but i want to do it for all the files at a time since the number is in thousands.
awk '/PATTERN/{i++}i' file (6 Replies)
Discussion started by: anurupa777
6 Replies
7. UNIX for Dummies Questions & Answers
Hi experts, I have a tab-delimited file with one column containing values separated by a comma. I wish to duplicate the entire line for every value in that comma-delimited field.
For example:
$cat file
4444 4444 4444 4444
9990 2222,7777 6666 2222 ... (3 Replies)
Discussion started by: torchij
3 Replies
8. Shell Programming and Scripting
I have a header-detail file that goes like this:
SHP00288820131021110921
ORDER0156605920131021110921INMMMMFN
DETAIL0004 4C2Z 10769 AAFC 0000009600000094 4C2Z 10769 AAFC 0000672107 OIL
DETAIL0002 ER3Z 14300 E 0000001300000012 ER3Z 14300 E 0000672107 OIL... (3 Replies)
Discussion started by: rbaggio666
3 Replies
9. Shell Programming and Scripting
Dear community,
I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns
The data are like this:
Region 23/11/2014 09:11:36 41752
Medio 23/11/2014 03:11:38 4132
Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies
10. Shell Programming and Scripting
Hi,
I have tried to remove dublicate lines based on first column with pipe delimiter . but i ma not able to get some uniqu lines
Command : sort -t'|' -nuk1 file.txt
Input :
38376KZ|09/25/15|1.057
38376KZ|09/25/15|1.057
02006YB|09/25/15|0.859
12593PS|09/25/15|2.803... (2 Replies)
Discussion started by: parithi06
2 Replies
LEARN ABOUT DEBIAN
fprev
NEW(1) [nmh-1.5] NEW(1) NAME
new - report on folders with new messages fnext - set current folder to next folder with new messages fprev - set current folder to previous folder with new messages unseen - scan new messages in all folders with new messages SYNOPSIS
new [sequences] [-mode mode] [-folders foldersfile] [-version] [-help] fnext is equivalent to new -mode fnext fprev is equivalent to new -mode fprev unseen is equivalent to new -mode unseen DESCRIPTION
New in its default mode produces a one-line-per-folder listing of all folders containing messages in the listed sequences or in the sequences listed in the profile entry "Unseen-Sequence". Each line contains the folder, the number of messages in the desired sequences, and the message lists from the .mh_sequences file. For example: foo 11.* 40-50 bar 380. 760-772 824-828 total 391. The `*' on foo indicates that it is the current folder. The last line shows the total number of messages in the desired sequences. New crawls the folder hierarchy recursively to find all folders, and prints them in lexicographic order. Override this behavior by provid- ing foldersfile containing the pre-sorted list of folders new should check, one per line. In fnext and fprev modes, new instead changes to the next or previous matching folder, respectively. In unseen mode, new executes scan sequences for each matching folder. FILES
$HOME/.mh_profile The user profile PROFILE COMPONENTS
Path: To determine the user's nmh directory Current-Folder: To find the default current folder Unseen-Sequence: The name of the unseen message sequence SEE ALSO
scan(1), mh-format(5) HISTORY
Based on Luke Mewburn's new (http://www.mewburn.net/luke/src/new). MH.6.8 11 June 2012 NEW(1)