Sponsored Content
Top Forums Shell Programming and Scripting How to find duplicate line in log file? Post 302424720 by wax_light on Wednesday 26th of May 2010 06:52:44 AM
Old 05-26-2010
wow that is really fast, thank you,

I will try it, I use Bourne Shell

I hope it will work
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to find Duplicate Records in a text file

Hi all pls help me by providing soln for my problem I'm having a text file which contains duplicate records . Example: abc 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452 abc 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452 tas 3420 3562 ... (1 Reply)
Discussion started by: G.Aavudai
1 Replies

2. Shell Programming and Scripting

How to : Find duplicate number from file? with bash

Thanks AVKlinux (6 Replies)
Discussion started by: avklinux
6 Replies

3. Shell Programming and Scripting

find out duplicate records in file?

Dear All, I have one file which looks like : account1:passwd1 account2:passwd2 account3:passwd3 account1:passwd4 account5:passwd5 account6:passwd6 you can see there're two records for account1. and is there any shell command which can find out : account1 is the duplicate record in... (3 Replies)
Discussion started by: tiger2000
3 Replies

4. Shell Programming and Scripting

Find c28 to next comma in line of log file

I have somelogfile.log where I'm trying to find only the numbers which would start at c28 on the third line from the bottom and end at the next comma, so for somelogfile.log: bunch of text - 0, some other text - 123 or bunch of text - 123, some other text - 192831923 where I want either the... (5 Replies)
Discussion started by: unclecameron
5 Replies

5. Shell Programming and Scripting

Find Duplicate records in first Column in File

Hi, Need to find a duplicate records on the first column, ANU4501710430989 0000000W20389390 ANU4501710430989 0000000W67065483 ANU4501130050520 0000000W80838713 ANU4501210170685 0000000W69246611... (3 Replies)
Discussion started by: Murugesh
3 Replies

6. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

7. Shell Programming and Scripting

Find duplicate based on 'n' fields and mark the duplicate as 'D'

Hi, In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'. In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies

8. Shell Programming and Scripting

How to find duplicate line in Linux?

Hi, Gurus, I need find the duplicate record in unix file. what command I should use for this. Thanks in advance (4 Replies)
Discussion started by: ken6503
4 Replies

9. Shell Programming and Scripting

Honey, I broke awk! (duplicate line removal in 30M line 3.7GB csv file)

I have a script that builds a database ~30 million lines, ~3.7 GB .cvs file. After multiple optimzations It takes about 62 min to bring in and parse all the files and used to take 10 min to remove duplicates until I was requested to add another column. I am using the highly optimized awk code: awk... (34 Replies)
Discussion started by: Michael Stora
34 Replies

10. UNIX for Dummies Questions & Answers

Log file - Delete duplicate line & keep last date

Hello All ! I need your help on this case, I have a csv file with this: ITEM105;ARI FSR;2016-02-01 08:02;243 ITEM101;ARI FSR;2016-02-01 06:02;240 ITEM032;RNO TLE;2016-02-01 11:03;320 ITEM032;RNO TLE;2016-02-02 05:43;320 ITEM032;RNO TLE;2016-02-01 02:03;320 ITEM032;RNO... (2 Replies)
Discussion started by: vadim-bzh
2 Replies
LOGSAVE(8)						      System Manager's Manual							LOGSAVE(8)

NAME
logsave - save the output of a command in a logfile SYNOPSIS
logsave [ -asv ] logfile cmd_prog [ ... ] DESCRIPTION
The logsave program will execute cmd_prog with the specified argument(s), and save a copy of its output to logfile. If the containing directory for logfile does not exist, logsave will accumulate the output in memory until it can be written out. A copy of the output will also be written to standard output. If cmd_prog is a single hyphen ('-'), then instead of executing a program, logsave will take its input from standard input and save it in logfile logsave is useful for saving the output of initial boot scripts until the /var partition is mounted, so the output can be written to /var/log. OPTIONS
-a This option will cause the output to be appended to logfile, instead of replacing its current contents. -s This option will cause logsave to skip writing to the log file text which is bracketed with a control-A (ASCII 001 or Start of Header) and control-B (ASCII 002 or Start of Text). This allows progress bar information to be visible to the user on the console, while not being written to the log file. -v This option will make logsave to be more verbose in its output to the user. AUTHOR
Theodore Ts'o (tytso@mit.edu) SEE ALSO
fsck(8) E2fsprogs version 1.44.1 March 2018 LOGSAVE(8)
All times are GMT -4. The time now is 06:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy