Sponsored Content
Full Discussion: Extract unique files
Top Forums Shell Programming and Scripting Extract unique files Post 302850137 by Scrutinizer on Wednesday 4th of September 2013 02:34:13 AM
Old 09-04-2013
Assuming the lines are in chronological order within the file and there are no spaces in the file names, try
Code:
awk '{A[$NF]=$0} END{for(i in A) print A[i]}' file

This User Gave Thanks to Scrutinizer For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh scripting: Extract 1 most recent record for unique key

I'm loading multiple delimited files into an Oracle DB using sqlldr on Unix. I would like to get only the most recent record per each unique key. There may be multiple updates for each key, but I only want the most recent one. There is a date column in my delimited files, so I'm using cat to... (2 Replies)
Discussion started by: OPTIMUS_prime
2 Replies

2. Shell Programming and Scripting

extract unique pattern from large text file

Hi All, I am trying to extract data from a large text file , I want to extract lines which contains a five digit number followed by a hyphen , like 12345- , i tried with egrep ,eg : egrep "+" text.txt but which returns all the lines which contains any number of digits followed by hyhen ,... (19 Replies)
Discussion started by: shijujoe
19 Replies

3. UNIX for Dummies Questions & Answers

Extract Unique Values from file

Hello all, I have a file with following sample data 2009-08-26 05:32:01.65 spid5 Process ID 86:214 owns resources that are blocking processes on Scheduler 0. 2009-08-26 05:32:01.65 spid5 Process ID 86:214 owns resources that are blocking processes on Scheduler 0. 2009-08-26... (5 Replies)
Discussion started by: simonsimon
5 Replies

4. Shell Programming and Scripting

Unique files in a given directory

I keep all my files on a NAS device and copy files from it to usb or local storage when needed. The bad part about this is that I often have the same file on numerous places. I'd like to write a script to check if the files in a given directory exist in another. An example: say I have a... (7 Replies)
Discussion started by: cue
7 Replies

5. Shell Programming and Scripting

Extract unique filenames

Hi Unix Gurus, In a script, I am trying to extract unique text from a set of filenames. I have certain files like below in a directory: OPEN_INV_01012011.xls OPEN_INV_01022011.xls OPEN_INV_01032011.xls CLOSE_INV_01012011.xls CLOSE_INV_01022011.xls I need to extract just "OPEN_INV_" ... (4 Replies)
Discussion started by: shankar1dada
4 Replies

6. Shell Programming and Scripting

Create shell script to extract unique information from one file to a new file.

Hi to all, I got this content/pattern from file http.log.20110808.gz mail1 httpd: Account Notice: close igchung@abc.com 2011/8/7 7:37:36 0:00:03 0 0 1 mail1 httpd: Account Information: login sastria9@abc.com proxy sid=gFp4DLm5HnU mail1 httpd: Account Notice: close sastria9@abc.com... (16 Replies)
Discussion started by: Mr_47
16 Replies

7. UNIX for Dummies Questions & Answers

getting unique lines from 2 files

hi i have used comm -13 <(sort 1.txt) <(sort 2.txt) option to get the unique lines that are present in file 2 but not in file 1. but some how i am getting the entire file 2. i would expect few but not all uncommon lines fro my dat. is there anything wrong with the way i used the command? my... (1 Reply)
Discussion started by: anurupa777
1 Replies

8. UNIX for Dummies Questions & Answers

Extract unique combination of rows from text files

Hi Gurus, I have 100 tab-delimited text files each with 21 columns. I want to extract only 2nd and 5th column from each text file. However, the values in both 2bd and 5th column contain duplicate values but the combination of these values in a row are not duplicate. I want to extract only those... (3 Replies)
Discussion started by: Unilearn
3 Replies

9. Shell Programming and Scripting

Extract UNIque records from File

Hi, I have a file with 20GB Pipe Delimited file where i have too many duplicate records. I need an awk script to extract the unique records from the file and put it into another file. Kindly help. Thanks, Arun (1 Reply)
Discussion started by: Arun Mishra
1 Replies

10. Shell Programming and Scripting

Extract lines with unique value using a specific column

Hi there, I need a help with extracting data from tab delimited file which look like this #CHROM POS ID REF ALT Human Cow Dog Mouse Lizard chr2 3033 . G C 0/0 0/0 0/0 1/1 0/0 chr3 35040 . G T 0/0 0/0 ./. 1/1 0/1 chr4 60584 . T G 1/1 1/1 0/1 1/1 0/0 chr10 7147815 . G A 0/0 1/1 0/0 0/0... (9 Replies)
Discussion started by: houkto
9 Replies
rcsfreeze(1)															      rcsfreeze(1)

NAME
rcsfreeze - freeze a configuration of sources checked in under RCS SYNOPSIS
rcsfreeze [name] DESCRIPTION
rcsfreeze assigns a symbolic revision number to a set of RCS files that form a valid configuration. The idea is to run rcsfreeze each time a new version is checked in. A unique symbolic name (C_number, where number is increased each time rcsfreeze is run) is then assigned to the most recent revision of each RCS file of the main trunk. An optional name argument to rcsfreeze gives a symbolic name to the configuration. The unique identifier is still generated and is listed in the log file but it will not appear as part of the symbolic revision name in the actual RCS files. A log message is requested from the user for future reference. The shell script works only on all RCS files at one time. All changed files must be checked in already. Run rcsclean(1) first and see whether any sources remain in the current directory. RESTRICTIONS
rcsfreeze does not check whether any sources are checked out and modified. Although both source file names and RCS file names are accepted, they are not paired as usual with RCS commands. Error checking is rudimentary. rcsfreeze is just an optional example shell script, and should not be taken too seriously. See CVS for a more complete solution. FILES
version number log messages, most recent first IDENTIFICATION
Author: Stephan v. Bechtolsheim SEE ALSO
co(1), rcs(1), rcsclean(1), rlog(1) rcsfreeze(1)
All times are GMT -4. The time now is 12:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy