Help with awk for selecting lines in a file avoiding repetition
Hello,
I am using Awk in UBUNTU 12.04.
I have a file as following with 48,432,354 lines and 4 fields.
The file has this structure (There are repetitions of the first column in several lines)
What I want to have as an outcome is this:
That is to have a file only with the first lines of the first file. To mention, I have a file only with the list of names of the first file like this if that can be useful.
Thank you very much in advance.
Last edited by Franklin52; 11-26-2012 at 05:50 AM..
Reason: Please use code tags for data and code samples
Hello,
I need to select the 3 lines above as well as below a search string, including the search string.
I have been trying various combinations using sed command without any success.
Can anuone help please.
Thanking (2 Replies)
Hi All,
I am creating a script that sends log data from text files to a Database and I will like to read sugestions, as I think that there might be better ways to achive this than with my shell script; maybe perl or I don't know, but I will like to read some sugestions.
The log is from... (10 Replies)
Hi all,
I have a file with special characters like this
file1
691775025 ýÄqJ8^Z^Y{ 2004-08-23E P 100.00
45585025 0527541139295037342008-07-25OEP 100.00
6983025 ýB<9D>x<^F^Xb 2004-11-16SPP 100.00
I need a sed command to print the lines which don't have special characters.ie., only line 2... (9 Replies)
Hi All,
Is there a way to get a range of lines from a file??? I want to search through a set of scripts and need to select the group of lines which do the FTP.
Say,
Line1
Line2
ftp SERVER
user UNAME PASS
send FILE_TO_BE_SENT
close
Line3
Line4
Line5
ftp SERVER1
user USER1 PASS1... (6 Replies)
Say I wanted to select the 5th line of a file without knowing the context of the file. Would I use grep and pipe it into wc or is there a more simple way of doing this? (3 Replies)
Hello experts,
I am new to this group and to 'SED' and 'AWK'. I have data (text file) with 5 columns (C_1-5) and 100s of lines (only 10 lines are shown below as an example). I have to find or select only the id numbers (C-1) of specific lines with '90' in the same line (of C_3) AND with '20' in... (6 Replies)
I have a file which contains five hundred thousand lines (500,000). I want to select lines of every 1000 lines from that file using AWK. I can think something like this in bash as below:
for lines in {0..500000..1000}
do
........
........
done
But I want to use AWK. I do not know how to... (4 Replies)
I am new to scripting/programming, so I apologize for any novice questions.
I have a tab delimited text file that was saved from excel xls file.
I am trying to select only the third column using awk command. My command line is as below:
cat test.txt | awk '{print $3}'
However, above... (8 Replies)
Ladles and Jellyspoons,I am trying to use, unsucessfully I might add, awk to strip a large block of information from and audit output.The format resembles the following:-----------------------------------------------------------Event: execveTime: ... (3 Replies)
Discussion started by: proc1269
3 Replies
LEARN ABOUT ULTRIX
uniq
uniq(1) General Commands Manual uniq(1)Name
uniq - report repeated lines in a file
Syntax
uniq [-udc[+n][-n]] [input[output]]
Description
The command reads the input file comparing adjacent lines. In the normal case, the second and succeeding copies of repeated lines are
removed; the remainder is written on the output file. Note that repeated lines must be adjacent in order to be found. For further infor-
mation, see
Options
The n arguments specify skipping an initial portion of each line in the comparison:
-n Skips specified number of fields. A field is defined as a string of non-space, non-tab characters separated by tabs and spaces from its
neighbors.
+n Skips specified number of characters in addition to fields. Fields are skipped before characters.
-c Displays number of repetitions, if any, for each line.
-d Displays only lines that were repeated.
-u Displays only unique (nonrepeated) lines.
If the -u flag is used, just the lines that are not repeated in the original file are output. The -d option specifies that one copy of
just the repeated lines is to be written. The normal mode output is the union of the -u and -d mode outputs.
The -c option supersedes -u and -d and generates an output report in default style but with each line preceded by a count of the number of
times it occurred.
See Alsocomm(1), sort(1)uniq(1)