I tried 'uniq' that only show uniq strings first, but don't know how to show uniq chunks. It only works for lines. And then I tried sed.
So now I could find the duplicated one and replaced with "-", but failed to remove the chunk.
If my post is not appropriate, I will remove it soon.
Best,
Jaeyoung
Moderator's Comments:
Please use CODE tags when displaying sample input, sample output, and code segments.
Last edited by Don Cragun; 05-26-2016 at 02:01 AM..
Reason: Add CODE tags.
I can not figure out this one, so I turn to unix.com for help, I have a file, in which there are some lines containing continuously duplicate columns, like the following
adb abc abc asd adfj
123 123 123 345
234 444 444 444 444 444 23
and the output I want is
adb abc asd adfj
123 345... (5 Replies)
hi there,
i have a text file like that one:
I like to delete the second block with the Start and End Line!
Does anyone have a idea?
Thanks for your help,
Roland (4 Replies)
cat file.txt
fvnuiehuewf
ruevhxncvkjrh
zxjvurhfuwe
jkhvBEGINvfnvf
ijrgioe
Trying to delete a line that has the pattern "BEGIN"
cat sedtest
filename=file.txt
pattern=BEGIN
sed "/^$pattern/d" "$filename" (9 Replies)
Input:
:: gstreamer
:: xine-lib
:: xine-lib-extras
Output should be:
gstreamer xine-lib xine-lib-extras
How can it be done with sed or perl? (12 Replies)
Hello
I have a unique problem of needing to delete large files slowly off of an XSan.
I was wondering if there is a script I could use to delete 100gb chunks of files and folders that get placed in to a watch folder, slowly so as not to disrupt the other users. I would like to use Automator in... (0 Replies)
hi,
Here is excerpt from my xml file
<!-- The custom module to do the authentication for LDAP
-->
</login-module>
<login-module code="com.nlayers.seneca.security.LdapLogin" flag="sufficient">
<module-option... (1 Reply)
First time poster, but the forum has saved my bacon more times than... Lots.
Anyway, I have a text file, and wanted to use Awk (or any other sensible program) to print out overlapping sections, or arbitrary length. To describe by example, for file
1
2
3
4
5
etc...
I want the out put... (3 Replies)
Hi gurus,
I wanted to split main file in 20 files with 2500 lines in each file. My main file conatins total 2500*20 lines. Following awk I made, but it is breaking with error.
awk '{ for (i = 1; i <= 20; i++) { starts=2500*$i-1; ends=2500*$i; NR>=starts && NR<=ends {f=My$i".txt"; print >> f;... (10 Replies)
Dear all,
I always appreciate your help.
I would like to delete lines containing duplicated strings in the second column.
test.txt
658 invert_d2e_q_reg_0_/Qalu_ecl_zlow_e 0.825692
659 invert_d2e_q_reg_0_/Qalu_byp_rd_data_e 0.825692
660 invert_d2e_q_reg_0_/Qalu_byp_rd_data_e 0.825692... (1 Reply)
Discussion started by: jypark22
1 Replies
LEARN ABOUT FREEBSD
uniq
UNIQ(1) BSD General Commands Manual UNIQ(1)NAME
uniq -- report or filter out repeated lines in a file
SYNOPSIS
uniq [-c | -d | -u] [-i] [-f num] [-s chars] [input_file [output_file]]
DESCRIPTION
The uniq utility reads the specified input_file comparing adjacent lines, and writes a copy of each unique input line to the output_file. If
input_file is a single dash ('-') or absent, the standard input is read. If output_file is absent, standard output is used for output. The
second and succeeding copies of identical adjacent input lines are not written. Repeated lines in the input will not be detected if they are
not adjacent, so it may be necessary to sort the files first.
The following options are available:
-c Precede each output line with the count of the number of times the line occurred in the input, followed by a single space.
-d Only output lines that are repeated in the input.
-f num Ignore the first num fields in each input line when doing comparisons. A field is a string of non-blank characters separated from
adjacent fields by blanks. Field numbers are one based, i.e., the first field is field one.
-s chars
Ignore the first chars characters in each input line when doing comparisons. If specified in conjunction with the -f option, the
first chars characters after the first num fields will be ignored. Character numbers are one based, i.e., the first character is
character one.
-u Only output lines that are not repeated in the input.
-i Case insensitive comparison of lines.
ENVIRONMENT
The LANG, LC_ALL, LC_COLLATE and LC_CTYPE environment variables affect the execution of uniq as described in environ(7).
EXIT STATUS
The uniq utility exits 0 on success, and >0 if an error occurs.
COMPATIBILITY
The historic +number and -number options have been deprecated but are still supported in this implementation.
SEE ALSO sort(1)STANDARDS
The uniq utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'') as amended by Cor. 1-2002.
HISTORY
A uniq command appeared in Version 3 AT&T UNIX.
BSD December 17, 2009 BSD