This appears to work for the example text:
so
lists lines from input where they don't match the lines in file (each line is a substring, obviously)
is a process substitution allowing another command to be used
instead of a file
gives us the list of _idxx
lists duplicated lines (uniq lines are thrown away).
I have a datafile that is formatted as fixed.
I know that each line should contain 880 characters.
I want to separate the file into 2 files, one that has lines with 880 characters and the other file with everything else.
Is this possible ? (9 Replies)
Hi All,
I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space.
I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
hi
my problem is little complicated one. i have 2 files which appear like this
file 1
abbsss:aa:22:34:as akl abc 1234
mkilll:as:ss:23:qs asc abc 0987
mlopii:cd:wq:24:as asd abc 7866
file2
lkoaa:as:24:32:sa alk abc 3245
lkmo:as:34:43:qs qsa abc 0987
kloia:ds:45:56:sa acq abc 7805
i... (5 Replies)
Hello,
I need some sort of way to extract every date contained in a file, and count how many of those dates there are.
Here are the specifics:
The date format I'm looking for is mm/dd/yyyy
I only need to look after line 45 in the file (that's where the data begins)
The columns of... (2 Replies)
Hello guys. I really hope someone will help me with this one..
So, I have to write this script who:
- creates a file home/student/vmdisk of 10 mb
- formats that file to ext3
- mounts that partition to /mnt/partition
- creates a file /mnt/partition/data. In this file, there will... (1 Reply)
Hello to all, happy new year 2013!
May somebody could help me, is about a very similar problem to the problem I've posted here where the member rdrtx1 and bipinajith helped me a lot.
https://www.unix.com/shell-programming-scripting/211147-map-values-blocks-single-line-2.html
It is very... (3 Replies)
file1:
file2:
I need to find matches for any lines in file1 that appear in file2. Desired output is '>' plus the file1 term, followed by the line after the match in file2 (so the title is a little misleading):
This is honestly beyond what I can do without spending the whole night on it, so I'm... (2 Replies)
I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
For some reason I am having difficulty performing what should be a fairly easy task. I would like to print lines of a file that have a unique value in the first field. For example, I have a large data-set with the following excerpt:
PS003,001 MZMWR/ L-DWD// *
PS003,001... (4 Replies)
Discussion started by: jvoot
4 Replies
LEARN ABOUT X11R4
uniq
UNIQ(1) User Commands UNIQ(1)NAME
uniq - report or omit repeated lines
SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]]
DESCRIPTION
Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output).
With no options, matching lines are merged to the first occurrence.
Mandatory arguments to long options are mandatory for short options too.
-c, --count
prefix lines by the number of occurrences
-d, --repeated
only print duplicate lines, one for each group
-D print all duplicate lines
--all-repeated[=METHOD]
like -D, but allow separating groups with an empty line; METHOD={none(default),prepend,separate}
-f, --skip-fields=N
avoid comparing the first N fields
--group[=METHOD]
show all items, separating groups with an empty line; METHOD={separate(default),prepend,append,both}
-i, --ignore-case
ignore differences in case when comparing
-s, --skip-chars=N
avoid comparing the first N characters
-u, --unique
only print unique lines
-z, --zero-terminated
line delimiter is NUL, not newline
-w, --check-chars=N
compare no more than N characters in lines
--help display this help and exit
--version
output version information and exit
A field is a run of blanks (usually spaces and/or TABs), then non-blank characters. Fields are skipped before chars.
Note: 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use 'sort -u' without
'uniq'. Also, comparisons honor the rules specified by 'LC_COLLATE'.
AUTHOR
Written by Richard M. Stallman and David MacKenzie.
REPORTING BUGS
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Report uniq translation bugs to <http://translationproject.org/team/>
COPYRIGHT
Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO comm(1), join(1), sort(1)
Full documentation at: <http://www.gnu.org/software/coreutils/uniq>
or available locally via: info '(coreutils) uniq invocation'
GNU coreutils 8.28 January 2018 UNIQ(1)