10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I have a csv file which contains some millions of lines in it.
The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line).
I don't want to use any pattern from the Header as I have some... (7 Replies)
Discussion started by: sudhakar T
7 Replies
2. Shell Programming and Scripting
hello all
in my bash script I have a file and I only want to keep the lines that appear twice in the file.Is there a way to do this?
thanks in advance! (4 Replies)
Discussion started by: vlm
4 Replies
3. UNIX for Advanced & Expert Users
Hi,
I copied the contents of a binary file into a .text file using hd (hexdump) command. The data in binary file is such that I get in many places like following
00000250 00 00 00 00 3f 2d 91 68 3f 69 fb e7 00 00 00 00 |....?-.h?i......|
00000260 00 00 00 00 00 00 00 00 00 00 00 00 00... (2 Replies)
Discussion started by: KidD312
2 Replies
4. Shell Programming and Scripting
Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file.
I wrote the following script, but does not work. What is the problem?
The output file should only contain five lines:
Later! (5 Replies)
Discussion started by: Ernst
5 Replies
5. UNIX for Advanced & Expert Users
Hi All,
I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space.
I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies
6. Shell Programming and Scripting
Hi All,
I am trying to remove the duplicate entries in a file and print them just once. For example, if my input file has:
00:44,37,67,56,15,12
00:44,34,67,56,15,12
00:44,58,67,56,15,12
00:44,35,67,56,15,12
00:59,37,67,56,15,12
00:59,34,67,56,15,12
00:59,35,67,56,15,12... (7 Replies)
Discussion started by: faiz1985
7 Replies
7. UNIX for Dummies Questions & Answers
I have a log file "logreport" that contains several lines as seen below:
04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Discussion started by: Nysif Steve
18 Replies
8. UNIX for Dummies Questions & Answers
Hi,
I am having a file which contains many duplicate lines. I wanted to redirect these duplicate lines into another file.
Suppose I have a file called file_dup.txt which contains some line as
file_dup.txt
A100-R1
ACCOUNTING-CONTROL
ACTONA-ACTASTOR
ADMIN-AUTH-STATS
ACTONA-ACTASTOR... (3 Replies)
Discussion started by: zing_foru
3 Replies
9. UNIX for Advanced & Expert Users
Hi,
I have a file with duplicate lines in it. I want to keep only the duplicate lines and delete the non duplicates. Can some one please help me?
Regards
Narayana Gupta (3 Replies)
Discussion started by: guptan
3 Replies
10. Shell Programming and Scripting
I am doing KSH script to remove duplicate lines in a file. Let say the file has format below.
FileA
1253-6856
3101-4011
1827-1356
1822-1157
1822-1157
1000-1410
1000-1410
1822-1231
1822-1231
3101-4011
1822-1157
1822-1231
and I want to simply it with no duplicate line as file... (5 Replies)
Discussion started by: Teh Tiack Ein
5 Replies
shell_builtins(1) shell_builtins(1)
NAME
shell_builtins, case, for, foreach, function, if, repeat, select, switch, until, while - shell command interpreter built-in commands
The shell command interpreters csh(1), ksh(1), and sh(1) have special built-in commands. The commands case, for, foreach, function, if,
repeat, select, switch, until, and while are commands in the syntax recognized by the shells. They are described in the Commands section of
the manual pages of the respective shells. The remaining commands listed in the table below are built into the shells for reasons such as
efficiency or data sharing between command invocations. They are described on their respective manual pages.
|
Command | Shell
alias |csh, ksh
bg |csh, ksh, sh
break |csh, ksh, sh
case |csh, ksh, sh
cd |csh, ksh, sh
chdir |csh, sh
continue |csh, ksh, sh
dirs |csh
echo |csh, ksh, sh
eval |csh, ksh, sh
exec |csh, ksh, sh
exit |csh, ksh, sh
export |ksh, sh
false |ksh
fc |ksh
fg |csh, ksh, sh
for |ksh, sh
foreach |csh
function |ksh
getopts |ksh, sh
glob |csh
goto |csh
hash |ksh, sh
hashstat |csh
history |csh
if |csh, ksh, sh
jobs |csh, ksh, sh
kill |csh, ksh, sh
let |ksh
limit |csh
login |csh, ksh, sh
logout |csh, ksh, sh
nice |csh
newgrp |ksh, sh
nohup |csh
notify |csh
onintr |csh
popd |csh
print |ksh
pushd |csh
pwd |ksh, sh
read |ksh, sh
readonly |ksh, sh
rehash |csh
repeat |csh
return |ksh, sh
select |ksh
set |csh, ksh, sh
setenv |csh
shift |csh, ksh, sh
source |csh
stop |csh, ksh, sh
suspend |csh, ksh, sh
switch |csh
test |ksh, sh
time |csh
times |ksh, sh
trap |ksh, sh
true |ksh
type |ksh, sh
typeset |ksh
ulimit |ksh, sh
umask |csh, ksh, sh
unalias |csh, ksh
unhash |csh
unlimit |csh
unset |csh, ksh, sh
unsetenv |csh
until |ksh, sh
wait |csh, ksh, sh
whence |ksh
while |csh, ksh, sh
Bourne Shell, sh, Special Commands
Input/output redirection is now permitted for these commands. File descriptor 1 is the default output location. When Job Control is
enabled, additional Special Commands are added to the shell's environment.
In addition to these built-in reserved command words, sh also uses:
: No effect; the command does nothing. A zero exit code is returned.
.filename Read and execute commands from filename and return. The search path specified by PATH is used to find the directory con-
taining filename.
C shell, csh
Built-in commands are executed within the C shell. If a built-in command occurs as any component of a pipeline except the last, it is exe-
cuted in a subshell. In addition to these built-in reserved command words, csh also uses:
: Null command. This command is interpreted, but performs no action.
Korn Shell, ksh, Special Commands
Input/Output redirection is permitted. Unless otherwise indicated, the output is written on file descriptor 1 and the exit status, when
there is no syntax error, is zero.
Commands that are preceded by one or two * (asterisks) are treated specially in the following ways:
1. Variable assignment lists preceding the command remain in effect when the command completes.
2. I/O redirections are processed after variable assignments.
3. Errors cause a script that contains them to abort.
4. Words, following a command preceded by ** that are in the format of a variable assignment, are expanded with the same rules as a vari-
able assignment. This means that tilde substitution is performed after the = sign and word splitting and file name generation are not
performed.
In addition to these built-in reserved command words, ksh also uses:
* : [ arg ... ] The command only expands parameters.
* .file [ arg ..Read the complete file then execute the commands. The commands are executed in the current shell environment. The search
path specified by PATH is used to find the directory containing file. If any arguments arg are given, they become the posi-
tional parameters. Otherwise, the positional parameters are unchanged. The exit status is the exit status of the last com-
mand executed. the loop termination test.
intro(1), alias(1), break(1), cd(1), chmod(1), csh(1), echo(1), exec(1), exit(1), find(1), getoptcvt(1), getopts(1), glob(1), hash(1), his-
tory(1), jobs(1), kill(1), ksh(1), let(1), limit(1), login(1), logout(1), newgrp(1), nice(1), nohup(1), print(1), pwd(1), read(1), read-
only(1), set(1), sh(1), shift(1), suspend(1), test(1B), time(1), times(1), trap(1), typeset(1), umask(1), wait(1), chdir(2), chmod(2),
creat(2), umask(2), getopt(3C), profile(4), environ(5)
29 Jun 2005 shell_builtins(1)