01-17-2007
count=0
cat {1} | while read line
do
count=`echo "${count} + 1" | bc`
cc=`head -${count} ${1} |grep -c ${line}`
if [ ${cc} -eq 2 ]
then
:
else
echo ${line}
fi
done
Would this work ? I have not tested it but it should work. save it as a file , give it execute permission and then use it as "script filename".
All the best. It's a simple logic but i think that would work for you.
Regds,
Kaps
10 More Discussions You Might Find Interesting
1. Windows & DOS: Issues & Discussions
I use sftp batch script to copy file from a remote server to local machine. Problem is when I rerun this shell after copying txt file it do not allow me to copy txt file again.
My question is , is this confrigration problem of remote server. beace same shell work fie with the other remote... (3 Replies)
Discussion started by: Jamil Qadir
3 Replies
2. Shell Programming and Scripting
How would I write a command(s) to read from a file (list) that looks like this: 29847374384 and grep from a second file (list) that looks like this: 29847374384, jkdfkjdf,3833,ddd:confused: (1 Reply)
Discussion started by: smellylizzard
1 Replies
3. Shell Programming and Scripting
I'm a biologist trying to analyse some data and I'll appreciate some help with the following problem. I have a column of characters which I'll like to delete the duplicated characters in each line and report only the unique one.No sorting should be done. E.g.
The original data:
GTG
CTC
CTC... (5 Replies)
Discussion started by: ivpz
5 Replies
4. Shell Programming and Scripting
Hi,
I have to programming a program in shell script which combine a lot of .txt files. But in all of these files the program had to delete the first both lines.
Because I don't know anything about shell script I need your help.
Does anyone have a command or a hint, where I can look for?
... (1 Reply)
Discussion started by: mkrol
1 Replies
5. Homework & Coursework Questions
Hi,
I have to programming a program in shell script which combine a lot of .txt files. But in all of these files the program had to delete the first both lines.
Because I don't know anything about shell script I need your help.
Does anyone have a command or a hint, where I can look for?
... (1 Reply)
Discussion started by: mkrol
1 Replies
6. UNIX for Dummies Questions & Answers
Hi,
I have a small problem with my script. I have everything in order but it doesnt seem to compare anything less than 1 correctly. If the input is more than 1, then the results is correct.
If the input is 0.xxx (anything) it returns erroneous results. Pls help
input=0.12
if ; then
... (7 Replies)
Discussion started by: streddy
7 Replies
7. Shell Programming and Scripting
Hi,
I`m a total newbie, well my requirement is that i have 2 files
I want to identify which countries i do not currently have in db..
how can i use the grep or another command to find this file ..
i want to match all-countries.txt with countries-in-db.txt so the output is equal to... (11 Replies)
Discussion started by: beanbaby
11 Replies
8. Shell Programming and Scripting
Hi,
I have files with this kind of format (separator is space):
A1 B1 C1 D1 E1 F1 D1 C1 G1 H1
A2 B2 C2 D2 E2 F2 D2 C2 G2 H2
A3 B3 C3 D3 E3 F3 G3 D3 C3 H3
A4 B4 C4 D4 E4 F4 G4 D4 C4 H4
I want the output to be:
A1 B1 E1 F1 G1 H1
A2 B2 E2 F2 G2 H2
A3 B3 E3 F3 G3 H3
A4 B4 E4 F4 G4... (12 Replies)
Discussion started by: Gr4wk
12 Replies
9. UNIX for Dummies Questions & Answers
Hi, I have very old files in my server like from 2012 and i want to delete them,
Please help.
Thanks in advance.. (2 Replies)
Discussion started by: nanz143
2 Replies
10. Shell Programming and Scripting
Hi,
I have the following output from an Oracle SQL statement and I want to remove duplicated column values.
I know it is possible using Oracle analytical/statistical functions but unfortunately I don't know how to use any of those.
So now, I've gone to PLAN B using awk/sed maybe or any... (5 Replies)
Discussion started by: newbie_01
5 Replies
rl(1) User Commands rl(1)
NAME
rl - Randomize Lines.
SYNOPSIS
rl [OPTION]... [FILE]...
DESCRIPTION
rl reads lines from a input file or stdin, randomizes the lines and outputs a specified number of lines. It does this with only a single
pass over the input while trying to use as little memory as possible.
-c, --count=N
Select the number of lines to be returned in the output. If this argument is omitted all the lines in the file will be returned in
random order. If the input contains less lines than specified and the --reselect option below is not specified a warning is printed
and all lines are returned in random order.
-r, --reselect
When using this option a single line may be selected multiple times. The default behaviour is that any input line will only be
selected once. This option makes it possible to specify a --count option with more lines than the file actually holds.
-o, --output=FILE
Send randomized lines to FILE instead of stdout.
-d, --delimiter=DELIM
Use specified character as a "line" delimiter instead of the newline character.
-0, --null
Input lines are terminated by a null character. This option is useful to process the output of the GNU find -print0 option.
-n, --line-number
Output lines are numbered with the line number from the input file.
-q, --quiet, --silent
Be quiet about any errors or warnings.
-h, --help
Show short summary of options.
-v, --version
Show version of program.
EXAMPLES
Some simple demonstrations of how rl can help you do everyday tasks.
Play a random sound after 4 minutes (perfect for toast):
sleep 240 ; play `find /sounds -name '*.au' -print | rl --count=1`
Play the 15 most recent .mp3 files in random order.
ls -c *.mp3 | head -n 15 | rl | xargs --delimiter='
' play
Roll a dice:
seq 6 | rl --count 2
Roll a dice 1000 times and see which number comes up more often:
seq 6 | rl --reselect --count 1000 | sort | uniq -c | sort -n
Shuffle the words of a sentence:
echo -n "The rain in Spain stays mainly in the plain."
| rl --delimiter=' ';echo
Find all movies and play them in random order.
find . -name '*.avi' -print0 | rl -0 | xargs -n 1 -0 mplayer
Because -0 is used filenames with spaces (even newlines and other unusual characters) in them work.
BUGS
The program currently does not have very smart memory management. If you feed it huge files and expect it to fully randomize all lines it
will completely read the file in memory. If you specify the --count option it will only use the memory required for storing the specified
number of lines. Improvements on this area are on the TODO list.
The program uses the rand() system random function. This function returns a number between 0 and RAND_MAX, which may not be very large on
some systems. This will result in non-random results for files containing more lines than RAND_MAX.
Note that if you specify multiple input files they are randomized per file. This is a different result from when you cat all the files and
pipe the result into rl.
COPYRIGHT
Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008 Arthur de Jong.
This is free software; see the license for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Version 0.2.7 Jul 2008 rl(1)