[bash]Stripping lines from a list


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers [bash]Stripping lines from a list
# 1  
Old 03-28-2008
[bash]Stripping lines from a list

Hello!

I have a script that is (among other things) doing the following:
Code:
list=/tmp/list1.txt
ncftpls -u <user> -p <password> -x "-l1" server.domain.tld > $list
cat $list | nl
echo "Choose file: "
read file
cat /tmp/list1.txt | nl | grep $file | sed -e "s/$file//g" -e "s/ //g" | column -t > /tmp/list2.txt
cat /tmp/list2.txt

With this, i want to choose a file that is located on a server and download it.
the sed lines was given to me on a guy on another forum who said it should do the trick. And it did, somewhat.
In the directory, i have around 40 files.
if i choose 1 - 9, it gives me a list with several lines like this

Code:
Choose file:
9
2 full_mysql_dump_2008-03-24_0:00.sql.gz full_mysql_dump_2008-03-25_06:00.sql.gz 10 full_mysql_dump_2008-03-25_0:00.sql.gz 18 full_mysql_dump_2008-03-26_0:00.sql.gz 1 full_mysql_dump_2008-03-26_12:00.sql.gz 2 full_mysql_dump_2008-03-28_06:00.sql.gz 30 full_mysql_dump_2008-03-28_0:00.sql.gz 35 full_mysql_dump_2008-03-2_00:00.sql.gz 3

but if i choose a higher number, it gives me this:
Code:
Choose file:
33
full_mysql_dump_2008-03-28_18:00.sql.gz

Just as i want it to do.

In this other script that i am working on, i am using the exact same thing, but this time with another directory on the same remote server, where it is only 2 files. (there will be more files there later).
The result is like the first example.
It doesn't cut away all the other lines that i don't want to have there.

Any of you guys who may know what is wrong, or maybe another solution for my problem?

Last edited by noratx; 03-28-2008 at 10:58 PM.. Reason: correcting typos
# 2  
Old 03-29-2008
No offense, but I'm sorry, that's a really hideous script. I guess the problem is that you are grepping without anchoring the grep, so it prints any line which contains the indicated number anywhere.

Here is a refactored version, with the following changes.

* cat file | command is known as "Useless Use of Cat". Changed those.
* grep | sed similarly refactored to use only sed, anchored at the beginning
* You use $file and /tmp/file1.txt interchangeably. Standardize on the former.
* Since you are not using file1 without the nl, doing the nl once and saving it in the file.
* Remove the temp file when done.
* The second temp file is apparently unnecessary, unless you are only showing part of the script.

Code:
list=/tmp/list1.txt
trap 'rm $list' 0  # remove temp file at exit
trap 'exit 127' 1 2 3 5 15  # remove temp file if interrupted
ncftpls -u <user> -p <password> -x "-l1" server.domain.tld | nl > $list
echo "Choose file: "
# maybe display the list to the user:
# pr -bt2 "$list"
read file
sed "s/^ *$file //g" | column -t

# 3  
Old 03-29-2008
No offence taken. =)
I'm pretty much a newbie, so writing hideous scripts is what you do at the beginning (i think).

As you mentioned, i was only showing a part of the total script.
Here is the whole funcion in the script (the rest is working as it should).
Also, the script that i wrote here in the beginning, is a snippet from a testing script that i used. Before i make major changes in the original script (that worked before i started to change it from backing up my MySQL database to backing up my home directory), i make minor testing scripts just to see if the changes works as i want the to. Because of that, some names (like the list names) may not be the same.

Code:
list() {
list=/tmp/beacon.list
$ncftpls -u <user> -p <password> -x "-l1" <server> > $list
cat $list | nl
echo "Choose file: "
read file
list_to_var=`cat $list | nl | grep $file | sed -e "s/$file//g" -e "s/ //g" | column -t`
echo $list_to_var
break
$ncftpget -u <user> -p <password> <host> /tmp backup/homes/$list_to_var
echo "Dumpfile retreived."
echo "Do you want to restore it?  Y/N: "
read unpack_
case $unpack_ in
        [yY]*)
                echo "unpacking..."
                $bunzip2 $dump_path$list_to_var
                tarfile=tarfile.txt
                echo "$list_to_var" > $dump_path$tarfile
                tar_var="`awk '/tar/{gsub(/.bz2/, "")};{print}' $dump_path$tarfile`"
                echo $tar_var
                break
                $tar -tf $dump_path$tar_var > $dump_path$tar_list.txt
                tar_list=$dump_path$tar_list.txt
                cat $tar_list | awk -F/ '{print $1 "/" $2}' | nl
                echo "Choose which directory to extract by entering the correponding number or * for everything : "
                read tar_path
                echo "Enter the path where you want to unpack"
                echo "Eg. '/home', '/tmp/' or '.'"
                echo "Enter path :"
                read extract_path
                case $tar_path in
                        '')
                                $tar -xvf $dump_path$tar_var -C $extract_path;;
                        *)
                                extract=`cat $tar_list | awk -F/ '{print $1 "/" $2}' | nl | grep $tar_path| sed -e "s/$tar_path//g" -e "s/ //g" | column -t`
                                $tar -xvf $dump_path$tar_var $extract -C $extract_path;;
                esac
                echo "The directory/s $tar_list unpacked to $extract_path!"
                echo "Do you want to delete the file $tar_var? Y/N: "
                read delete_tar_var
                case $delete_tar_var in
                        [yY]*)
                                rm $dump_path$tar_var
                                echo "$tar_var deleted!";;
                        [nN]*)
                                echo "";;
                esac
                unpack;;
        [nN]*)
                echo "Ok, lets quit!" ;;
esac
rm $dump_path$list
rm $dump_pathtarfile.txt
rm $dump_pathtar_list.txt
}

The paths and programs are entered in the beginning of the total script (like ncftpls=/usr/local/bin/ncftpls), in case you would wonder why i use $ncftpls and such.
# 4  
Old 03-29-2008
Quote:
Originally Posted by noratx
No offence taken. =)
I'm pretty much a newbie, so writing hideous scripts is what you do at the beginning (i think).
Yeah, sorry about the tone there.

The rest of the script really looks quite reasonable, apart from the inexplicable urge to spoon-feed nl with cat, as if it couldn't read the file itself. Ditto for awk etc.

Another minor nit I have is about how you capture stuff with backticks only to echo back the output to the user. Just take out the backticks and let the output go to standard output if you want the user to see it. (Echoing also has some subtle side effects, which you might or might not want. You avoid a lot of quoting quandaries if you just don't do that.)

Quote:
The paths and programs are entered in the beginning of the total script (like ncftpls=/usr/local/bin/ncftpls), in case you would wonder why i use $ncftpls and such.
Frankly, I think that's a dubious practice -- just make sure your PATH is set up correctly, and use the actual program names without putting them in variables. That's easier to port to a different system where the paths are different, too.

All in all, not too hideous at all. Hope this helps.

Last edited by era; 03-29-2008 at 09:42 AM.. Reason: Remark on echo `output from backticks` too
# 5  
Old 03-31-2008
I got a problem with the code you canged.
To test it before changing it in my script, i created a new script with the code, just to test it.
Code:
list=/tmp/list1.txt
list2=/tmp/list2.txt
trap 'rm $list $list2' 0  # remove temp file at exit
trap 'exit 127' 1 2 3 5 15  # remove temp file if interrupted
ncftpls -u <username> -p <password> -x "-l1" server.domain.tld | nl > $list
echo "Choose file: "
# maybe display the list to the user:
pr -bt2 "$list"
read file
sed "s/^ *$file //g" | column -t > $list2
cat $list2

-b for pr does not exist in my version of pr, i tryed without it, but since i don't know what it does, i don't know if it was better or not (it continued even though the -b option did not exist).
When i chhose the file, it just stops and nothing happends.
I also added list2 and a cat of the list, just so i can see what happends, but nothing is really happening.

Last edited by noratx; 03-31-2008 at 12:12 PM.. Reason: spelling corrections. (pt instead of pr)
# 6  
Old 03-31-2008
The lack of a -b option to pr should not matter a lot.

Forgot to add the file argument to sed, sorry about that.

Code:
sed "s/^ *$file //g" "$list" | column -t

if something doesn't work, figure out where it stops working by peeling off some layers from the end.
# 7  
Old 04-03-2008
Hello!

Now a new problem has appeared.
When i do
Code:
sed "s/^ *2 //g" "/tmp/list1.txt" | column -t

the result is:
Code:
1  home.tar.bz2
2  itsak.se.homes.2008-03-28_01-30.tar.gz
3  itsak.se.homes.2008-04-01_01-30.tar.gz

And this is already what is in list1.txt.

I tried to change it to this:
Code:
sed "s/1//g" "/tmp/list1.txt" | column -t

and then i get this:
[CODE]
home.tar.bz2
2 itsak.se.homes.2008-03-28_0-30.tar.gz
3 itsak.se.homes.2008-04-0_0-30.tar.gz
[/CODE/
This is allmost what i want, only that the rest:
2 itsak.se.homes.2008-03-28_0-30.tar.gz
3 itsak.se.homes.2008-04-0_0-30.tar.gz
should be removed from the file as well.

Also, when i do "sed "s/1//g" "/tmp/list1.txt" | column -t"
it also removes the numbers from the dates, which is not good.
If the date in the file is removed, then the script won't be able to download the file.
The exact filenames in this example should be:
"itsak.se.homes.2008-03-28_01-30.tar.gz"
and
"itsak.se.homes.2008-04-01_01-30.tar.gz".

Sorry if i behave like a total newbe and asks many questions, but i am trying to learn. (And to learn even better, i just bought a "sed & awk reference book") =)
 
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Stripping ret of the lines in a file (sed question)

Hi all, I didn't use SED for 20 years and was never an expert. So my current knowledge is about zero. Please be patient with me. I'm neither a native speaker. I have a huge dictionary file and want the rest of the lines stripped. Everything after (and including) the "/" should be stripped. I... (2 Replies)
Discussion started by: Hinnerk2005
2 Replies

2. Shell Programming and Scripting

How to find the X highest values in a list depending on the values of another list with bash/awk?

Hi everyone, This is an exemple of inpout.txt file (a "," delimited text file which can be open as csv file): ID, Code, Value, Store SP|01, AABBCDE, 15, 3 SP|01, AABBCDE, 14, 2 SP|01, AABBCDF, 13, 2 SP|01, AABBCDE, 16, 3 SP|02, AABBCED, 15, 2 SP|01, AABBCDF, 12, 3 SP|01, AABBCDD,... (1 Reply)
Discussion started by: jeremy589
1 Replies

3. Shell Programming and Scripting

For x in a b c; { list; } (in lieu of do list; done) works (in bash) - why and how?

In this post I came across the cited construct. It works! while ... { list; } does not. man bash does not mention it (or, better, I didn't see it). Any reason for / behind this? Am I missing something? (5 Replies)
Discussion started by: RudiC
5 Replies

4. Shell Programming and Scripting

Removing lines from one list from another list.

Hello, I was wondering if there was an easy way to take lines from a single-column list, and remove them from a second single-column list. For example, I want to remove the contents of list 1 from list 2. How would I do this? Contents of list 1: server1a server2b server3c server4a... (2 Replies)
Discussion started by: LinuxRacr
2 Replies

5. UNIX for Dummies Questions & Answers

Stripping down binaries

Hello, I am the CEO of Grand Tech Corporation. We are launching Linux NT and forgive me, but I do not know how to strip binaries down in Mandriva Linux. Can someone tell me a way to?:b: (2 Replies)
Discussion started by: Linux NT
2 Replies

6. Shell Programming and Scripting

Bash script - stripping away characters that can't be used in filenames

I want to create a temp file which is named based on a search string. The search string may contain spaces or characters that aren't supposed to be used in filenames so I want to strip those out. My thought was to use 'tr' with but the result is the opposite of what I want: $ echo "test... (5 Replies)
Discussion started by: mglenney
5 Replies

7. Shell Programming and Scripting

stripping out non-numeric values in a list

hi all, i'm very new to scripting and have the folllowing issue. I have used a few commands to get a list of numbers, but I need to strip away the non-numeric ones, and then need a total of all values. any ideas? root@unixserver # cat myfile | awk '{print $8}'| sort -rn 1504 1344 896 704... (2 Replies)
Discussion started by: badoshi
2 Replies

8. UNIX for Dummies Questions & Answers

merging 2 lines with awk and stripping first two words

Hey all i am pretty new to awk... here my problem. My input is something like this: type: NSR client; name: pegasus; save set: /, /var, /part, /part/part2, /testpartition, /foo/bar,... (9 Replies)
Discussion started by: bazzed
9 Replies
Login or Register to Ask a Question