Extracting the file name from the specified URL


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extracting the file name from the specified URL
# 1  
Old 04-09-2012
Extracting the file name from the specified URL

Hello Everyone,

I am trying to write a shell script(or Perl Script) that would do the following:

I have a file that contains the following lines:
File:
Code:
https://ims-svnus.com/dev/DB/trunk/feeds/templates/shell_script.txt -r860
https://ims-svnus.com/dev/DB/trunk/feeds/perl/templates/perl_script.txt -r867
https://ims-svnus.com/dev/DB/trunk/cfg/temlpates/cfg_file.txt -r868
...so on

Script:
Code:
#!/bin/ksh
file=$1
path=/home/filter/
while read line
do
echo "Getting $line"
svn export $line /home/filter/feed/templates/<filename>   --> need to Extract the file name from the url
done < $file
exit

Here:
I need to extract the file name from the above file that contains the lines:
using perl or shell script:

Code:
https://ims-svnus.com/dev/DB/trunk/feeds/templates/shell_script.txt -r860
https://ims-svnus.com/dev/DB/trunk/feeds/perl/templates/perl_script.txt -r867
https://ims-svnus.com/dev/DB/trunk/cfg/temlpates/cfg_file.txt -r868
...so on

Example:
Code:
if s/templates/
<then extract the file name as:>
shell_script.txt
so that the script would save along with the file name in that directory.

Could someone please help me out to extract the filename from the above url.

It would be greatly appreciated for your time and help.
# 2  
Old 04-09-2012
Code:
cat FILE
https://ims-svnus.com/dev/DB/trunk/feeds/templates/shell_script.txt -r860
https://ims-svnus.com/dev/DB/trunk/feeds/perl/templates/perl_script.txt -r867
https://ims-svnus.com/dev/DB/trunk/cfg/temlpates/cfg_file.txt -r868

perl -lpe 's!.*/(.*)\s+.*!$1!' FILE
shell_script.txt
perl_script.txt
cfg_file.txt

The same with
Code:
sed 's!.*/\(.*\)  *.*!\1!' FILE

This User Gave Thanks to yazu For This Post:
# 3  
Old 04-09-2012
Try:
Code:
while read url rev; do
  echo "${url##*/}"
done < "$file"

This User Gave Thanks to Scrutinizer For This Post:
# 4  
Old 04-09-2012
An easy approach:


You can use the basename command for the same purpose very conveniently.Here is its syntax:

basename <absolute pathname>

It will extract the filename from the absolute path.

Example:

Code:
$$ basename /home/user/abc/def/ghi/...../aa.txt
aa.txt

Thus the desired output is obtained.

This User Gave Thanks to Epsilon For This Post:
# 5  
Old 04-09-2012
Thanks a lot for all your quick replies.This would really help me out.

Seems like my script is still to be performed:

In the file , I would have the following:
Code:
https://ims-svnus.com/dev/DB/trunk/bin/feeds/templates/shell_script.txt -r860
https://ims-svnus.com/dev/DB/trunk/bin/perl/templates/perl_script.txt -r867
https://ims-svnus.com/dev/DB/trunk/cfg/temlpates/cfg_file.txt -r868
...so on..

I will have the following directories already created:
/home/filter/prod/bin
/home/filter/prod/cfg

I need to read line by line from the above file and do the following:
1) Somehow split the line based on the word "Templates" and then store the filename(ex: shell_script.txt) into a variable.($filename)
2) The other part of the splitted line should be stored in another variable.($path)
3) $path = reverse $path -- to extract the directory feeds, cfg ($1) (The directory should be created if it doesn't exists. i.e. /home/filter/prod/bin/feeds
4) then svn export <$line> <save to /home/filter/prod/$path/$filename)
i.e. the extracted file should save under that directory.

Script
Code:
$ver=$opts{"v"};
$environment=$opts{"e"};

$vob_dir="https://ims-svnus.com/dev/DB/trunk/"
$dest_tmpl_path="/home/filter/prod/";
$dest_ovrd_path="/home/filter/prod/ovrd/";

($full_path, $version) = split('trunk', $ver);
        $temp = reverse $full_path;
 ($file) = split (/\//, $temp);
        $file = reverse $file;
$full_path =~ /^${vob_dir}(.*)${file}$/;  -- not sure whether this is correct
        $full_path = $1;
        $full_path =~ s/templates\///;
$dest_tmpl_path .= "$full_path";
        $dest_ovrd_path .= "$full_path";

        `mkdir -p $dest_tmpl_path` if (! -e $dest_tmpl_path );
        `mkdir -p $dest_ovrd_path` if (! -e $dest_ovrd_path );
}

Could someone please help out in this. Would really appreciate your help.
# 6  
Old 04-09-2012
Try:
Code:
while read url rev
do 
  name=${url##*/templates/}
  base=${url##*/trunk/}
  dir=${base%%/templates/*}
  echo "$name  $dir"
done < "$file"

There's a small typo on the third line of your sample..
This User Gave Thanks to Scrutinizer For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Extracting URL with domain

I have a file like this: http://article.wn.com/view/2010/11/26/IV_drug_policy_feels_HIV_patients_Red_Cross/ http://aidsjournal.com/,www.cfpa.org.cn/page1/page2 , www.youtube.com http://seattletimes.nwsource.com/html/jerrybrewer/2013517803_brewer25.html... (1 Reply)
Discussion started by: csim_mohan
1 Replies

2. Shell Programming and Scripting

Extracting the column containing URL from a text file

I have the file like this: Timestamp URL Text 1331635241000 http://example.com Peoples footage at www.test.com,http://example4.com 1331635231000 http://example1.net crack the nuts http://example6.com 1331635280000 http://example2.net ... (3 Replies)
Discussion started by: csim_mohan
3 Replies

3. Shell Programming and Scripting

Extracting the column containing URL from a text file

I have the file like this: Timestamp URL Text 1331635241000 http://example.com Peoples footage at www.test.com,http://example4.com 1331635231000 http://example1.net crack the nuts http://example6.com 1331635280000 http://example2.net ... (0 Replies)
Discussion started by: csim_mohan
0 Replies

4. Shell Programming and Scripting

Extracting the column containing URL from a text file

I have the file like this: Timestamp URL Text 1331635241000 http://example.com Peoples footage at www.test.com,http://example4.com 1331635231000 http://example1.net crack the nuts http://example6.com 1331635280000 http://example2.net ... (0 Replies)
Discussion started by: csim_mohan
0 Replies

5. Shell Programming and Scripting

Reading URL using Mechanize and dump all the contents of the URL to a file

Hello, Am very new to perl , please help me here !! I need help in reading a URL from command line using PERL:: Mechanize and needs all the contents from the URL to get into a file. below is the script which i have written so far , #!/usr/bin/perl use LWP::UserAgent; use... (2 Replies)
Discussion started by: scott_cog
2 Replies

6. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

7. Shell Programming and Scripting

Extracting anchor text and its URL from HTML files in BASH

Hi All, I have some HTML files and my requirement is to extract all the anchor text words from the HTML files along with their URLs and store the result in a separate text file separated by space. For example, <a href="/kid/stay_healthy/">Staying Healthy</a> which has /kid/stay_healthy/ as... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

8. Web Development

Regex to rewrite URL to another URL based on HTTP_HOST?

I am trying to find a way to test some code, but I need to rewrite a specific URL only from a specific HTTP_HOST The call goes out to http://SUB.DOMAIN.COM/showAssignment/7bde10b45efdd7a97629ef2fe01f7303/jsmodule/Nevow.Athena The ID in the middle is always random due to the cookie. I... (5 Replies)
Discussion started by: EXT3FSCK
5 Replies

9. Shell Programming and Scripting

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

10. UNIX for Advanced & Expert Users

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies
Login or Register to Ask a Question