The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
bash while read how to remove \n character papasj Shell Programming and Scripting 3 05-25-2009 10:24 PM
Remove duplicate text dejavu88 Shell Programming and Scripting 7 06-08-2008 05:37 PM
Remove html tags with bash dejavu88 Shell Programming and Scripting 4 05-22-2008 02:58 PM
remove lines in text starting with . (period) Movomito UNIX for Dummies Questions & Answers 11 04-24-2008 11:19 AM
command find returned bash: /usr/bin/find: Argument list too long yacsil Shell Programming and Scripting 1 12-15-2003 06:38 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 04-04-2008
Movomito Movomito is offline
Registered User
  
 

Join Date: Apr 2008
Posts: 27
bash find and remove text

Here's the story:
I have two txt files of filenames one is like this

W00CHZ0103340-I1CZ31
W00CHZ0103340-I1CZ32
W00CHZ0103340-I1CZ33
W00CHZ0103341-I1CZ35
W00CHZ0103342-I1CZ46
W00CHZ0103343-I1CZ37
W00CHZ0103344-I1CZ39
W00CHZ0103345-I1CZ43
W00CHZ0103345-I1CZ44
...
the other like this

W00CHZ0103340/images/W00CHZ0103340-I1CZ31/
W00CHZ0103340/images/W00CHZ0103340-I1CZ32/
W00CHZ0103340/images/W00CHZ0103340-I1CZ33/
W00CHZ0103341/images/W00CHZ0103341-I1CZ35/
W00CHZ0103342/images/W00CHZ0103342-I1CZ46/
W00CHZ0103343/images/W00CHZ0103343-I1CZ37/
W00CHZ0103344/images/W00CHZ0103344-I1CZ39/
W00CHZ0103345/images/W00CHZ0103345-I1CZ43/
W00CHZ0103345/images/W00CHZ0103345-I1CZ44/

I want to remove all the text up to and including images/ and then the final / then i would be left with identical lists. On list comes from an ls command and the other from some type of manifest that i am checking against. I need to do this so that i can then compare the two for unique entries and thereby find out what's missing. Please Help me
  #2 (permalink)  
Old 04-04-2008
djsal djsal is offline
Registered User
  
 

Join Date: Apr 2004
Location: Long Island Ny
Posts: 23
Quote:
Originally Posted by Movomito View Post
Here's the story:
I have two txt files of filenames one is like this

W00CHZ0103340-I1CZ31
W00CHZ0103340-I1CZ32
W00CHZ0103340-I1CZ33
W00CHZ0103341-I1CZ35
W00CHZ0103342-I1CZ46
W00CHZ0103343-I1CZ37
W00CHZ0103344-I1CZ39
W00CHZ0103345-I1CZ43
W00CHZ0103345-I1CZ44
...
the other like this

W00CHZ0103340/images/W00CHZ0103340-I1CZ31/
W00CHZ0103340/images/W00CHZ0103340-I1CZ32/
W00CHZ0103340/images/W00CHZ0103340-I1CZ33/
W00CHZ0103341/images/W00CHZ0103341-I1CZ35/
W00CHZ0103342/images/W00CHZ0103342-I1CZ46/
W00CHZ0103343/images/W00CHZ0103343-I1CZ37/
W00CHZ0103344/images/W00CHZ0103344-I1CZ39/
W00CHZ0103345/images/W00CHZ0103345-I1CZ43/
W00CHZ0103345/images/W00CHZ0103345-I1CZ44/

I want to remove all the text up to and including images/ and then the final / then i would be left with identical lists. On list comes from an ls command and the other from some type of manifest that i am checking against. I need to do this so that i can then compare the two for unique entries and thereby find out what's missing. Please Help me
on the second file, run this:

#!/bin/bash
cat secondfile.txt | while read line
do
FN=`echo $line | awk -F\/ '{print $(NF-1)}' `
echo $FN
done
  #3 (permalink)  
Old 04-04-2008
Franklin52 Franklin52 is online now Forum Staff  
Moderator
  
 

Join Date: Feb 2007
Posts: 4,322
With sed:

Code:
sed 's!.*\(.*\)/!\1!' file2
or with awk:

Code:
awk -F/ '{print $3}' file2
Regards
  #4 (permalink)  
Old 04-05-2008
Movomito Movomito is offline
Registered User
  
 

Join Date: Apr 2008
Posts: 27
now i have just one more question. I left work already where i need to try and use this script but it will drive me nuts all weekend if i don't get this thing licked. Will the above responses work even if the parameters following the W above change? for example sometimes the filenames change to W23704/images...etc It always starts with W but the proceeding characters can be a hadge podge of numbers and letters and not always the same number. Again, any help is really appreciated. I just want to know that i've got it come monday.
  #5 (permalink)  
Old 04-05-2008
Franklin52 Franklin52 is online now Forum Staff  
Moderator
  
 

Join Date: Feb 2007
Posts: 4,322
Quote:
Originally Posted by Movomito View Post
now i have just one more question. I left work already where i need to try and use this script but it will drive me nuts all weekend if i don't get this thing licked. Will the above responses work even if the parameters following the W above change? for example sometimes the filenames change to W23704/images...etc It always starts with W but the proceeding characters can be a hadge podge of numbers and letters and not always the same number. Again, any help is really appreciated. I just want to know that i've got it come monday.
No problem. The sed command prints the piece between the last 2 slashes and the awk command changes the field seperator to a slash and prints the 3th field.

Regards
  #6 (permalink)  
Old 04-05-2008
cfajohnson's Avatar
cfajohnson cfajohnson is offline Forum Advisor  
Shell programmer, author
  
 

Join Date: Mar 2007
Location: Toronto, Canada
Posts: 2,361
Quote:
Originally Posted by djsal View Post
on the second file, run this:

#!/bin/bash
cat secondfile.txt | while read line
do
FN=`echo $line | awk -F\/ '{print $(NF-1)}' `
echo $FN
done

There's no need for cat, and calling awk for each and every line is ridiculously inefficient.

Adding to the inefficiency is storing the reault of each awk call in a variable with command substitution then printing the variable. Why not let awk's output be printed directly?

All it takes is a single call to awk, with the filename as an argument:
Code:
awk -F/ '{ print $(NF-1) }' secondfile.txt
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 10:57 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0