bash find and remove text


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting bash find and remove text
# 1  
Old 04-04-2008
bash find and remove text

Here's the story:
I have two txt files of filenames one is like this

W00CHZ0103340-I1CZ31
W00CHZ0103340-I1CZ32
W00CHZ0103340-I1CZ33
W00CHZ0103341-I1CZ35
W00CHZ0103342-I1CZ46
W00CHZ0103343-I1CZ37
W00CHZ0103344-I1CZ39
W00CHZ0103345-I1CZ43
W00CHZ0103345-I1CZ44
...
the other like this

W00CHZ0103340/images/W00CHZ0103340-I1CZ31/
W00CHZ0103340/images/W00CHZ0103340-I1CZ32/
W00CHZ0103340/images/W00CHZ0103340-I1CZ33/
W00CHZ0103341/images/W00CHZ0103341-I1CZ35/
W00CHZ0103342/images/W00CHZ0103342-I1CZ46/
W00CHZ0103343/images/W00CHZ0103343-I1CZ37/
W00CHZ0103344/images/W00CHZ0103344-I1CZ39/
W00CHZ0103345/images/W00CHZ0103345-I1CZ43/
W00CHZ0103345/images/W00CHZ0103345-I1CZ44/

I want to remove all the text up to and including images/ and then the final / then i would be left with identical lists. On list comes from an ls command and the other from some type of manifest that i am checking against. I need to do this so that i can then compare the two for unique entries and thereby find out what's missing. Please Help me
# 2  
Old 04-04-2008
Quote:
Originally Posted by Movomito
Here's the story:
I have two txt files of filenames one is like this

W00CHZ0103340-I1CZ31
W00CHZ0103340-I1CZ32
W00CHZ0103340-I1CZ33
W00CHZ0103341-I1CZ35
W00CHZ0103342-I1CZ46
W00CHZ0103343-I1CZ37
W00CHZ0103344-I1CZ39
W00CHZ0103345-I1CZ43
W00CHZ0103345-I1CZ44
...
the other like this

W00CHZ0103340/images/W00CHZ0103340-I1CZ31/
W00CHZ0103340/images/W00CHZ0103340-I1CZ32/
W00CHZ0103340/images/W00CHZ0103340-I1CZ33/
W00CHZ0103341/images/W00CHZ0103341-I1CZ35/
W00CHZ0103342/images/W00CHZ0103342-I1CZ46/
W00CHZ0103343/images/W00CHZ0103343-I1CZ37/
W00CHZ0103344/images/W00CHZ0103344-I1CZ39/
W00CHZ0103345/images/W00CHZ0103345-I1CZ43/
W00CHZ0103345/images/W00CHZ0103345-I1CZ44/

I want to remove all the text up to and including images/ and then the final / then i would be left with identical lists. On list comes from an ls command and the other from some type of manifest that i am checking against. I need to do this so that i can then compare the two for unique entries and thereby find out what's missing. Please Help me
on the second file, run this:

#!/bin/bash
cat secondfile.txt | while read line
do
FN=`echo $line | awk -F\/ '{print $(NF-1)}' `
echo $FN
done
# 3  
Old 04-04-2008
With sed:

Code:
sed 's!.*\(.*\)/!\1!' file2

or with awk:

Code:
awk -F/ '{print $3}' file2

Regards
# 4  
Old 04-05-2008
now i have just one more question. I left work already where i need to try and use this script but it will drive me nuts all weekend if i don't get this thing licked. Will the above responses work even if the parameters following the W above change? for example sometimes the filenames change to W23704/images...etc It always starts with W but the proceeding characters can be a hadge podge of numbers and letters and not always the same number. Again, any help is really appreciated. I just want to know that i've got it come monday.
# 5  
Old 04-05-2008
Quote:
Originally Posted by Movomito
now i have just one more question. I left work already where i need to try and use this script but it will drive me nuts all weekend if i don't get this thing licked. Will the above responses work even if the parameters following the W above change? for example sometimes the filenames change to W23704/images...etc It always starts with W but the proceeding characters can be a hadge podge of numbers and letters and not always the same number. Again, any help is really appreciated. I just want to know that i've got it come monday.
No problem. The sed command prints the piece between the last 2 slashes and the awk command changes the field seperator to a slash and prints the 3th field.

Regards
# 6  
Old 04-05-2008
Quote:
Originally Posted by djsal
on the second file, run this:

#!/bin/bash
cat secondfile.txt | while read line
do
FN=`echo $line | awk -F\/ '{print $(NF-1)}' `
echo $FN
done

There's no need for cat, and calling awk for each and every line is ridiculously inefficient.

Adding to the inefficiency is storing the reault of each awk call in a variable with command substitution then printing the variable. Why not let awk's output be printed directly?

All it takes is a single call to awk, with the filename as an argument:
Code:
awk -F/ '{ print $(NF-1) }' secondfile.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Bash to remove find and remove specific extension

The bash below executes and does find all the .bam files in each R_2019 folder. However set -x shows that the .bam extension only gets removed from one .bam file in each folder (appears to be the last in each). Why is it not removing the extension from each (this is $SAMPLE)? Thank you :). set... (4 Replies)
Discussion started by: cmccabe
4 Replies

2. Shell Programming and Scripting

How to remove the text between all curly brackets from text file?

Hello experts, I have a text file with lot of curly brackets (both opening { & closing } ). I need to delete them alongwith the text between opening & closing brackets' pair. For ex: Input:- 59. Rh1 Qe4 {(Qf5-e4 Qd8-g8+ Kg6-f5 Qg8-h7+ Kf5-e5 Qh7-e7+ Ke5-f5 Qe7-d7+ Qe4-e6 Qd7-h7+ Qe6-g6... (6 Replies)
Discussion started by: prvnrk
6 Replies

3. Shell Programming and Scripting

Bash to select text and apply it to a selected file in bash

In the bash below I am asking the user for a panel and reading that into bed. Then asking the user for a file and reading that into file1.Is the grep in bold the correct way to apply the selected panel to the file? I am getting a syntax error. Thank you :) ... (4 Replies)
Discussion started by: cmccabe
4 Replies

4. Shell Programming and Scripting

awk to skip lines find text and add text based on number

I am trying to use awk skip each line with a ## or # and check each line after for STB= and if that value in greater than or = to 0.8, then at the end of line the text "STRAND BIAS" is written in else "GOOD". So in the file of 4 entries attached. awk tried: awk NR > "##"' "#" -F"STB="... (6 Replies)
Discussion started by: cmccabe
6 Replies

5. Shell Programming and Scripting

remove ] followed by newline in bash

I need to remove ] followed by newline character to convert lines like: line1] line2 into: line1line2 (4 Replies)
Discussion started by: locoroco
4 Replies

6. Shell Programming and Scripting

Find and add/replace text in text files

Hi. I would like to have experts help on below action. I have text files in which page nubmers exists in form like PAGE : 1 PAGE : 2 PAGE : 3 and so on there is other text too. I would like to know is it possible to check the last occurance of Page... (6 Replies)
Discussion started by: lodhi1978
6 Replies

7. Shell Programming and Scripting

[bash help]Adding multiple lines of text into a specific spot into a text file

I am attempting to insert multiple lines of text into a specific place in a text file based on the lines above or below it. For example, Here is a portion of a zone file. IN NS ns1.domain.tld. IN NS ns2.domain.tld. IN ... (2 Replies)
Discussion started by: cdn_humbucker
2 Replies

8. Shell Programming and Scripting

Bash snippet to find files based on a text file?

Evening all. I'm having a terrible time with a script I've been working on for a few days now... Say I have a text file named top10song.tm2, with the following in it: kernkraft 400 Imagine i kissed a girl Thriller animals hallelujah paint it black psychosocial Oi to the world... (14 Replies)
Discussion started by: DJ Charlie
14 Replies

9. Shell Programming and Scripting

find, copy and replace text in bash or sh

here is my prob .. i have a very large text files and i need to locate specific lines, copy them and then replace a single word in the replaced text example find all lines that contain '/etc', copy the line immediately below (not at the end of the file) and then replace '/etc' with '/root'... (1 Reply)
Discussion started by: jmvbxx
1 Replies

10. Shell Programming and Scripting

command find returned bash: /usr/bin/find: Argument list too long

Hello, I create a file touch 1201093003 fichcomp and inside a repertory (which hava a lot of files) I want to list all files created before this file : find *.* \! -maxdepth 1 - newer fichcomp but this command returned bash: /usr/bin/find: Argument list too long but i make a filter all... (1 Reply)
Discussion started by: yacsil
1 Replies
Login or Register to Ask a Question