Visit Our UNIX and Linux User Community


How best to remove certain characters from filenames and folders recursively


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How best to remove certain characters from filenames and folders recursively
# 1  
Old 05-14-2012
How best to remove certain characters from filenames and folders recursively

hello,

I'm trying to figure out which tool is best for recursively renaming and files or folders using the characters \/*?<>| in their name. I've tried many examples that use Bash, Python and Perl, but I'm not much of a programmer I seem to have hit a roadblock.

Does anyone have any examples on how best to do this?

Thanks in advance!

Ryan
# 2  
Old 05-16-2012
First, I'm pretty sure a / cannot be in a filename. Second, what would you replace the * and ? and " with?

I would approach that like this (GNU tools specific):
Code:
find /targetdir -type f -print0 | xargs -rl0 rename_bad_filename

where rename_bad_filename is a perl script that is executable and in your PATH...
Code:
#!/usr/bin/perl
$newf = $oldf=$ARGV[0];
# replace any non alpha-numeric characters, excluding / . and - with an underscore
$newf =~ s/[^a-zA-Z0-9.\/-]/_/g;
exit 0 if $newf eq $oldf;
print "Renaming \"$oldf\" to \"$newf\"...";
rename $oldf,$newf ;
print ($! ? "Error: $!\n" : "OK\n");

Note: no error checking; existing files are clobbered.

Example output:
Code:
$ rename_bad_filename  2l4kj2l^%.
Renaming "2l4kj2l^%." to "2l4kj2l__."...Error: No such file or directory
$ rename_bad_filename  2l4kj2l.
$ rename_bad_filename  2l4kj2l.\!
Renaming "2l4kj2l.!" to "2l4kj2l._"...Error: No such file or directory
$ rename_bad_filename  \[2-7\]-ldap-reordered
Renaming "[2-7]-ldap-reordered" to "_2-7_-ldap-reordered"...OK

# 3  
Old 05-16-2012
Hey Otheus,

Thanks for the response and the help. I'm able to get your script working for some reason. Don't I need to specifically specify perl in the command line? A script I found that seems to be kind of working. I'm just not sure how to define which variables to replace. The file types that Box doesn't support are \/*?”<>| Like you, I don't think I've ever see a forward slash in a file name, but Box said to make sure it's not there.

Code:
#!/usr/bin/perl

use strict;

sub processDir
{
    my $dir=shift;
    opendir(DIR, $dir);
    my @files=grep {! /^\.\.?$/} readdir(DIR);
    closedir(DIR);
    foreach my $file(@files)
    {
        if(-d "$dir/$file") 
        {
            processDir("$dir/$file");
        }
        my $newfile=$file;
        $newfile=~ s/[,& '\(\)]/_/g; #Search for ',','&','<','>','*','?','|','"',':', "'", '(', ')' in filenames and replace them with underscores.
        if( $newfile ne $file )
        {
            print "Renaming \"$dir/$file\" to \"$dir/$newfile\"\n";
            rename "$dir/$file","$dir/$newfile" or warn("Problems renaming $dir/$file --> $dir/$newfile: $!\n"); 
        }
    }
}

my $dir=shift;
if(!defined($dir))
{
    $dir=".";
}
processDir($dir);


At the command prompt I entered perl rename2.pl Desktop/BoxMigraiton/FILESHARE
# 4  
Old 05-16-2012
Problem with renaming files and folders recursively is if you start renaming your folders, then the paths you're looking inside are changing on the go... So you need to process the insides first.

Code:
#!/bin/bash


find . -depth -name "*[,&<>*?|\":'()]*" |     # Find all files or folders containing 'bad' characters.
while read FILEDIR                            # Read them line-by-line.
do
        DIR="${FILEDIR%/*}"                   # Get the folder its inside
        FILE="${FILEDIR/*\/}"                 # Get the plain name.
        NEWFILE="${FILE//[,&<>*?|\":\'()]/_}" # Substitute _ for bad things.
        echo mv "$DIR/$FILE" "$DIR/$NEWFILE"  # Rename it.
done

Remove the 'echo' once you're sure it does what you want.
This User Gave Thanks to Corona688 For This Post:
# 5  
Old 05-16-2012
This shell script looks awesome. Just what I was looking for. Thanks again.

So if I wanted to only replace these characters:
Code:
\/*?”<>|

which line would I define that in?
# 6  
Old 05-16-2012
There's two expressions actually. Look for the comments with 'bad' in them.

The one in 'find' has * on the beginning and end because the expression has to match the entire file/dir name for find to print it... The bash expression on the other hand only matches the part you want to change.
# 7  
Old 05-16-2012
I think I understand. So both lines, should look like:
Code:
find . -depth -name "*\/*?”<>|*"

|
and
Code:
NEWFILE="${FILE//[\/*?”<>|]/_}"

Did I get that right?

Then I should run
Code:
sh renamefiles.sh BoxMigraiton/FILESHAREWITHBADFILENAMES

, correct?

Last edited by prometheon123; 05-16-2012 at 06:46 PM..

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to compare partial filenames in two folders and delete duplicates

Background: I use a TV tuner card to capture OTA video files (.mpeg) and then my Plex Media Server automatically optimizes the files (transcodes for better playback) and places them in a new directory. I have another Plex Library pointing to the new location for the optimized .mp4 files. This... (2 Replies)
Discussion started by: shaky
2 Replies

2. Shell Programming and Scripting

Change filenames recursively

Hello, I made a mistake in a script and now need to go back and change allot of filenames. I need to change "v4" in filenames to "v3". I was thinking of something like this. #!/bin/bash FILELIST=$(ls -f -R *) for FILE in $FILELIST do # create new filename ... (5 Replies)
Discussion started by: LMHmedchem
5 Replies

3. Shell Programming and Scripting

recursively going through folders and subdirectorys and running delete from crontab

Hio, So I have a crontab delete of older files setup. This script works fine if I run them by each individual directory. Problem is there are so many thousands of files and hundreds of directories and sub directories that I need to recursively have it go through and delete files by directory... (2 Replies)
Discussion started by: vsekvsek
2 Replies

4. Shell Programming and Scripting

ftp script to copy folders recursively

hi all I want a script that will use ftp to copy folder and sub folders from source server to current server. if i use -r switch then it just copies folders for 5 level. (1 Reply)
Discussion started by: kashif.live
1 Replies

5. Shell Programming and Scripting

HELP! Need to compare 2 folders on 2 different systems, and copy unmatched filenames to other folder

This has been tearing my hair out. I need to: 1: compare server1:/data/archive/ to server2:/data/archive/ (through rsync, ssh, etc) 2: filenames that don't match, get copied (scp) to server2:/data/ server1 and server2 have ssh, scp, rsync access between eachother. Is there any option in... (3 Replies)
Discussion started by: damang111
3 Replies

6. UNIX for Dummies Questions & Answers

find & remove characters in filenames

I have a group of files in different directories with characters such as " ? : in the file names. How do I find these files and remove these characters on mass? Thanks (19 Replies)
Discussion started by: barrydocks
19 Replies

7. Shell Programming and Scripting

Change all filenames under different folders...

Hi, all: I'd love to use shell script to change all filenames under different folders once for all: I've got over 100 folders, in each of them, there is a file named "a.ppm". I wanna change all these "a.ppm" to "b.ppm", and still . Visually, the directory structure looks like: and hope... (1 Reply)
Discussion started by: jiapei100
1 Replies

8. Shell Programming and Scripting

RSYNC script to transfer folders recursively without overwriting via FTP

Hi all, I would need a bash script to sync/transfer folders recursively via FTP/RSYNC (I initially planned to use FTP but I heard RSYNC would fit a lot better for this job(?)) The situation: 3 different Linux servers 1. source 2. destination - Samba 3. Server where the script runs on ... (2 Replies)
Discussion started by: thibautp
2 Replies

9. UNIX for Dummies Questions & Answers

delete recursively contents of folders

hi, I've a folder structure like : /home/project/LIBNAMEA/FILE1 /home/project/LIBNAMED/FILE2 /home/project/LIBNAMEC/FILE3 /home/project/LIBNAMED/FILE4 /home/project/LIBNAMEX/FILE5 (there is no relation in the letters after the project/ ) and i need to delete the files keeping... (5 Replies)
Discussion started by: jtmartins
5 Replies

10. Shell Programming and Scripting

remove special characters from filename recursively

hi: i have several thousand files from users and of course they use all kind of characters on filenames. I have things like: My special report (1999 ) Lisa & Jack's work.doc crazy. How do I remove all this characters in the current dir and subdirs too? Thanks. (3 Replies)
Discussion started by: jason7
3 Replies

Featured Tech Videos