Sponsored Content
Top Forums Shell Programming and Scripting Split a folder with huge number of files in n folders Post 302906661 by AlokKumbhare on Sunday 22nd of June 2014 05:08:05 PM
Old 06-22-2014
Quote:
Originally Posted by Don Cragun
What OS are you using?

Do any of the 350,000 files you're moving have names that contain any space, tab, or newline characters?
Hi Don,

I am on Ubuntu Server 12.04 LTS. The files do not contain any special characters except _

Actually the format is something like

<xyz>_n1_n2_n3.pbx

where <xyz> is a prefix containing only alphabets [a-z A-Z]
n1 is a number less than 10
n2 is a number less than 1000
n3 is a number less than 10000
pbx is the extension

---------- Post updated at 04:08 PM ---------- Previous update was at 04:06 PM ----------

Quote:
Originally Posted by MadeInGermany
Aren't 350 000 files too many arguments for for i in *?
Safer and faster is
Code:
find . -type f |
while read i

Thanks.. Although I did not get an error for too many files with for I in *, find seems faster and as you mentioned safer if the number of files increase.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

delete all folders/files and keep only the last 10 in a folder

Hi, I want to write a script that deletes all folders and keep the last 10 recent folders. I know the following: ls -ltr will sort the folders from old to recent. ls -ltr | awk '{print $9}' will list the folder names (with a blank line at the beginning) I want to get the 10th folder from... (3 Replies)
Discussion started by: melanie_pfefer
3 Replies

2. Shell Programming and Scripting

Split a huge data into few different files?!

Input file data contents: >seq_1 MSNQSPPQSQRPGHSHSHSHSHAGLASSTSSHSNPSANASYNLNGPRTGGDQRYRASVDA >seq_2 AGAAGRGWGRDVTAAASPNPRNGGGRPASDLLSVGNAGGQASFASPETIDRWFEDLQHYE >seq_3 ATLEEMAAASLDANFKEELSAIEQWFRVLSEAERTAALYSLLQSSTQVQMRFFVTVLQQM ARADPITALLSPANPGQASMEAQMDAKLAAMGLKSPASPAVRQYARQSLSGDTYLSPHSA... (7 Replies)
Discussion started by: patrick87
7 Replies

3. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

4. Shell Programming and Scripting

Move all files but not folders to a new folder

Hi, I have a sub directory with a number of files and folders. What i want is a subdirectory with just folders and not files for cleanliness sake. So I want to move the files into the new folder but keep the folders in the same place. Move all files (but not folders) to new folder. I am... (4 Replies)
Discussion started by: Hopper_no1
4 Replies

5. Shell Programming and Scripting

How to delete a huge number of files at a time

I met a problem on HPUX with 64G RAM and 20 CPU. There are 5 million files with file name from file0000001.dat to file9999999.dat, in the same directory, and with some other files with random names. I was trying to remove all the files from file0000001.dat to file9999999.dat at the same time.... (9 Replies)
Discussion started by: lisp21
9 Replies

6. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

7. Shell Programming and Scripting

moving files from one folder to many folders

I have a more than 10 K files in a folder. They are accumulated in a period of more than an year (Say from 13th July 2010 to 4th June 2011). I need to perform housekeeping on this. The requirement is to create a folder like 13Jul2010,14July2010,......3June2011,4June2010 and then from the main... (2 Replies)
Discussion started by: realspirituals
2 Replies

8. Shell Programming and Scripting

Symlink all files from one folder into all found folders

Hi. I have a folder which contains my application. I then have a flexible number of folders in another directory, called “sites”. It looks like this: -Application -- Test.html -- CSS --- Style.css -Sites --Site1 --Site2 I want to symlink all the files in the application folder... (1 Reply)
Discussion started by: Spadez
1 Replies

9. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

10. Shell Programming and Scripting

Moving files and folders to another folder

I recently bought Synology server and realised it can run scripts. I would need fairly simple script which moves all files and folders from ARCHIVE folder to WORKING folder. I would also need to maintain folder structure as each of the folders may contain subfolders. How would I go about it as I am... (1 Reply)
Discussion started by: ###
1 Replies
OD(1)									FSF								     OD(1)

NAME
od - dump files in octal and other formats SYNOPSIS
od [OPTION]... [FILE]... od --traditional [FILE] [[+]OFFSET [[+]LABEL]] DESCRIPTION
Write an unambiguous representation, octal bytes by default, of FILE to standard output. With more than one FILE argument, concatenate them in the listed order to form the input. With no FILE, or when FILE is -, read standard input. All arguments to long options are mandatory for short options. -A, --address-radix=RADIX decide how file offsets are printed -j, --skip-bytes=BYTES skip BYTES input bytes first -N, --read-bytes=BYTES limit dump to BYTES input bytes -s, --strings[=BYTES] output strings of at least BYTES graphic chars -t, --format=TYPE select output format or formats -v, --output-duplicates do not use * to mark line suppression -w, --width[=BYTES] output BYTES bytes per output line --traditional accept arguments in traditional form --help display this help and exit --version output version information and exit Traditional format specifications may be intermixed; they accumulate: -a same as -t a, select named characters -b same as -t oC, select octal bytes -c same as -t c, select ASCII characters or backslash escapes -d same as -t u2, select unsigned decimal shorts -f same as -t fF, select floats -h same as -t x2, select hexadecimal shorts -i same as -t d2, select decimal shorts -l same as -t d4, select decimal longs -o same as -t o2, select octal shorts -x same as -t x2, select hexadecimal shorts For older syntax (second call format), OFFSET means -j OFFSET. LABEL is the pseudo-address at first byte printed, incremented when dump is progressing. For OFFSET and LABEL, a 0x or 0X prefix indicates hexadecimal, suffixes may be . for octal and b for multiply by 512. TYPE is made up of one or more of these specifications: a named character c ASCII character or backslash escape d[SIZE] signed decimal, SIZE bytes per integer f[SIZE] floating point, SIZE bytes per integer o[SIZE] octal, SIZE bytes per integer u[SIZE] unsigned decimal, SIZE bytes per integer x[SIZE] hexadecimal, SIZE bytes per integer SIZE is a number. For TYPE in doux, SIZE may also be C for sizeof(char), S for sizeof(short), I for sizeof(int) or L for sizeof(long). If TYPE is f, SIZE may also be F for sizeof(float), D for sizeof(double) or L for sizeof(long double). RADIX is d for decimal, o for octal, x for hexadecimal or n for none. BYTES is hexadecimal with 0x or 0X prefix, it is multiplied by 512 with b suffix, by 1024 with k and by 1048576 with m. Adding a z suffix to any type adds a display of printable characters to the end of each line of output. --string without a number implies 3. --width without a number implies 32. By default, od uses -A o -t d2 -w 16. AUTHOR
Written by Jim Meyering. REPORTING BUGS
Report bugs to <bug-coreutils@gnu.org>. COPYRIGHT
Copyright (C) 2002 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU- LAR PURPOSE. SEE ALSO
The full documentation for od is maintained as a Texinfo manual. If the info and od programs are properly installed at your site, the com- mand info od should give you access to the complete manual. od (coreutils) 4.5.3 February 2003 OD(1)
All times are GMT -4. The time now is 05:26 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy