Sponsored Content
Top Forums Shell Programming and Scripting Using find in a directory containing large number of files Post 302545336 by panyam on Monday 8th of August 2011 05:37:06 AM
Old 08-08-2011
Code:
 
Is this correct?
 
cat $page | sed -e 's/<.*>//g' $page>$page.txt

It should be 
 
sed -e 's/<.*>//g' $page>$page.txt

This User Gave Thanks to panyam For This Post:
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

moving large number of files

I have a task to move more than 35000 files every two hours, from the same directory to another directory based on a file that has the list of filenames I tried the following logics (1) find . -name \*.dat > list for i in `cat list` do mv $i test/ done (2) cat list|xargs -i mv "{}"... (7 Replies)
Discussion started by: bryan
7 Replies

2. UNIX for Dummies Questions & Answers

Problem using find with prune on large number of files

Hi all; I'm having a problem when want to list a large number of files in current directory using find together with the prune option. First i used this command but it list all the files including those in sub directories: find . -name "*.dat" | xargs ls -ltr Then i modified the command... (2 Replies)
Discussion started by: ashikin_8119
2 Replies

3. Shell Programming and Scripting

Need help combining large number of text files

Hi, i have more than 1000 data files(.txt) like this first file format: 178.83 554.545 179.21 80.392 second file: 178.83 990.909 179.21 90.196 etc. I want to combine them to the following format: 178.83,554.545,990.909,... 179.21,80.392,90.196,... (7 Replies)
Discussion started by: mr_monocyte
7 Replies

4. Shell Programming and Scripting

Concatenation of a large number of files

Hellow i have a large number of files that i want to concatenate to one. these files start with the word 'VOICE_' for example VOICE_0000000000 VOICE_1223o23u0 VOICE_934934927349 I use the following code: cat /ODS/prepaid/CDR_FLOW/MEDIATION/VOICE_* >> /ODS/prepaid/CDR_FLOW/WORK/VOICE ... (10 Replies)
Discussion started by: chriss_58
10 Replies

5. Shell Programming and Scripting

Find line number of bad data in large file

Hi Forum. I was trying to search the following scenario on the forum but was not able to. Let's say that I have a very large file that has some bad data in it (for ex: 0.0015 in the 12th column) and I would like to find the line number and remove that particular line. What's the easiest... (3 Replies)
Discussion started by: pchang
3 Replies

6. UNIX for Dummies Questions & Answers

Delete large number of files

Hi. I need to delete a large number of files listed in a txt file. There are over 90000 files in the list. Some of the directory names and some of the file names do have spaces in them. In the file, each line is a full path to a file: /path/to/the files/file1 /path/to/some other/files/file 2... (4 Replies)
Discussion started by: inakajin
4 Replies

7. Shell Programming and Scripting

How to count number of files in directory and write to new file with number of files and their name?

Hi! I just want to count number of files in a directory, and write to new text file, with number of files and their name output should look like this,, assume that below one is a new file created by script Number of files in directory = 25 1. a.txt 2. abc.txt 3. asd.dat... (20 Replies)
Discussion started by: Akshay Hegde
20 Replies

8. Shell Programming and Scripting

Sftp large number of files

Want to sftp large number of files ... approx 150 files will come to server every minute. (AIX box) Also need make sure file has been sftped successfully... Please let me know : 1. What is the best / faster way to transfer files? 2. should I use batch option -b so that connectivity will be... (3 Replies)
Discussion started by: vegasluxor
3 Replies

9. Shell Programming and Scripting

Find Large Files Recursively From Specific Directory

Hi. I found many scripts in the web of achieving this. But I like to use this one find /EDWH-DMT03 -xdev -size +10000 -exec ls -la {} \;|sort -n -k 5 > LARGE.rst But the problem is, why it still list out files with 89 bytes as the output? Is there anything wrong with the command? My... (7 Replies)
Discussion started by: aimy
7 Replies
digitaglinktree(1)						Linux User's Manual						digitaglinktree(1)

NAME
digitaglinktree - Export tag structure of photos in digikam to the filesystem. SYNOPSIS
digitaglinktree -l taglinkdir | -A archivedir -d database [-r rootdir] [-H|-f|-a|-v|-C] DESCRIPTION
digitaglinktree will create a linktree for all photos in a digikam database that have tags set on them. Tags (like eg. "family", "events", ...) are used in digikam to create virtual folders containing images that all have one or more tags assigned. Please note: Photos that have no tags at all assigned are silently ignored by this script. The program will not modify or even touch your original photos managed by digikam. The script can be used in two ways: If you call it using Option -l taglinkdir the script will create the user specified directory taglinkdir and inside this directory it will create sub directories for digikam tags set on the photos. Inside these subdirectories it will finally place symbolic or hard links (see -H) to photos having the tags in question. As a result you will see the tags of your pho- tos as folders and in these folders you will find links to your original photos. In this way you can access the collection of all images that share a certain tag by changing directory to the folder with the tags name created by this script. This allows you e.g. to run JAlbum a photo album software that needs to find the pictures to be put into a web album in the filesystem because JAlbum cannot access digikams virtual folders directly. The second way of calling this script is the so called archive-mode by setting option -A archiveDir. Archive mode is thought for people who want to archive tagged photos independently of digikams root directories and the photos therein. This way you can put your photos and their tag structure in eg. a tar archive and send it to a friend, who can look at the photos via their tag structure. In this mode the script creates the directory given as parameter to -A and in this directory two more subdirectories. One named Photos and a second named Tags. The Photos directory contains hard links to your original photos, and the Tags directory con- tains a subdirectory for each Tag used by any of your photos. Inside this subdirectory there are links (either symbolic or hard links) to the files in the Photos directory. This way the archive directory needs nearly no additional space on your harddisk and you have an ar- chive that allows you or a friend to easily look at the photos tag structure. Another benefit from using this script is that you have kind of a backup of your tag settings for all of your photos. The backup is simply the directory structure containing links to the original images that wear the tags. This could become important if for whatever reason the digikam.db file gets corrupted or even lost. COMMAND-LINE OPTIONS -l taglinkdir Parameter taglinkdir denotes a directory in which the tag structure of all your photos stored in rootdir will be exported to by creating subdirectories for each tag and placing symbolic links in these subdirectories that point to the original photo wearing the tags. If calling the script with option -l taglinkDir you also have to specify options -r rootdir as well as -d database. -A archivedirectory archivedirectory denotes a directory into which the script will export the photos and their tag structure. -A has to be used together with option -r rootdir if using digikam version < 0.10, as well as -d database else the script will terminate. Inside the archive directory the script will create a Photos and a Tags directory. It will put hard links in the Photos directory that point to your original photos. By using hard links you are independent of changes in your digikam root directory but on the other hand you are limited to one filesystem. So the directory given by -r rootdir and the directory specified for -A archivedir have to be one the same filesystem. If using digikam in version >= 0.10 you cannot specify -r rootdir . Instead the root paths of all albums are taken from digikams database directly. However still the requirement holds, that archivedir has to be on the same filesystem like all the root directories containing photos you defined in digikam. If one of digikams root directories is on another filesystem this one will not be processed, since hardlinking the photos inside the Photos directory would not work in this case! The Tags subdirectory will contain links to the files in the Photos directory. This way you have one archive directory that is completely self contained. You can tar it, send it to a friend or just put it somewhere for archivel or backup purposes. Usually only those photos will be archived that have a digikam tag set on them. By using option -C however you can perform a complete ar- chive. See -C for more infos. -d database database is the complete path including the filename to digikams photo database which usually can be found in digikams root direc- tory. The files name is usually digikam.db . -r rootdir rootdir denotes the digikam base directory containing all your photos if you are using digikam in a version before 0.10. If you are using digikam 0.10 or newer you must not use this option. If you do it anyway you will see a hint that the option given will be ignored because digikams version 0.10 database contains the root directories of all albums defined in digikam. -C When the script is called with option -A archivedir only those photos will be archived (by placing links) in the Photos subdirectory of archivedir that have at least one digikam tag set. By setting option -C all photos will be archived to archivedir no matter if they have a tag set or not. Note: This only changes the contents of the Photos subdirectory not of the Tags subdirectory in the archivedir directory. -a By default the script will try to create relative symbolic links from the directory taglinkdir set by option -l to the photo files under rootdir given by option -r. Using this option will result in absolute symbolic links beeing created instead of relative ones. -H By default the script will create soft (symbolic) links from the Tag-Tree to the photos. By setting option -H the script will use hard links instead. Please note that hard links can only be created inside one filesystem. So your photos and the Tag tree have to be one the same filesystem. If not you will see a warning about this problem and the script will not run. -f In digikam photos can have hierachical tags (tags that have subtags). In this case digitaglinktree would by default add a directory for the tag and a subdirectory for each of the subtags of this tag. By setting -f a subtag is treated like a regular tag just as its parent tag so digitaglinktree will create all subdirectories for tags and subtags at the same level independent of the tag - subtag hierarchy. -v Prints the scripts version number and exits. CONFIGURATION
By default this script will run with all photo databases created by digikam version 0.10 as well as older version like 0.9 and 0.8. If you still have digikam version 0.7 then you have to reconfigure the script itself. You have to reconfigure the script by setting the path to the sqlite binary that is used by the script to query the digikam database digikam.db. Since very old digikam version use sqlite in version 2, but later digikam versions need sqlite version 3 you have to take care to install the correct version of sqlite for the installed digikam version and to set the path to the correct sqlite executable in the scripts head: Choose $SQLITE="/usr/bin/sqlite3"; for digikam version 0.8x and 0.9x and 0.10x $SQLITE="/usr/bin/sqlite"; for digikam version 0.7x. EXAMPLE
A call to digitaglinktree is shown below: digiTagLinktree -l /home/user/tags -d /home/user/photos/digikam.db In this example we assume that you are running digikam version 0.10 or higher so that no option -r was used to specify the photo root dir. Instead this information will automatically be fetched from digikams database. In case you want to run the script on a digikam database that was created by digikam version 0.9 or earlier you have to use -r to specify the root directory where you keep all your photos that are managed by digikam: digiTagLinktree -r /home/user/photos -l /home/user/tags -d /home/user/photos/digikam.db In this example digikams photo root denoted by -r is /home/user/photos. Option -l /home/user/tags tells the script that all the subdirectories and symbolic links will be placed in the directory /home/user/tags. The folder was chosen so that the tags-directory is not under digikams photo root. You may put the tags folder inside digikams photoroot but this is not the preferred method. Because the link directory contains only links this tag structure does hardly need any additional space on your harddisk. AUTHORS
digitaglinktree was written by Rainer Krienke <krienke at uni-koblenz.de> 16 Aug 2006 digitaglinktree(1)
All times are GMT -4. The time now is 12:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy