Sponsored Content
Top Forums UNIX for Advanced & Expert Users I need some help matching my file database to my filesystem. Post 302188545 by charlesm on Wednesday 23rd of April 2008 07:39:26 PM
Old 04-23-2008
I need some help matching my file database to my filesystem.

Hi There I have a big problem, but maybe a simple question.

I'm attempting something that is turning out to be a huge job, but maybe it could be simpler if I knew some more advanced commands or techniques.

My problem is this: I work for a company that has a primary file server with 27T capacity. I have a database keeping track of the files on this server that reports that only ~10T total when I sum the file sizes.

I need to recover the wasted space by deleting files that are not referenced in the database, and then prevent this storage leak from occuring again. I know that my predecessor has "lost" the database before on at least on occasion and restored an out of date backup.

Some details:
The file ids are stored on the file server as the mysql autoincrement id like this:

id -> padded with leading 0's and broken into 2 digit folders, each file is stored in a unique folder.
1194649 -> /ifs/data/00/00/01/19/46/49/filename.ext

So far I have made a list of all directories on the file server with `find -type d` and outputted a list of all file id's. One id per line.

I tried to get unique ids that do not exist in the database and the file server directory tree by doing:

Code:
cat file_ids_from_db file_ids_from_filesystem | sort | uniq -u > list_of_unique_ids

This didn't work, I tested a sample of the ids on the database before trying to delete the files and I found matches.

My question is this: What is the best way to do a left join between these files, only getting ids that are in the filesystem and not the database?

Another question is why am I doing it this way, is there a better solution?

Thank you in advance for any help. Smilie

-Charles
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Compress file into another filesystem/folder

Hi all, how can I compress file into another folder/file system?? What is the syntax like?? Thank you. (1 Reply)
Discussion started by: wkk1020
1 Replies

2. SCO

create filesystem on file

Hello, iam pretty new to SCO, installed it yesterday in vm. Now, i'd like to create a filesystem on a file like you can do on lnx or bsd. But seems not possible for me to do so on SCO OpenServer 6.0.0. Thats what i get: bash-3.00# uname -a SCO_SV scosysv 5 6.0.0 i386 bash-3.00# mkfs... (0 Replies)
Discussion started by: user5
0 Replies

3. HP-UX

Filesystem on file

How do i create a filesystem on a file ? In linux ( and solaris i think ) i would use losetup to attatch the file to a device, and then run mkfs as usual. But as far as i can understand theres no such functionality in HP-UX. Any pointers at all would be great. (1 Reply)
Discussion started by: jahlinder
1 Replies

4. Solaris

Filesystem - error when extend the filesystem

Hi all, currently , my root filesystem already reach 90 ++% I already add more cylinder in the root partition as below Part Tag Flag Cylinders Size Blocks 0 root wm 67 - 5086 38.46GB (5020/0/0) 80646300 1 swap wu 1 - ... (11 Replies)
Discussion started by: SmartAntz
11 Replies

5. Shell Programming and Scripting

Calculate total space, total used space and total free space in filesystem names matching keyword

Good afternoon! Im new at scripting and Im trying to write a script to calculate total space, total used space and total free space in filesystem names matching a keyword (in this one we will use keyword virginia). Please dont be mean or harsh, like I said Im new and trying my best. Scripting... (4 Replies)
Discussion started by: bigben1220
4 Replies

6. AIX

Extending database filesystem

Ok, I'm a newbie to AIX so I just want to make sure my thinking is correct with regards to this issue. Any insight you guys could give would be appreciated. A main question I have is should I shutdown the database to extend this LV? The filesystem the dba wants to extend is /dev/fslv83 ... (3 Replies)
Discussion started by: Steelysteel
3 Replies

7. UNIX for Dummies Questions & Answers

hwo to find shared filesystem and local filesystem in AIX

Hi, I wanted to find out that in my database server which filesystems are shared storage and which filesystems are local. Like when I use df -k, it shows "filesystem" and "mounted on" but I want to know which one is shared and which one is local. Please tell me the commands which I can run... (2 Replies)
Discussion started by: kamranjalal
2 Replies

8. Solaris

Solaris Filesystem vs. Windows FileSystem

Hi guys! Could you tell me what's the difference of filesystem of Solaris to filesystem of Windows? I need to compare both. I have read some over the net but it's so much technical. Could you explain it in a more simpler term? I am new to Solaris. Hope you help me guys. Thanks! (4 Replies)
Discussion started by: arah
4 Replies

9. AIX

Mount Filesystem in AIX Unable to read /etc/filesystem

Dear all, We are facing prolem when we are going to mount AIX filesystem, the system returned the following error 0506-307The AFopen call failed : A file or directory in the path name does not exist. But when we ls filesystems in the /etc/ directory it show -rw-r--r-- 0 root ... (2 Replies)
Discussion started by: m_raheelahmed
2 Replies
pop3d::dbox(n)						      Tcl POP3 Server Package						    pop3d::dbox(n)

NAME
pop3d::dbox - Simple mailbox database for pop3d SYNOPSIS
package require Tcl 8.2 package require pop3d::dbox ?1.0? ::pop3d::dbox::new ?dbName? dbName option ?arg arg ...? dbName destroy dbName base base dbName add mbox dbName remove mbox dbName move old new dbName list dbName exists mbox dbName locked mbox dbName lock mbox dbName unlock mbox dbName stat mbox dbName size mbox msgId dbName dele mbox msgList storageCmd get mbox msgId DESCRIPTION
The package pop3d::dbox provides simple/basic mailbox management facilities. Each mailbox object manages a single base directory whose sub- directories represent the managed mailboxes. Mails in a mailbox are represented by files in a mailbox directory, where each of these files contains a single mail, both headers and body, in RFC822 conformant format. Any mailbox object following the interface described below can be used in conjunction with the pop3 server core provided by the package pop3d. It is especially possible to directly use the objects created by this package in the storage callback of pop3 servers following the same interface as servers created by the package pop3d. ::pop3d::dbox::new ?dbName? This command creates a new database object with an associated global Tcl command whose name is dbName. The command dbName may be used to invoke various operations on the database. It has the following general form: dbName option ?arg arg ...? Option and the args determine the exact behavior of the command. The following commands are possible for database objects: dbName destroy Destroys the mailbox database and all transient data. The directory associated with the object is not destroyed. dbName base base Defines the base directory containing the mailboxes to manage. If this method is not called none of the following methods will work. dbName add mbox Adds a mailbox of name mbox to the database. The name must be a valid path component. dbName remove mbox Removes the mailbox specified through mbox, and the mails contained therein, from the database. This method will fail if the speci- fied mailbox is locked. dbName move old new Changes the name of the mailbox old to new. dbName list Returns a list containing the names of all mailboxes in the directory associated with the database. dbName exists mbox Returns true if the mailbox with name mbox exists in the database, or false if not. dbName locked mbox Checks if the mailbox specified through mbox is currently locked. dbName lock mbox This method locks the specified mailbox for use by a single connection to the server. This is necessary to prevent havoc if several connections to the same mailbox are open. The complementary method is unlock. The command will return true if the lock could be set successfully or false if not. dbName unlock mbox This is the complementary method to lock, it revokes the lock on the specified mailbox. dbName stat mbox Determines the number of messages in the specified mailbox and returns this number. This method fails if the mailbox mbox is not locked. dbName size mbox msgId Determines the size of the message specified through its id in msgId, in bytes, and returns this number. The msgId has to be in the range "1 ... [dbName stat]" or this call will fail. If stat was not called before this call, size will assume that there are zero messages in the mailbox. dbName dele mbox msgList Deletes the messages whose numeric ids are contained in the msgList from the mailbox specified via mbox. The msgList must not be empty or this call will fail. The numeric ids in msgList have to be in the range "1 ... [dbName stat]" or this call will fail. If stat was not called before this call, dele will assume that there are zero messages in the mailbox. storageCmd get mbox msgId Returns a handle for the specified message. This handle is a mime token following the interface described in the documentation of package mime. The token is read-only. In other words, the caller is allowed to do anything with the token except to modify it. The msgId has to be in the range "1 ... [dbName stat]" or this call will fail. If stat was not called before this call, get will assume that there are zero messages in the mailbox. KEYWORDS
pop3, internet, network, protocol, rfc1939 pop3d 1.0 pop3d::dbox(n)
All times are GMT -4. The time now is 03:05 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy