Sponsored Content
Top Forums Programming Pipe usage error while visiting directories Post 303032909 by Corona688 on Tuesday 26th of March 2019 02:05:19 PM
Old 03-26-2019
If it's saying bad file descriptor it probably means it. Without seeing your actual code, I can't tell why you're closing a bad file descriptor, you should print the FD's to stderr when you open a pipe, and print them to stderr again when you try and close it to see what's going on. But I have some further comments.

fork() is pointless. Disks don't multithread. Forcing it to read 19 directories at once will make your disk run 19 times slower. You already benefit from the caching and read-ahead built into the OS, too.

Second, there's a system function for what you want to do, ftw() It operates depth-first, so every time you see a new second-level folder, you'll know everything afterwards will be within that folder until it leaves.

Code:
#include <ftw.h>
#include <sys/stat.h>
#include <stdio.h>
#include <string.h>

struct FTW {
        int base;
        int level;
};

struct result_t {
        char name[256];
        long int size;
} result[64];

int last_result=0;

int ftw_callback(const char *fpath, const struct stat *sb,
        int typeflag, struct FTW *ftwbuf);

int main(void) {
        int n;
        const char *ROOT="./";
        strcpy(result[0].name, ROOT);
        ftw(ROOT, ftw_callback, 8);

        for(n=0; n<=last_result; n++)
                printf("%s\t%ld bytes\n", result[n].name, result[n].size);
}

int ftw_callback(const char *fpath, const struct stat *sb,
        int typeflag, struct FTW *ftwbuf) {

        // Found a new second-level folder
        if((typeflag == FTW_D) && (ftwbuf->level == 1)) {
                last_result++;
                strcpy(result[last_result].name, fpath);
        }

        if(typeflag == FTW_F)
        {
                int res=last_result;
                // Special case for level-1 files, those are in ROOT
                if(ftwbuf->level == 1) res=0;
                result[res].size += sb->st_size;
        }

        return(0);
}


Last edited by Corona688; 03-26-2019 at 03:18 PM..
This User Gave Thanks to Corona688 For This Post:
 

10 More Discussions You Might Find Interesting

1. HP-UX

how can I find cpu usage memory usage swap usage and logical volume usage

how can I find cpu usage memory usage swap usage and I want to know CPU usage above X% and contiue Y times and memory usage above X % and contiue Y times my final destination is monitor process logical volume usage above X % and number of Logical voluage above can I not to... (3 Replies)
Discussion started by: alert0919
3 Replies

2. AIX

broken pipe error

Hi, I am working on AIX 5.3 . I have client-server program which is in ProC.while sending packet to server i am getting error as broken pipe and program exiting. please help?/? (1 Reply)
Discussion started by: ajaysahoo
1 Replies

3. UNIX for Dummies Questions & Answers

du - Disk Usage for only files and NOT directories.

Hello, Could any one help me how to find the Disk Usage for all the files in the running directory and the sub directories without the disk usage of the directory. I mean to say, i need only the file names without the size of the directories. See, i used this command du -a .|sort... (3 Replies)
Discussion started by: RRVARMA
3 Replies

4. AIX

How to monitor the IBM AIX server for I/O usage,memory usage,CPU usage,network..?

How to monitor the IBM AIX server for I/O usage, memory usage, CPU usage, network usage, storage usage? (3 Replies)
Discussion started by: laknar
3 Replies

5. Programming

Broken Pipe error

All, I am using the below code The C code : if ((fp2=fopen(szout_fname,"r"))==NULL) { sprintf(stream_ptr1,"cat %s | sort -t, -rn -k 11,11 | awk -F\",\" '{ \ if ( \$3 ==\"%s\" ) {print... (0 Replies)
Discussion started by: arunkumar_mca
0 Replies

6. Programming

Pipe error

hi guys, o have a big error in this program but i cant solve someone ?! #include <stdlib.h> #include <stdio.h> #include <unistd.h> #include <string.h> int main(int argc, char *argv){ int cont = 2, posicao; char geraArquivo= "|cat>>", espaco=" "; char nomeArquivo, comando,... (11 Replies)
Discussion started by: beandj
11 Replies

7. Shell Programming and Scripting

Broken Pipe error

Hello while doing sftp over server "A" , i am getting a broken pipe error i.e cat: write error: Broken pipe what does that mean? please let me know if you want any other info on this.. (3 Replies)
Discussion started by: urfrnddpk
3 Replies

8. Shell Programming and Scripting

listing file date and time without usage of pipe

Hi , I am using below code to list the 6th,7th and 8th field of the file ls -lrt test | awk '{print $6,$7,$8}' output: Nov 21 19:34 Now the problem here is that I want to do it without the usage of pipes as its now allowed in my production environment Please let me know... (6 Replies)
Discussion started by: harish612
6 Replies

9. UNIX for Dummies Questions & Answers

broken pipe error

I'm new to scripting, and this forum has been invaluable in helping me out. I'm hoping I can get some personal help now though. I have a korn script that takes a list of servers and either telnets or sshs into it (only some are set up for ssh). What I'm doing now is trying to telnet first, and... (10 Replies)
Discussion started by: aimeet
10 Replies

10. Shell Programming and Scripting

Capture error before pipe

Hi, I have a script that runs a tar command to standard out then pipes to a gzip: tar cfE - * | gzip -c > OUT.gz At the moment, even if the tar fails (e.g. because of lack of disk space), the gzip still runs successfully. Is there a way to make the whole line exit with a non-zero error... (6 Replies)
Discussion started by: Catullus
6 Replies
FTW(3)							     Linux Programmer's Manual							    FTW(3)

NAME
ftw, nftw - file tree walk SYNOPSIS
#include <ftw.h> int ftw(const char *dirpath, int (*fn) (const char *fpath, const struct stat *sb, int typeflag), int nopenfd); #define _XOPEN_SOURCE 500 /* See feature_test_macros(7) */ #include <ftw.h> int nftw(const char *dirpath, int (*fn) (const char *fpath, const struct stat *sb, int typeflag, struct FTW *ftwbuf), int nopenfd, int flags); DESCRIPTION
ftw() walks through the directory tree that is located under the directory dirpath, and calls fn() once for each entry in the tree. By default, directories are handled before the files and subdirectories they contain (preorder traversal). To avoid using up all of the calling process's file descriptors, nopenfd specifies the maximum number of directories that ftw() will hold open simultaneously. When the search depth exceeds this, ftw() will become slower because directories have to be closed and reopened. ftw() uses at most one file descriptor for each level in the directory tree. For each entry found in the tree, ftw() calls fn() with three arguments: fpath, sb, and typeflag. fpath is the pathname of the entry, and is expressed either as a pathname relative to the calling process's current working directory at the time of the call to ftw(), if dirpath was expressed as a relative pathname, or as an absolute pathname, if dirpath was expressed as an absolute pathname. sb is a pointer to the stat structure returned by a call to stat(2) for fpath. typeflag is an integer that has one of the following values: FTW_F fpath is a regular file. FTW_D fpath is a directory. FTW_DNR fpath is a directory which can't be read. FTW_NS The stat(2) call failed on fpath, which is not a symbolic link. If fpath is a symbolic link and stat(2) failed, POSIX.1-2001 states that it is undefined whether FTW_NS or FTW_SL (see below) is passed in typeflag. To stop the tree walk, fn() returns a nonzero value; this value will become the return value of ftw(). As long as fn() returns 0, ftw() will continue either until it has traversed the entire tree, in which case it will return zero, or until it encounters an error (such as a malloc(3) failure), in which case it will return -1. Because ftw() uses dynamic data structures, the only safe way to exit out of a tree walk is to return a nonzero value from fn(). To allow a signal to terminate the walk without causing a memory leak, have the handler set a global flag that is checked by fn(). Don't use longjmp(3) unless the program is going to terminate. nftw() The function nftw() is the same as ftw(), except that it has one additional argument, flags, and calls fn() with one more argument, ftwbuf. This flags argument is formed by ORing zero or more of the following flags: FTW_ACTIONRETVAL (since glibc 2.3.3) If this glibc-specific flag is set, then nftw() handles the return value from fn() differently. fn() should return one of the fol- lowing values: FTW_CONTINUE Instructs nftw() to continue normally. FTW_SKIP_SIBLINGS If fn() returns this value, then siblings of the current entry will be skipped, and processing continues in the parent. FTW_SKIP_SUBTREE If fn() is called with an entry that is a directory (typeflag is FTW_D), this return value will prevent objects within that directory from being passed as arguments to fn(). nftw() continues processing with the next sibling of the directory. FTW_STOP Causes nftw() to return immediately with the return value FTW_STOP. Other return values could be associated with new actions in the future; fn() should not return values other than those listed above. The feature test macro _GNU_SOURCE must be defined (before including any header files) in order to obtain the definition of FTW_ACTIONRETVAL from <ftw.h>. FTW_CHDIR If set, do a chdir(2) to each directory before handling its contents. This is useful if the program needs to perform some action in the directory in which fpath resides. FTW_DEPTH If set, do a post-order traversal, that is, call fn() for the directory itself after handling the contents of the directory and its subdirectories. (By default, each directory is handled before its contents.) FTW_MOUNT If set, stay within the same file system (i.e., do not cross mount points). FTW_PHYS If set, do not follow symbolic links. (This is what you want.) If not set, symbolic links are followed, but no file is reported twice. If FTW_PHYS is not set, but FTW_DEPTH is set, then the function fn() is never called for a directory that would be a descendant of itself. For each entry in the directory tree, nftw() calls fn() with four arguments. fpath and sb are as for ftw(). typeflag may receive any of the same values as with ftw(), or any of the following values: FTW_DP fpath is a directory, and FTW_DEPTH was specified in flags. All of the files and subdirectories within fpath have been processed. FTW_SL fpath is a symbolic link, and FTW_PHYS was set in flags. FTW_SLN fpath is a symbolic link pointing to a nonexistent file. (This occurs only if FTW_PHYS is not set.) The fourth argument that nftw() supplies when calling fn() is a structure of type FTW: struct FTW { int base; int level; }; base is the offset of the filename (i.e., basename component) in the pathname given in fpath. level is the depth of fpath in the directory tree, relative to the root of the tree (dirpath, which has depth 0). RETURN VALUE
These functions return 0 on success, and -1 if an error occurs. If fn() returns nonzero, then the tree walk is terminated and the value returned by fn() is returned as the result of ftw() or nftw(). If nftw() is called with the FTW_ACTIONRETVAL flag, then the only nonzero value that should be used by fn() to terminate the tree walk is FTW_STOP, and that value is returned as the result of nftw(). CONFORMING TO
POSIX.1-2001, SVr4, SUSv1. POSIX.1-2008 marks ftw() as obsolete. NOTES
POSIX.1-2001 note that the results are unspecified if fn does not preserve the current working directory. The function nftw() and the use of FTW_SL with ftw() were introduced in SUSv1. On some systems ftw() will never use FTW_SL, on other systems FTW_SL occurs only for symbolic links that do not point to an existing file, and again on other systems ftw() will use FTW_SL for each symbolic link. For predictable control, use nftw(). Under Linux, libc4 and libc5 and glibc 2.0.6 will use FTW_F for all objects (files, symbolic links, FIFOs, etc.) that can be stat'ed but are not a directory. The function nftw() is available since glibc 2.1. FTW_ACTIONRETVAL is glibc-specific. EXAMPLE
The following program traverses the directory tree under the path named in its first command-line argument, or under the current directory if no argument is supplied. It displays various information about each file. The second command-line argument can be used to specify characters that control the value assigned to the flags argument when calling nftw(). #define _XOPEN_SOURCE 500 #include <ftw.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <stdint.h> static int display_info(const char *fpath, const struct stat *sb, int tflag, struct FTW *ftwbuf) { printf("%-3s %2d %7jd %-40s %d %s ", (tflag == FTW_D) ? "d" : (tflag == FTW_DNR) ? "dnr" : (tflag == FTW_DP) ? "dp" : (tflag == FTW_F) ? "f" : (tflag == FTW_NS) ? "ns" : (tflag == FTW_SL) ? "sl" : (tflag == FTW_SLN) ? "sln" : "???", ftwbuf->level, (intmax_t) sb->st_size, fpath, ftwbuf->base, fpath + ftwbuf->base); return 0; /* To tell nftw() to continue */ } int main(int argc, char *argv[]) { int flags = 0; if (argc > 2 && strchr(argv[2], 'd') != NULL) flags |= FTW_DEPTH; if (argc > 2 && strchr(argv[2], 'p') != NULL) flags |= FTW_PHYS; if (nftw((argc < 2) ? "." : argv[1], display_info, 20, flags) == -1) { perror("nftw"); exit(EXIT_FAILURE); } exit(EXIT_SUCCESS); } SEE ALSO
stat(2), fts(3), readdir(3) COLOPHON
This page is part of release 3.53 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. Linux 2010-09-20 FTW(3)
All times are GMT -4. The time now is 02:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy