Cannot correctly connect multi-stage C command pipe (among others) (FYI: a lot of code)


 
Thread Tools Search this Thread
Homework and Emergencies Homework & Coursework Questions Cannot correctly connect multi-stage C command pipe (among others) (FYI: a lot of code)
# 1  
Old 03-31-2013
Cannot correctly connect multi-stage C command pipe (among others) (FYI: a lot of code)

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!

1. The problem statement, all variables and given/known data:

We are supposed to write a C program that parses a command line, separates it into each command (further separates each command into its argument vectors), and then creates a multi-stage execution pipeline.

Example: "ls -l | grep ^d | wc -l"

The program must also print the results to a LOGFILE of each command created, the PIDs of each command, and the exit status of all commands. Finally, the program needs to wait for each program to execute, kill each command if Cntl-C is hit while the pipeline is executing and start over, or simply end the program if Cntl-C is hit beforehand.

2. Relevant commands, code, scripts, algorithms:

This is the code we have to work with. I apologize for the super-long massive block of code, and any problems with formatting or displaying this properly here, but this is what we must work with, and believe me, no one is more frustrated trying to comprehend this than me, since I am not very good at C programming:

file: piper.c

Code:
/***********************************************************************************************

 CSci 4061 Spring 2013
 Assignment# 3: Piper program for executing pipe commands 

 Student name: [hidden for confidentiality], [hidden for confidentiality]
 Student ID:   [hidden for confidentiality], [hidden for confidentiality]
 X500 id: [hidden for confidentiality], [hidden for confidentiality]

 Operating system on which you tested your code: Linux
 CSELABS machine: [hidden for confidentiality]

 GROUP INSTRUCTION:  Please make only ONLY one submission when working in a group.

***********************************************************************************************/

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <string.h>
#include <ctype.h>
#include <unistd.h>
#include <errno.h>
#include <signal.h>

#define DEBUG

#define MAX_INPUT_LINE_LENGTH 2048 // Maximum length of the input pipeline command
                                   // such as "ls -l | sort -d +4 | cat "
#define MAX_CMDS_NUM 8           // maximum number of commands in a pipe list
                                   // In the above pipeline we have 3 commands
#define MAX_CMD_LENGTH 256         // A command has no more than 255 characters


volatile int interrupt = 0;//used for handling Cntl-C
FILE *logfp;

int run=1,run2=1;
int prev[2],curr[2];
int num_cmds = 0;
char *cmds[MAX_CMDS_NUM+1];
char *cmds2[MAX_CMDS_NUM+1][256];
int cmd_pids[MAX_CMDS_NUM];
int cmd_status[MAX_CMDS_NUM];
int stat;
int printCount=1,success=0;

/*******************************************************************************/
/*   The function parse_command_line will take a string such as
     ls -l | sort -d +4 | cat | wc
     given in the char array commandLine, and it will separate out each pipe
     command and store pointer to the command strings in array "cmds"
     For example:
     cmds[0] will point to string "ls -l"
     cmds[1] will point to string "sort -d +4"
     cmds[2] will point to string "cat"
     cmds[3] will point to string "wc"

     This function will write to the LOGFILE above information.
*/
/*******************************************************************************/

int parse_command_line (char commandLine[MAX_INPUT_LINE_LENGTH], char* cmds[MAX_CMDS_NUM])
{
        char delims[] = "|";
	char *result = NULL;
	result = strtok( commandLine, delims );
	int writErr,i=0;
        while( (result != NULL) && i<8 )
	{
	    cmds[i]=result;
	    i++;
	    num_cmds++;
	    result = strtok( NULL, delims );
	}
	cmds[i]=NULL;
   	if (num_cmds>MAX_CMDS_NUM)//is the number of commands greater than 8? if so, too many to work with!!!
   	{
		printf("Error! Too many commands passed in!\n");
		exit(666);
	}
   
	return num_cmds;
}

/*******************************************************************************/
/*  parse_command takes command such as  
    sort -d +4
    It parses a string such as above and puts command program name "sort" in
    argument array "cmd" and puts pointers to ll argument string to argvector
    It will return  argvector as follows
    command will be "sort"
    argvector[0] will be "sort"
    argvector[1] will be "-d"
    argvector[2] will be "+4"
*/
/*******************************************************************************/

void parse_command(char input[MAX_CMD_LENGTH], char command[MAX_CMD_LENGTH], char *argvector[MAX_CMD_LENGTH])
{
	char delims[] = " ";
	char *result = NULL;
	result = strtok( input, delims );
	int i=0;
	int argcount=0;
        command=result;
	argvector[i]=result;
	i++;
	while( ((result = strtok( NULL, delims )) != NULL) && i<8 )
	{
	    argvector[i]=result;
	    i++;  
	}
	argvector[i]=NULL;
}


/*******************************************************************************/
/*  The function print_info will print to the LOGFILE information about all    */
/*  processes  currently executing in the pipeline                             */
/*  This printing should be enabled/disabled with a DEBUG flag                 */
/*******************************************************************************/

void print_info(char* cmds[MAX_CMDS_NUM],
		int cmd_pids[MAX_CMDS_NUM],
		int cmd_stat[MAX_CMDS_NUM],
		int num_cmds)
{
	int writErr,i;
	
	if((logfp=fopen("LOGFILE","w"))==NULL)
	{
		perror("Error opening LOGFILE:");
		exit(666);
	}
	#ifdef DEBUG	
		if(printCount==1)
		{	
			for(i=0;i<num_cmds;i++)
			{
				if((writErr=(fprintf(logfp,"Command %d info: %s\n",i,cmds[i])))<0)//can we write command summary to LOGFILE?
		   		{
					perror("Error writing to LOGFILE:");
		 			exit(666);
		    		}
			}
			if((writErr=(fprintf(logfp,"Number of commands: %d\n",num_cmds)))<0)//can we write number of commands to LOGFILE?
			{
				perror("Error writing to LOGFILE:");
				exit(666);
			}
			if((writErr=(fprintf(logfp,"\n")))<0)//can we write number of commands to LOGFILE?
			{
				perror("Error writing to LOGFILE:");
				exit(666);
			}
   
	 		if((writErr=fprintf(logfp,"PID          COMMAND\n"))<0)
			{
				perror("Error writing to LOGFILE:");
				exit(666);
			}
			for(i=0;i<num_cmds;i++)
			{
				if((writErr=(fprintf(logfp,"%d     %s\n",cmd_pids[i],cmds[i])))<0)
				{
					perror("Error writing to LOGFILE:");
					exit(666);			
				}
			}
		}
		else
		{	
			for(i=0;i<num_cmds;i++)
			{
				if((writErr=(fprintf(logfp,"Command %d info: %s\n",i,cmds[i])))<0)//can we write command summary to LOGFILE?
		   		{
					perror("Error writing to LOGFILE:");
		 			exit(666);
		    		}
			}
			if((writErr=(fprintf(logfp,"Number of commands: %d\n",num_cmds)))<0)//can we write number of commands to LOGFILE?
			{
				perror("Error writing to LOGFILE:");
				exit(666);
			}
			if((writErr=(fprintf(logfp,"\n")))<0)//can we write number of commands to LOGFILE?
			{
				perror("Error writing to LOGFILE:");
				exit(666);
			}
   			if((writErr=fprintf(logfp,"PID     COMMAND\n"))<0)
			{
				perror("Error writing to LOGFILE:");
				exit(666);
			}
			for(i=0;i<num_cmds;i++)
			{
				if((writErr=(fprintf(logfp,"%d		%s\n",cmd_pids[i],cmds[i])))<0)
				{
					perror("Error writing to LOGFILE:");
					exit(666);			
				}
			}
			if((writErr=(fprintf(logfp,"\n")))<0)//can we write number of commands to LOGFILE?
			{
				perror("Error writing to LOGFILE:");
				exit(666);
			}
	 		if((writErr=fprintf(logfp,"PID     COMMAND			EXIT STATUS\n"))<0)
			{
				perror("Error writing to LOGFILE:");
				exit(666);
			}
			for(i=0;i<num_cmds;i++)
			{
				if((writErr=(fprintf(logfp,"%d     %s				%d\n",cmd_pids[i],cmds[i],cmd_stat[i])))<0)
				{
					perror("Error writing to LOGFILE:");
					exit(666);			
				}
			}
		}	
	#endif
	fclose(logfp);	
}

/*******************************************************************************/
/*     The create_command_process  function will create a child process        */
/*     for the i'th command                                                    */
/*     The list of all pipe commands in the array "cmds"                       */
/*     the argument cmd_pids contains PID of all preceding command             */
/*     processes in the pipleine.  This function will add at the               */
/*     i'th index the PID of the new child process.                            */
/*******************************************************************************/

void create_command_process (char currComm[MAX_CMD_LENGTH],   // Command line to be processed
                     int cmd_pids[MAX_CMDS_NUM],          // PIDs of preceding pipeline processes
                                                          // Insert PID of new command processs
		             int i)                               // commmand line number being processed
				
{
	pid_t childpid;
	char command[MAX_CMD_LENGTH];
	char *argvector[MAX_CMD_LENGTH];
	char *test=NULL;
	char testDel[] = " ";
	char *temp[9];
	char cmdsTest[256];//temp array for storing the command name by itself
	strcpy(cmdsTest,currComm);
	test = strtok(cmdsTest,testDel);
	int j,k,writErr;
	parse_command(currComm,command,argvector);
	/*PROBLEM EXISTS SOMEWHERE AFTER THIS LINE IN THIS COMMAND*/
	if(childpid=fork())//if more than one process, and this is the parent process....
	{
		if((cmd_pids[i]=(int)childpid)==-1)//did it fork correctly?
		{
			perror("Error forking!");
			exit(666);
		}
		close(prev[0]);//close previous filedescriptor in
		close(prev[1]);//close previous filedescriptor out

	}
	else
	{
		if(prev[1]!=-1)//if previous pipe has a valid item
		{
			dup2(prev[1],0);
		}
		if(curr[0]!=-1)//if current pipe has a valid item
		{
			dup2(curr[0],0);
		}
		close(prev[0]);
		close(prev[1]);
		close(curr[0]);
		close(curr[1]);
		execvp(test,argvector);//execute the process....
	}	
	prev[0]=curr[0];
	prev[1]=curr[1];
	curr[0]=curr[1]=-1;
}
/********************************************************************************/
/*   The function waitPipelineTermination waits for all of the pipeline         */
/*   processes to terminate.                                                    */
/********************************************************************************/

void waitPipelineTermination ()
{
	wait(&stat);
}

/********************************************************************************/
/*  This is the signal handler function. It should be called on CNTRL-C signal  */
/*  if any pipeline of processes currently exists.  It will kill all processes  */
/*  in the pipeline, and the piper program will go back to the beginning of the */
/*  control loop, asking for the next pipe command input.                       */
/********************************************************************************/

void killPipeline( int signum )
{
   printf("CNTL-C detected!!!\n");
   interrupt = 1;	
}

/********************************************************************************/

int main(int ac, char *av[])
{

  int i, pipcount,writErr;

  //check usage
  if (ac > 1)
  {
    printf("\nIncorrect use of parameters\n");
    printf("USAGE: %s \n", av[0]);
    exit(1);
  }

  /* Set up signal handler for CNTRL-C to kill only the pipeline processes  */

  if((logfp = fopen("LOGFILE", "w")) == NULL)//
  {
	perror("Error opening logfile:");
	exit(666);
  }
  while (run==1)
  {
     signal(SIGINT, SIG_DFL );
     pipcount = 0;

     /*  Get input command file name form the user */
     char pipeCommand[MAX_INPUT_LINE_LENGTH];

     fflush(stdout);
     printf("Give a list of pipe commands: ");
     gets(pipeCommand); 
     char* terminator = "quit";
     printf("You entered : list of pipe commands  %s\n", pipeCommand);
     if ( strcmp(pipeCommand, terminator) == 0  )
     {
        fflush(logfp);
        fclose(logfp);
        printf("Goodbye!\n");
        exit(0);
     }  
     num_cmds = parse_command_line( pipeCommand, cmds);
    /*  SET UP SIGNAL HANDLER  TO HANDLE CNTRL-C                         */
     signal(SIGINT, killPipeline); 
     if(interrupt==1)
     {
	for(i=0;i<num_cmds;i++)
	{
		kill(cmd_pids[i],SIGKILL);
	}
	interrupt=0;
     }	    
    /*  num_cmds indicates the number of command lines in the input file */

    /* The following code will create a pipeline of processes, one for   */
    /* each command in the given pipe                                    */
    /* For example: for command "ls -l | grep ^d | wc -l "  it will      */
    /* create 3 processes; one to execute "ls -l", second for "grep ^d"  */
    /* and the third for executing "wc -l"                               */
    prev[0]=-1;
    prev[1]=-1;
    curr[0]=-1;
    curr[1]=-1;
    		 
    for(i=0;i<num_cmds;i++)
    {
         /*  CREATE A NEW PROCESS EXECUTE THE i'TH COMMAND    */
         /*  YOU WILL NEED TO CREATE A PIPE, AND CONNECT THIS NEW  */
         /*  PROCESS'S stdin AND stdout  TO APPROPRIATE PIPES    */
	 char testy[256];//used to invoke command creation for each command
  	 pipcount = num_cmds-1;
	 strcpy(testy,cmds[i]);
         if(pipcount>0)
	 {
		 if(stat=pipe(curr))//did a pipe get created?
		 {
			perror("Error creating pipe:");
			exit(666);
		 }
	 }
	 create_command_process(testy, cmd_pids, i);//make the command
	 cmd_status[i]=stat;
	 pipcount--;
    }
    print_info(cmds, cmd_pids, cmd_status, num_cmds);//print pipeline info
    waitPipelineTermination();
    printCount++;
    print_info(cmds, cmd_pids, cmd_status, num_cmds);//print pipeline info (again)
    success+=1;
    if(success==1)//are we done?	
    	run=0;
  }
  fclose(logfp);
  return 0;
} //end main

/*************************************************/

3. The attempts at a solution (include all code and scripts):

I ran the command "ls | grep ^d | wc -l" in the shell, and then ran it again in my program and these are the results from both (first shell, then program):

Code:
user@ubuntu:~/Dropbox/CSCI4061/HW3/Examples$ ls | grep ^d | wc -l
0
user@ubuntu:~/Dropbox/CSCI4061/HW3/Examples$ ./piper04
Give a list of pipe commands: ls | grep ^d | wc -l
You entered : list of pipe commands  ls | grep ^d | wc -l
0
LOGFILE      child08.c	       parent.c.txt  pipeline	      process-fan.c
child.c      executeCommand.c  parent02.c    pipeline.c       process-tree.c
child.c.txt  fan	       parent04.c    pipeline.c~
child02.c    input.txt	       parent05.c    piper04
child04.c    now	       parent06.c    printpid.c
child06.c    parent.c	       parent08.c    process-chain.c

4. Complete Name of School (University), City (State), Country, Name of Professor, and Course Number (Link to Course):

University of Minnesota, Twin Cities, Minneapolis, MN, USA, Professor: Anand Tripathi, Computer Science (CSCI) 4061: Intro to Operating Systems, http://www-users.cselabs.umn.edu/cla...2013/csci4061/

(when I posted this thread, I could not access the website, but that's definitely it)

Note: Without school/professor/course information, you will be banned if you post here! You must complete the entire template (not just parts of it).

I have heard an expert C programmer call this "one pig of an assignment." Because I have not been able to get the piping working correctly, I have not had time to see if my Cntl-C, wait() calls, or logfile printing works correctly, and this is due tonight at midnight. Hopefully someone is on here given how today is Easter (when most of us should be with our families anyway), but a lot of this code was given to us from the TAs, and even then it didn't work (I had to edit some places here just to get it running).

SmilieSmilie

I have been working on this for three weeks and cannot figure out what is happening, so I am very desperate here. I have also had problems on other boards because users who are a lot more knowledgeable with C programming than I am are a little too snide and harsh with me, someone who is already as nervous as a newbie usually is, and also has deep emotional problems that make it hard to interact with the public because of being on the receiving end of overzealous or even downright arrogant remarks that easily can get taken as attacks (even if the person who said it wasn't intending it). I am not looking for trouble, but please... go easy on me, okay?

Any help here would be greatly appreciated! Thank you and Happy Easter! (if you celebrate it, that is, otherwise.. Happy Chocolate Day, I guess??)

P.S. If it would help or if my post is incomprehensible, I can provide the assignment description upon request.

---------- Post updated at 10:44 AM ---------- Previous update was at 10:39 AM ----------

Sometimes when I run my own code, running something "ls | grep ^d | wc -l" would print the results of "ls" before printing the rest (what prints out is not uniform in its order), but if I do something like:

"ls | wc"

The shell prompt will print this:

Code:
user@ubuntu:~/Dropbox/CSCI4061/HW3/Examples$ ls | wc
     26      26     276

But my program does this:

Code:
user@ubuntu:~/Dropbox/CSCI4061/HW3/Examples$ ./piper04
Give a list of pipe commands: ls | wc
You entered : list of pipe commands  ls | wc
      0       0       0
user@ubuntu:~/Dropbox/CSCI4061/HW3/Examples$ LOGFILE      child08.c	       parent.c.txt  pipeline	      process-fan.c
child.c      executeCommand.c  parent02.c    pipeline.c       process-tree.c
child.c.txt  fan	       parent04.c    pipeline.c~
child02.c    input.txt	       parent05.c    piper04
child04.c    now	       parent06.c    printpid.c
child06.c    parent.c	       parent08.c    process-chain.c

# 2  
Old 03-31-2013
That you can see their output means they're not being redirected properly.

You must dup2 the read-end of the previous pipe over the stdin of the process, and dup2 the write-end of the next pipe over the stdout of the process, for it to be part of the pipe chain. Otherwise its output will go direct to the terminal like everything else.

You must also close all original pipes in the child, leaving only the duplicates, or they may freeze when they're done instead of quitting.

Try simplifying your example. Right now it's very large and complex. Try something that just uses arrays for input instead of a full parser, then add it back on once you get it to work.
# 3  
Old 03-31-2013
Also, exit(666) makes no sense. It only takes numbers 1-127.
# 4  
Old 03-31-2013
Re:

Yeah, I know, sorry the code is so huge, I guess I had to post the whole code so the mods don't come down on me for not filling out the whole thread template, but anyway... I wish I had time to simplify the code, now I'm just trying to get this thing going. I changed the code to this (just including the problem area in the "create_command_process" function):

Code:
if(childpid=fork())//if more than one process, and this is the parent process....
 {
  if((cmd_pids[i]=(int)childpid)==-1)//did it fork correctly?
  {
   perror("Error forking!");
   exit(666);
  }
  close(prev[0]);//close previous filedescriptor in
  close(prev[1]);//close previous filedescriptor out
  close(curr[0]);
  close(curr[1]);
 }
 else
 {
  if(prev[1]!=-1)//if previous pipe has a valid item
  {
      dup2(prev[0],0);
      dup2(prev[1],1);
      close(prev[0]);
      close(prev[1]);
  }
  if(curr[0]!=-1)//if current pipe has a valid item
  {
      dup2(curr[0],0);
      dup2(curr[1],1);
      close(curr[0]);
      close(curr[1]);
  }
    execvp(test,argvector);//execute the process....
 }

When I run it with "ls | wc", absolute NO output to the shell console comes back (even though it's writing stuff to my logfile). I will work on this some more, but I don't know if this means it's working now? Aren't I like supposed to get a set of numbers back for "ls | wc"? (running it in shell spits out "16 16 150" in my current folder). Then again, the assignment description says this, and I quote:

" 
The standard output stream is used and used only for printing command execution results.
"

So I take this to mean I shouldn't have to worry enough about it printing anything straight to the console?


---------- Post updated at 01:12 PM ---------- Previous update was at 01:11 PM ----------

Quote:
Originally Posted by Corona688
Also, exit(666) makes no sense. It only takes numbers 1-127.
Oh, really? I didn't know that. I just heard that any non-zero number fed into an exit() call indicated failure so I use 666 to be a wise guy for obvious reasons, bahahaha. I'll change it, though, to be something in that valid range. Thanks.
# 5  
Old 03-31-2013
Code:
      dup2(prev[0],0);
      dup2(prev[1],1);

You are attaching the reading end of the pipe to standard input. This makes sense.

You are attaching the writing end of the pipe to standard output. This makes sense.

However, you are doing both to the same pipe. This does not make sense. To picture the loop you've made:

Code:
 /-->grep---\
 \----------/

Where what you actually want is:

Code:
command1 --> command2 --> command3

command1 writes into the write end of pipe A. command2 reads from pipe A, writes to pipe B. command3 reads from pipe B, writes to the terminal.

Last edited by Corona688; 03-31-2013 at 03:26 PM..
# 6  
Old 04-01-2013
Re:

Since I'm having so much trouble comprehending how piping works in C, and since the professor said we were free to make our own version of the code from scratch, I decided to abandon the pipe/fork/dup2 approach and instead just use popen, after finding an example of that on the 'Net, and that did the trick. But thanks for the help. Hopefully come test time, I will better understand how dup2 and all that works.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Egrep -v command not filtering correctly

Hello guys, I have an issue when trying to do an egrep -v on a file, let me show you. I want to filter the last column as to where it filters out the columns with asterisks and zeros ( * and 0 ) it is working properly up to a certain point where I have a value of '10000' which is also getting... (3 Replies)
Discussion started by: evergreen
3 Replies

2. UNIX for Beginners Questions & Answers

How to compile a Datastage Job using Execute Command Stage or Routines in Datastage 11?

I am trying to compile the datastage jobs using the Execute Command stage in datastage 11 or any Routines if possible. My datastage is on Unix machine. So, How can I Compile a datastage job in UNIX from command line or any Routines. Please help me in doing so. Thank you. (1 Reply)
Discussion started by: elena jessi
1 Replies

3. Solaris

Can I run repair on lot of blocks with single command ?

Hi, I have Solaris-10 OS on T5220. Both local disks were mirrored under SVM. Somehow when one disk gone bad (c0t1d0), other disk (c0t0d0) also got lot of bad block. We have readable data only on c0t0d0, but as soon as server comes after, it hangs when I run few commands because of read errors,... (1 Reply)
Discussion started by: solaris_1977
1 Replies

4. UNIX for Dummies Questions & Answers

[Solved] Why code run not correctly

Hi there can anyone help me here is my code echo "Type in a positive number" read X I=2 while do if then echo "It is not prime" break else if then echo "It is prime" break else I=$(( $I + 1)) fi fi (4 Replies)
Discussion started by: FUTURE_EINSTEIN
4 Replies

5. UNIX for Advanced & Expert Users

command taking lot of time to execute

Hi, I am running the following command, and it tries to delete some dn from ldap, however, it takes lot of time before it finally request LDAP server to delete it. I am trying to find why it is taking lot of time. Could you anyone help me in this regard. I have copies the pstack output, and... (3 Replies)
Discussion started by: john_prince
3 Replies

6. UNIX for Dummies Questions & Answers

Date command to obtain the last month is not working correctly..

Hello, I could not find the exactly same post here.. so I will explain what I did to get the last month using date command. I used date +%Y-%m -d "-1 months" to get the last month. However, the returned value of above command on 2009/10/31 was 2009 10 and not 2009 09.. and the... (9 Replies)
Discussion started by: tigersk
9 Replies

7. Shell Programming and Scripting

find command takes a lot of time ( can I skip directories)

I have a file called "library" with the following content libnxrdbmgr.a libnxrdbmgr.so libnxtk.a libnxtk.so libora0d_nsc_osi.so I am trying to locate if these libraries are on my machine or not. find command runs for about few seconds and hangs after this. Can someone please help me and... (3 Replies)
Discussion started by: knijjar
3 Replies

8. Shell Programming and Scripting

How to get exit code in a pipe-lined command?

I have a question about how to get the exit code of the first command when it appears in a pipe-lined command. For example, I have the following script: grep abc dddd | tee -a log if ] then echo "ERROR!" fi In the above script, ] is supposed to test the exit code of "grep abc... (3 Replies)
Discussion started by: pankai
3 Replies

9. Solaris

Solaris has a lot of bugs, fstream wont write to file correctly!

I've got a c++ program that works fine on Linux, compiles on Solaris fine with g++, but will not write to a fstream correctly in a class object. And I've run into numerous other bugs in the disk management. Jon (4 Replies)
Discussion started by: Joncamp
4 Replies
Login or Register to Ask a Question