Writing Your Own Shell

The object of this assignment is to gain experience with some advanced programming techniques like process creation and control, file descriptors, signals and possibly pipes. To do this, you will be writing your own command shell - much like csh, bash or the DOS command prompt.

You are already given a complete parser and the skeleton code of a shell here.The parser code is in parser.c and the shell code needs to be completed in shell.c

The tar file of the parser and shell skeleton code contains a Makefile that will get you started compiling on a UNIX/Linux platform. Download the tar file on your local machine and then execute the command tar -xf shellWithParser.tar.gz to uncompress the contents. Then simply cd shellWithParser to get into the shellWithParser directory and type make to run the compilation. It will generate an executable file called shell from shell.c. You can execute the shell by typing ./shell on the command line. If you want to delete all the object and executable files, you can simply type the command make clean.

If you cannot get your code compiled, probably you are missing some library packages. If you are in Ubuntu, try to type the two commands below to install the packages.

        sudo apt-get install build-essential
        sudo apt-get install libreadline-dev

You can work on this assignment either alone or in pairs. If you work alone, you will be asked to implement all the required features. If you work with a teammate, you need to additionally implement two optional features.

Remember to include a README file to describe your assignment's status, which features do not work yet, who is your teammate, and what optional features you have implemented. Anything that is helpful for grading is welcome.

Basic Pseudocode for a Shell

From experience using a command shell, you should be able to write basic pseudocode for a shell. (Note: The pseducode below uses the UNIX style fork/exec not the Windows style CreateProcess/WaitForSingleObject.). The following is an example about where to implement a built-in command, where to execute an external command, and how to put an external command into background.

The parser function parse() is already fully implemented in parse.c. It will return a user-defined structure parseinfo which stores all the command information like command name, command arguments, background or not, piped filename etc. The subfunctions isBuiltInCommand() and isBackgroundJob() can extract that information from the parseinfo structure. The subfunctions executeBuiltInCommand() and executeCommand() need to be further implemented. You need to know how to run a linux command inside a program, and also you need to know some OS system calls that may be needed to implement the built-in commands. If you don't know the usage of those system calls, typing "man systemcallname" will help you a lot. Besides, googling some sample code of those system calls can also give you some idea of how it should be used.

     
 
     int main (int argc, char **argv)
     {
	int childPid;
	char * cmdLine;
	parseInfo *info;
	while (1){

	        cmdLine= readline(">"); //GNU readline;
		
		info = parse(cmdLine);

		record command in history list (GNU readline history ?)
 
		if ( isBuiltInCommand(info)){
		    executeBuiltInCommand(info);
		} else {		
		     childPid = fork();
		     if (childPid == 0){
			executeCommand(info); //calls execvp  
			
		     } else {
			if (isBackgroundJob(info)){
			        record in list of background jobs
			} else {
				waitpid (childPid);

			}		
		    }
	        }
     }

Required Features

Between this simple pseudocode and full featured shells, there are many useful features missing. Below is a list of the features you should support in your shell at a minimum. (Note: These are not necessarily in order of easy to hard, don't feel that you have to implement them in the order as they are.):

1. The prompt you print should indicate the current working directory. For example:
   

   The directory: /usr/foo/bar%
   

   It may also indicate other things like machine name or username or any other information you would like.

   Try getcwd(char * buf, size_t size) .

2. You should allow the user to specify commands either by relative or absolute pathnames. To read in the command line, you may want to consider the readline function from the GNU readline library as it supports user editing of the command line.

   Linux TIP! Example of a relative command df, same command with absolute path: /bin/df. To find the absolute path of a command use which, which ls gives /bin/ls

Try execvp it will search the path automatically for you. The first argument should be a pointer to the command string and the second argument should be a pointer to an array which contains the command string as arg[0] and the other arguments as arg[1] through arg[n].

3. You do not need to support setting of environment variables. However, you may find it useful to know about these variables especially PATH which is the list of directories in which to look for a specified executable. You may use execvp to have the system search the PATH inherited by your own shell from its parent.

   Linux TIP! Use printenv PATH to see what's in your executable path

4. You should be able to redirect STDIN and STDOUT for the new processes by using < and >. For example, foo < infile > outfile would create a new process to run foo and assign STDIN for the new process to infile and STDOUT for the new process to outfile. In many real shells it gets much more complicated than this (e.g. >> to append, > to overwrite, >& redirect STDERR and STDOUT, etc.)! (WARNING: I am told that I/O redirection may be quite tricky on Windows. We may substitute a different feature here for Windows) You also do not have to support I/O redirection for built-in commands (it shouldn't be too hard but you don't have to do it.)

   Note: one redirect in each direction is fine, not ls > foo < foo2

First open the file (use open or creat, open read only for infiles and creat writable for outfiles ) and then use dup2. 0 is the filedescriptor for STDIN and 1 is the file descriptor for STDOUT.
Examples:
dup2 (fdFileYouOpened, fileno(stdin))
dup2 (fdFileYouOpened, fileno(stdout))

5. You should be able to place commands in the background with an & at the end of the command line. (You do not need to support moving processes between the foreground and the background (ex. bg and fg). You also do not need to support putting built-in commands in the background.)

Try waitpid(pid, status, options).

6. You should maintain a history of commands previously issued. The number of previous commands recorded can be a compile time constant of at least 10. This is a FIFO list, you should start numbering them with 1 and then when you exhaust your buffer you should discard the lowest number *BUT* keep incrementing the number of the next item. For example, if storing 10 commands, when the 11th is issued, you would be recording commands 2 through 11.

   Linux TIP! The history command can show you the remembered commands by the linux shell, available with the up and down arrows

7. A user should be able to repeat a previously issued command by typing !number where number indicates which command to repeat. The number can be negative, which indicates the command prior to the last. !-2 would mean to repeat the one next to the last command. !1 would mean repeat the command numbered 1 in the list of command returned by history.

   Note: You can probably think of better syntax for this, but I thought it was good to stay as close as possible to syntax used by real shells

8. A built-in command is one for which no new process is created but instead the functionality is build directly into the shell itself. You should support the following built-in commands: jobs, cd, history, exit and kill.
   • jobs provide a numbered list of processes currently executing in the background.

   Try waitpid with WNOHANG option to check without blocking. You can either check when jobs called or each time through the main while loop.

   • cd should change the working directory.
   • history should print the list of previously executed commands. The list of commands should include be numbered such that the numbers can be used with ! to indicate a command to repeat.
   • exit should terminate your shell process.
   • kill %num should terminate the process numbered in the list of background processes returned by jobs (by sending it a SIGKILL signal).

     Note: Usually kill num refers to the process with ProcessId num; while kill %num refers to the process in the jobs list with number num

     Try kill (pid, SIGKILL) .

   • I would also *highly* recommend the built-in function help that lists the available built-in commands and their syntax. (If you don't follow the syntax expected, then a help function would let the graders proceed anyway.)
9. If the user chooses to exit while there are background processes, notify the user that these background processes exist, do not exit and return to the command prompt. The user must kill the background processes before exiting.
10. You may assume that each item in the command string is separated on either side by at least one space (e.g. prog > outfile rather than prog>outfile).

Optional Features

If you are working in a team, you must either port your shell to Windows (1. below) or implement any two of the other features (2.through 13.). If you enjoy this assignment would like to add more advanced features to your shell, you can earn extra credit. Here are some sugguestions:

1. Port the code to run on Windows using CreateProcess/WaitForSingleObject.
2. You could support optional parameters to some of the built-in commands. For example, history -s num could set the size of the history buffer and history num could return the last num commands. You could also support additional built-in commands like which, pushd/popd or alias. If you make modifcations of this type, I would recommend help command to return more detailed information on a single command.
3. You could support | , a pipe, between two processes. For example, foo | bar would send the STDOUT of foo to the STDIN of bar using a pipe. You may want to start by supporting pipes only between two processes before considering longer chains. Longer chains will probably require something like handle process n and then recursively handle the other n-1.
4. You could implement more advanced I/O redirection as described above (>&, >!, etc.).
5. You could implement the built-in shell functions, fg and bg, to move processes between the background and the foreground.
6. You could support the editing of shell variables with built-in shell functions like printenv and setenv.
7. I wouldn't recommend it :-), but you could even write support for shell programming (e.g. if/then, while, for constructs).
8. Tab completion and command prompt editing. The GNU readline library makes this easy.
9. Up and down errors to scroll through the history list. The GNU history library makes easy.
10. Adding the builtin function ls on Windows.
11. You could relax the parsing constraints on the command line itself (e.g. correctly recognize the space free command prog>outfile).
12. Terminal support (messy!)(You may notice that some programs like more or pine are aware of the screen real estate they need - that requires terminal emulation support.
13. You could also try porting it to yet another OS. (PalmOS?)

Any advanced shell feature is likely to earn you some extra credit, but you can earn it only after you've finished the required functions.

Examples

Here show an example of the format and syntax of the shell in operation.

Here is a tar file that contains a Makefile and two programs that illustrate the use of the dup system call to support I/O redirection. Note that the file f1 is present in the directory and is needed as the input file for the forkDupExec program. That program when run will produce f2 an identical copy of f1 by forking off a new process, reassigning its stdout and stdin and execing cat.

Here is sample code showing the usage of system calls that you may use in this lab. They are good references for you to basically understand C programming and how to use Linux system calls. If you don't know where to start on this lab project, they will help give you a kickstart.

Submitting Your Assignment

Submit your work to the AFS directory /afs/cu/class/cs444/sp11/shell/your_username.

Copy all your files into the AFS homework directory. You can use the linux command:

      scp  shellWithParser/*   your_name@polaris.clarkson.edu:/afs/cu/class/cs444/sp11/shell/your_name  

Don't create any subdirectories in the AFS homework directory.

Don't compress your files in the AFS homework directory.

Your AFS homework directory should include all the source code and a README file which describes your project status, lists the features that have been implemented, and so on.
Note: If you are working with a partner, you need to additionally implement two optional features listed as above or port your shell to Windows. The README should additionally contain both teammate names and which features were additionally implemented.

Helpful Resources

Here are a list of resources you may find helpful. Feel free to send mail suggesting others.

Linux/UNIX

GNU C Library

GNU history library (for the required functionality might be easier without it?)

Linux System Calls

MANPAGE
The following system functions are likely to be helpful (consult the man pages for details):
fork, exec, execvp, wait, waitpid, kill, dup, pipe, strncmp, strlen, malloc, free, getcwd, chdir , open, close, readline, gets, fgets, getchar, signal

Debugging with GDB

Windows

MSDN

Sample I/O redirection