UNIX for Dummies Questions & Answers

Hi All,

I'm relatively new to Unix scripting and am trying to get my head around piping.

I'm trying to take a header record from one file and prepend it to another file. I've done this by creating several temp files but i'm wondering if there is a cleaner way to do this.

I'm thinking something along the lines of using two sed commands to first pull out the header using that as the input to prepend the file. So far i've got this:

Code:

 
sed -n 1,31p $fileOne | sed -i '1i<prependText>' $fileTwo

Is it possible to fill the <prependText> section with the output of the first sed command? i.e. store it in a variable and then reference it?

Kind Regards

Something like this?

Code:

awk 'NR==FNR && NR>31{next}1' file1 file2 > newfile

Try

Code:

sed -i '1i\'<(sed -n 1,31p file1) file2

Hi hergp,

I tried your solution but strangly it inserts the txt "/dev/fd/63" into my file instead of the header record. If I extract the sed command in the () it outputs correctly. Not sure what's going on here

Regards

Insert a space beween ' and <.

Hmm, it works here (without the -i option) on CentOS 6.4:

Code:

$ cat file1
head1
head2
head3
head4
$ cat file2
line1
line2
line3
$ sed '1i\'<(sed -n 1,2p file1) file2
head1
head2
line1
line2
line3

No problem. Picture a Unix-process like a garden hose: you put something in on top, something happens inside, then something comes out on bottom. Piping now means: connect the end of one of these hoses to the beginning of the other: something goes into hose one, is processed, the result pours out of hose one and into hose two, is processed again and the result comes out on bottom.

Yes, there is. In fact you should not use pipes at all for this. See below.


Without even looking at your solution: you should combine the two sed-statements into one. It is relatively "costly" (in terms of machine time, memory, ...) to start a new process while it is "cheap" to let it run further when it already runs. If it is possible to combine two sed-processes into one (and with sed this is always possible) you should do so, because the resulting single process takes a lot less time, memory, resources, whatever to complete than the two ones.


No, this is not possible, at least not within the same line. You would have to run the first process, let it fill the variable, only then issue the command that starts the second process. The reason is that the processes are started one after the other, but the shell starting them evaluates the commandline at once. To understand the process consider the following:

Code:

var1="abc"
var2="123"
cmd1 -x "$var1" | cmd2 -y "$var2"

The first thing the shell does is to notice that there are variables to evaluate. Therefore in a first pass ALL variables are replaced with their content:

Code:

cmd1 -x "abc" | cmd2 -y "123"

Only then the processes themselves are started. You can easily observe this process by setting "set -xv" in your shell and running this code (switch off again with "set +xv"). Because all the variables are evaluated (the shell term is "expanded") at the same time the following will NOT work:

Code:

var1="var2"
var2="123"
cmd -x $$var1

One might expect that this results in "$var" being evaluated to "var2", then the resulting "$var2" being evaluated to "123", but this is not the case as it would require a second pass: all variables are evaluated at the same time and - loosely speaking - a variable coming too late to the party is simply ignored. (In fact there is a way to circumvent this with the keyword "eval", but this is another story.)

Now to the problem itself: Try the following:

Code:

prep="<prepend-text>"
sed -n '1,31 {;
           1 s/^/'"$prep"'/
           p
          }' /path/to/input > path/to/output

You can have rules to apply to a range of lines, like you did, but you can also nest these rules: one rule for lines 1-31 and another one for only line 1.

I hope this helps.

bakunin