Output of sed command to another sed command


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Output of sed command to another sed command
# 1  
Old 12-05-2013
Output of sed command to another sed command

Hi All,

I'm relatively new to Unix scripting and am trying to get my head around piping.

I'm trying to take a header record from one file and prepend it to another file. I've done this by creating several temp files but i'm wondering if there is a cleaner way to do this.

I'm thinking something along the lines of using two sed commands to first pull out the header using that as the input to prepend the file. So far i've got this:

Code:
 
sed -n 1,31p $fileOne | sed -i '1i<prependText>' $fileTwo

Is it possible to fill the <prependText> section with the output of the first sed command? i.e. store it in a variable and then reference it?

Kind Regards
# 2  
Old 12-05-2013
Something like this?
Code:
awk 'NR==FNR && NR>31{next}1' file1 file2 > newfile

# 3  
Old 12-05-2013
Try
Code:
sed -i '1i\'<(sed -n 1,31p file1) file2

# 4  
Old 12-05-2013
Quote:
Originally Posted by hergp
Try
Code:
sed -i '1i\'<(sed -n 1,31p file1) file2

Hi hergp,

I tried your solution but strangly it inserts the txt "/dev/fd/63" into my file instead of the header record. If I extract the sed command in the () it outputs correctly. Not sure what's going on here

Regards
# 5  
Old 12-05-2013
Insert a space beween ' and <.
# 6  
Old 12-05-2013
Hmm, it works here (without the -i option) on CentOS 6.4:
Code:
$ cat file1
head1
head2
head3
head4
$ cat file2
line1
line2
line3
$ sed '1i\'<(sed -n 1,2p file1) file2
head1
head2
line1
line2
line3

# 7  
Old 12-05-2013
Quote:
Originally Posted by BigCroyd
I'm relatively new to Unix scripting and am trying to get my head around piping.
No problem. Picture a Unix-process like a garden hose: you put something in on top, something happens inside, then something comes out on bottom. Piping now means: connect the end of one of these hoses to the beginning of the other: something goes into hose one, is processed, the result pours out of hose one and into hose two, is processed again and the result comes out on bottom.

Quote:
Originally Posted by BigCroyd
I'm trying to take a header record from one file and prepend it to another file. I've done this by creating several temp files but i'm wondering if there is a cleaner way to do this.
Yes, there is. In fact you should not use pipes at all for this. See below.


Quote:
Originally Posted by BigCroyd
I'm thinking something along the lines of using two sed commands to first pull out the header using that as the input to prepend the file.
Without even looking at your solution: you should combine the two sed-statements into one. It is relatively "costly" (in terms of machine time, memory, ...) to start a new process while it is "cheap" to let it run further when it already runs. If it is possible to combine two sed-processes into one (and with sed this is always possible) you should do so, because the resulting single process takes a lot less time, memory, resources, whatever to complete than the two ones.


Quote:
Originally Posted by BigCroyd
Is it possible to fill the <prependText> section with the output of the first sed command? i.e. store it in a variable and then reference it?
No, this is not possible, at least not within the same line. You would have to run the first process, let it fill the variable, only then issue the command that starts the second process. The reason is that the processes are started one after the other, but the shell starting them evaluates the commandline at once. To understand the process consider the following:

Code:
var1="abc"
var2="123"
cmd1 -x "$var1" | cmd2 -y "$var2"

The first thing the shell does is to notice that there are variables to evaluate. Therefore in a first pass ALL variables are replaced with their content:

Code:
cmd1 -x "abc" | cmd2 -y "123"

Only then the processes themselves are started. You can easily observe this process by setting "set -xv" in your shell and running this code (switch off again with "set +xv"). Because all the variables are evaluated (the shell term is "expanded") at the same time the following will NOT work:

Code:
var1="var2"
var2="123"
cmd -x $$var1

One might expect that this results in "$var" being evaluated to "var2", then the resulting "$var2" being evaluated to "123", but this is not the case as it would require a second pass: all variables are evaluated at the same time and - loosely speaking - a variable coming too late to the party is simply ignored. (In fact there is a way to circumvent this with the keyword "eval", but this is another story.)

Now to the problem itself: Try the following:

Code:
prep="<prepend-text>"
sed -n '1,31 {;
           1 s/^/'"$prep"'/
           p
          }' /path/to/input > path/to/output

You can have rules to apply to a range of lines, like you did, but you can also nest these rules: one rule for lines 1-31 and another one for only line 1.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

sed Command Output

I want to know how this command output will be sed 's/^"\(.*\)"$/\1/' <filename> (3 Replies)
Discussion started by: abhilashnair
3 Replies

2. Shell Programming and Scripting

sed command is saving output as blank file

Hi, I am working on a script where I am adding adding colors to few of the info in the output. Now , after that is done , I see colour codes in log files which I don't want to see.:mad::mad::mad::mad: So , I tried using sed command in script as below which gives me o/p (new.log) as blank file... (7 Replies)
Discussion started by: Dream4649
7 Replies

3. Shell Programming and Scripting

sed command not getting the right output

Hello. Ive spent probably a hour on this problem, and cant figure it out. Let me explain the problem. I run head -n5 datasets/q13data.txt and get this : $$001011<-:::$$<-::: ' GreenWHITE<-::3.1415<-:::"BLACK fubar<-:::phi<-:::foochi $$$Yellow->:::'<-:::VOIDTue taochi->::$->:::Blue"... (12 Replies)
Discussion started by: jozo95
12 Replies

4. Shell Programming and Scripting

Help with Passing the Output of grep to sed command - to find and replace a string in a file.

I have a file example.txt as follows :SomeTextGoesHere $$TODAY_DT=20140818 $$TODAY_DT=20140818 $$TODAY_DT=20140818I need to automatically update the date (20140818) in the above file, by getting the new date as argument, using a shell script. (It would even be better if I could pass... (5 Replies)
Discussion started by: SriRamKrish
5 Replies

5. Shell Programming and Scripting

How to append a value to the output after using sed command?

Hi All, I have a file where I am converting newlines to comma separated values but I would like to append zero if the output is empty Here is the command I am using sed -n -e 'H;${x;s/\n/,/g;s/^,//;p;}' test1.txt test1.txt will have comma seperated values but sometimes this file can be... (6 Replies)
Discussion started by: rajeevm
6 Replies

6. Shell Programming and Scripting

Abnormality while piping tr command output to sed

i have a file seperated each line seperated by newline. For example alpha beta gamma i am trying to replace the newlines to "," but dont want , present at the end of the line so i am trying the below one liner . but not sure whats wrong but its not working cat myfile | tr -s '\n' ',' | sed... (9 Replies)
Discussion started by: chidori
9 Replies

7. Shell Programming and Scripting

Help needed with file output awk sed command - please

Hi I have a file that contains lines starting with a particular string plus a Colon: I need to output all these lines but only what comes after the colon Can you pelase assist? Example of lines in the file: com.ubs.f35.cashequities/cashequities: 1 2 ... (5 Replies)
Discussion started by: mnassiri
5 Replies

8. Shell Programming and Scripting

sed command works from cmd line to standard output but will not write to file

Hi all .... vexing problem here ... I am using sed to replace some special characters in a .txt file: sed -e 's/_<ED>_/_355_/g;s/_<F3>_/_363_/g;s/_<E1>_/_341_/g' filename.txt This command replaces <ED> with , <F3> with and <E1> with . When I run the command to standard output, it works... (1 Reply)
Discussion started by: crumplecrap
1 Replies

9. UNIX for Dummies Questions & Answers

Using grep output as input for sed command

Hi, I would like to know if this is possible, and if so what can i do to make this work. I would like to grep a line X from fileA and then use the output to replace a word Y in fileB. grep "line X" fileA | sed -e 's/Y/X/g' > outfile this statement does not work, as i do not know how to... (7 Replies)
Discussion started by: cavanac2
7 Replies

10. Shell Programming and Scripting

Loop with sed command to replace line with sed command in it

Okay, title is kind of confusion, but basically, I have a lot of scripts on a server that I need to replace a ps command, however, the new ps command I'm trying to replace the current one with pipes to sed at one point. So now I am attempting to create another script that replaces that line. ... (1 Reply)
Discussion started by: cbo0485
1 Replies
Login or Register to Ask a Question