search and replace dynamic data in a shell script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting search and replace dynamic data in a shell script
# 1  
Old 10-17-2003
search and replace dynamic data in a shell script

Hi,

I have a file that looks something like this:

...
0,6,256,87,0,0,0,1187443420
0,6,438,37,0,0,0,1187443380
0,2,0,0,0,10,0,1197140320
0,3,0,0,0,10,0,1197140875
0,2,0,0,0,23,0,1197140332
0,3,0,0,0,23,0,1197140437
0,2,0,0,0,17,0,1197140447
0,3,0,0,0,17,0,1197140543
0,2,0,0,0,30,0,1197140237
0,3,0,0,0,30,0,1197140231
0,6,375,98,0,0,0,1187443420
0,6,222,11,0,0,0,1187443380
...

The number in the last column is dynamic. "0,2,0,0,0" is always followed by "0,3,0,0,0" What I need to do is as follows:

- I need to find the first line that starts out with 0,2,0,0,0
- Compare the column 6 to mynumber_1 (10) and if they match I move on to the next line
- Make sure in the next line the column 6 is also 10
- This time, I do the same thing with mynumber_2
- I repeat until mynumber_4 (30 in this case) is found
- Add two lines of my own data in between:

0,3,0,0,0,30,0,1197140231 and 0,6,375,98,0,0,0,1187443420

- Basically, the end result should be new file with two more lines of data in the right place
- In next iteration, I just have to get to the two new lines that I added and update the column6

I am thinking of doing the following:

- cat each line and see if it starts with 0,2,0,0,0. Is there a easy way to do this? In Perl, you can check to see if your variable CONTAINS certain data. Can you do this in shell?

- If it doesn't start out with 0,2,0,0,0 then cat $line >> new_file

- If it does, check the subsequent lines until my last number is matched up in column 6.

- If it fails during these subsequent checks, move on to the next line and start over with my_number1

Does this sound like a good algorithm? Any suggestions? I am currently stuck at checking each line to see if it starts out with 0,2,0,0,0. Thanks.
# 2  
Old 10-17-2003
cat $line
will look for a file whose name is in the variable $line. I don't think that you want that.

I don't understand your algorithm at all. But let's say that I have data like this:
1,2,3
0,0,7
0,2,4
0,0,0
8,0,0
And I only want the lines where the first two numbers are 0. Now my simplified example could be solved very easily, but I'm going to show how to read those numbers and check the values....

Code:
#! /usr/bin/ksh
exec < inputfile
exec > outputfile
IFS=,
while read n1 n2 n3 ; do
          if (( n1 == 0 && n2 == 0)) ; then
                      echo ${n1},${n2},${n3}
          fi
done
exit 0

I hope this gets you pointed in the right direction.
# 3  
Old 10-18-2003
i also have a similar problem that i spose could also come under "search and replace dynamic data in a shell script"
I am a mac user and am fairly new to the comand line and so don't know alot about shell scripts.
I want to know how to make a script that can either search an xml doc for a certain string or pattern or whatever(more specifically the value of a certain tag) and replace it with another or simply replace line #... with a specific string of text.

I've gotton so far as getting grep to find and output the text i want to replace(if that's of any use, i don't know) but i am lacking the knowledge of what to do next.

What should i do?
# 4  
Old 10-19-2003
Well, Nat, the first thing you should do is to acquaint yourself with our search function. You can get some ideas by looking at some old threads. Here are a few....

Replace text in match files

Replace string and delete extra characters

String Substitution Question
# 5  
Old 10-19-2003
Re: search and replace dynamic data in a shell script

Quote:
Originally posted by csejl

- I need to find the first line that starts out with 0,2,0,0,0
- Compare the column 6 to mynumber_1 (10) and if they match I move on to the next line
- Make sure in the next line the column 6 is also 10
- This time, I do the same thing with mynumber_2
- I repeat until mynumber_4 (30 in this case) is found
- Add two lines of my own data in between:

0,3,0,0,0,30,0,1197140231 and 0,6,375,98,0,0,0,1187443420

- Basically, the end result should be new file with two more lines of data in the right place
- In next iteration, I just have to get to the two new lines that I added and update the column6

I am thinking of doing the following:

- cat each line and see if it starts with 0,2,0,0,0. Is there a easy way to do this? In Perl, you can check to see if your variable CONTAINS certain data. Can you do this in shell?

- If it doesn't start out with 0,2,0,0,0 then cat $line >> new_file

- If it does, check the subsequent lines until my last number is matched up in column 6.

- If it fails during these subsequent checks, move on to the next line and start over with my_number1

Does this sound like a good algorithm? Any suggestions? I am currently stuck at checking each line to see if it starts out with 0,2,0,0,0. Thanks.

First of all, your explanation is very confusing. You didn't explain what my_number_1(10) was. In any case, it seems that what you are trying to do is simple to do in Perl. To see if a line starts with "0,2,0,0,0" is trivial in Perl:

if ($line =~ /^0\,2\,0\,0\,0/) {

}

From what I can make of your algorithm, it seems like you want to do something along the lines of:

open(INPUT, "< inputfile");
open(OUTPUT, "> outputfile");

while ($line = <INPUT>) {

chomp $line; # remove the carriage return
if ($line =~ /^0\,2\,0\,0\,0/) {
#look for 0,2,0,0,0 at the beginning of the line

@array = split(/,/ , $line); # split the line into parts

if ($array[5] eq $mynumber_4) {

$nextline = <INPUT>; # get the next line
chomp $nextline; # remove the carriage return

print OUTPUT "$line\n$nextline\n";

## add your own 2 lines here
print OUTPUT $my_newline1;
print OUTPUT $my_newline2;

next;
}

if ($array[5] eq $mynumber_1) {

$nextline = <INPUT>; # get the next line
chomp $nextline; # remove the carriage return

@array2 = split(/,/ , $nextline); #split the next line
if ($array2[5] eq $mynumber_1) {
print OUTPUT "$line\n$nextline\n";
}
} else {
## didn't find 0,2,0,0,0 .... write directly to output file
print OUTPUT "$line\n";
}

}


}

close(INPUT);
close(OUTPUT);

Again, this is the best I could make out from what you asked. I also left out a bunch of things like checking error conditions and delcaring variables as 'my'. In any case, it should be simple to do in Perl. I would recommend reading into Perl some more or get an experienced Perl programmer to help you out.
# 6  
Old 10-19-2003
kowrip,

Thanks for your help. But if you read my post carefully, you'll see that I almost explicitly declined to do this in perl. I just have to do this in shell that's all... :-p

Perderabo,

That definitely helped. I guess I didn't do a good job of describing what I am faced with. The problem is actually a lot more complicated than what I posted. I was just trying to get an idea of how I'd get started. I have written 200 lines of code so far and counting...

At any rate, thanks everyone for your help!
# 7  
Old 10-19-2003
Quote:
Originally posted by csejl
kowrip,

Thanks for your help. But if you read my post carefully, you'll see that I almost explicitly declined to do this in perl. I just have to do this in shell that's all... :-p

Perderabo,

That definitely helped. I guess I didn't do a good job of describing what I am faced with. The problem is actually a lot more complicated than what I posted. I was just trying to get an idea of how I'd get started. I have written 200 lines of code so far and counting...

At any rate, thanks everyone for your help!
Oops. I completely misread what you typed. You typed:

"In Perl, you can check to see ...."

I read this as:

"In Perl, CAN YOU check to see .... ?"

Sorry again. I feel like an idiot now. Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help needed with shell script to search and replace a set of strings among the set of files

Hi, I am looking for a shell script which serves the below purpose. Please find below the algorithm for the same and any help on this would be highly appreciated. 1)set of strings need to be replaced among set of files(directory may contain different types of files) 2)It should search for... (10 Replies)
Discussion started by: Amulya
10 Replies

2. Shell Programming and Scripting

Search and replace data

FILENAME: (Orig File) lab computer11node1 { hardware ok 19:50:56:bd:03:c2; MAC-ADDRESS 10.1.1.1; } lab computer11node2 { hardware ok 19:50:56:bd:03:c3; MAC-ADDRESS 10.1.1.2; } FILENAME2: computer11node1 11:50:56:81:33:c5 computer11node2 11:50:56:81:33:c6 computer11node3 11:50:56:81:33:c7... (5 Replies)
Discussion started by: kenshinhimura
5 Replies

3. Shell Programming and Scripting

Dynamic search and replace

Hi, I am trying to search and replace a value in a file. File contains data which is stated below ruby ./scripts/CreateUsage.rb Cloud_Computing_001 Cloud_Computing_001 is the 3rd column of the file. I want to replace this Cloud_Computing_001 with the runtime value. Next time i want to... (4 Replies)
Discussion started by: Chandresh Kumar
4 Replies

4. UNIX for Dummies Questions & Answers

Shell script for search and replace by field

Hi, I have an input file with below data and rules file to apply search and replace by each field in the input based on exact value or pattern. Could you please help me with unix script to read input file and rules file and then create the output and reject files based on the rules file. Input... (13 Replies)
Discussion started by: chandrath
13 Replies

5. Shell Programming and Scripting

Reading the data from CSV and performing search through shell script

Hello, I am working on building a script that does the below actions together in my Linux server. 1) First, have to read the list of strings mentioned in CSV and store it in the shell script 2) Second, pick one by one from the string list, and search a particular folder for files that... (2 Replies)
Discussion started by: vikrams
2 Replies

6. Shell Programming and Scripting

awk/sed to search & replace data in first column

Hi All, I need help in manipulating the data in first column in a file. The sample data looks like below, Mon Jul 18 00:32:52 EDT 2011,NULL,UAT Jul 19 2011,NULL,UAT 1] All field in the file are separated by "," 2] File is having weekly data extracted from database 3] For eg.... (8 Replies)
Discussion started by: gr8_usk
8 Replies

7. Shell Programming and Scripting

Using sed to search and replace data

Hi, Kindly need your expertise in this problem. I have to search and replace data. The problem is, the data is in the same format but slightly different content. What I need is sed commands that can work for those "slightly different content". input: ... (3 Replies)
Discussion started by: Alvin123
3 Replies

8. Shell Programming and Scripting

search and replace with data from another file

Can someone help me in solving the problem below. I have the following two files template_file ------------ ...other data.. ...other data.. FILE_NAME= ...other data.. ...other data.. list_file ---------- <file_name1> <file_name2> <file_name3> I need to produce another output... (3 Replies)
Discussion started by: paruthiveeran
3 Replies

9. Shell Programming and Scripting

sed - dynamic search and replace

Hi all, I have a data file formatted as in the following line: Achadd 0:35 1:35 2:35 3:40 4:40 5:40 I need the minutes converted to seconds; I wrote a script, min2sec, to do so for one datapoint. I was hoping to use sed as in the following code to call this script and... (4 Replies)
Discussion started by: x-375HK-x
4 Replies
Login or Register to Ask a Question