Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to read file, and replace certain string with another string?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to read file, and replace certain string with another string?
# 1  
Old 12-04-2014
How to read file, and replace certain string with another string?
Hi all, the value in the following file is just an example. It could be a different value/network addresses.

Here is my example of initial output in a file name net.txt

Initial Output, net.txt
Quote:
10.0.0.0/8
172.16.1.0/16
192.168.1.0/24
The goal is to produce the following format which is to convert from CIDR to Netmask formatting.

Quote:
10.0.0.0 255.0.0.0
172.16.1.0 255.255.0.0
192.168.1.0 255.255.255.0
My current draft script which is not perfect is to use “sed” to replace “/24” with “255.255.255.0” format. This is part of network subnetting conversion.

This is to print the initial net.txt value and save the final output to net-mask.txt
Quote:
#!/bin/bash
awk '{
print "$1;
}' net.txt > net-mask.txt
Quote:
sed 's/\/8/ 255.0.0.0/'
sed 's/\/16/ 255.255.0.0/'
sed 's/\/24/ 255.255.255.0/'
or
Quote:
sed 's#/8# 255.0.0.0#'
sed 's#/16# 255.255.0.0#'
sed 's#/24# 255.255.255.0#'
Or course the sed script for CIDR to netmask table above is not complete. The full list of the conversion can be found here. Netmask / CIDR Translation Table

Please let me know where is the best place to put “sed” into my shell script file.
If there is better way to do this, please let me know. Thanks in advance.
Last edited by type8code0; 12-05-2014 at 12:05 AM..
# 2  
Old 12-05-2014
Quote:
Originally Posted by type8code0
Hi all, the value in the following file is just an example. It could be a different value/network addresses.
Here is my example of initial output in a file name net.txt
Initial Output, net.txt

The goal is to produce the following format which is to convert from CIDR to Netmask formatting.
My current draft script which is not perfect is to use “sed” to replace “/24” with “255.255.255.0” format. This is part of network subnetting conversion.

This is to print the initial net.txt value and save the final output to net-mask.txt
or
Or course the sed script for CIDR to netmask table above is not complete. The full list of the conversion can be found here. Netmask / CIDR Translation Table

Please let me know where is the best place to put “sed” into my shell script file.
If there is better way to do this, please let me know. Thanks in advance.
Hello type8code0,

Following may help you in same.
Code:
 awk -F"/" '{sub("8","255.0.0.0",$2);sub("16","255.255.0.0",$2);sub("24","255.255.255.0",$2);print}' Input_file

Output will be as follows.
Code:
10.0.0.0 255.0.0.0
172.16.1.0 255.255.0.0
192.168.1.0 255.255.255.0

EDIT: Also kindly use code tags for commands and codes not quotes, quotes you can use to address any user's speicific post etc purposes like I have used above for your post, you can go through the forum rules in below link.

https://www.unix.com/misc.php?do=cfrules


Thanks,
R. Singh
Last edited by RavinderSingh13; 12-05-2014 at 01:00 AM.. Reason: Added suggestion to use code tags in place of quotes for user
This User Gave Thanks to RavinderSingh13 For This Post:
type8code0
# 3  
Old 12-05-2014
Quote:
#!/bin/bash
awk '{
print "$1;
}' net.txt > net-mask.txt
There is a syntax error in the above code: The quote in front of $1 is superfluous.
Apart from that, in this particular case (input file contains only one column) the above code is essentially the same as cp net.txt net-mask.txt Smilie

Quote:
Please let me know where is the best place to put “sed” into my shell script file.
If you plan to use it for modification of net-mask.txt, then it certainly should be placed below the awk command.

Code:
#!/bin/bash
awk '{
print $1;
}' net.txt > net-mask.txt

sed -i 's/\/8/ 255.0.0.0/' net-mask.txt
sed -i 's/\/16/ 255.255.0.0/' net-mask.txt
sed -i 's/\/24/ 255.255.255.0/' net-mask.txt

Note I used sed's -i option for inplace editing, otherwise things get complicated, e.g.
Code:
sed 's/\/8/ 255.0.0.0/' net-mask.txt > net-mask.temp
mv net-mask.temp net-mask.txt
sed 's/\/16/ 255.255.0.0/' net-mask.txt > net-mask.temp
mv net-mask.temp net-mask.txt
sed 's/\/24/ 255.255.255.0/' net-mask.txt > net-mask.temp
mv net-mask.temp net-mask.txt

Quote:
And of course, if there is better way to do this, please let me know.
Sure.
Code:
#!/bin/bash

awk -F'/' '
    $2==8  { print $1 " 255.0.0.0" }
    $2==16 { print $1 " 255.255.0.0" }
    $2==24 { print $1 " 255.255.255.0" }
' net.txt > net-mask.txt

I think you get the idea now and will be able to extend this code to match other CIDRs Smilie
Last edited by junior-helper; 12-05-2014 at 01:20 AM.. Reason: grammar
This User Gave Thanks to junior-helper For This Post:
type8code0
# 4  
Old 12-05-2014
This task is almost impossible in awk as it doesn't have bit operations. But, in (recent) bash, you could try this:
Code:
while IFS="/" read A B
        do DL=$'\t'
           printf "%s/%s" $A $B 
           for i in 24 16 8 0
                do printf "%s%d" "$DL" $(((2**32-1 ^ (2**(32-B)-1) >> $i) & 255))
                   DL="."
                done
           printf "\n"
        done < file
10.0.0.0/8        255.0.0.0
10.0.0.0/10       255.192.0.0
172.16.1.0/16     255.255.0.0
172.16.1.0/18     255.255.192.0
192.168.1.0/24    255.255.255.0
192.168.1.0/32    255.255.255.255

This User Gave Thanks to RudiC For This Post:
type8code0
# 5  
Old 12-05-2014
My original solution, from your original thread, still does exactly what you say you want. Did you ever try it?

Code:
#!/bin/bash

while IFS="/" read IP S
do
        M=$(( 0xffffffff ^ ((1 << (32-S)) -1) ))
        echo "$IP netmask $(( (M>>24) & 0xff )).$(( (M>>16) & 0xff )).$(( (M>>8) & 0xff )).$(( M & 0xff ))"
done < infile > outfile

This User Gave Thanks to Corona688 For This Post:
type8code0
# 6  
Old 12-05-2014
Quote:
Originally Posted by RavinderSingh13
Hello type8code0,

Following may help you in same.
Code:
 awk -F"/" '{sub("8","255.0.0.0",$2);sub("16","255.255.0.0",$2);sub("24","255.255.255.0",$2);print}' Input_file

Output will be as follows.
Code:
10.0.0.0 255.0.0.0
172.16.1.0 255.255.0.0
192.168.1.0 255.255.255.0

EDIT: Also kindly use code tags for commands and codes not quotes, quotes you can use to address any user's speicific post etc purposes like I have used above for your post, you can go through the forum rules in below link.

https://www.unix.com/misc.php?do=cfrules


Thanks,
R. Singh
Cool. I’ll just need to add the rest of CIDR to Netmask conversion.
I’ll share the full bash script later once it’s complete.
I’ll take note on code vs quotes. Thanks again I really appreciate it. Smilie

Here is the sample of the output.
Code:
me@box:~/$ cat net.txt
10.0.0.0/8
172.16.1.0/16
192.168.1.0/24
me@box:~/$ awk -F"/" '{sub("8","255.0.0.0",$2);sub("16","255.255.0.0",$2);sub("24","255.255.255.0",$2);print}' net.txt
10.0.0.0 255.0.0.0
172.16.1.0 255.255.0.0
192.168.1.0 255.255.255.0
me@box:~/$

---------- Post updated at 01:22 AM ---------- Previous update was at 01:09 AM ----------

Quote:
Originally Posted by junior-helper
There is a syntax error in the above code: The quote in front of $1 is superfluous.
Apart from that, in this particular case (input file contains only one column) the above code is essentially the same as cp net.txt net-mask.txt Smilie

If you plan to use it for modification of net-mask.txt, then it certainly should be placed below the awk command.

Code:
#!/bin/bash
awk '{
print $1;
}' net.txt > net-mask.txt

sed -i 's/\/8/ 255.0.0.0/' net-mask.txt
sed -i 's/\/16/ 255.255.0.0/' net-mask.txt
sed -i 's/\/24/ 255.255.255.0/' net-mask.txt

Note I used sed's -i option for inplace editing, otherwise things get complicated, e.g.
Code:
sed 's/\/8/ 255.0.0.0/' net-mask.txt > net-mask.temp
mv net-mask.temp net-mask.txt
sed 's/\/16/ 255.255.0.0/' net-mask.txt > net-mask.temp
mv net-mask.temp net-mask.txt
sed 's/\/24/ 255.255.255.0/' net-mask.txt > net-mask.temp
mv net-mask.temp net-mask.txt

Sure.
Code:
#!/bin/bash

awk -F'/' '
    $2==8  { print $1 " 255.0.0.0" }
    $2==16 { print $1 " 255.255.0.0" }
    $2==24 { print $1 " 255.255.255.0" }
' net.txt > net-mask.txt

I think you get the idea now and will be able to extend this code to match other CIDRs Smilie
Awesome! You’re definitely senior-helper. Thanks again for your help I really appreciate it.

Here is the full code. Hopefully others will get benefit from this too.
Code:
#!/bin/bash
################################################
# Credit and special thanks to "junior-helper": 
# Usage: ./cidr2subnet-junior-helper
################################################

awk -F'/' '
$2== 32 { print $1 " 255.255.255.255 " }
$2== 31 { print $1 " 255.255.255.254 " }
$2== 30 { print $1 " 255.255.255.252 " }
$2== 29 { print $1 " 255.255.255.248 " }
$2== 28 { print $1 " 255.255.255.240 " }
$2== 27 { print $1 " 255.255.255.224 " }
$2== 26 { print $1 " 255.255.255.192 " }
$2== 25 { print $1 " 255.255.255.128 " }
$2== 24 { print $1 " 255.255.255.0 " }
$2== 23 { print $1 " 255.255.254.0 " }
$2== 22 { print $1 " 255.255.252.0 " }
$2== 21 { print $1 " 255.255.248.0 " }
$2== 20 { print $1 " 255.255.240.0 " }
$2== 19 { print $1 " 255.255.224.0 " }
$2== 18 { print $1 " 255.255.192.0 " }
$2== 17 { print $1 " 255.255.128.0 " }
$2== 16 { print $1 " 255.255.0.0 " }
$2== 15 { print $1 " 255.254.0.0 " }
$2== 14 { print $1 " 255.252.0.0 " }
$2== 13 { print $1 " 255.248.0.0 " }
$2== 12 { print $1 " 255.240.0.0 " }
$2== 11 { print $1 " 255.224.0.0 " }
$2== 10 { print $1 " 255.192.0.0 " }
$2== 9 { print $1 " 255.128.0.0 " }
$2== 8 { print $1 " 255.0.0.0 " }
$2== 7 { print $1 " 254.0.0.0 " }
$2== 6 { print $1 " 252.0.0.0 " }
$2== 5 { print $1 " 248.0.0.0 " }
$2== 4 { print $1 " 240.0.0.0 " }
$2== 3 { print $1 " 224.0.0.0 " }
$2== 2 { print $1 " 192.0.0.0 " }
$2== 1 { print $1 " 128.0.0.0 " }
' net.txt > net-mask.txt

And here is the output
Code:
me@box:~/$ cat net.txt
10.0.0.0/8
172.16.1.0/16
192.168.1.0/24
me@box:~/$

me@box:~/$ ./cidr2subnet-junior-helper

me@box:~/$ cat net-mask.txt
10.0.0.0 255.0.0.0
172.16.1.0 255.255.0.0
192.168.1.0 255.255.255.0
me@box:~/$

Next project is to reverse this process which is to convert from netmask to CIDR form Smilie

---------- Post updated at 01:34 AM ---------- Previous update was at 01:22 AM ----------

Quote:
Originally Posted by RudiC
This task is almost impossible in awk as it doesn't have bit operations. But, in (recent) bash, you could try this:
Code:
while IFS="/" read A B
        do DL=$'\t'
           printf "%s/%s" $A $B 
           for i in 24 16 8 0
                do printf "%s%d" "$DL" $(((2**32-1 ^ (2**(32-B)-1) >> $i) & 255))
                   DL="."
                done
           printf "\n"
        done < file
10.0.0.0/8        255.0.0.0
10.0.0.0/10       255.192.0.0
172.16.1.0/16     255.255.0.0
172.16.1.0/18     255.255.192.0
192.168.1.0/24    255.255.255.0
192.168.1.0/32    255.255.255.255

Many thanks RudiC Smilie
This forum is super cool. One question, multiple solutions provided and each of them is unique.
And here is the output

Code:
me@box:~/$ cat net.txt
10.0.0.0/8
172.16.1.0/16
192.168.1.0/24
me@box:~/$
me@box:~/$ cat ./cidr2subnet-RudiC
################################################
# Credit and special thanks to "RudiC":
# Usage: ./cidr2subnet-RudiC
################################################

while IFS="/" read A B
        do DL=$'\t'
           printf "%s/%s" $A $B
           for i in 24 16 8 0
                do printf "%s%d" "$DL" $(((2**32-1 ^ (2**(32-B)-1) >> $i) & 255))
                   DL="."
                done
           printf "\n"
        done < net.txt
me@box:~/$

me@box:~/$ ./cidr2subnet-RudiC
10.0.0.0/8      255.0.0.0
172.16.1.0/16   255.255.0.0
192.168.1.0/24  255.255.255.0
me@box:~/$

I just have one question on this script. Is it possible to remove /8,/16, and /24 from the output and use a space instead of tab?
Please let me know if there is a way to reverse this process from CIDR to netmask formatting.

Example:
Initial Input
Quote:
10.0.0.0 255.0.0.0
172.16.1.0 255.255.0.0
192.168.1.0 255.255.255.0
Final Output
Quote:
10.0.0.0/8
172.16.1.0/16
192.168.1.0/24
---------- Post updated at 01:52 AM ---------- Previous update was at 01:34 AM ----------

Quote:
Originally Posted by Corona688
My original solution, from your original thread, still does exactly what you say you want. Did you ever try it?

Code:
#!/bin/bash

while IFS="/" read IP S
do
        M=$(( 0xffffffff ^ ((1 << (32-S)) -1) ))
        echo "$IP netmask $(( (M>>24) & 0xff )).$(( (M>>16) & 0xff )).$(( (M>>8) & 0xff )).$(( M & 0xff ))"
done < infile > outfile

Your solution is perfect. Thanks again. I noticed you've added "< infile > outfile" to this script. I'm new to the shell script and it's so fun learning it by solving daily problem and make things easier.

I've updated the input and output file name and here is the sample of the output. Please let me know how to reverse this process as well from netmask to CIDR formatting.

Code:
me@box:~/$ cat net.txt
10.0.0.0/8
172.16.1.0/16
192.168.1.0/24
me@box:~/$
me@box:~/$ cat cidr2subnet-Corona688
#!/bin/bash

################################################
# Credit and special thanks to "Corona688":
# Usage: ./cidr2subnet-Corona688
################################################

while IFS="/" read IP S
do
        M=$(( 0xffffffff ^ ((1 << (32-S)) -1) ))
        echo "$IP $(( (M>>24) & 0xff )).$(( (M>>16) & 0xff )).$(( (M>>8) & 0xff )).$(( M & 0xff ))"
done < net.txt > net-mask.txt
me@box:~/$
me@box:~/$ ./cidr2subnet-Corona688
me@box:~/$
me@box:~/$ cat net-mask.txt
10.0.0.0 255.0.0.0
172.16.1.0 255.255.0.0
192.168.1.0 255.255.255.0
me@box:~/$

Last edited by type8code0; 12-05-2014 at 01:42 PM..
# 7  
Old 12-05-2014
To remove the /nn, replace printf "%s/%s" $A $B by printf "%s" $A
For the reverse operation, try
Code:
while IFS="     ." read -a VAL
          do N=0
             DL=( . . . / )
             for ((J=0;J<=3;J++))
                  do printf "%d%s" ${VAL[J]} "${DL[J]}"
                     for ((I=7;I>=0;I--))
                          do N=$(( N + (VAL[J+4]>>I)%2 ))
                          done
                  done
             printf "%d\n" $N
          done < file
10.0.0.0/8
10.0.0.0/10
172.16.1.0/16
172.16.1.0/17
172.16.1.0/18
172.16.1.0/19
192.168.1.0/24
192.168.1.0/32

This User Gave Thanks to RudiC For This Post:
type8code0
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Search partial string in a file and replace the string - UNIX

I have the below string which i need to compare with a file and replace this string in the file which matches closely. Can anyone help me on this. string(Scenario 1)- user::r--,user::ourfrd:r-- String(Scenario 2)- user::r-- File **** # file: /local/Desktop/myfile # owner: me # group:... (6 Replies)
Discussion started by: sarathy_a35
6 Replies

2. Shell Programming and Scripting

Script to replace a string with pattern read from a file

I have two files blocks.txt and rules.txt. In blocks.txt i have the following entries Linux1 Linux2 Linux3 ..... Linux10 In rules.txt i have the lines where a filename pattern starts like 'blk-name.*' I want to replace 'blk-name' with the names read from blocks.txt file I tried... (2 Replies)
Discussion started by: Jag02
2 Replies

3. Shell Programming and Scripting

Replace string in XML file with awk/sed with string from another

Sorry for the long/weird title but I'm stuck on a problem I have. I have this XML file: </member> <member> <name>TransactionID</name> <value><string>123456789123456</string></value> </member> <member> <name>Number</name> ... (9 Replies)
Discussion started by: cozzin
9 Replies

4. UNIX for Dummies Questions & Answers

Search for a string,delete the line and replace with new string in a file

Hi Everyone, I have a requirement in ksh where i have a set of files in a directory. I need to search each and every file if a particular string is present in the file, delete that line and replace that line with another string expression in the same file. I am very new to unix. Kindly help... (10 Replies)
Discussion started by: Pradhikshan
10 Replies

5. Shell Programming and Scripting

Perl script to read string from file#1 and find/replace in file#2

Hello Forum. I have a file called abc.sed with the following commands; s/1/one/g s/2/two/g ... I also have a second file called abc.dat and would like to substitute all occurrences of "1 with one", "2 with two", etc and create a new file called abc_new.dat sed -f abc.sed abc.dat >... (10 Replies)
Discussion started by: pchang
10 Replies

6. Shell Programming and Scripting

perl- read search and replace string from the file

Dear all, I have a number of files and each file has two sections separated by a blank line. At the top section, I have lines which describes the values of the alphabetical characters, # s #; 0.123 # p #; 12.3 # d #; -2.33 # f #; 5.68 <blank line> sssssss spfdffff sdfffffff Now I... (4 Replies)
Discussion started by: sasharma
4 Replies

7. Shell Programming and Scripting

sed or awk command to replace a string pattern with another string based on position of this string

here is what i want to achieve... consider a file contains below contents. the file size is large about 60mb cat dump.sql INSERT INTO `table1` (`id`, `action`, `date`, `descrip`, `lastModified`) VALUES (1,'Change','2011-05-05 00:00:00','Account Updated','2012-02-10... (10 Replies)
Discussion started by: vivek d r
10 Replies

8. Shell Programming and Scripting

replace (sed?) a string in file with multiple lines (string) from variable

Can someone tell me how I can do this? e.g: a=$(echo -e wert trewt ertert ertert ertert erttert erterte rterter tertertert ert) How do i replace the STRING with $a? I try this: sed -i 's/STRING/'"$a"'/g' filename.ext but this don' t work (2 Replies)
Discussion started by: jforce
2 Replies

9. Shell Programming and Scripting

replace (sed?) a single line/string in file with multiple lines (string) from another file??

Can someone tell me how I can do this? e.g: Say file1.txt contains: today is monday the 22 of NOVEMBER 2010 and file2.txt contains: the 11th month of How do i replace the word NOVEMBER with (5 Replies)
Discussion started by: tuathan
5 Replies

10. Shell Programming and Scripting

Using sed to replace a string in file with a string in a variable that contains spaces

Hi, i call my shell like: my_shell "my project name" my script: #!/bin/bash -vx projectname=$1 sed s/'PROJECT_NAME ='/'PROJECT_NAME = '$projectname/ <test_config_doxy >temp cp temp test_config_doxy the following error occurres: sed s/'PROJECT_NAME ... (2 Replies)
Discussion started by: vivelafete
2 Replies
Login or Register to Ask a Question