Using sed or awk to replace digits in files


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Using sed or awk to replace digits in files
# 1  
Old 10-04-2019
Using sed or awk to replace digits in files

Hello;

I am not good at file and stream editing. I need to replace a few digits in two files. The lines in files looks like this:

Line in the first file,
Code:
/dw300/data/obe/2019273.L800JR.1909.273

Line in second file,
Code:
1|2019273.L800JR.1909.273

I will write a function to connect to database and select Last Julian Business Day and Julian Date digits to get two output digits as:

Code:
Last Julian Business Day      Last Julian Date Digits
2019304                       1910.304

Then I need to use the first number 2019304 to replace 2019273 in these two files and replace 1909.273 with 1910.304 in two files too.

After replacement, the line in two files should looks like this:
Line in the first file,
Code:
/dw300/data/obe/2019304.L800JR.1910.304

Line in second file,
Code:
1|2019304.L800JR.1910.304

Function output digits can be in one select statement or separate select statement. Output digits will be used in sed or awk command to replace old digits.

I am learning sed or awk to manipulate and edit files. Please help me to figure out how to use either sed or awk to replace these digits in the files. If you can include the logic or simple explanation on your command, it will be greatly appreciated.

My Unix environment is Solaris 11 and ksh shell.

Thanks for your advice and help.
# 2  
Old 10-04-2019
Hi, All;

I also tested using sed to replace the digits in the one line as:

Code:
sed 's/[0-9]\{7\}/2019304/' test_file1.txt,

---replacement is entered at this circumstance.

I got output as:
Code:
 /dw300/data/obe/2019304.L800JR.1909.273

Now the problem is I couldn't replace 1909.273 with the same command logic. I need to figure out how sed can search to the pattern as digit.digit. What is the delimiter I should use here? Please provide your advice. Thanks.

Last edited by duke0001; 10-04-2019 at 05:10 PM..
# 3  
Old 10-04-2019
Try [0-9]\{4\}\.[0-9]\{3\}
The dot needs to be escaped with a backslash, since it is a special character in regular expressions
# 4  
Old 10-05-2019
Scrutinizer:

Thanks for advice. I have tested with your input and command like this:
Code:
sed 's/[0-9]\{7\}/2019304/; s/[0-9]\{4\}\.[0-9]\{3\}/1910.304/' test_file1.txt

Then output is what I need.
Code:
/dw300/data/obe/2019304.L800JR.1910.304

At this time, replacement digits is manually entered. In real situation, I will use parameter for replacement digits.

Thanks a lot for your help. I would welcome other experts to comment on my command to see whether there are better ways to do this work.

Last edited by duke0001; 10-05-2019 at 04:46 PM..
# 5  
Old 10-05-2019
You could try an alternate approach like this, piping the output of your database command into an awk script, who processes it as stdin:

Code:
some_database_command |             
awk '
  NR==FNR {                                    # When reading the first file (stdin in this case)
    if(FNR==2) {                               # When we encounter the second line
      business_day=$1                          # Save the values
      date_digits=$2
    }
    next                                       # Do not process the rest in case of the first file.
  }

  /L800JR/ {                                   # for the two input file if the line contains "L800JR"
    split($NF,F,".")                           # split the last field on the dot character
    $NF=business_day "." F[2] "." date_digits  # Recreate the last field using the second split field
  }

  {
    print > (FILENAME ".new")                  # print the two input files to "filename".new
  }
' - FS=/ OFS=/ file1 FS=\| OFS=\| file2        # Read stdin (-) as the first "file" and use "/" and
                                               # "|" as field separators for the two files respectively


Last edited by Scrutinizer; 10-05-2019 at 07:50 PM..
This User Gave Thanks to Scrutinizer For This Post:
# 6  
Old 10-06-2019
Scrutinizer:

I used this SQL query to fetch business_day and Date_digits into shell variables jdate1 and jdate2. Then use sed to replace them. It worked well.
If using my SQL query, how it work with your awk code?. My query like this:

Code:
jdate1=`sqlplus -s /nolog <<EOF
connect / as sysdba
set pagesize 0 feedback off heading off
SELECT TO_CHAR(DW_ADHOC.F_FIND_LAST_BUSINESS_DAY, 'YYYYDDD') "Last_Business_Day" from dual;
exit
EOF`

This output is : 2019304

Code:
jdate2=`sqlplus -s /nolog <<EOF
connect / as sysdba
set pagesize 0 feedback off heading off
SELECT TO_CHAR(DW_ADHOC.F_FIND_LAST_BUSINESS_DAY, 'YY')||to_char(to_date(TO_CHAR(DW_ADHOC.F_FIND_LAST_BUSINESS_DAY, 'DDD'), 'j'), 'MM')
||'.'||TO_CHAR(DW_ADHOC.F_FIND_LAST_BUSINESS_DAY, 'DDD') "Julian Date Digits"  FROM dual;
exit
EOF`

This output is: 1910.304,

Then I used:
Code:
sed 's/[0-9]\{7\}/'$jdate1'/; s/[0-9]\{4\}\.[0-9]\{3\}/'$jdate2'/' test_file1.txt > test_file1_new.txt

Code:
sed 's/[0-9]\{7\}/'$jdate1'/; s/[0-9]\{4\}\.[0-9]\{3\}/'$jdate2'/' test_file2.txt > test_file2_new.txt

The two files has been replaced with correct Julian date digits in variables. I redirect output to a new file because Solaris do not support sed -i to direct change in place. Then I can overwrite the file with mv back to original name for another Application to use. I want to learn from you how to use your code awk to do the work. Thanks.

Last edited by duke0001; 10-07-2019 at 06:55 PM..
# 7  
Old 10-07-2019
Scrutinizer:

I posted some feedback for your to review. Thanks for your advice and help.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Help with awk or sed Command to Replace Text in Files

Hello Everyone, I have many files like so: file1.txt file2.txt file3.txt Within each file I have many lines of random text separated by commas like so: abcAAA,123,defAA,456777,ghiA,789 jklB,101,mnoBBB,11211,pqrB,13111 stuCC,415,vwxCCCC,161,yzaC,718 I am trying to use SED or AWK to... (4 Replies)
Discussion started by: D3U5X
4 Replies

2. UNIX for Beginners Questions & Answers

sed / awk script to delete the two digits from first 3 digits

Hi All , I am having an input file as stated below 5728 U_TOP_LOGIC/U_CM0P/core/u_cortexm0plus/u_top/u_sys/u_core/r03_q_reg_20_/Q 011 611 U_TOP_LOGIC/U_CM0P/core/u_cortexm0plus/u_top/u_sys/u_core/r04_q_reg_20_/Q 011 3486... (4 Replies)
Discussion started by: kshitij
4 Replies

3. Shell Programming and Scripting

Replace my perl with awk or sed

My code below will print only the email address from all lines. I want to convert it with sed or awk.. also what if i just want to find only filenames. cat LIS_EMAIL | perl -wne'while(/+@+\w+/g){print "$&\n"}' Hoping to extract the filename such us .exe, .bin. From file that has scrambled... (8 Replies)
Discussion started by: invinzin21
8 Replies

4. UNIX for Beginners Questions & Answers

Replace using sed or awk

Hi, Need a help to replace a word if a pattern is found between the delimiters preferably using SED or AWK. below is the sample file that iam dealing with, need to match pattern 'application' if found replace the whole word between the delimiters and also print the lines that don't match.... (1 Reply)
Discussion started by: tech_frk
1 Replies

5. UNIX for Advanced & Expert Users

How to replace last 8 digits?

Hi, How I can replace last 8 ZEROS with 22991231? 19523479811841494432A2013052700000000 19523479811730333980A2013052700000000 19523479811417044397A2013052700000000 19523479811205895810C2013010120130131 A9523479811205895810A2013020120130228 19523479811205895810I2013030120130331... (9 Replies)
Discussion started by: jnrohit2k
9 Replies

6. UNIX for Dummies Questions & Answers

replace 0.00 with awk/sed

Hi I have a problem when i use awk or sed to replace characters in file. For example when I want to replace line like this : 00000O120100512 1.70 1.59 0.00 +7.280 I want to get a new line : 0000000O120100512 1.70 1.59 13.56 +7.280 In ksh : awk... (1 Reply)
Discussion started by: Artur
1 Replies

7. Shell Programming and Scripting

Replace Strings with sed or awk

Hello i need some help with the usage of sed. Situation : 2 textfiles, file.in , file.out In the first textfile which is called file.in are the words for the substitution. Every word is in a new-line like : Firstsub Secondsub Thridsub ... In the second textflie wich is called file.out is... (5 Replies)
Discussion started by: Kingbruce
5 Replies

8. Shell Programming and Scripting

Sed help to replace and then awk

Sed help echo "(200 rows affected)" | sed -e '/\(//p' | sed -e '/\)//p' | awk '{print $1}' I want output as "200" Please help me correct (2 Replies)
Discussion started by: pinnacle
2 Replies

9. Shell Programming and Scripting

Using awk and sed to replace contents

So I am working on command line and I have a file that is spaced by tabs like: one countMe two countMEtoo three COUNTMEthree What I want to do is read in that file, and replace the second column contents with the length of the string in that column. one 7 two 10... (14 Replies)
Discussion started by: silkiechicken
14 Replies

10. Shell Programming and Scripting

Replace tr/sed with awk

There is a service that runs that we call multi-streaming that calls a shell script multiple times simultaneously. In this shell script is the following line: tr '\r' '\n' < $POLLFILE.OUT | sed '/0000000000000016000A/d' > $POLLFILE When I run this manually it produces the desired results, but... (6 Replies)
Discussion started by: philplasma
6 Replies
Login or Register to Ask a Question