Change numbers in flat file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Change numbers in flat file
# 8  
Old 11-28-2013
Quote:
Originally Posted by Akshay Hegde
@ Don Thanks you, I was in hurry forgot to usesprintf and I too noticed expected is 30.0001 and actual result is 29.99999

@faka please post your requirement clearly
Sorry, the real espected value there is 29.99999

---------- Post updated at 04:39 PM ---------- Previous update was at 04:37 PM ----------

Quote:
But, 54.24425 - 24.24426 is +29.99999; not the +30.00001 you said you wanted. And the 3rd value you had in Variable is 24.24225; not the 24.25225 you said you wanted in the output.

Note also that your 1st message in this thread says that your input file is file1.text; not file1.txt. The script I gave you used the filename you specified.
You `re right sorry for the error there.
# 9  
Old 11-28-2013
Quote:
Originally Posted by faka
Sorry, the real espected value there is 29.99999

---------- Post updated at 04:39 PM ---------- Previous update was at 04:37 PM ----------



You `re right sorry for the error there.
OK. So, did the script I gave you provide the output you want? If not, what did it do wrong?
# 10  
Old 11-28-2013
A ksh-93 script for your newest requirement:
Code:
#!/bin/ksh
set -f
Variable=(54.24425 35.24535 25.53231 24.24225 35.24535 28.350132)
i=0
while read line
do
  out=""
  for w in $line
  do
    # require a dot and a digit, and all characters in [-+.0-9]
    if [[ "$w" == *.* && "$w" == *[0-9]* && "$w" != *[^-+.0-9]* ]]
    then
      w2=${Variable[$((i++))]}
      w="$w2 $( printf "%+.5f" $((w2 - w)) )"
    fi
    out="$out${out:+ }$w"
  done
  echo "$out"
done < file1.txt


Last edited by MadeInGermany; 11-28-2013 at 03:36 PM.. Reason: require a dot and a digit
# 11  
Old 11-28-2013
Your latest set of revisions to the 1st message in this thread still has grossly inconsistent output. (Some lines have a trailing space; most do not. Some lines have two spaces as field separators; some have one space. The values in the Variables do not always match the corresponding field in your output file.

It looks like MadeInGermany's script captured most of the intent of your latest changes, but missed two points:
  1. You added a new line at the start of the desired output.
  2. Sometimes you want 5 digits after the decimal point in your output and sometimes you want 6.
The script below consistently only uses a single space as the field separator, does not add any trailing space, adds the new line you added as the first line of your output file, and adjusts the output to have the same number of digits after the decimal point as the longest input value for that line:
Code:
awk -v Variable='54.24425 35.24535  25.53231 24.24225 35.24535 28.350132' '
# dpm(v1, v2) returns the maximum number of digits after the decimal point in
#       the two given values.
function dpm(v1, v2,        n1, n2) {
        n1 = length(v1) - index(v1, ".")
        n2 = length(v2) - index(v2, ".")
        return(n1 > n2 ? n1 : n2)
}
BEGIN { split(Variable, vlist)
        print "New File generated"
}
NF > 2 { p = dpm(vlist[++v], $NF)
        $(NF + 1) = sprintf("%+.*f", p, vlist[v] - $NF)
        $(NF - 1) = vlist[v]
}
1' file1.text > file2.txt

The output it saves in file2.txt is when your latest sample input is in file1.text is:
Code:
New File generated
Data 1
Value One 54.24425 +29.99999
Value Two 35.24535 -10.00000
Other values in the list 25.53231 -59.71304

Data 2
Value One 24.24225 -20.00000
Value Two 35.24535 +10.00000
Other values of new 28.350132 -26.785328

# 12  
Old 11-29-2013
Quote:
Originally Posted by Don Cragun
Your latest set of revisions to the 1st message in this thread still has grossly inconsistent output. (Some lines have a trailing space; most do not. Some lines have two spaces as field separators; some have one space. The values in the Variables do not always match the corresponding field in your output file.

It looks like MadeInGermany's script captured most of the intent of your latest changes, but missed two points:
  1. You added a new line at the start of the desired output.
  2. Sometimes you want 5 digits after the decimal point in your output and sometimes you want 6.
The script below consistently only uses a single space as the field separator, does not add any trailing space, adds the new line you added as the first line of your output file, and adjusts the output to have the same number of digits after the decimal point as the longest input value for that line:
Code:
awk -v Variable='54.24425 35.24535  25.53231 24.24225 35.24535 28.350132' '
# dpm(v1, v2) returns the maximum number of digits after the decimal point in
#       the two given values.
function dpm(v1, v2,        n1, n2) {
        n1 = length(v1) - index(v1, ".")
        n2 = length(v2) - index(v2, ".")
        return(n1 > n2 ? n1 : n2)
}
BEGIN { split(Variable, vlist)
        print "New File generated"
}
NF > 2 { p = dpm(vlist[++v], $NF)
        $(NF + 1) = sprintf("%+.*f", p, vlist[v] - $NF)
        $(NF - 1) = vlist[v]
}
1' file1.text > file2.txt

The output it saves in file2.txt is when your latest sample input is in file1.text is:
Code:
New File generated
Data 1
Value One 54.24425 +29.99999
Value Two 35.24535 -10.00000
Other values in the list 25.53231 -59.71304

Data 2
Value One 24.24225 -20.00000
Value Two 35.24535 +10.00000
Other values of new 28.350132 -26.785328

Don thanks for your help, but I have some problems with the results of the code.
I have added data to the original question to don`t get wrong answers.
In the real example I have a mix of text data in the file and in some cases have one word, there it`s a problem to the script however in the other cases with more than one word works printing only the variable value, not the operation result.
original data
Code:
awk -v Variable='20.90000 21.45000 28.25000 29.35000 1.75000 2.75000 8.30000 9.80000 1.61140 1.66910 0.91310 0.89940 102.88000 101.40000 1.35200 1.36900 12.25710 7.60000 2.58430 2.13270 4508.34000 4318.94000 2.73230 2.73230 16097.33000 1219.47000 1219.47000' '
# dpm(v1, v2) returns the maximum number of digits after the decimal point in
#       the two given values.
function dpm(v1, v2,        n1, n2) {
        n1 = length(v1) - index(v1, ".")
        n2 = length(v2) - index(v2, ".")
        return(n1 > n2 ? n1 : n2)
}
BEGIN { split(Variable, vlist)
        print "New File generated"
}
NF > 2 { p = dpm(vlist[++v], $NF)
        $(NF + 1) = sprintf("%+.*f", p, vlist[v] - $NF)
        $(NF - 1) = vlist[v]
}
1' file1 > file2

file 1
Code:
Values
Data 20.90000
Data1 21.45000
Value 1
Data 28.25000
Data1 29.35000
Value 2
Data 1.75000
Data1 2.75000
Value 3
Data1 8.30000
Data2 9.80000
Values data text in the text with in
Data1 1.61140
Data2  1.66910
Values text in the text
Data1 0.91310
Data2 0.89940
Data in the text
Data1 102.88000
Data2 101.40000
Data in the example
Data1 1.35200
Data2 1.36900
Text example data with text
Data1 12.25710
Data2 7.60000
Data with text
Data1 2.58430
Data2 2.13270
Data of the text
Data1 4508.34000
Data2 4318.94000
Values in data
Data1 2.73300
Data2 2.73300
Data in the text
Data1
Data2 16097.33000
Data in all text
Data1 1219.47000
Data2 1219.47000

Result of the script
Code:
Values
Data 20.90000
Data1 21.45000
Value 1
Data 28.25000
Data1 29.35000
Value 2
Data 1.75000
Data1 2.75000
Value 3
Data1 8.30000
Data2 9.80000
Values data text in the text with 20.90000 +20.90000
Data1 1.61140
Data2  1.66910
Values text in the 21.45000 +21.45000
Data1 0.91310
Data2 0.89940
Data in the 28.25000 +28.25000
Data1 102.88000
Data2 101.40000
Data in the 29.35000 +29.3500000
Data1 1.35200
Data2 1.36900
Text example data with 1.75000 +1.75000
Data1 12.25710
Data2 7.60000
Data with 2.75000 +2.75000
Data1 2.58430
Data2 2.13270
Data of the 8.30000 +8.30000
Data1 4508.34000
Data2 4318.94000
Values in 9.80000 +9.80000
Data1 2.73300
Data2 2.73300
Data in the 1.61140 +1.61140
Data1
Data2 16097.33000
Data in all 1.66910 +1.66910
Data1 1219.47000
Data2 1219.47000

Update: I haved modified the original source of your code and I get some differences.
Code:
awk -v Variable='20.90000 21.45000 28.25000 29.35000 1.75000 2.75000 8.30000 9.80000 1.61140 1.66910 0.91310 0.89940 102.88000 101.40000 1.35200 1.36900 12.25710 7.60000 2.58430 2.13270 4508.34000 4318.94000 2.73230 2.73230 16097.33000 1219.47000 1219.47000' '
# dpm(v1, v2) returns the maximum number of digits after the decimal point in
#       the two given values.
function dpm(v1, v2,        n1, n2) {
        n1 = length(v1) - index(v1, ".")
        n2 = length(v2) - index(v2, ".")
        return(n1 > n2 ? n1 : n2)
}
BEGIN { split(Variable, vlist)
        print "New File generated"
}
NF > 1 { p = dpm(vlist[++v], $NF)
        $(NF + 1) = sprintf("%+.*f", p, vlist[v] - $NF)
        $(NF - 1) = vlist[v]
}
1' file1 > file2

result file
Code:
Values
Data 20.90000 +0.00000
Data1 21.45000 +0.00000
Value 28.25000 +27.25000
Data 29.35000 +1.10000
Data1 1.75000 -27.60000
Value 2.75000 +0.75000
Data 8.30000 +6.55000
Data1 9.80000 +7.05000
Value 1.61140 -1.38860
Data1 1.66910 -6.63090
Data2 0.91310 -8.88690
Values data text in the text with 0.89940 +0.89940
Data1 102.88000 +101.26860
Data2 101.40000 +99.73090
Values text in the 1.35200 +1.35200
Data1 1.36900 +0.45590
Data2 12.25710 +11.35770
Data in the 7.60000 +7.60000
Data1 2.58430 -100.29570
Data2 2.13270 -99.26730
Data in the 4508.34000 +4508.3400000
Data1 4318.94000 +4317.58800
Data2 2.73230 +1.36330
Text example data with 2.73230 +2.73230
Data1 16097.33000 +16085.07290
Data2 1219.47000 +1211.87000
Data with 1219.47000 +1219.47000
Data1  -2.58430
Data2  -2.13270
Data of the  +0.0000
Data1  -4508.34000
Data2  -4318.94000
Values in  +0.0000
Data1  -2.73300
Data2  -2.73300
Data in the  +0.0000
Data1
Data2  -16097.33000
Data in all  +0.0000
Data1  -1219.47000
Data2  -1219.47000

Update 2: I haved update the file1 to replace the space separator inter words with one _ and I get the right value in the output, how I can change the code to accept the space separator between words without limit of words before the number?
Thanks again

---------- Post updated at 03:13 PM ---------- Previous update was at 09:39 AM ----------

Another Problem
Im trying to get the variable value from an assignation of a cat of a file and I cant get the values inside the variable.
In awk -v Variable='value' where value cant load from
I have the variable with the values: example $NUMBERS='20.90000 21.45000 28.25000 29.35000 1.75000 2.75000 8.30000 9.80000 1.61520 1.67080 0.91100 0.89800 103.11000 101.58000 1.35140 1.36840 12.04630 7.73540 2.58430 2.13270 4517.90000 4325.33000 2.73300 2.73300 16097.33000 1225.98000 1225.98000'
If I add $NUMBERS to Variable='$NUMBERS' I get a problem I don`t know why.
If I do echo $NUMBERS I get in STDOUT the correct values.

Thanks in advance

Last edited by faka; 11-29-2013 at 01:14 PM.. Reason: Update
# 13  
Old 11-29-2013
It is no surprise that my script doesn't work with your new data. This is a completely different problem from what you posted in your original message in this thread (in fact from any of the 12 different versions of that message). In your original problem, any line with three or more "words" on the line had a number as the last word and that word was to be modified by the list you supplied in the variable named Variable. So my script copied lines with less than three words to the output without changing them and made the change you requested to the last word on all lines with three or more words.

The fact that you have also changed your last message six times is also a bit disconcerting.

You say that the script isn't working with your new data, but you haven't shown us what output you want. You have 27 values in Variable and you have 30 lines in your sample input file that have a last word that is entirely numeric. How is the script supposed to determine which lines are to be modified? If you can tell me how to recognize lines that need to be modified, it can be made to handle any character you want as a field separator. But you have to CLEARLY describe your input file format and your output file format.

Note also that your 1st set of requirements had all replacement values with 5 digits after the decimal point and you wanted 5 digits to be printed after the decimal point for the values that were calculated and did not add any lines to your output file. Your 2nd set of requirements had some replacement values with 5 digits and some with 6 and you wanted the calculated value to be printed with the same number of digits after the decimal points as were in the replacement value for that line and requested that the line:
Code:
New File generated

be added as the first line to appear in the output file.

Now it looks like you're back to only having 5 digits after the decimal point. Can the script assume that all computed values should be printed with 5 digits, or should it determine the number of digits to be printed based on the number of digits being used in each number involved in the calculation on that line?

Do you want any lines to be added to your output file? If so, what should they be?

When you have a variable that is to be expanded by the shell, it needs to be inside double quotes; not single quotes. I.e.:
Code:
awk -v Variable="$NUMBERS"

not:
Code:
awk -v Variable='$NUMBERS'

But, it would be more efficient to just let awk read the contents of your list of replacement values from the file that you are using to set the variable. What is the name of the file that you are using to initialize the value of $NUMBERS? Inside that file, is each value on a separate line, or are there multiple values on input lines? If there are multiple values on lines, what is the separator between values?

In the future, if you want our help, please post sample input that is representative of the input you want to process, clearly state the algorithm to be used to convert your input to the output you want, and provide sample output that EXACTLY corresponds to the sample input and the algorithm you specify. Constantly changing requirements wastes our time and your time.
# 14  
Old 11-29-2013
Quote:
Originally Posted by Don Cragun
It is no surprise that my script doesn't work with your new data. This is a completely different problem from what you posted in your original message in this thread (in fact from any of the 12 different versions of that message). In your original problem, any line with three or more "words" on the line had a number as the last word and that word was to be modified by the list you supplied in the variable named Variable. So my script copied lines with less than three words to the output without changing them and made the change you requested to the last word on all lines with three or more words.
Don, you are right in all that you sayed but I need the explanation of the script to handle the results and the input file learning at the same time. If I dont have any information about the script I cant understand clearly what are it doing.
Quote:
The fact that you have also changed your last message six times is also a bit disconcerting.
Right also but itīs clear my last message its sometimes difficult to explain all the needs and a bit more in the first message, because the needs are not only one.
Quote:
You say that the script isn't working with your new data, but you haven't shown us what output you want. You have 27 values in Variable and you have 30 lines in your sample input file that have a last word that is entirely numeric.
Don,the last message show the file what gets 27 values from a plain Variable='...0.00000....' and show in the output the 27 values from the variable in the right place that I was queried.
Quote:
How is the script supposed to determine which lines are to be modified? If you can tell me how to recognize lines that need to be modified, it can be made to handle any character you want as a field separator. But you have to CLEARLY describe your input file format and your output file format.
Simple, I want to replace ONLY the DIGITS and DOTS of the original file, not the text.
Quote:
Note also that your 1st set of requirements had all replacement values with 5 digits after the decimal point and you wanted 5 digits to be printed after the decimal point for the values that were calculated and did not add any lines to your output file. Your 2nd set of requirements had some replacement values with 5 digits and some with 6 and you wanted the calculated value to be printed with the same number of digits after the decimal points as were in the replacement value
Don 5 or 6 decimal points are fine I need to know how parametrize this value also.
Quote:
for that line and requested that the line:
Code:
New File generated

be added as the first line to appear in the output file.
I donīt want to add these line.

Quote:
Now it looks like you're back to only having 5 digits after the decimal point. Can the script assume that all computed values should be printed with 5 digits, or should it determine the number of digits to be printed based on the number of digits being used in each number involved in the calculation on that line?
I like to determine the decimal point based on the original value of the file but when the value in the variable are the same of the original or the calculation itīs 0.00000 I donīt want to show any in the output.
Quote:
Do you want any lines to be added to your output file? If so, what should they be?
Not clear the question, can you explain me a bit more?

Quote:
When you have a variable that is to be expanded by the shell, it needs to be inside double quotes; not single quotes. I.e.:
Code:
awk -v Variable="$NUMBERS"

not:
Code:
awk -v Variable='$NUMBERS'

Ok I got it.
Quote:
But, it would be more efficient to just let awk read the contents of your list of replacement values from the file that you are using to set the variable. What is the name of the file that you are using to initialize the value of $NUMBERS?
the file itīs named varnw1
Quote:
Inside that file, is each value on a separate line, or are there multiple values on input lines? If there are multiple values on lines, what is the separator between values?
The file only content the numbers separated by space between values itīs in one line all the values
Quote:
In the future, if you want our help, please post sample input that is representative of the input you want to process, clearly state the algorithm to be used to convert your input to the output you want, and provide sample output that EXACTLY corresponds to the sample input and the algorithm you specify. Constantly changing requirements wastes our time and your time.
Got it, sorry for the mistakes in the thread.
Thank you again!
Regards!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Change numbers

Hallo This is the content of the file 3 4 5 6 7 8 9 10 11 12 And I want the following output 1 2 3 4 5 6 7 (4 Replies)
Discussion started by: thailand
4 Replies

2. Shell Programming and Scripting

Change the numbers

Hi friends, i need a command which can be used to change the values in file. I have file which contain some values. Data_Center_costing=XXXX Financial_loss=XXXX operational_cost=XXX I am not aware about the values of XXXX, They may be 4 digit or less/more than that. but i need these... (12 Replies)
Discussion started by: Nakul_sh
12 Replies

3. Shell Programming and Scripting

Adding (as in arithmetic) to numbers in columns in file, and writing new file with new numbers

Hi again. Sorry for all the questions — I've tried to do all this myself but I'm just not good enough yet, and the help I've received so far from bartus11 has been absolutely invaluable. Hopefully this will be the last bit of file manipulation I need to do. I have a file which is formatted as... (4 Replies)
Discussion started by: crunchgargoyle
4 Replies

4. Shell Programming and Scripting

Trying to take file numbers from a file, pass them to sed to change strings in corresponding lines

I have a bunch of file numbers in the file 'test': I'm trying the above command to change all the instances of "H" to "Na+" in the file testsds.pdb at the line numbers indicated in the file 'test'. I've tried the following and various similar alternatives but nothing is working: cat test |... (3 Replies)
Discussion started by: crunchgargoyle
3 Replies

5. Shell Programming and Scripting

How to change the format of the date column in a flat file?

Hi, i have a flat file namely temp.txt with this data below ID|name|contact_date 101|Kay|2013-12-26 102|let|2013-12-26 I need to modify the date data in the flat file into MM/DD/YYYY HH24:MI:SS format let me know the code for this. Thank you! (5 Replies)
Discussion started by: srikanth_sagi
5 Replies

6. Shell Programming and Scripting

Regarding change in column numbers after some commands

Hi All, I was using some commands to: replace a column by a constant string character awk -v a=CA 'NF>1{ $3=a; print; } ' $line>$line"_1" to copy a column and paste it in another place awk '$5=$2" "$5' $line>$line"_2" to delete the extra columns awk '{for(i=1;i<=NF;i++)... (9 Replies)
Discussion started by: CAch
9 Replies

7. Shell Programming and Scripting

Change numbers in a file, incrementing them

Hello, Here's a file of mine: key1:431 key2:159 key3:998 I need to change these keys to something bigger - and I actually need to shift them all by a range of 3. The output would be: key1:434 key2:162 key3:1001 I can't find the propper sed/awk line that would alter all my... (4 Replies)
Discussion started by: fzd
4 Replies

8. Shell Programming and Scripting

read numbers from file and output which numbers belongs to which range

Howdy experts, We have some ranges of number which belongs to particual group as below. GroupNo StartRange EndRange Group0125 935300 935399 Group2006 935400 935476 937430 937459 Group0324 935477 935549 ... (6 Replies)
Discussion started by: thepurple
6 Replies

9. AIX

how do I change major-minor numbers of disk devices

Good evening ... does anyone of you know how to change major/minor numbers of disk devices ? I had to migrate from raid1 to raid5 and this messed up my ASM cluster - I know which devices should have which IDs to match the content - but I have no idea how to change it. Any help would be... (2 Replies)
Discussion started by: zxmaus
2 Replies

10. Shell Programming and Scripting

script to change filename with numbers

ok, this one is definitely too hard for my shell-script-skills. Hopefully, there is somebody who can help me with this: I have a folder with files in it named 0.ppm 10.ppm 2.ppm ... 5.ppm 50.ppm 55.ppm ... 355.ppm 360.ppm etc. As you will notice, the order in which the files are... (5 Replies)
Discussion started by: silversurfer202
5 Replies
Login or Register to Ask a Question