Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Removal of extra spaces in *.log files to allow extraction of frequencies

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removal of extra spaces in *.log files to allow extraction of frequencies
# 8  
Old 04-21-2013
You can put these two lines in any shell script, and it will correct the formatting:
Code:
$ cat atoms.sh
sed -e "s/ Atom  AN/Atom AN/" -e "s/^     /   /" atoms.txt > /tmp/temp.x
mv /tmp/temp.x atoms.txt

I changed the first substitution a little to ensure that the correction is only applied once. So if the log file gets corrected once, and then new text is appended to the end, the previously corrected section will not get "corrected again".

Are you sure there are no other lines that start with five blanks? As the script is right now, any line that starts with five blanks will be switched to start with three blanks.
# 9  
Old 04-21-2013
Actually, there are lines throughout the log file that have five spaces prior to the data. I was wondering if there was a way to set the initial search string such as the __Atom__AN where the underscores are spaces and then successive line corrections for the atom numbers that have five spaces in front of them.
# 10  
Old 04-21-2013
What do the other lines look like that that have five spaces at the start? If you are referring to the data lines you already sent, there is no problem (I think), because my understanding is you want to make those lines to end up with three spaces. And that is exactly what the sed command does.

In other words, are there any lines that start with five spaces that you do NOT want to change? If so, then could you post what they look like?

There is a way to do what you ask (start the changes at the line with ATOM AN) but it is more complex, and no point doing that if not needed.
# 11  
Old 04-21-2013
The following are just a few examples:

Code:
Integral symmetry usage will be decided dynamically.
     1530128 words used for storage of precomputed grid.
Keep R1 ints in memory in canonical form, NReq=44454758.

Code:
 ITU= 1 0 0
     Eigenvalues --- 0.00208 0.00230 0.01092 0.01339 0.01389
     Eigenvalues --- 0.02487 0.03769 0.03800 0.04351 0.10382
     Eigenvalues --- 0.14831 0.15999 0.16000 0.16012 0.16016
     Eigenvalues --- 0.34023 0.34720 0.48005 0.53596 0.85053
     Eigenvalues --- 1000.00000

Code:
     1 Zn 0.000000
     2 O 2.267745 0.000000
     3 O 3.157518 5.392657 0.000000
     4 H 2.183471 4.414552 0.979583 0.000000
     5 C 3.739606 5.989831 1.243183 1.863198 0.000000

# 12  
Old 04-22-2013
Here is an alternative to preserve the other lines starting with five spaces. The following two lines can be added to any shell script. The first substitution is the same as before (_Atom__AN -> Atom_AN). The second substitution does the 5->3 blank change, starting with the Atom_AN line (nothing happens) and ending with the next line that does NOT start with a blank. "^ " means five spaces at start of line. "[^ ]" means single character that is NOT a blank.

Code:
$ cat ./atoms.sh
sed "s/ Atom  AN /Atom AN /" atoms.txt > /tmp/t1.x
sed "/Atom AN /,/^[^ ]/ {s/^     /   /}" /tmp/t1.x > atoms.txt

Code:
$ cat atoms.txt
1 2 3
A A A
Frequencies -- 73.6186 95.0148 177.9910
Red. masses -- 2.5506 3.7026 3.3055
Frc consts -- 0.0081 0.0197 0.0617
IR Inten -- 7.9374 9.9457 8.1890
  Atom  AN X Y Z X Y Z X Y Z
     1 30 0.00 0.00 0.02 -0.07 -0.08 0.00 0.04 -0.02 0.00
     2 8 0.00 0.00 -0.11 0.28 0.09 0.00 0.12 -0.01 0.00
     3 8 0.00 0.00 0.25 -0.06 0.00 0.00 -0.13 0.24 0.00
4 5 6
A A A
Frequencies -- 231.0559 251.4928 255.6673
Red. masses -- 2.8839 1.1192 1.0754
Frc consts -- 0.0907 0.0417 0.0414
IR Inten -- 82.8162 113.2879 160.2404
  Atom  AN X Y Z X Y Z X Y Z
     1 30 0.10 -0.01 0.00 0.00 0.00 0.00 0.00 0.00 -0.01
     2 8 -0.03 0.13 0.00 0.00 0.00 0.06 0.00 0.00 0.06
     3 8 -0.18 -0.13 0.00 0.00 0.00 0.05 0.00 0.00 0.01
Integral symmetry usage will be decided dynamically.
     1530128 words used for storage of precomputed grid.
Keep R1 ints in memory in canonical form, NReq=44454758.
     1 Zn 0.000000
     2 O 2.267745 0.000000
     3 O 3.157518 5.392657 0.000000
     4 H 2.183471 4.414552 0.979583 0.000000
     5 C 3.739606 5.989831 1.243183 1.863198 0.000000
ITU= 1 0 0
     Eigenvalues --- 0.00208 0.00230 0.01092 0.01339 0.01389
     Eigenvalues --- 0.02487 0.03769 0.03800 0.04351 0.10382
     Eigenvalues --- 0.14831 0.15999 0.16000 0.16012 0.16016
     Eigenvalues --- 0.34023 0.34720 0.48005 0.53596 0.85053
     Eigenvalues --- 1000.00000

Code:
$ ./atoms.sh

Code:
$ cat atoms.txt
1 2 3
A A A
Frequencies -- 73.6186 95.0148 177.9910
Red. masses -- 2.5506 3.7026 3.3055
Frc consts -- 0.0081 0.0197 0.0617
IR Inten -- 7.9374 9.9457 8.1890
 Atom AN X Y Z X Y Z X Y Z
   1 30 0.00 0.00 0.02 -0.07 -0.08 0.00 0.04 -0.02 0.00
   2 8 0.00 0.00 -0.11 0.28 0.09 0.00 0.12 -0.01 0.00
   3 8 0.00 0.00 0.25 -0.06 0.00 0.00 -0.13 0.24 0.00
4 5 6
A A A
Frequencies -- 231.0559 251.4928 255.6673
Red. masses -- 2.8839 1.1192 1.0754
Frc consts -- 0.0907 0.0417 0.0414
IR Inten -- 82.8162 113.2879 160.2404
 Atom AN X Y Z X Y Z X Y Z
   1 30 0.10 -0.01 0.00 0.00 0.00 0.00 0.00 0.00 -0.01
   2 8 -0.03 0.13 0.00 0.00 0.00 0.06 0.00 0.00 0.06
   3 8 -0.18 -0.13 0.00 0.00 0.00 0.05 0.00 0.00 0.01
Integral symmetry usage will be decided dynamically.
     1530128 words used for storage of precomputed grid.
Keep R1 ints in memory in canonical form, NReq=44454758.
     1 Zn 0.000000
     2 O 2.267745 0.000000
     3 O 3.157518 5.392657 0.000000
     4 H 2.183471 4.414552 0.979583 0.000000
     5 C 3.739606 5.989831 1.243183 1.863198 0.000000
ITU= 1 0 0
     Eigenvalues --- 0.00208 0.00230 0.01092 0.01339 0.01389
     Eigenvalues --- 0.02487 0.03769 0.03800 0.04351 0.10382
     Eigenvalues --- 0.14831 0.15999 0.16000 0.16012 0.16016
     Eigenvalues --- 0.34023 0.34720 0.48005 0.53596 0.85053
     Eigenvalues --- 1000.00000

# 13  
Old 04-22-2013
When I put both code strings into a script file, the part of the code after 'sed' is a red font color. I was under the impression that if the code displays red then it is 'broken'? I may not understand the color schemes very clearly. The beginning of my scripts start with #!/bin/bash and when I source the script it doesn't do anything. I put the actual log file name into the code and I also tried *.log but neither produced the modified log file.
# 14  
Old 04-22-2013
Code:
if the code displays red then it is 'broken'?

Don't worry about the editor color schemes. Your editor doesn't know anything about sed.
Code:
when I source the script it doesn't do anything

You need to be more clear. Please copy / paste exactly what you did, with the code tags.
Code:
I also tried *.log

Do not do that. Just use the name of the log file.
Code:
neither produced the modified log file

Again, you need to copy / paste what you did. That's the only way I can tell anything.

A word of advice: Make a copy of the log file, and work on the copy. If you make a mistake, you can damage the log file. Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep string causes extra spaces

Hello, I have an xml file and my aim is to grab each line in keywords file and search the string in another file. When keyword is found in xml file,I expect the script to go to previous line in the xml file and grab the string/value between two strings. It's almost working with an error. tab... (6 Replies)
Discussion started by: baris35
6 Replies

2. Shell Programming and Scripting

Removing extra unwanted spaces

hi, i need to remove the extra spaces in the filed. Sample: abc~bd ~bkd123 .. 1space abc~badf ~bakdsf123 .. 2space abc~bqed ~bakuowe .. 3space output: abc~bd ~bkd123 .. 1space abc~badf~bakdsf123 .. 2space abc~bqed~bakuowe .. 3space i used the following command, (2 Replies)
Discussion started by: anshaa
2 Replies

3. Shell Programming and Scripting

What extra Parameters I can use for archiving log files

Hello All, I have developed a script which takes following parameter from the input file to archive log files 1)Input Path 2)File pattern(*.csv) 3)Number of days(+1) Following is the algorithm of my script Read the input file go to that path and search for particular n days older... (3 Replies)
Discussion started by: mitsyjohn
3 Replies

4. Shell Programming and Scripting

Remove of extra spaces from the trailing

HI, I need the help from the experts like I have created one file with text like: a b c d e f g h i j k l So my question is that i have to write the script in which like in the first sentence it will take only one space after d and remove all the extra space in the end.I dont... (8 Replies)
Discussion started by: bhanudhingra
8 Replies

5. Shell Programming and Scripting

Help with removal of spaces between operators and operands

Hi I'm trying to remove blank spaces in expressions and function calls.. Consider the following example printf ("Hello"); a = a + b; I'm trying to eliminate space in between the function name and the opening brace. And also eliminate space between operators and operands.. That is, I'm... (19 Replies)
Discussion started by: abk07
19 Replies

6. Shell Programming and Scripting

Help with removal of blank spaces from the second field!

Hi everyone.. I'm trying to eliminate multiple whitespaces from a file.. I must make use of shell script to eliminate whitespaces.. Take a look at the sample file 1 int main() 2 { 3 int a,b; 4 printf("Enter the values of a and b"); 5 scanf("%d%d",&a,&b); 6 if(a>b) ... (6 Replies)
Discussion started by: abk07
6 Replies

7. Shell Programming and Scripting

Help with removal of blank spaces in a file

Hello.. I have a text file. I want to remove all the blank spaces(except tab) from the file.. I tried using sed command as shown below sed 's/ //g' file1 But the problem with the above command is that it also eliminates 'tab' which is between the columns.. For example if the contents... (7 Replies)
Discussion started by: abk07
7 Replies

8. UNIX for Dummies Questions & Answers

selective removal of blank spaces in string

Hi, I'm a newbie to shell scripting and I have the following problem: I need all spaces between two letters or a letter and a number exchanged for an underscore, but all spaces between a letter and other characters need to remain. Searching forums didn't help... One example for clarity: ... (3 Replies)
Discussion started by: Cpt_Cell
3 Replies

9. Shell Programming and Scripting

How to remove extra spaces from a string??

Hi, I have a string like this and i want to remove extra spaces that exists between the words. Here is the sentence. $string="The small DNA genome of hepadnaviruses is replicated by reverse transcription via an RNA intermediate. This RNA "pregenome" contains ... (2 Replies)
Discussion started by: vanitham
2 Replies

10. UNIX for Dummies Questions & Answers

To remove the extra spaces in unix

Hi... I am quite new to Unix and would like an issue to be resolved. I have a file in the format below; 4,Reclaim,ECXTEST02,abc123,Harry Potter,5432 6730 0327 5469,0603,,MC,,1200,EUR,sho-001,,1,,,abc123,1223 I would like my output to be as follows; 4,Reclaim,ECXTEST02,abc123,Harry... (4 Replies)
Discussion started by: Sho
4 Replies
Login or Register to Ask a Question