I have a relatively large text file (25,000K) consisting of records of data. For each record, I need to create a new line based on what is already there.
Every record has a block that looks like,
The data tag lines > <ID>, etc, are the same for each record (or should be). The data on the line below the tag varies. I need to make a new field called
> <SOURCE_SOURCE_ID>
That is the data from > <SOURCE> concatenated with > <SOURCE_ID> separated with an underscore.
The record above would look like,
This is quite a bit beyond the things I normally do with shell scripts and I'm not sure where to start. I presume this would be some kind of while read line that looks for > <SOURCE> and captures the next line, looks for > <SOURCE_ID> and captures the next line, makes up the new variable, and makes an insert. All other lines would just be printed. This seems like manipulating an output stream, which I know how to do in cpp, but not in bash.
I decided to try this first. I ran this from the command line adding my file name at the end.
I let it run for a while and it doesn't seem to do anything. There is no change to the file KHHscaffolds_7108.sdf and no output to the terminal. Should I be redirecting to a new file or something like that? Is it just taking a long time to run since it is processing the entire file in one pass?
I also tried,
and this finishes quickly, but the output file is the same as the input.
LMHmedchem
---------- Post updated at 03:21 PM ---------- Previous update was at 03:12 PM ----------
Alright, after reading a bit about awk RS, I see the meaning of RS=
This is the correct usage,
The empty space after RS= fooled me a bit there. This worked very well. I am always amazed at how fast these things can work, even on a large file. I am sure that would have taken me a few hundred lines in cpp and I doubt it would have run nearly as fast.
Morning ..
I have a file with approximately 1000 lines. I want to check that the file contains, for example, 100 lines.
Something like whats given below is ugly. And even if I create a function I have to call it 100 times.
I may need to look through multiple files at times.
Is there a... (4 Replies)
Hi Experts,
Let us take a text file,say items.txt having the following data
jar
bottle
gum
tube
cereal
bag
I want to add the content of items.txt to another file say
#many lines not necessary
ingredients
#many line not necesary
ingredients
I want to append the data in... (3 Replies)
Dear all,
I have a file with two columns - the first column is increasing every 50, the second column is just count (e.g. 5). However, when count is zero, no line is present.
Sample:
1950 7
2000 14
2050 7
2100 13
2150 10
2200 9
2250 7
2300 8
2350 7... (1 Reply)
I have a file that needs to be parsed into multiple files every time there line contains a number 1. the problem i face is the lines are random and the file size is random. an example is that on line 4, 65, 187, 202 & 209 are number 1's so there has to be file breaks between all those to create 4... (6 Replies)
I need some help with adding lines to file and substitute a pattern.
Ok I have a file:
#cat names.txt
name: John Doe
stationed: 1
name: Michael Sweets
stationed: 41
.
.
.
And would like to change it to:
name: John Doe
employed
permanently
stationed: 1-office (7 Replies)
Hi,
I have a requirement to search for a string in a large log file along with few lines before and after the the string. The following script was sufficient to search such an entry.
STRING_TO_GREP="$1"
FILE_TO_GREP="$2"
NUMBER_OF_LINES_BEFORE=$3
NUMBER_OF_LINES_AFTER=$4
for i in `grep... (3 Replies)
Hi all,
I have a positional text file that comes from some source application. Before it is processed by destination application I have to add some header (suffix) to every record(line) in the file.
e.g.
Actual File
...............
AccountDetails
AcNO Name Amount
1234 John 26578
5678... (3 Replies)
Hello,
I need to select the 3 lines above as well as below a search string, including the search string.
I have been trying various combinations using sed command without any success.
Can anuone help please.
Thanking (2 Replies)