02-26-2014
Wow, three very nice approaches using three powerful tools (awk integrates best into the existing analysis scripts but the python and perl solutions look very promising too) - I will now test them on different and (much) larger sets of input files/patterns and then probably stick to the fastest one.
Thanks again, your input was extremely helpful and sure motivates me to get deeper into this kind of scripting.
Cheers!
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Experts,
I am a beginner to Unix Shell Scripting
We have source as a flat file which contains CTRL+F character as the delimiter. We need to count the number of records in the file (CTRL+F) to perform file validation
Following command being used:
awk '{cnt+=gsub(//,"&")}END {print cnt}'... (4 Replies)
Discussion started by: srivijay81
4 Replies
2. Shell Programming and Scripting
I couldn't figure out how to use sed or any other shell to do the following. Can anyone help? Thanks.
If seeing a string (e.g., TODAY) in the line,
replace a string in the line above (e.g, replace "Raining" with "Sunny")
and replace a string in the line below (e.g., replace "Reading" with... (7 Replies)
Discussion started by: sprinner
7 Replies
3. Shell Programming and Scripting
I have an xml file that is stripped down to output that looks bacically like;
<!-- TABLEA header -->
<tablea>
some fields
</tablea>
<!-- TABLEB header -->
<!-- TABLEC header -->
<tablec>
some fields
</tablec>
I want to remove the header... (3 Replies)
Discussion started by: Griffs_Revenge
3 Replies
4. Shell Programming and Scripting
Sample file:
This is line one,
this is another line,
this is the PRIMARY INDEX line
l ;
This is another line
The command should find the line with “PRIMARY INDEX” and remove the last character from the line preceding it (in this case , comma) and remove the first character from the line... (5 Replies)
Discussion started by: KC_Rules
5 Replies
5. Shell Programming and Scripting
Hi friends,
This is sed & awk type question.
I have a text file which has numbers spread all over the file. I want to sum the series of numbers whenever i find it and produce an output file with the sum. For example
###start of input text file ####
abc
def
ghi
1
2
3
4
kjld
random... (3 Replies)
Discussion started by: kaaliakahn
3 Replies
6. Shell Programming and Scripting
I am trying to find a line in a file ("Replace_Flag") and replace it with a variable which hold a multi lined file.
myVar=`cat myfile`
sed -e 's/Replace_Flag/'$myVar'/' /pathto/test.file
myfile:
cat
dog
boy
girl
mouse
house
test.file:
football
hockey
Replace_Flag
baseball
... (4 Replies)
Discussion started by: bblondin
4 Replies
7. Shell Programming and Scripting
Hi
I know sed and awk has options to give range of line numbers, but
I need to replace pattern in specific lines
Something like
sed -e '1s,14s,26s/pattern/new pattern/' file name
Can somebody help me in this....
I am fine with see/awk/perl
Thank you in advance (9 Replies)
Discussion started by: dani777
9 Replies
8. Shell Programming and Scripting
Hi,
I have a log file which has sessionids in it, each block in the log starts with a date entry, a block may be a single line or multiple lines. I need to sed (or awk) out the lines/blocks with that start with a date and include the session id.
The files are large at several Gb.
My... (3 Replies)
Discussion started by: andyatit
3 Replies
9. Shell Programming and Scripting
'Hi
I'm using the following code to extract the lines(and redirect them to a txt file) after the pattern match. But the output is inclusive of the line with pattern match.
Which option is to be used to exclude the line containing the pattern?
sed -n '/Conn.*User/,$p' > consumers.txt (11 Replies)
Discussion started by: essem
11 Replies
10. UNIX for Beginners Questions & Answers
In the awk piped to sed below I am trying to format file by removing the odd xxxx_digits and whitespace after, then move the even xxxx_digit to the line above it and add a space between them. There may be multiple lines in file but they are in the same format. The Filename_ID line is the last line... (4 Replies)
Discussion started by: cmccabe
4 Replies
LEARN ABOUT DEBIAN
mic-image-writer
MIC-IMAGE-WRITER(1) General Commands Manual MIC-IMAGE-WRITER(1)
NAME
mic-image-writer - Write a live image to a USB stick
SYNOPSIS
mic-image-writer [options] [image file]
DESCRIPTION
mic-image-writer is a simple yet very helpful tool, it can help you write a live image to a USB stick, it is safer than dd and has a good
progress indicator, it has two work modes, console and GUI, you can explicitly use -c | --console and -g | --gui to force it to enter
console or gui mode, by default, it will smartly decide this automatically.
It just writes an image to the whole USB stick, so the original data on your USB stick will be overwritten, mic-image-manager has a more
powerful GUI tool for this case, it can write a live image to a specified partition, the old data on that partition will keep intact.
OPTIONS
-h, --help show this help message
-c, --console Run in console mode
-g, --gui Run in GUI mode
EXAMPLES
Write a Molib live image to your USB disk:
mic-image-writer your-2.1-final.img
EXIT STATUS
mic-image-convertor returns a zero exist status if it succeeds, otherwise return non-zero and print error message.
AUTHOR
Yi Yang, Anas Nashif, Jianfeng Ding
SEE ALSO
mic-image-creator(1), mic-convertor(1), mic-chroot(1), mic-livecd-iso-to-disk(1), mic-image-manager(1)
perl v5.12.3 2011-05-31 MIC-IMAGE-WRITER(1)