Isolate and Extract a Pattern Substring (Digits Only)
Hi guys,
I have a text file report generated from egrepping multiple files.
The text files themselves are obtianed after many succesive refinements, so they contain already the desired number, but this is surrounded by unwanted characters, newlines, spaces, it is not always at the start of the line, as can be seen in sample below:
$ egrep [0-7]\{7}
dte--0072.txt:1596223
dte--0073.txt:1560379
dte--0075.txt:!!! !!�!!!!!! !!�! !�� !!!!!!!!!! !!?! 1623749
dte--0076.txt:1596014
dte--0077.txt: 1791213
dte--0078.txt: 1767933
dte--0079.txt:_____1777023
What I need to generate is a clean report that looks like this:
desired clean report
dte--0072.txt:1596223
dte--0073.txt:1560379
dte--0075.txt:1623749
dte--0076.txt:1596014
dte--0077.txt:1791213
dte--0078.txt:1767933
dte--0079.txt:1777023
How can I do this? I am too new to regex, so I was hoping maybe someone can help with negating the expression, or a sed oneliner.
Note: The string is always of the same pattern:
- digits only
- the same number of digits (in this report a 7digit number)
- there are no spaces or any other signs between the digit pattern, it is like 1234567
- only need the digits (nothing before or after the number pattern)
- would prefer to operate the command directly on the multiple files, as in the egrep, so the report file is already preserving the filenames on the same line with the contained number string
Can you please help?
Thanks!
Last edited by netfreighter; 03-22-2009 at 04:58 AM..
Try this. It uses sed to remove everything after the colon that's not a digit.
Ken, many thanks for that.
I did try the command, and it removes most characters trailing AFTER the number, except "_" underscore; however the characters BEFORE the digit pattern are still there, as can be seen from sample output:
So now what is left is to remove the noise before the number.
Perhaps there is a way to rather extract "only just what is a 7 digit pattern", rather than provide for all the possible trailing or preceding symbols? The files are obtained from non-English languages so there may be funny non-printing symbols that need to be removed.
I am surprised that the last line got through. There must be something there that I'm not seeing.
But yes, we can pick out explicitly 7 characters. In fact, if all the filenames are similar enough you can drop egrep and use only sed. Try this:
The escaped parentheses capture and copy .*txt: to \1 and [0-9]\{7\} to \2. The rest is dropped.
SED command or REGEX to extract only the number from a textfile
Hello and thank you both for the answers.
I apologise for the delay in my answer but been overwhelmed with tasks lately. It is now time to turn to this project and finish off.
I have tried both commands, but they only partially remove the unwanted characters.
I also discovered that they are not question marks, those are just replacement characters because the console terminal does not have enough characters to display the actual signs.
So, first I run
then
and the output file is showing matching lines that are only cleaned BEFORE the number, while after the number trailing characters remain:
repfile:dte--0055.txt:?! !?!!!!?! !!?! !!?! !!?!! !!!!!!!!! 001431616
repfile:dte--0056.txt:?? !?!!!!?! !???!!!?! 001548532______
repfile:dte--0057.txt:0015817
repfile:dte--0058.txt:!!!! ??!?? !!?! !!? )??? !!!!!!?! 001438615
repfile:dte--0059.txt:0016327
repfile:dte--0060.txt:!)!> !?!!!!?? ??!? !!!! ??! ??!? !?! 001467161
I opened the file in TextEditapp in MacOSX and I see strange characters like
"ª ´!!! ´ï)!ª´´´´(ª ´? ´!´ ______ " (may not show correctly but it's symbols that look like superscript and foreign letters)
Since these nnumbers are going to be extracted from multilingual files, non-English writing, thenn it is hard to predict what symbols are to be encountered, so to cleanup the number I gues I would need a SED command or REGEX to extract only the number from a textfile.
Some command that negates "anything else that is NOT a 9-digit number to be removed"
I have a feeling that egrep or a regular rexpression could do that, but do not know where to look.
Maybe some character classes like [punct] can be used?
What the final result should look like:
repfile:0057.txt:001581743
repfile:0058.txt:001438615
repfile:0059.txt:001632790
repfile:0060.txt:001467161
Where that number is the only such number available, one per each textfile.
Thanks!
Last edited by netfreighter; 04-03-2009 at 11:31 AM..
I also tried
but $1 or $2 only map to "words", whichmeans those digit squences that have some separator or blank, therefore only some or the desired numbers are filtered out, the others that are surrounded by grabage characters without a break or space remain unfiltered.
Any idea how to awk the desired PATTERN only?
How can I extract digits at the end of a string in UNIX shell scripting or perl?
cat file.txt
abc_d123_4567.txt
A246_B789.txt
B123cc099.txt
a123_B234-012.txt
a13.txt
What can I do here? Many thanks.
cat file.txt | sed "s/.txt$//" | ........
4567
789
099
012
13 (11 Replies)
Hello,
I have a log file with logs such as
01/05/2017 10:23:41 : file.log.38: database error, MODE=SINGLE, LEVEL=critical, STATE: 01170255 (mode main
how can i use perl to extract the 8-digit number below from the string
01170255
Thanks (7 Replies)
I would like to extract a digit from $0 starting 2,30 to 3,99 or 2.30 to 3.99
Can somebody fix this?
awk --re-interval '{if($0 ~ /{1}{2}/) {print FILENAME, substr($0,index($0,/{1}{2}/) , 4)}}'input
abcdefg sdlfkj 3,29 g. lasdfj
alsdfjasl 2.86 gr. slkjds sldkd
lskdjfsl sdfkj kdjlksj 3,34 g... (4 Replies)
Hello,
I need to construct a pattern to match the below string (especially the timestamp at the beginning)
20101222100436_temp.dat
The below pattern works _temp.dat
However I am trying find if there are any other better representations.
I tried {14}, but it did not work.
I am on... (5 Replies)
Hi all,
i have such string stored in a variable
var1 = 00000120
i want the o/p
var1 = 120
is it possible to have such o/p in ksh/bash ...
thanx in advance for the help
sonu (3 Replies)
Hi,
I've been searching for a quick way to do this with sed, but to no avail.
I have a file containing a long series of (windows) file paths that are separated by the pattern '@'. I would like to extract each file path so that I can later assign a variable to each path.
Here is the file:... (2 Replies)
I have a shell script that uses wget to grab a bunch of html from a url.
URL_DATA=`wget -qO - "$URL1"`
I now have a string $URL_DATA that I need to pull a substring out of..say I had the following in my string
<p><a href="/scooby/929011567.html">Dog pictures check them out! -</a><font... (3 Replies)
I have a log file that ends in a ".xxx" where xxx are digits but I don't necessarily know what digits they are. The log file rotates automatically and is auto-incrementing - starting at .001.
So the example would be:
file-name.005
If the file ends in .005 and the log rotates, it logically... (2 Replies)