How can I extract digits at the end of a string in UNIX shell scripting?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How can I extract digits at the end of a string in UNIX shell scripting?
# 1  
Old 11-21-2018
How can I extract digits at the end of a string in UNIX shell scripting?

How can I extract digits at the end of a string in UNIX shell scripting or perl?
Code:
cat file.txt
abc_d123_4567.txt
A246_B789.txt
B123cc099.txt
a123_B234-012.txt
a13.txt


What can I do here? Many thanks.
Code:
cat file.txt | sed "s/.txt$//" | ........
4567
789
099
012
13


Last edited by vbe; 11-21-2018 at 04:36 AM.. Reason: code tags
# 2  
Old 11-21-2018
Code:
$ < file.txt sed 's/.*[^0-9]\([0-9][0-9]*\).txt$/\1/'
4567
789
099
012
13

or
Code:
$ sed 's/.*[^0-9]\([0-9][0-9]*\).txt$/\1/' file.txt
4567
789
099
012
13

Andrew
This User Gave Thanks to apmcd47 For This Post:
# 3  
Old 11-21-2018
Thanks for your reply

------ Post updated at 02:43 AM ------

And I want to increment a Variable but it doesnt work.
How can extract digits at the end of a string and increment variable value by 1 in Unix shell scripting or perl?
Thanks again.

e.g. a123_B234-012.txt
Code:
echo "a123_B234-012.txt\c" | sed "s/.txt$//" | sed "s/[0-9]\{1,10\}$//" && echo "`echo "a123_B234-012.txt" | sed 's/.*[^0-9]\([0-9][0-9]*\).txt$/\1/'`+1\c"  && echo ".txt"

I want the output:
Code:
a123_B234-013.txt

------ Post updated at 03:13 AM ------

e.g.
Code:
cat file.txt
abc_d123_4567.txt
A246_B789.txt
B123cc099.txt
a123_B234-012.txt
a13.txt


Output
Code:
abc_d123_4568.txt
A246_B790.txt
B123cc100.txt
a123_B234-013.txt
a14.txt


Last edited by mingch; 11-21-2018 at 11:12 PM.. Reason: Code tags
# 4  
Old 11-22-2018
A few things:

Quote:
Originally Posted by mingch
e.g. a123_B234-012.txt
Code:
echo "a123_B234-012.txt\c" | sed "s/.txt$//" | sed "s/[0-9]\{1,10\}$//" && echo "`echo "a123_B234-012.txt" | sed 's/.*[^0-9]\([0-9][0-9]*\).txt$/\1/'`+1\c"  && echo ".txt"

I want the output:
Code:
a123_B234-013.txt

The first is: any command involving sed .... | sed ... is (99.99999% of the times) wrong: sed is perfectly capable of handling several commands in a script with loops, branches and everything a programming language offers. Use that instead of pipes.

Second: you don't need sed at all for this. You can do this with shell-internal variable manipulation. Notice that calling a program is "expensive" in terms of time and resources: the OS needs to load the program, start a sub-process, and so on. Variable manipulation happens within the shell and foregoes all this. The worst thing one sees all the time is:

Code:
while read LINE ; do
     echo "$LINE" | sed '.....'
     ....
done

If your file is long and you replace the sed with variable manipulation it might be ten more lines to write but it will run in 1% of the time.

Parameter expansion (this is the "official" name for the mechanism) works like this:

Code:
echo ${variable}

This you already know: ${variable} produces the content unaltered.

Code:
echo ${variable%pattern} ; echo ${variable%%pattern}

This cuts off <pattern> from the end of the variables contents. This is basically the same as sed "s/.txt$//". Try it out:

Code:
$ myfile="abc_d123_4567.txt"
$ echo ${myfile%.txt}
abc_d123_4567

You may wonder what the difference between ${variable%pattern} and ${variable%%pattern} is: it is "longest match" ("%%") and "shortest match" ("%"). Suppose we want to cut off not only ".txt" but any extension. We could do so by:

Code:
$ myfile="abc_d123_4567.txt"
$ echo ${myfile%.*}
abc_d123_4567

But a filename could contain several dots like this, then there would be a difference between longest and shortest match:
Code:
myfile="abc.d123.4567.txt"
$ echo ${myfile%.*}
abc.d123.4567
$ echo ${myfile%%.*}
abc

Notice that all these manipuations do NOT change the value of the variable! They just change what is displayed. If you want the effect to be lasting you would have to do:

Code:
myfile="abc.d123.4567.txt"
$ myfile=${myfile%.*}
echo ${myfile}
abc_d123_4567

There is another expansion which cuts off not from the end but from the beginning of a string: ${variable#pattern} and ${variable##pattern}. It works the same way as the "%" otherwise. For instance, you often need to split a pathname ("/some/path/to/a/file.name") into the path and the filename:

Code:
myfile="/some/path/to/a/file.name"
$ echo "The filename is: ${myfile##*/}"
The filename is: file.name
echo "The path to the file is: ${myfile%/*}"
The path to the file is: /some/path/to/a

There are even more of these expansions which can selectively change patterns within a string, conditionally assign values to variables and more. I hope to have piqued your interest. You are probably eager now to try this to solve your problem yourself.

I hope this helps.

bakunin
These 4 Users Gave Thanks to bakunin For This Post:
# 5  
Old 11-22-2018
I am a beginner in shell script, thank you very much indeed.
# 6  
Old 11-22-2018
For your latest request - increment the last number in the file names - , on top of the "parameter expansion" proposed by bakunin, "arithmetic expansion" could help. Its syntax is (( variable / constant / operator ... )), and it can be mixed with other expansions. For the first two of your sample data, this might work:



Code:
#!/bin/bash
while read FN
  do    TMP=${FN%.txt}
        TMP=${TMP##*[A-Za-z_]}
        echo ${FN%_*}_$((++TMP)).${FN#*.}
   done < file.txt
abc_d123_4568.txt
A246_790.txt

As the structure of the other names is different, above simple approach will fail, and it needs to be adapted, but mayhap you have a good starting point from it.

Last edited by RudiC; 11-22-2018 at 09:25 AM..
This User Gave Thanks to RudiC For This Post:
# 7  
Old 11-22-2018
I tried to create the following scripts but it can't increment the last number from 012 to 013, anyone help? Many thanks.

Code:
#!/bin/sh
cat file.txt | while read FN
  do
    FNPATH=${FN%/*}
    FNFILE=${FN##*/}
    BNFILE=`echo "${FNFILE%.txt}" | sed "s/[0-9]\{1,10\}$//"`
    OLDNUM=`echo "${FNFILE%.txt}" | sed 's/.*[^0-9]\([0-9][0-9]*\)/\1/'`
    NEWNUM=$(($OLDNUM+1))
    NEWFILE="${BNFILE}${NEWNUM}.txt"
   echo "OLD file is ${FN}, NEW file is $FNPATH/${NEWFILE}"
  done

Output
Code:
OLD file is /tmp/path1/abc_d123_4567.txt, NEW file is /tmp/path1/abc_d123_4568.txt
OLD file is /tmp/path2/A246_B789.txt, NEW file is /tmp/path2/A246_B790.txt
OLD file is /tmp/path12/B123cc099.txt, NEW file is /tmp/path12/B123cc100.txt
OLD file is /tmp/path12/a123_B234-012.txt, NEW file is /tmp/path12/a123_B234-13.txt
OLD file is /tmp/path13/a13.txt, NEW file is /tmp/path13/a14.txt
OLD file is /tmp/path13/1.txt, NEW file is /tmp/path13/2.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract n-digits from string in perl

Hello, I have a log file with logs such as 01/05/2017 10:23:41 : file.log.38: database error, MODE=SINGLE, LEVEL=critical, STATE: 01170255 (mode main how can i use perl to extract the 8-digit number below from the string 01170255 Thanks (7 Replies)
Discussion started by: james2009
7 Replies

2. UNIX for Dummies Questions & Answers

Extract directory name from the full directory path in UNIX using shell scripting

My input is as below : /splunk/scrubbed/rebate/IFIND.REBTE.WROC.txt /splunk/scrubbed/rebate/IFIND.REBTE.WROC.txt /splunk/scrubbed/loyal/IFIND.HELLO.WROC.txt /splunk/scrubbed/triumph/ifind.triumph.txt From the above input I want to extract the file names only . Basically I want to... (5 Replies)
Discussion started by: IshuGupta
5 Replies

3. Shell Programming and Scripting

How to extract text from STRING to end of line?

Hi I have a very large data file with several hundred columns and millions of lines. The important data is in the last set of columns with variable numbers of tab delimited fields in front of it on each line. Im currently trying sed to get the data out - I want anything beetween :RES and... (4 Replies)
Discussion started by: Manchesterpaul
4 Replies

4. Shell Programming and Scripting

How to add trailer record at the end of the flat file in the unix ksh shell scripting?

Hi, How to add trailer record at the end of the flat file in the unix ksh shell scripting can you please let me know the procedure Regards Srikanth (3 Replies)
Discussion started by: srikanth_sagi
3 Replies

5. Shell Programming and Scripting

BASH: remove digits from end of string

Hi there, im sure this is really simple but i have some strings like this e1000g123001 e1000g0 nge11101 nge3and i want to create two variables ($DRIVER and $INSTANCE). the first one containing the alpha characters that make up the first part of the string, e.g. e1000g or nge and the... (9 Replies)
Discussion started by: rethink
9 Replies

6. Shell Programming and Scripting

help with Scripting - trying to search for string and extract next few characters

Hi I am new to world on unix scripting so any assistance would be gratefully appreciated, I am trying to write a script which reads through a file, reads in line by line, searches for a pattern, copies string after it and then to do a search and replace elsehwere in the line, so the... (7 Replies)
Discussion started by: LonJ_80
7 Replies

7. Shell Programming and Scripting

extract digits from a string in unix

Hi all, i have such string stored in a variable var1 = 00000120 i want the o/p var1 = 120 is it possible to have such o/p in ksh/bash ... thanx in advance for the help sonu (3 Replies)
Discussion started by: sonu_pal
3 Replies

8. Shell Programming and Scripting

Search and remove digits (if exist) from end of the string

Hi Experts, Here is what I am trying to do. 1) say I have a file with below strings database1 database2 database3 data10gdb1 data10gdb2 databasewithoutdigit 2) I want to get the below output. (- if there is any digit at the end of the string, I need to remove it) (- Any... (3 Replies)
Discussion started by: shail_boy
3 Replies

9. Shell Programming and Scripting

Need UNIX shell scripting end to end information

Hi, I would like to learn shell scripting in UNIX. Can any one please give me the support and share the information/documents with me. If any documents please post it to aswanikumar_nimmagadda@yahoo.co.in Thanks in advance...!!! (3 Replies)
Discussion started by: aswani_n
3 Replies

10. Shell Programming and Scripting

Extract digits at end of string

I have a string like xxxxxx44. What's the best way to extract the digits (one or more) in a ksh script? Thanks (6 Replies)
Discussion started by: offirc
6 Replies
Login or Register to Ask a Question