The good news is: you do not need sed for this at all. In fact, as long as you are not manipulating a data stream you usually should not use it at all and use "variable expansion" instead. It is the shells equivalent of substr(), trim(), strtok() and similar functions found in other high-level-languages.
The first thing you want to do is to cut off the extension ".csv" from the filename:
This works the following way: ${variable%pattern} cuts off the pattern from the end of the variables content if it is found. "pattern" is everything you could use as a filename pattern in the shell: "*" would mean any number of any characters (like in "file*"), "?" means any one character, etc. Notice also that this only DISPLAYS the resulting string, it does NOT CHANGE the content of the variable! If you want to change it lastingly you need to assign the new value:
Code:
file="${file%.csv}"
Since you want to cut off the last two zeroes either we can do that in a single run:
The next thing we want is the date to be cut off but we need to preserve it, so we assign a new variable with a copy of "$file"s contents, but with everything up to the last underscore removed. For this there is another expansion which works like the one i showed you but it cuts off from the beginning instead of the end:
Code:
file="JAN_DAT_TES1_2018071814"
echo "${file##*_}"
Notice that i used "##" instead of "#". "##" and its companion "%%" cut off the longest possible match whereas "#" and "%" cut off the shortest possible match. That means:
In this case: cut off the other parts with what i have showed you:
Code:
for file in /some/where/*csv ; do
path="${file%/*}" # remove the filename, leaving the path "/some/where/JAN_DAT_TES1_201807181400.csv" => "/some/where"
file="${file%00.csv}" # remove the extension and the last "00" "/some/where/JAN_DAT_TES1_201807181400.csv" => "/some/where/JAN_DAT_TES1_2018071814"
date="${file##*_}" # extract the date: "/some/where/JAN_DAT_TES1_2018071814" => "2018071814"
file="${file%_${date}}" # remove the date from the filename: "/some/where/JAN_DAT_TES1_2018071814" => "/some/where/JAN_DAT_TES1"
file="${file##*_}" # remove everything up to the last "_" from the filename: "/some/where/JAN_DAT_TES1" => "TES1"
echo "my file is: $path/$file , my date is: $date"
done
Thank you!!. It helped.
If you could pl tell me what was the error in the sed code, it will help me to learn better.
Code:
sed -e 's/JAN_DAT_(.*\)\_([0-9]\{10\}\).*.csv/\1 \2/g'
The first thing is: sed-patterns are always "greedy", which means they match the longest possible part of the line. A pattern like A.*B will NOT match the part marked bold but the WHOLE of the following string:
Code:
AXYZBBBBBBBBBBBBBBB
Therefore /JAN_DAT_(.*\) is already at least problematic if not wrong. What you want to do is to match up to the next underscore, so you better exclude the underscore from the matching set:
Code:
/JAN_DAT_\([^_]*\)_....
In the example above, you would not write A.*B but A[^B]*B ("A, followed by any number of non-Bs, followed by B") to match only from A to the next following B.
The second thing was the unescaped "(", which i corrected (marked in red). The same goes for the second opening bracket, before the "[0-9]". You see, sed uses "BRE", "basic regular expressions", unlike i.e. awk, which uses "ERE"s, "extended regular expressions". In BRE you have to escape brackets whereas in ERE you don't. I do not know which sed you use, some are able to use EREs too, but i wouldn't recommend it, even if they can. It is easier to use BREs, which work everywhere than writing a script on one system only to have it fail on another just because the one sed has a non-standard extension and the other doesn't.
I hope this helps.
bakunin
These 5 Users Gave Thanks to bakunin For This Post:
Hi,
I am running a script sample.sh in bash environment .In the script i am using sed and awk commands which when executed individually from terminal they are getting executed normally but when i give these sed and awk commands in the script it is giving the below errors :-
./sample.sh: line... (12 Replies)
Hi,
I am trying to run below script where my input file contains
URL 1:Port1
URL 2:Port2
URL 3:Port3
IFS=" "
#Loop through the file
HSFILE=/home/Temp/Components.txt
for line in $HSFILE ;do
Url=$(awk -F":" '{print $1}' $HSFILE)
Port=$(awk -F":" '{print $2}' $HSFILE)
echo... (3 Replies)
When i pass parameter to sed command its throwing error as “SED cannot be parsed”.sed command is working if I pass directly.please tell me how to handle sed command with parameter.Please check below.
n=14
`sed 's/'OCCURS $n TIMES'//g' main.txt >sam1.txt`-Not working
`sed 's/OCCURS 14 TIMES//g'... (2 Replies)
sort.sh
------
ls -lSr|cat -n/*gives the detailed description o files with a serial number concatenated*/
i=0
n=10
j=$n
if
then
while
do
(sh -C sub.sh $i $j)&
((i++))
((j--))
done
fi
if
then
while
do
(sh -C sub.sh $i $j)& (1 Reply)
Hello All,
I have something like below
LDC100/rel/prod/libinactrl.a
LAA2000/rel/prod/libinactrl.a
I want to remove till first forward slash that is outputshould be as below
rel/prod/libinactrl.a
rel/prod/libinactrl.a
How can I do that ??? (8 Replies)
Hi all,
Suppose that I want to update the db.password entry in the below properties file,
db.username=admin
db.password=qhKkBno2U5GEq5k/dnmGPA== //I want this line to be replaced by: "db.password=abc/123/"
db.host=server
db.port=22
db.sid=database
However when... (2 Replies)
I have list of data I have cut down to format:
I am using sed command to remove the sed 's/
Returns error:
sed: -e expression #1, char 5: unterminated `s' command
Full code line is:
cat textFile | cut -d ' ' -f 4 | cut ':' -f 1 | sed 's/
Thanks,
Please use next time code tags... (2 Replies)
Hi,
Could some one help me please ?
When I execute this command
# paste -s -d"^m" swagent.log | sed '{s/\"\./\"\.\n/g}' > swagent.new
I get this error.
sed: Function {s/\"\./\"\.\n/g} cannot be parsed.
Many Thanks in advance
Regards,
Nagu (1 Reply)
Dear friends,
please give me the solution to the following query.
If a file contains multiple tags of same name, then how to get the required string between the tags, in which the string begins with "W/X"
i.e., file1.txt contains following text(please note that all the following tags are in... (1 Reply)
01-07-2019
