Error while using sed command


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
# 1  
Error while using sed command

I have a few csv files in a directory and i am using sed command for processing the results from the filename.Details are given below:
Code:
Directory: /tmp/test/output
Files: JAN_DAT_TES1_201807181400.csv
         JAN_DAT_TES2_201807181500.csv

I want to get the output as
Code:
/tmp/test/output/TES1  2018071814
/tmp/test/output/TES2  2018071815

Itried with following code,but its giving error as Unmatched ) or \)
Code:
find /tmp/test/output -type f -name "*.csv" | sed -e 's/JAN_DAT_(.*\)\_([0-9]\{10\}\).*.csv/\1 \2/g'

Kindly help to correct the mistake in the above command
# 2  
Quote:
Originally Posted by Jag02
Code:
Files: JAN_DAT_TES1_201807181400.csv
         JAN_DAT_TES2_201807181500.csv

I want to get the output as
Code:
/tmp/test/output/TES1  2018071814
/tmp/test/output/TES2  2018071815

The good news is: you do not need sed for this at all. In fact, as long as you are not manipulating a data stream you usually should not use it at all and use "variable expansion" instead. It is the shells equivalent of substr(), trim(), strtok() and similar functions found in other high-level-languages.

The first thing you want to do is to cut off the extension ".csv" from the filename:

Code:
file="JAN_DAT_TES1_201807181400.csv"
echo "${file%.csv}"

This works the following way: ${variable%pattern} cuts off the pattern from the end of the variables content if it is found. "pattern" is everything you could use as a filename pattern in the shell: "*" would mean any number of any characters (like in "file*"), "?" means any one character, etc. Notice also that this only DISPLAYS the resulting string, it does NOT CHANGE the content of the variable! If you want to change it lastingly you need to assign the new value:

Code:
file="${file%.csv}"

Since you want to cut off the last two zeroes either we can do that in a single run:

Code:
file="JAN_DAT_TES1_201807181400.csv"
echo "${file%00.csv}"

The next thing we want is the date to be cut off but we need to preserve it, so we assign a new variable with a copy of "$file"s contents, but with everything up to the last underscore removed. For this there is another expansion which works like the one i showed you but it cuts off from the beginning instead of the end:

Code:
file="JAN_DAT_TES1_2018071814"
echo "${file##*_}"

Notice that i used "##" instead of "#". "##" and its companion "%%" cut off the longest possible match whereas "#" and "%" cut off the shortest possible match. That means:

Code:
file="JAN_DAT_TES1_2018071814"
echo "${file#*_}"     # gives "DAT_TES1_2018071814"
echo "${file##*_}"    # gives "2018071814"

Now, putting it all together (notice that you can use a variable as pattern too!):

Code:
for file in /some/where/*csv ; do
     file="${file%00.csv}"
     date="${file##*_}"
     file="${file%_${date}}"
     echo "my file is: $file , my date is: $date"
done

I hope this helps.

bakunin
# 3  
Thank you. But i would like the output to be
Code:
/tmp/test/output/TES1  2018071814

The above code gives output as
Code:
/tmp/test/JAN_DAT_TES1 2018071814

# 4  
Quote:
Originally Posted by Jag02
Thank you. But i would like the output to be
Code:
/tmp/test/output/TES1  2018071814

The above code gives output as
Code:
/tmp/test/JAN_DAT_TES1 2018071814

In this case: cut off the other parts with what i have showed you:

Code:
for file in /some/where/*csv ; do
     path="${file%/*}"        # remove the filename, leaving the path  "/some/where/JAN_DAT_TES1_201807181400.csv" => "/some/where"
     file="${file%00.csv}"    # remove the extension and the last "00"  "/some/where/JAN_DAT_TES1_201807181400.csv" => "/some/where/JAN_DAT_TES1_2018071814"
     date="${file##*_}"       # extract the date: "/some/where/JAN_DAT_TES1_2018071814" => "2018071814"
     file="${file%_${date}}"  # remove the date from the filename: "/some/where/JAN_DAT_TES1_2018071814"  => "/some/where/JAN_DAT_TES1"
     file="${file##*_}"       # remove everything up to the last "_" from the filename: "/some/where/JAN_DAT_TES1"  => "TES1"

     echo "my file is: $path/$file , my date is: $date"
done

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 5  
Thank you!!. It helped.
If you could pl tell me what was the error in the sed code, it will help me to learn better.
# 6  
Quote:
Originally Posted by Jag02
Thank you!!. It helped.
If you could pl tell me what was the error in the sed code, it will help me to learn better.
Code:
sed -e 's/JAN_DAT_(.*\)\_([0-9]\{10\}\).*.csv/\1 \2/g'

The first thing is: sed-patterns are always "greedy", which means they match the longest possible part of the line. A pattern like A.*B will NOT match the part marked bold but the WHOLE of the following string:

Code:
AXYZBBBBBBBBBBBBBBB

Therefore /JAN_DAT_(.*\) is already at least problematic if not wrong. What you want to do is to match up to the next underscore, so you better exclude the underscore from the matching set:

Code:
/JAN_DAT_\([^_]*\)_....

In the example above, you would not write A.*B but A[^B]*B ("A, followed by any number of non-Bs, followed by B") to match only from A to the next following B.

The second thing was the unescaped "(", which i corrected (marked in red). The same goes for the second opening bracket, before the "[0-9]". You see, sed uses "BRE", "basic regular expressions", unlike i.e. awk, which uses "ERE"s, "extended regular expressions". In BRE you have to escape brackets whereas in ERE you don't. I do not know which sed you use, some are able to use EREs too, but i wouldn't recommend it, even if they can. It is easier to use BREs, which work everywhere than writing a script on one system only to have it fail on another just because the one sed has a non-standard extension and the other doesn't.

I hope this helps.

bakunin
These 5 Users Gave Thanks to bakunin For This Post:
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #628
Difficulty: Easy
In PHP, True !== 1;
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed and awk giving error ./sample.sh: line 13: sed: command not found

Hi, I am running a script sample.sh in bash environment .In the script i am using sed and awk commands which when executed individually from terminal they are getting executed normally but when i give these sed and awk commands in the script it is giving the below errors :- ./sample.sh: line... (12 Replies)
Discussion started by: satishmallidi
12 Replies

2. Shell Programming and Scripting

Error in sed command

Hi, I am trying to run below script where my input file contains URL 1:Port1 URL 2:Port2 URL 3:Port3 IFS=" " #Loop through the file HSFILE=/home/Temp/Components.txt for line in $HSFILE ;do Url=$(awk -F":" '{print $1}' $HSFILE) Port=$(awk -F":" '{print $2}' $HSFILE) echo... (3 Replies)
Discussion started by: sharsour
3 Replies

3. Shell Programming and Scripting

sed command error

When i pass parameter to sed command its throwing error as “SED cannot be parsed”.sed command is working if I pass directly.please tell me how to handle sed command with parameter.Please check below. n=14 `sed 's/’OCCURS $n TIMES’//g' main.txt >sam1.txt`-Not working `sed 's/OCCURS 14 TIMES//g'... (2 Replies)
Discussion started by: srivalli
2 Replies

4. Shell Programming and Scripting

sed command error-URGENT

sort.sh ------ ls -lSr|cat -n/*gives the detailed description o files with a serial number concatenated*/ i=0 n=10 j=$n if then while do (sh -C sub.sh $i $j)& ((i++)) ((j--)) done fi if then while do (sh -C sub.sh $i $j)& (1 Reply)
Discussion started by: dishak
1 Replies

5. Shell Programming and Scripting

sed returns error "sed: -e expression #1, char 18: unterminated `s' command"

Hello All, I have something like below LDC100/rel/prod/libinactrl.a LAA2000/rel/prod/libinactrl.a I want to remove till first forward slash that is outputshould be as below rel/prod/libinactrl.a rel/prod/libinactrl.a How can I do that ??? (8 Replies)
Discussion started by: anand.shah
8 Replies

6. Shell Programming and Scripting

sed: command garbled error

Hi all, Suppose that I want to update the db.password entry in the below properties file, db.username=admin db.password=qhKkBno2U5GEq5k/dnmGPA== //I want this line to be replaced by: "db.password=abc/123/" db.host=server db.port=22 db.sid=database However when... (2 Replies)
Discussion started by: isaacniu
2 Replies

7. UNIX for Dummies Questions & Answers

sed error unterminated `s' command

I have list of data I have cut down to format: I am using sed command to remove the sed 's/ Returns error: sed: -e expression #1, char 5: unterminated `s' command Full code line is: cat textFile | cut -d ' ' -f 4 | cut ':' -f 1 | sed 's/ Thanks, Please use next time code tags... (2 Replies)
Discussion started by: maximus73
2 Replies

8. Shell Programming and Scripting

sed command garbled error

sed: command garbled: s/ri="*"/ri=" what i did is you can see below sed "s/ri=\"*\"/ri=\"$newri\"/" $2 > output how to solve this (2 Replies)
Discussion started by: pasricha.kunal
2 Replies

9. Shell Programming and Scripting

sed command error

Hi, Could some one help me please ? When I execute this command # paste -s -d"^m" swagent.log | sed '{s/\"\./\"\.\n/g}' > swagent.new I get this error. sed: Function {s/\"\./\"\.\n/g} cannot be parsed. Many Thanks in advance Regards, Nagu (1 Reply)
Discussion started by: srnagu
1 Replies

10. Shell Programming and Scripting

sed: command garbled error.....

Dear friends, please give me the solution to the following query. If a file contains multiple tags of same name, then how to get the required string between the tags, in which the string begins with "W/X" i.e., file1.txt contains following text(please note that all the following tags are in... (1 Reply)
Discussion started by: swamymns
1 Replies

Featured Tech Videos