Hi all, I'm trying to use bash to create a basic parser for some text, because basic grep/sed/awk have all my needs covered.
So, I'm creating the chain of grep/sed I need and trying the execute that on the data I gather.
The problem is, it's not working as it does when hardcoded.
e.g.
Code:
#!/bin/sh
##a b c
##a b d
##a "b" x
##a p "b"
me=$0
data=`cat $0`
func1="grep 'a b'"
func2="grep ^##"
func3="grep ## | grep b"
echo "$data" | grep 'a b'
#echo "$data" | $func1
So, the last two lines should be the same.
using the 2nd to last line with grep 'a b'
Code:
beomagi@VBMagi ~ $ ./test.sh
##a b c
##a b d
func1="grep 'a b'"
echo "$data" | grep 'a b'
using the variable for grep 'a b'
Code:
beomagi@VBMagi ~ $ ./test.sh
grep: b': No such file or directory
But variables work in the case of the $func2 variable in my test script.
Code:
beomagi@VBMagi ~ $ ./test.sh
##a b c
##a b d
##a "b" x
##a p "b"
So what's the deal? I have text in a variable, and want to filter it using a string of grep/sed/awk - but once there's any spaces or quotes in my filter's variable, it breaks.
In this case, try removing the quotes of the last executed line from [ICODE]"$data"[/ICODE].
Also, cat will produce output already, so there is no need to echo this.
And to make it even simpler try this as your last line:
[CODE]$func1 $me[/CODE]
hth
EDIT:
Sry looked good at first sight.
---------- Post updated at 04:07 ---------- Previous update was at 03:40 ----------
One should not parse itself, this can cause incredible forks.
Despite said thing, i tried your code.
While i fail to understand why you cant use func1/2/3 from your original script, i found a work around for you.
Code:
#!/bin/sh
##a b c
##a b d
##a "b" x
##a p "b"
me="$0"
func1="grep ^##"
str1="a b"
str2="\"b"
$func1 $me | grep "$str2"
Note that the `` changed to $(), that is because the later is easier to manage.
Also note that str2 contains an escaped quote-char for its search pattern.
Hope this helps
Last edited by sea; 03-07-2015 at 10:12 PM..
Reason: Code fix - me-string
Thanks.
Yeah I got that far, but I'm hoping to be able to dynamically chain multiple greps/seds/awks etc together.
The original intent was to parse webpages - using grep/sed etc is actually simple for this.
Here's the intent - I automate getting stuff - like parsing through comics and getting them for me.
I'm looking for make it more dynamic.
e.g. here's how I'm getting one of my comics:
If I want to define all data for a comic, I specify a url, and filter rules.
rooturl:
http;//www.mangahere.co/manga/kingdom/
Chaps:
grep "a class" | grep kingdom | sed 's@<.*href="@@g' | sed 's@".*>@@g' | sed 's-\ --g'
Page:
grep "option value" | sed 's-^.*value="--g' | sed 's-"\ .*--g'
Image:
grep jpg | grep "render(this)" | sed 's-<img.*src="--g' | sed 's-".*--g'
Chapters (lists pages) - filter:grep "a class" | grep kingdom | sed 's@<.*href="@@g' | sed 's@".*>@@g' | sed 's-\ --g'
Code:
beomagi@Ganymede ~
$ curl -s http://www.mangahere.co/manga/kingdom/ | grep "a class" | grep kingdom | sed 's@<.*href="@@g' | sed 's@".*>@@g' | sed 's-\ --g'
http://www.mangahere.co/manga/kingdom/v37/c423/
http://www.mangahere.co/manga/kingdom/v37/c422/
http://www.mangahere.co/manga/kingdom/v37/c421/
http://www.mangahere.co/manga/kingdom/v37/c420/
http://www.mangahere.co/manga/kingdom/v37/c419/
http://www.mangahere.co/manga/kingdom/v37/c418/
http://www.mangahere.co/manga/kingdom/v37/c417/
http://www.mangahere.co/manga/kingdom/v37/c416/
http://www.mangahere.co/manga/kingdom/v37/c415/
http://www.mangahere.co/manga/kingdom/v37/c414/
Pages (has link to image)- filter:grep "option value" | sed 's-^.*value="--g' | sed 's-"\ .*--g'
And lastly from here
Image filter:grep jpg | grep "render(this)" | sed 's-<img.*src="--g' | sed 's-".*--g'
Code:
beomagi@Ganymede ~
$ curl -s http://www.mangahere.co/manga/kingdom/v37/c422/4.html | grep jpg | grep "render(this)" | sed 's-<img.*src="--g' | sed 's-".*--g'
http://z.mhcdn.net/store/manga/8198/37-422.0/compressed/g004.jpg?v=11425389828
So the idea is to create templates for a script to follow through for various comics. So If I want to add comics, I just have to come up with a url to initially follow, and rules to define pages. The grep/sed etc would need to be dynamic. I've done it in other languages, but bash is kinda convenient, and more than anything now it's irking me that I can't figure out why I can't make a system call echoing a variable and piping to grep.
#!/bin/sh
##apple soda
##banana split
##peach cobbler
##apple pie
##apple computer
##zuchinni casserole
##mango apple smoothie
##adam's apple
##app installer
##whatsapp
##application manager
export me=$0
datacmd="cat $me | grep ^##"
echo $datacmd
data=`$datacmd`
output:
Code:
beomagi@VBMagi ~ $ ./test.sh
cat ./test.sh | grep ^##
cat: |: No such file or directory
cat: grep: No such file or directory
cat: ^##: No such file or directory
The first line of the output
"cat ./test.sh | grep ^##" is from the line echo $datacmd
The next 3 lines of errors all come from this : data=`$datacmd`
Yet if I directly drop this into the prompt:
Code:
beomagi@VBMagi ~ $ cat ./test.sh | grep ^##
##apple soda
##banana split
##peach cobbler
##apple pie
##apple computer
##zuchinni casserole
##mango apple smoothie
##adam's apple
##app installer
##whatsapp
##application manager
It works.
i.e. cat ./test.sh | grep ^## works,
BUT if it's a in a string type variable and I execute it as a system call it barfs.
Why?
---------- Post updated at 09:33 PM ---------- Previous update was at 09:14 PM ----------
I'm balder now.
I think that helped.
I found out that there's "eval" and I can even call bash if i want to execute my string.
backticks just don't seem to cut it for complex calls.
Code:
#!/bin/sh
##apple soda
##banana split
##peach cobbler
##apple pie
##apple computer
##zuchinni casserole
##mango apple smoothie
##adam's apple
##app installer
##whatsapp
##application manager
datacmd="cat $me | grep ^##"
echo $datacmd
data=`eval "$datacmd"`
echo "$data"
gives:
Code:
beomagi@VBMagi ~ $ ./test.sh
cat ./test.sh | grep ^##
##apple soda
##banana split
##peach cobbler
##apple pie
##apple computer
##zuchinni casserole
##mango apple smoothie
##adam's apple
##app installer
##whatsapp
##application manager
We are calling the bind system call as below
bind(sfd, (struct sockaddr *) &addr, sizeof(struct sockaddr_un));Why there is difference in third parameter getting the sizeof as "struct sockaddr_un", wherein the 2nd parametere we are passing it as "(struct sockaddr *)"?
Regards,
Sajjan (2 Replies)
Trying to figure out a load issue with a webserver. I have traced a php script and noticed the following
connect(4, {sa_family=AF_INET, sin_port=htons(3306), sin_addr=inet_addr("XX.XX.XX.XX")}, 16) = -1 EINPROGRESS (Operation now in progress) <0.000035>
poll(, 1, 2000) = 1 () <0.000120>... (5 Replies)
hi everyone
i wrote a system call and compiled the kernel succesfully...
my system call is in a file in the kernel folder named my_syscall1.c (kernel/my_syscall1.c)
the header file for this system call i added it in the folder include like this include/my_syscall1/my_syscall1.h
my problem is... (2 Replies)
I have a cgi script which is called after certain time interval, which has this:
system ("ls -l /tmp/cgic* | grep -v \"cgicsave.env\" | awk '{print $5}'");
During the execution of this script,the output is 0 sometimes. But due to this the system call is not working at all and doesnt o/p... (2 Replies)
Hi
I'm studing the system call. I've written a small program that return the time spent in doing some operations. Now I'd like to write one that return the time spent in user mode of a process.
I'm reading that i should use the tms struct:
clock_t times(struct tms *buf);
struct tms {... (2 Replies)
hi,
where can I find the detail information about the syscall in binary instructions of linux/mips.
for example, in linux/mips:
li v0, 4140
syscall
it's a syacall of "lseek" , but how can I find that which registers will be used in this syscall , and the meaning of the arguments in the... (2 Replies)
Hi,
Ho do I differentiate system call from library call?
for example if I am using chmod , how do I find out if it is a system call or library call?
Thanks
Muru (2 Replies)