Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

System call oddity

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting System call oddity
# 1  
Old 03-07-2015
Code System call oddity
Hi all, I'm trying to use bash to create a basic parser for some text, because basic grep/sed/awk have all my needs covered.

So, I'm creating the chain of grep/sed I need and trying the execute that on the data I gather.

The problem is, it's not working as it does when hardcoded.


e.g.
Code:
#!/bin/sh
##a b c
##a b d
##a "b" x
##a p "b"

me=$0
data=`cat $0`
func1="grep 'a b'"
func2="grep ^##"
func3="grep ## | grep b"
echo "$data" | grep 'a b'
#echo "$data" | $func1


So, the last two lines should be the same.

using the 2nd to last line with grep 'a b'
Code:
beomagi@VBMagi ~ $ ./test.sh 
##a b c
##a b d
func1="grep 'a b'"
echo "$data" | grep 'a b'

using the variable for grep 'a b'
Code:
beomagi@VBMagi ~ $ ./test.sh 
grep: b': No such file or directory


But variables work in the case of the $func2 variable in my test script.
Code:
beomagi@VBMagi ~ $ ./test.sh 
##a b c
##a b d
##a "b" x
##a p "b"


So what's the deal? I have text in a variable, and want to filter it using a string of grep/sed/awk - but once there's any spaces or quotes in my filter's variable, it breaks.

Ideas?
# 2  
Old 03-07-2015
Hi, welcome to the forum Smilie

In this case, try removing the quotes of the last executed line from [ICODE]"$data"[/ICODE].
Also, cat will produce output already, so there is no need to echo this.

And to make it even simpler try this as your last line:
[CODE]$func1 $me[/CODE]

hth

EDIT:
Sry looked good at first sight.

---------- Post updated at 04:07 ---------- Previous update was at 03:40 ----------

One should not parse itself, this can cause incredible forks.
Despite said thing, i tried your code.
While i fail to understand why you cant use func1/2/3 from your original script, i found a work around for you.

Code:
#!/bin/sh
##a b c
##a b d
##a "b" x
##a p "b"

me="$0"
func1="grep ^##"
str1="a b"
str2="\"b"
$func1 $me | grep "$str2"

Note that the `` changed to $(), that is because the later is easier to manage.
Also note that str2 contains an escaped quote-char for its search pattern.

Hope this helps
Last edited by sea; 03-07-2015 at 10:12 PM.. Reason: Code fix - me-string
# 3  
Old 03-07-2015
Thanks.
Yeah I got that far, but I'm hoping to be able to dynamically chain multiple greps/seds/awks etc together.

The original intent was to parse webpages - using grep/sed etc is actually simple for this.

Here's the intent - I automate getting stuff - like parsing through comics and getting them for me.

I'm looking for make it more dynamic.

e.g. here's how I'm getting one of my comics:


If I want to define all data for a comic, I specify a url, and filter rules.
rooturl:
http;//www.mangahere.co/manga/kingdom/
Chaps:
grep "a class" | grep kingdom | sed 's@<.*href="@@g' | sed 's@".*>@@g' | sed 's-\ --g'
Page:
grep "option value" | sed 's-^.*value="--g' | sed 's-"\ .*--g'
Image:
grep jpg | grep "render(this)" | sed 's-<img.*src="--g' | sed 's-".*--g'


Chapters (lists pages) - filter:grep "a class" | grep kingdom | sed 's@<.*href="@@g' | sed 's@".*>@@g' | sed 's-\ --g'
Code:
beomagi@Ganymede ~
$ curl -s http://www.mangahere.co/manga/kingdom/ | grep "a class" | grep kingdom | sed 's@<.*href="@@g' | sed 's@".*>@@g' | sed 's-\ --g'
http://www.mangahere.co/manga/kingdom/v37/c423/
http://www.mangahere.co/manga/kingdom/v37/c422/
http://www.mangahere.co/manga/kingdom/v37/c421/
http://www.mangahere.co/manga/kingdom/v37/c420/
http://www.mangahere.co/manga/kingdom/v37/c419/
http://www.mangahere.co/manga/kingdom/v37/c418/
http://www.mangahere.co/manga/kingdom/v37/c417/
http://www.mangahere.co/manga/kingdom/v37/c416/
http://www.mangahere.co/manga/kingdom/v37/c415/
http://www.mangahere.co/manga/kingdom/v37/c414/

Pages (has link to image)- filter:grep "option value" | sed 's-^.*value="--g' | sed 's-"\ .*--g'
Code:
beomagi@Ganymede ~
$ curl -s http://www.mangahere.co/manga/kingdom/v37/c422/ | grep "option value" | sed 's-^.*value="--g' | sed 's-"\ .*--g'
http://www.mangahere.co/manga/kingdom/v37/c422/
http://www.mangahere.co/manga/kingdom/v37/c422/2.html
http://www.mangahere.co/manga/kingdom/v37/c422/3.html
http://www.mangahere.co/manga/kingdom/v37/c422/4.html
http://www.mangahere.co/manga/kingdom/v37/c422/5.html
http://www.mangahere.co/manga/kingdom/v37/c422/6.html
http://www.mangahere.co/manga/kingdom/v37/c422/7.html
http://www.mangahere.co/manga/kingdom/v37/c422/8.html
http://www.mangahere.co/manga/kingdom/v37/c422/9.html

And lastly from here
Image filter:grep jpg | grep "render(this)" | sed 's-<img.*src="--g' | sed 's-".*--g'
Code:
beomagi@Ganymede ~
$ curl -s http://www.mangahere.co/manga/kingdom/v37/c422/4.html | grep jpg | grep "render(this)" | sed 's-<img.*src="--g' | sed 's-".*--g'
        http://z.mhcdn.net/store/manga/8198/37-422.0/compressed/g004.jpg?v=11425389828



So the idea is to create templates for a script to follow through for various comics. So If I want to add comics, I just have to come up with a url to initially follow, and rules to define pages. The grep/sed etc would need to be dynamic. I've done it in other languages, but bash is kinda convenient, and more than anything now it's irking me that I can't figure out why I can't make a system call echoing a variable and piping to grep.
# 4  
Old 03-09-2015
Ok, here's a simpler example.

very simple script

Code:
#!/bin/sh
##apple soda
##banana split
##peach cobbler
##apple pie
##apple computer
##zuchinni casserole
##mango apple smoothie
##adam's apple
##app installer
##whatsapp
##application manager

export me=$0
datacmd="cat $me | grep ^##"
echo $datacmd
data=`$datacmd`

output:
Code:
beomagi@VBMagi ~ $ ./test.sh 
cat ./test.sh | grep ^##
cat: |: No such file or directory
cat: grep: No such file or directory
cat: ^##: No such file or directory

The first line of the output
"cat ./test.sh | grep ^##" is from the line echo $datacmd

The next 3 lines of errors all come from this : data=`$datacmd`

Yet if I directly drop this into the prompt:
Code:
beomagi@VBMagi ~ $ cat ./test.sh | grep ^##
##apple soda
##banana split
##peach cobbler
##apple pie
##apple computer
##zuchinni casserole
##mango apple smoothie
##adam's apple
##app installer
##whatsapp
##application manager

It works.

i.e. cat ./test.sh | grep ^## works,
BUT if it's a in a string type variable and I execute it as a system call it barfs.

Why?Smilie

---------- Post updated at 09:33 PM ---------- Previous update was at 09:14 PM ----------

I'm balder now.

I think that helped.

I found out that there's "eval" and I can even call bash if i want to execute my string.

backticks just don't seem to cut it for complex calls.

Code:
#!/bin/sh
##apple soda
##banana split
##peach cobbler
##apple pie
##apple computer
##zuchinni casserole
##mango apple smoothie
##adam's apple
##app installer
##whatsapp
##application manager

datacmd="cat $me | grep ^##"
echo $datacmd
data=`eval "$datacmd"`
echo "$data"


gives:
Code:
beomagi@VBMagi ~ $ ./test.sh 
cat ./test.sh | grep ^##
##apple soda
##banana split
##peach cobbler
##apple pie
##apple computer
##zuchinni casserole
##mango apple smoothie
##adam's apple
##app installer
##whatsapp
##application manager



Yaaayyy....
Last edited by beomagi; 03-09-2015 at 11:28 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Bind system call

We are calling the bind system call as below bind(sfd, (struct sockaddr *) &addr, sizeof(struct sockaddr_un));Why there is difference in third parameter getting the sizeof as "struct sockaddr_un", wherein the 2nd parametere we are passing it as "(struct sockaddr *)"? Regards, Sajjan (2 Replies)
Discussion started by: VSSajjan
2 Replies

2. Shell Programming and Scripting

system call

Trying to figure out a load issue with a webserver. I have traced a php script and noticed the following connect(4, {sa_family=AF_INET, sin_port=htons(3306), sin_addr=inet_addr("XX.XX.XX.XX")}, 16) = -1 EINPROGRESS (Operation now in progress) <0.000035> poll(, 1, 2000) = 1 () <0.000120>... (5 Replies)
Discussion started by: rajan007
5 Replies

3. Programming

need help with system call

hi everyone i wrote a system call and compiled the kernel succesfully... my system call is in a file in the kernel folder named my_syscall1.c (kernel/my_syscall1.c) the header file for this system call i added it in the folder include like this include/my_syscall1/my_syscall1.h my problem is... (2 Replies)
Discussion started by: demis87
2 Replies

4. Programming

system call

I have a cgi script which is called after certain time interval, which has this: system ("ls -l /tmp/cgic* | grep -v \"cgicsave.env\" | awk '{print $5}'"); During the execution of this script,the output is 0 sometimes. But due to this the system call is not working at all and doesnt o/p... (2 Replies)
Discussion started by: xs2punit
2 Replies

5. Programming

C:system call

Hi I'm studing the system call. I've written a small program that return the time spent in doing some operations. Now I'd like to write one that return the time spent in user mode of a process. I'm reading that i should use the tms struct: clock_t times(struct tms *buf); struct tms {... (2 Replies)
Discussion started by: Dedalus
2 Replies

6. Shell Programming and Scripting

system call

Hi, How to write a system calls in a script ? > cd $HOME > ls -ltr thanks in advance.. (10 Replies)
Discussion started by: hegdeshashi
10 Replies

7. Linux

system call problem

hi, where can I find the detail information about the syscall in binary instructions of linux/mips. for example, in linux/mips: li v0, 4140 syscall it's a syacall of "lseek" , but how can I find that which registers will be used in this syscall , and the meaning of the arguments in the... (2 Replies)
Discussion started by: zerocool_08
2 Replies

8. Programming

c system call

How the c compiler differentiates the system calls and function calls? (1 Reply)
Discussion started by: rangaswamy
1 Replies

9. UNIX for Advanced & Expert Users

how to differentiate system call from library call

Hi, Ho do I differentiate system call from library call? for example if I am using chmod , how do I find out if it is a system call or library call? Thanks Muru (2 Replies)
Discussion started by: muru
2 Replies

10. Programming

semclt system call ???

hi mates, What is the : semctl system call for? any example will be helpful and be appreciated. cya and thanx abdul (2 Replies)
Discussion started by: abdul
2 Replies
Login or Register to Ask a Question