Sponsored Content
Full Discussion: System call oddity
Top Forums Shell Programming and Scripting System call oddity Post 302937690 by beomagi on Saturday 7th of March 2015 10:04:00 PM
Old 03-07-2015
Thanks.
Yeah I got that far, but I'm hoping to be able to dynamically chain multiple greps/seds/awks etc together.

The original intent was to parse webpages - using grep/sed etc is actually simple for this.

Here's the intent - I automate getting stuff - like parsing through comics and getting them for me.

I'm looking for make it more dynamic.

e.g. here's how I'm getting one of my comics:


If I want to define all data for a comic, I specify a url, and filter rules.
rooturl:
http;//www.mangahere.co/manga/kingdom/
Chaps:
grep "a class" | grep kingdom | sed 's@<.*href="@@g' | sed 's@".*>@@g' | sed 's-\ --g'
Page:
grep "option value" | sed 's-^.*value="--g' | sed 's-"\ .*--g'
Image:
grep jpg | grep "render(this)" | sed 's-<img.*src="--g' | sed 's-".*--g'


Chapters (lists pages) - filter:grep "a class" | grep kingdom | sed 's@<.*href="@@g' | sed 's@".*>@@g' | sed 's-\ --g'
Code:
beomagi@Ganymede ~
$ curl -s http://www.mangahere.co/manga/kingdom/ | grep "a class" | grep kingdom | sed 's@<.*href="@@g' | sed 's@".*>@@g' | sed 's-\ --g'
http://www.mangahere.co/manga/kingdom/v37/c423/
http://www.mangahere.co/manga/kingdom/v37/c422/
http://www.mangahere.co/manga/kingdom/v37/c421/
http://www.mangahere.co/manga/kingdom/v37/c420/
http://www.mangahere.co/manga/kingdom/v37/c419/
http://www.mangahere.co/manga/kingdom/v37/c418/
http://www.mangahere.co/manga/kingdom/v37/c417/
http://www.mangahere.co/manga/kingdom/v37/c416/
http://www.mangahere.co/manga/kingdom/v37/c415/
http://www.mangahere.co/manga/kingdom/v37/c414/

Pages (has link to image)- filter:grep "option value" | sed 's-^.*value="--g' | sed 's-"\ .*--g'
Code:
beomagi@Ganymede ~
$ curl -s http://www.mangahere.co/manga/kingdom/v37/c422/ | grep "option value" | sed 's-^.*value="--g' | sed 's-"\ .*--g'
http://www.mangahere.co/manga/kingdom/v37/c422/
http://www.mangahere.co/manga/kingdom/v37/c422/2.html
http://www.mangahere.co/manga/kingdom/v37/c422/3.html
http://www.mangahere.co/manga/kingdom/v37/c422/4.html
http://www.mangahere.co/manga/kingdom/v37/c422/5.html
http://www.mangahere.co/manga/kingdom/v37/c422/6.html
http://www.mangahere.co/manga/kingdom/v37/c422/7.html
http://www.mangahere.co/manga/kingdom/v37/c422/8.html
http://www.mangahere.co/manga/kingdom/v37/c422/9.html

And lastly from here
Image filter:grep jpg | grep "render(this)" | sed 's-<img.*src="--g' | sed 's-".*--g'
Code:
beomagi@Ganymede ~
$ curl -s http://www.mangahere.co/manga/kingdom/v37/c422/4.html | grep jpg | grep "render(this)" | sed 's-<img.*src="--g' | sed 's-".*--g'
        http://z.mhcdn.net/store/manga/8198/37-422.0/compressed/g004.jpg?v=11425389828



So the idea is to create templates for a script to follow through for various comics. So If I want to add comics, I just have to come up with a url to initially follow, and rules to define pages. The grep/sed etc would need to be dynamic. I've done it in other languages, but bash is kinda convenient, and more than anything now it's irking me that I can't figure out why I can't make a system call echoing a variable and piping to grep.
 

10 More Discussions You Might Find Interesting

1. Programming

semclt system call ???

hi mates, What is the : semctl system call for? any example will be helpful and be appreciated. cya and thanx abdul (2 Replies)
Discussion started by: abdul
2 Replies

2. UNIX for Advanced & Expert Users

how to differentiate system call from library call

Hi, Ho do I differentiate system call from library call? for example if I am using chmod , how do I find out if it is a system call or library call? Thanks Muru (2 Replies)
Discussion started by: muru
2 Replies

3. Programming

c system call

How the c compiler differentiates the system calls and function calls? (1 Reply)
Discussion started by: rangaswamy
1 Replies

4. Linux

system call problem

hi, where can I find the detail information about the syscall in binary instructions of linux/mips. for example, in linux/mips: li v0, 4140 syscall it's a syacall of "lseek" , but how can I find that which registers will be used in this syscall , and the meaning of the arguments in the... (2 Replies)
Discussion started by: zerocool_08
2 Replies

5. Shell Programming and Scripting

system call

Hi, How to write a system calls in a script ? > cd $HOME > ls -ltr thanks in advance.. (10 Replies)
Discussion started by: hegdeshashi
10 Replies

6. Programming

C:system call

Hi I'm studing the system call. I've written a small program that return the time spent in doing some operations. Now I'd like to write one that return the time spent in user mode of a process. I'm reading that i should use the tms struct: clock_t times(struct tms *buf); struct tms {... (2 Replies)
Discussion started by: Dedalus
2 Replies

7. Programming

system call

I have a cgi script which is called after certain time interval, which has this: system ("ls -l /tmp/cgic* | grep -v \"cgicsave.env\" | awk '{print $5}'"); During the execution of this script,the output is 0 sometimes. But due to this the system call is not working at all and doesnt o/p... (2 Replies)
Discussion started by: xs2punit
2 Replies

8. Programming

need help with system call

hi everyone i wrote a system call and compiled the kernel succesfully... my system call is in a file in the kernel folder named my_syscall1.c (kernel/my_syscall1.c) the header file for this system call i added it in the folder include like this include/my_syscall1/my_syscall1.h my problem is... (2 Replies)
Discussion started by: demis87
2 Replies

9. Shell Programming and Scripting

system call

Trying to figure out a load issue with a webserver. I have traced a php script and noticed the following connect(4, {sa_family=AF_INET, sin_port=htons(3306), sin_addr=inet_addr("XX.XX.XX.XX")}, 16) = -1 EINPROGRESS (Operation now in progress) <0.000035> poll(, 1, 2000) = 1 () <0.000120>... (5 Replies)
Discussion started by: rajan007
5 Replies

10. Programming

Bind system call

We are calling the bind system call as below bind(sfd, (struct sockaddr *) &addr, sizeof(struct sockaddr_un));Why there is difference in third parameter getting the sizeof as "struct sockaddr_un", wherein the 2nd parametere we are passing it as "(struct sockaddr *)"? Regards, Sajjan (2 Replies)
Discussion started by: VSSajjan
2 Replies
PYP(1)							      General Commands Manual							    PYP(1)

NAME
pyp - The Pyed Piper: A Modern Python Alternative to awk, sed and Other Unix Text Manipulation Utilities SYNOPSIS
pyp [options] files ... DESCRIPTION
pyp, the Pyed Piper, is a command line tool for text manipulation. It is similar to awk and sed in functionality, but its subcommands are Python based, and thus more familiar to many programmers. It can operate both on a per-line base and on the complete input stream. Different features can be pipelined in a single command by using the pipe character familiar from shell commands. pyp backs up its input for reruns with modified commands, and can save commands as macros. On the downside, the rerun feature makes it unsuitable for continuous pipe operation. OPTIONS
These programs follow the usual GNU command line syntax, with long options starting with two dashes (`-'). A summary of options is included below. For a complete description, use --manual. -h, --help Show this help message and exit. -m, --manual Prints out extended help. -l, --macro_list Lists all available macros. -s MACRO_SAVE_NAME, --macro_save=MACRO_SAVE_NAME Saves current command as macro. use "#" for adding comments EXAMPLE: pyp -s "great_macro # prints first letter" "p[1]". -f MACRO_FIND_NAME, --macro_find=MACRO_FIND_NAME Searches for macros with keyword or user name. -d MACRO_DELETE_NAME, --macro_delete=MACRO_DELETE_NAME Deletes specified public macro. -g, --macro_group Specify group macros for save and delete; default is user. -t TEXT_FILE, --text_file=TEXT_FILE Specify text file to load. For advanced users, you should typically cat a file into pyp. -x, --execute Execute all commands. -c, --turn_off_color Prints raw, uncolored output. -u, --unmodified_config Prints out generic PypCustom.py config file. -b BLANK_INPUTS, --blank_inputs=BLANK_INPUTS Generate this number of blank input lines; useful for generating numbered lists with variable 'n'. -n, --no_input Use with command that generates output with no input; same as --dummy_input 1. -k, --keep_false Print blank lines for lines that test as False. default is to filter out False lines from the output. -r, --rerun Rerun based on automatically cached data from the last run. Use this after executing "pyp", pasting input into the shell, and hitting CTRL-D. SEE ALSO
awk(1), grep(1), sed(1). AUTHOR
pyp was written by Toby Rosen <tobyrosen@gmail.com>. This manual page was written by Khalid El Fathi <khalid@elfathi.fr>, for the Debian project (and may be used by others). March 19, 2012 PYP(1)
All times are GMT -4. The time now is 07:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy