SCRAPY(1) General Commands Manual SCRAPY(1)
NAME
scrapy - the Scrapy command-line tool
SYNOPSIS
scrapy [command] [OPTIONS] ...
DESCRIPTION
Scrapy is controlled through the scrapy command-line tool. The script provides several commands, for different purposes. Each command sup-
ports its own particular syntax. In other words, each command supports a different set of arguments and options.
OPTIONS
fetch [OPTION] URL
Fetch a URL using the Scrapy downloader
--headers
Print response HTTP headers instead of body
runspider [OPTION] spiderfile
Run a spider
--output=FILE
Store scraped items to FILE in XML format
settings [OPTION]
Query Scrapy settings
--get=SETTING
Print raw setting value
--getbool=SETTING
Print setting value, intepreted as a boolean
--getint=SETTING
Print setting value, intepreted as an integer
--getfloat=SETTING
Print setting value, intepreted as an float
--getlist=SETTING
Print setting value, intepreted as an float
--init Print initial setting value (before loading extensions and spiders)
shell URL | file
Launch the interactive scraping console
startproject projectname
Create new project with an initial project template
--help, -h
Print command help and options
--logfile=FILE
Log file. if omitted stderr will be used
--loglevel=LEVEL, -L LEVEL
Log level (default: None)
--nolog
Disable logging completely
--spider=SPIDER
Always use this spider when arguments are urls
--profile=FILE
Write python cProfile stats to FILE
--lsprof=FILE
Write lsprof profiling stats to FILE
--pidfile=FILE
Write process ID to FILE
--set=NAME=VALUE, -s NAME=VALUE
Set/override setting (may be repeated)
AUTHOR
Scrapy was written by the Scrapy Developers <scrapy-developers@googlegroups.com>.
This manual page was written by Ignace Mouzannar <mouzannar@gmail.com>, for the Debian project (but may be used by others).
October 17, 2009 SCRAPY(1)