工具简介

Cewl是一款采用Ruby开发的应用程序，你可以给它的爬虫指定URL地址和爬取深度，还可以添额外的外部链接，接下来Cewl会给你返回一个字典文件，你可以把字典用到类似John the Ripper这样的密码破解工具中。除此之外，Cewl还提供了命令行工具

工具命令

终端输入“cewl -h”查看帮助文档，工具会输出所有可接受的选项参数，以及对应的选项描述

──(kali㉿kali)-[~]
└─$ cewl -h                                                                              
CeWL 6.2.1 (More Fixes) Robin Wood (robin@digi.ninja) (https://digi.ninja/)
Usage: cewl [OPTIONS] ... <url>

    OPTIONS:
        -h, --help: Show help.
        -k, --keep: Keep the downloaded file.
        -d <x>,--depth <x>: Depth to spider to, default 2.
        -m, --min_word_length: Minimum word length, default 3.
        -x, --max_word_length: Maximum word length, default unset.
        -o, --offsite: Let the spider visit other sites.
        --exclude: A file containing a list of paths to exclude
        --allowed: A regex pattern that path must match to be followed
        -w, --write: Write the output to the file.
        -u, --ua <agent>: User agent to send.
        -n, --no-words: Don't output the wordlist.
        -g <x>, --groups <x>: Return groups of words as well
        --lowercase: Lowercase all parsed words
        --with-numbers: Accept words with numbers in as well as just letters
        --convert-umlauts: Convert common ISO-8859-1 (Latin-1) umlauts (ä-ae, ö-oe, ü-ue, ß-ss)
        -a, --meta: include meta data.
        --meta_file file: Output file for meta data.
        -e, --email: Include email addresses.
        --email_file <file>: Output file for email addresses.
        --meta-temp-dir <dir>: The temporary directory used by exiftool when parsing files, default /tmp.
        -c, --count: Show the count for each word found.
        -v, --verbose: Verbose.
        --debug: Extra debug information.

        Authentication
        --auth_type: Digest or basic.
        --auth_user: Authentication username.
        --auth_pass: Authentication password.

        Proxy Support
        --proxy_host: Proxy host.
        --proxy_port: Proxy port, default 8080.
        --proxy_username: Username for proxy, if required.
        --proxy_password: Password for proxy, if required.

        Headers
        --header, -H: In format name:value - can pass multiple.

    <url>: The site to spider.

输入以下命令可爬取指定页面信息,爬取的信息会打印在命令行，这里要注意这个数据没有保存只是打印在控制台，如果要报错可以使用“-w”命令指定保存文件

#基础命令
cewl <url>

──(kali㉿kali)-[~]
└─$ cewl https://authlab.digi.ninja/
CeWL 6.2.1 (More Fixes) Robin Wood (robin@digi.ninja) (https://digi.ninja/)
the
you
and
JWT
information
authentication
this
Lab
challenge
get
see
can
The
more
want
different
created
Robin
set
range
stuck
Auth
Wood
DigiNinja
challenges
covering
technologies
token
are
but
that
Visit
for
walkthrough
Back
home
user
signature
key
Authentication
some
not
have
login
from
with
Bypass
None
use
This
User

大多数的网站都有登录验证，所以爬取的数据很多时候只能爬到登录页面，这里可以使用“--header ”命令伪造登录信息（cookie）以绕过登录验证，具体示例如下：

cewl 192.168.3.52:28080/fdc -v --header Cookie:"jeeplus.session.id=13b502e098514d818e6c50ce589a87d1;"