Gobuster

Basic Usage
1. Command Syntax
2. Common Options
Directory and File Brute Forcing
1. Wordlist Attack
2. Extension Bruteforcing
DNS Subdomain Bruteforcing
1. DNS Wordlist Attack
2. Virtual Host Bruteforcing
Advanced Techniques
Combining Gobuster with Other Tools
1. Nmap and Gobuster
2. Burp Suite and Gobuster
Monitoring and Managing Scans
1. Output and Logging
2. Real-time Monitoring
Glossary

Basic Usage

Understanding Gobuster’s basic usage involves familiarizing yourself with its command syntax and common options. This foundational knowledge is essential before delving into more advanced functionalities.

Command Syntax

The general syntax for running Gobuster is as follows:

gobuster [mode] [options]

[mode]: Specifies the operation mode (dir, dns, vhost, etc.).
[options]: Flags and parameters that customize the scan.

Common Options

Below are some frequently used options in Gobuster commands:

-u [URL]: Specifies the target URL (used in dir and vhost modes).
-d [domain]: Specifies the target domain (used in dns mode).
-w [wordlist]: Path to the wordlist file.
-t [threads]: Number of concurrent threads (default is 10).
-o [file]: Outputs results to the specified file.
-x [extensions]: Comma-separated list of extensions to append.
-r: Follows redirects.
-s [codes]: Comma-separated list of HTTP status codes to include.
-k: Skips SSL/TLS certificate verification.
-v: Enables verbose mode, providing detailed output.
-b [codes]: Specifies HTTP status codes to exclude from the results.
-H [header]: Adds a custom header to the HTTP requests.
-e: Displays the full URL in the output.
-q: Quiet mode; suppresses non-error messages.
--no-error: Suppresses error messages.
--timeout [seconds]: Sets the request timeout in seconds.

Directory and File Brute Forcing

One of Gobuster’s primary functionalities is to brute-force directories and files on a web server. This process helps identify hidden or unlisted resources that could be potential entry points for further exploration.

Wordlist Attack

A wordlist attack involves using a predefined list of potential directory and file names to probe the target web server. This method is effective in uncovering hidden paths that are not publicly linked.

Example Command:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 50

Explanation:

dir: Sets the mode to directory/file brute-forcing.
-u http://target-website.com: Specifies the target URL.
-w /path/to/wordlist.txt: Path to the wordlist containing potential directory and file names.
-t 50: Utilizes 50 concurrent threads for faster scanning.

Sample Output:

/admin (Status: 301)
/images (Status: 200)
/login (Status: 200)
/uploads (Status: 403)

Extension Bruteforcing

Extension bruteforcing allows Gobuster to append specific file extensions to each entry in the wordlist. This technique is useful for discovering files with particular extensions, such as .php, .html, or .js.

Example Command:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -x php,html,js -t 50

Explanation:

-x php,html,js: Appends the extensions .php, .html, and .js to each wordlist entry.

Sample Output:

/index.php (Status: 200)
/about.html (Status: 200)
/script.js (Status: 200)

DNS Subdomain Bruteforcing

Beyond directories and files, Gobuster can be leveraged to discover DNS subdomains associated with a target domain. This capability is invaluable for expanding the attack surface during reconnaissance.

DNS Wordlist Attack

A DNS wordlist attack involves using a list of potential subdomain names to query the DNS records of a target domain. This helps identify active subdomains that might host additional services or applications.

Example Command:

gobuster dns -d example.com -w /path/to/dns-wordlist.txt -t 50

Explanation:

dns: Sets the mode to DNS subdomain brute-forcing.
-d example.com: Specifies the target domain.
-w /path/to/dns-wordlist.txt: Path to the DNS wordlist containing potential subdomain names.
-t 50: Utilizes 50 concurrent threads for faster scanning.

Sample Output:

api.example.com (Resolved: 192.168.1.10)
/blog.example.com (Resolved: 192.168.1.11)
/shop.example.com (Resolved: 192.168.1.12)

Virtual Host Bruteforcing

Virtual host bruteforcing You can use gobuster’s vhost mode to brute-force subdomains over HTTP. This is particularly useful when you rely on /etc/hosts to resolve the base domain. The vhost mode tests virtual hosts by injecting subdomain names into the HTTP Host header.

Example Command:

gobuster vhost -u http://192.168.1.100 -w /path/to/vhost-wordlist.txt -t 50

Explanation:

vhost: Sets the mode to virtual host brute-forcing.
-u http://192.168.1.100: Specifies the target IP address.
-w /path/to/vhost-wordlist.txt: Path to the wordlist containing potential virtual host names.
-t 50: Utilizes 50 concurrent threads for faster scanning.

How It Works

gobuster will append each word from the wordlist to the base domain and send HTTP requests using that subdomain in the Host header. If a server responds, it will identify the valid subdomain.

Sample Output:

admin.example.com (Status: 200)
/dev.example.com (Status: 301)
/test.example.com (Status: 404)

Filtering for HTTP 200 with Curl

If you want to simulate curl behavior for HTTP 200 responses, combine gobuster with curl to extract valid subdomains dynamically:

gobuster vhost -u example.com -w /path/to/vhost-wordlist.txt | awk '{print $2}' | while read host; do
    curl -I -H "Host: $host.example.com" example.com 2>/dev/null | grep "HTTP/1.1 200" && echo "$host.example.com"
done

How This Works

awk '{print $2}': Extracts subdomain names from gobuster output.
curl -I: Sends an HTTP HEAD request for each subdomain to verify the response.
grep "HTTP/1.1 200": Filters for responses with HTTP status 200.
&& echo "$host.example.com": Prints only the subdomains with HTTP 200 responses.

Sample Output

For a server that resolves shop.example.com with HTTP 200:

HTTP/1.1 200 OK
shop.example.com

Advanced Techniques

Gobuster offers several advanced features that enhance its effectiveness in various scenarios. These techniques allow for more nuanced and targeted scans, improving the quality of reconnaissance.

Recursive Searches

Recursive searches enable Gobuster to perform subsequent scans within discovered directories. This deepens the enumeration process by exploring the directory structure beyond the initial findings.

Example Command:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 50 -r

Explanation:

-r: Enables recursive scanning, allowing Gobuster to delve into newly discovered directories.

Sample Output:

/admin (Status: 301)
/admin/login (Status: 200)
/admin/dashboard (Status: 200)
/images (Status: 200)
/images/icons (Status: 200)
/images/logo.png (Status: 200)

Note: Recursive searches can significantly increase scan time and resource usage. Use them judiciously, especially on large or complex websites.

Handling HTTP Status Codes

Gobuster allows you to filter results based on HTTP status codes, helping you focus on relevant findings and ignore benign responses.

Including Specific Status Codes:

You can specify which HTTP status codes Gobuster should include in the results using the -s option.

Example Command:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 50 -s "200,204,301,302,307,401,403"

Explanation:

-s "200,204,301,302,307,401,403": Instructs Gobuster to display only responses with the specified status codes.

Excluding Specific Status Codes:

Alternatively, you can exclude certain status codes using the -b option.

Example Command:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 50 -b "404,500"

Explanation:

-b "404,500": Instructs Gobuster to omit responses with the specified status codes.

Rate Limiting and Timeouts

Managing the rate of requests and handling timeouts is crucial to avoid overwhelming the target server and to ensure Gobuster operates smoothly.

Setting Request Timeout:

Use the --timeout option to define how long Gobuster should wait for a response before considering the request as failed.

Example Command:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 50 --timeout 10

Explanation:

--timeout 10: Sets the request timeout to 10 seconds.

Adjusting Rate Limiting:

While Gobuster doesn’t have a direct rate-limiting option, you can control the scan speed by adjusting the number of threads with the -t option.

Example Command:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 20

Explanation:

-t 20: Reduces the number of concurrent threads to 20, effectively slowing down the scan.

Best Practices:

Respect Target’s Resources: Avoid setting the thread count too high, which can overwhelm the target server.
Use Timeouts Wisely: Set appropriate timeout values to prevent prolonged waiting periods during slow responses.

Combining Gobuster with Other Tools

Integrating Gobuster with other security tools can enhance your reconnaissance and penetration testing workflows, providing a more comprehensive assessment of the target environment.

Nmap and Gobuster

Nmap is a powerful network scanning tool that can identify open ports and services on a target system. Combining Nmap with Gobuster allows for targeted enumeration based on the discovered services.

Example Workflow:

Scan with Nmap:

First, identify open ports (e.g., HTTP on port 80 and HTTPS on port 443).
```
nmap -p 80,443 target-ip-address
```
Sample Output:
```
PORT    STATE SERVICE
80/tcp  open  http
443/tcp open  https
```
Attack with Gobuster:

Use Gobuster to enumerate directories on the discovered web services.
```
gobuster dir -u http://target-ip-address -w /path/to/wordlist.txt -t 50
gobuster dir -u https://target-ip-address -w /path/to/wordlist.txt -t 50 -k
```
Explanation:
- For HTTP (port 80), no additional options are needed.
- For HTTPS (port 443), the -k option skips SSL certificate verification, which is useful if the target has self-signed certificates.

Benefits:

Targeted Scanning: Focus enumeration efforts on active services, reducing unnecessary scan time.
Comprehensive Assessment: Provides a fuller picture of the target’s web infrastructure.

Burp Suite and Gobuster

Burp Suite is an integrated platform for performing security testing of web applications. While Gobuster is used for enumerating directories and subdomains, Burp Suite can be employed to analyze and exploit the discovered endpoints.

Example Workflow:

Discover Endpoints with Gobuster:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 50 -o gobuster-results.txt

Import Findings into Burp Suite:
- Open Burp Suite and navigate to the Target tab.
- Use the Import feature to load gobuster-results.txt.
- Analyze the discovered endpoints for vulnerabilities, such as SQL injection, Cross-Site Scripting (XSS), and more.

Benefits:

Enhanced Analysis: Use Burp Suite’s advanced features to perform detailed security assessments on enumerated endpoints.
Streamlined Workflow: Seamlessly transition from enumeration to exploitation within the same framework.

Monitoring and Managing Scans

Effective monitoring and management of Gobuster scans are essential to ensure efficient operation, accurate results, and the ability to troubleshoot issues as they arise.

Output and Logging

Gobuster provides options to save scan results to files, facilitating later analysis and record-keeping.

Output to a File:

Use the -o option to direct Gobuster’s output to a specified file.

Example Command:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 50 -o gobuster-results.txt

Explanation:

-o gobuster-results.txt: Saves the scan results to gobuster-results.txt.

Appending Results:

To append results to an existing file instead of overwriting it, use the -a option.

Example Command:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 50 -o gobuster-results.txt -a

Explanation:

-a: Appends the new scan results to gobuster-results.txt instead of overwriting.

Real-time Monitoring

While Gobuster doesn’t provide built-in real-time monitoring dashboards, you can utilize terminal multiplexers or logging tools to observe scan progress.

Using tee for Real-time and File Logging:

The tee command allows you to view Gobuster’s output in real-time while simultaneously saving it to a file.

Example Command:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 50 | tee gobuster-results.txt

Explanation:

| tee gobuster-results.txt: Pipes Gobuster’s output to tee, which displays it in the terminal and writes it to gobuster-results.txt.

Using screen or tmux for Session Management:

Terminal multiplexers like screen or tmux allow you to manage multiple terminal sessions, making it easier to monitor long-running scans.

Example with tmux:

Start a new tmux session:
```
tmux new -s gobuster-session
```

Run Gobuster within the tmux session:

gobuster dir -u http://target-website.com -w /path/to/wordlist.txt -t 50 -o gobuster-results.txt

Detach from the session:

Press Ctrl + B, then D.
Reattach to the session later:
```
tmux attach -t gobuster-session
```

Benefits:

Session Persistence: Allows scans to continue running even if the terminal is closed.
Concurrent Monitoring: View and manage multiple scans simultaneously.

Glossary

Below is a comprehensive list of commands, options, and terms used in Gobuster, along with their descriptions.

Command/Option	Description
`gobuster`	The main command to invoke Gobuster for various brute-forcing tasks.
`dir`	Mode for directory and file brute-forcing.
`dns`	Mode for DNS subdomain brute-forcing.
`vhost`	Mode for virtual host brute-forcing.
`-u [URL]`	Specifies the target URL (used in `dir` and `vhost` modes).
`-d [domain]`	Specifies the target domain (used in `dns` mode).
`-w [wordlist]`	Path to the wordlist file used for brute-forcing.
`-t [threads]`	Sets the number of concurrent threads (default is 10).
`-o [file]`	Outputs scan results to the specified file.
`-a`	Appends results to the output file instead of overwriting.
`-x [extensions]`	Comma-separated list of file extensions to append to wordlist entries (e.g., `php,html,js`).
`-r`	Enables recursive scanning, allowing Gobuster to explore newly discovered directories.
`-s [codes]`	Comma-separated list of HTTP status codes to include in the results (e.g., `200,301,302`).
`-b [codes]`	Comma-separated list of HTTP status codes to exclude from the results (e.g., `404,500`).
`-k`	Skips SSL/TLS certificate verification, useful for targets with self-signed certificates.
`-v`	Enables verbose mode, providing detailed output during the scan.
`-H [header]`	Adds a custom HTTP header to requests (e.g., `-H "Authorization: Bearer TOKEN"`).
`-e`	Displays the full URL in the output, including the protocol and domain.
`-q`	Enables quiet mode, suppressing non-error messages.
`--no-error`	Suppresses error messages, reducing console clutter.
`--timeout [seconds]`	Sets the request timeout in seconds, determining how long Gobuster waits for a response before timing out.
`FUZZ`	Placeholder used in headers or URLs where wordlist entries are inserted during scanning.
Modes
`dir`	Brute-forces directories and files on a web server.
`dns`	Brute-forces DNS subdomains for a given domain.
`vhost`	Brute-forces virtual hosts by manipulating the HTTP `Host` header.
Terms
Brute-forcing	The process of systematically checking all possible combinations to discover hidden resources.
Wordlist	A file containing a list of potential directory, file, or subdomain names used for brute-forcing.
HTTP Status Codes	Standard response codes indicating the result of an HTTP request (e.g., `200 OK`, `404 Not Found`).
Recursive Scanning	The ability to perform additional scans within discovered directories to uncover deeper layers of the website.
Multithreading	The capability to perform multiple operations concurrently, enhancing the speed of scans.
Timeout	The maximum duration Gobuster waits for a response before aborting the request.
Verbose Mode	A setting that provides detailed information about the scan’s progress and results.
Custom Headers	Additional HTTP headers that can be included in requests to manipulate or enhance the scanning process.
Output File	A file where scan results are saved for later analysis or record-keeping.

Table of Contents

Basic Usage

Command Syntax

Common Options

Directory and File Brute Forcing

Wordlist Attack

Extension Bruteforcing

DNS Subdomain Bruteforcing

DNS Wordlist Attack

Virtual Host Bruteforcing

How It Works

Filtering for HTTP 200 with Curl

How This Works

Sample Output

Advanced Techniques

Recursive Searches

Handling HTTP Status Codes

Rate Limiting and Timeouts

Combining Gobuster with Other Tools

Nmap and Gobuster

Burp Suite and Gobuster

Monitoring and Managing Scans

Output and Logging

Real-time Monitoring

Glossary