acidvegas/masscan
1
Fork 0
Code Issues Pull Requests Releases Activity

initial commit

Browse Source
This commit is contained in:
Dionysus
2026-06-19 07:07:20 +00:00
commit 33435ab4b4
258 changed files with 67880 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
paused.conf
.Makefile.swp
.vscode
vs10/.vs/

387
FORK.md Normal file
View File

@@ -0,0 +1,387 @@
# masscan fork — changelog
This is a modified fork of [robertdavidgraham/masscan](https://github.com/robertdavidgraham/masscan).
**Base commit:** `94e118c` (Merge pull request #844 from p-l-/fix-smb-segfault)
**Purpose of the fork:**
1. Remove every wire-level fingerprint that lets a passive observer say "this is masscan." All known signatures (TLS hardcoded random, ICMP `0x08..0x37` payload, UDP `masscan-test` literals, IP-ID XOR formula, default SYN template, banner-grab anomalies) have been replaced with values that look like a normal Linux or Windows host doing TLS 1.2 cert harvesting.
2. Greatly expand the set of services masscan can banner-grab out of the box — adds PostgreSQL, Cassandra, Modbus, Siemens S7, OPC UA, DNP3, EtherNet/IP, BACnet, plus 50+ HTTP/HTTPS bindings on alt ports (Grafana, Kibana, Prometheus, k8s, Docker, Elasticsearch, etc.).
3. Auto-discover and load nmap's `nmap-payloads` file if installed, giving UDP probe coverage on hundreds more ports without a CLI flag.
CLI is **100% backward-compatible** with upstream masscan. Drop in `bin/masscan` and existing config files / commands keep working. The OS-mimicry chosen at startup is silently picked between Linux (50%) and Windows (50%); no flag changes that.
---
## File-by-file changes
Diff stats vs. upstream:
| File | Lines added | Lines removed |
| --- | --- | --- |
| `src/main-conf.c` | +26 | -4 |
| `src/proto-banner1.c` | +229 | -0 |
| `src/proto-http.c` | +15 | -9 |
| `src/proto-ssl.c` | +84 | -29 |
| `src/stack-tcp-core.c` | +3 | -1 |
| `src/templ-payloads.c` | +57 | -10 |
| `src/templ-pkt.c` | +152 | -41 |
| **Total** | **+566** | **-94** |
---
### `src/templ-pkt.c` — Packet templates
This is the file that owns the byte-level templates masscan copies and rewrites at send time. Most of the fingerprint work landed here.
#### 1. Replaced the single SYN template with Linux + Windows variants
**Where:** Lines 30–141 (was: lines 30–54).
**What:**
- The original `default_tcp_template[]` (TTL=255, win=1024, DF clear, options = MSS=1460 only — every one of those is a known masscan tell) was replaced with two static templates:
- `linux_tcp_template[]` — TTL=64, win=64240, DF set, options = `MSS=1460, SAckOK, Timestamp(0/0), NOP, WScale=7` (full Linux 5.x/6.x SYN, 40-byte TCP header)
- `windows_tcp_template[]` — TTL=128, win=64240, DF set, options = `MSS=1460, NOP, WScale=8, NOP, NOP, SAckOK` (Windows 10/11 SYN, 32-byte TCP header)
- New function `mimicry_select_os()` picks one at startup using `time(NULL) ^ (getpid() << 16)` as the seed. Exported via `int g_masscan_mimic_os` (0=Linux, 1=Windows) so other code paths can match the chosen OS if needed.
- `default_tcp_template` is now a pointer set by `mimicry_select_os()`; `default_tcp_template_size` is the size of whichever was picked.
**Why:** The original masscan SYN signature (`win=1024 + TTL=255 + DF clear + opts=[MSS=1460]`) is detected by every "is this a scanner?" rule. Real Linux and Windows hosts never look like that.
**Callsite update at line 1525:** `template_packet_init()` now calls `mimicry_select_os()` before initializing the TCP template, then uses `default_tcp_template_size` instead of `sizeof(default_tcp_template) - 1`.
#### 2. IP-ID changed from XOR formula to random per packet
**Where:** Line 525 (function `tcp_create_packet`, IPv4 path) and line 1033 (function that builds outbound SYNs).
**Old:** `unsigned ip_id = ip_them.ipv4 ^ port_them ^ seqno;`
**New:** `unsigned ip_id = ((unsigned)rand() ^ ((unsigned)rand() << 8)) & 0xFFFF;`
**Why:** The XOR formula was masscan-unique and verifiable from a single packet. Real OS stacks emit random IP-IDs. Reply validation still works because masscan validates the SipHash cookie embedded in the TCP ackno, not the IP-ID.
#### 3. ICMP echo payload rewritten
**Where:** Lines 199–224 (the `default_icmp_ping_template[]` array).
**Old:** 48 bytes of `0x08, 0x09, 0x0a, …, 0x37` (ascending starting at 0x08).
**New:** 8 zero bytes (placeholder for `struct timeval` like Linux ping uses) followed by 40 bytes of `0x10, 0x11, …, 0x37` (Linux ping ascending pattern).
**Why:** The exact `0x08..0x37` 48-byte payload is masscan's ICMP signature — neither Linux ping nor Windows ping nor nmap produces that pattern.
#### 4. ICMP / UDP IP header TTL and DF flag
**Where:** Lines 200, 201, 237, 238 (ICMP echo and timestamp templates), and lines 65, 66 (UDP template).
**Old:** TTL=255 (0xFF), DF flag clear.
**New:** TTL=64 (0x40), DF flag set.
**Why:** Match Linux defaults instead of masscan's high-TTL no-DF combo.
#### 5. Banner-grab packet construction (`tcp_create_packet`)
**Where:** Lines 519–550 (the IPv4 branch start) and lines 605–614 (window write site).
**Two problems fixed in one block:**
(a) Non-SYN segments were inheriting the SYN template's full TCP option block. That meant ACK / PSH-ACK / FIN packets carried an `MSS=1460` option — which is illegal per RFC and a masscan-specific banner-grab tell.
**Fix:** Detect SYN vs non-SYN via `(flags & 0x02)`. For non-SYN, force `tcp_header_len = 20` (no options) and overwrite `px[offset_tcp+12]` to data-offset 5.
(b) Window was hardcoded to 1200 on every non-SYN packet (and 600 for the "small-window" kludge).
**Fix:** Window now written as 64240 (realistic post-handshake Linux/Windows value).
**Why:** A non-SYN segment carrying MSS, combined with window=1200, was the smoking-gun signature for masscan's `--banners` mode. Both gone now.
#### 6. Added `<time.h>` and `<unistd.h>` for `time()` / `getpid()`
**Where:** Lines 28–35.
**Why:** Needed by `mimicry_select_os()`. Wrapped in `#ifdef _WIN32` for Windows compat.
---
### `src/templ-payloads.c` — UDP probes
#### 1. Stripped masscan literal strings from UDP probes
**Where:** Lines 67–74 (ports 7, 17, 19), line 90 (port 69 TFTP), line 130 (port 138 NetBIOS), line 243 (port 1701 L2TP), line 255 (port 1900 SSDP).
**What:**
| Port | Old payload | New payload |
| --- | --- | --- |
| 7 echo | `"masscan-test 0x00000000"` | 12 `\n` bytes |
| 17 qotd | `"masscan-test"` | 12 `\n` bytes |
| 19 chargen | `"masscan-test"` | 12 `\n` bytes |
| 69 TFTP | filename `"masscan-test"` | filename `"anonymous_cf"` |
| 138 SMB browser | NetBIOS-encoded `MASSCAN-TEST<00>` | NetBIOS-encoded `WORKSTATION<00>` |
| 1701 L2TP | hostname `"masscan1"` | hostname `"vpnhost1"` |
| 1900 SSDP | UA `unix/1.0 UPnP/1.1 masscan/1.x` | UA `unix/1.0 UPnP/1.1 Linux/1.0.0` |
**Why:** Any one of those literals appearing in a UDP probe is sufficient to fingerprint masscan instantly.
#### 2. Added `dtls_set_random` callback
**Where:** New function at lines 51–69 (after the `PayloadUDP_Item` struct), wired into the UDP/443 DTLS probe at line 182.
**What:** Overwrites the 32 bytes of DTLS ClientHello `client_random` (at payload offset 27) with `rand()` bytes for every probe.
**Why:** The original DTLS template had a fixed hardcoded random `1d b1 e3 52 …` — a masscan signature. Now it's regenerated per probe like a real DTLS client.
#### 3. Added BACnet/IP Who-Is probe
**Where:** Lines 491–504 (new entry in `hard_coded_udp_payloads[]`).
**Port:** 47808 UDP.
**What:** 12-byte BVLC Original-Broadcast-NPDU containing a Who-Is APDU. Response is an I-Am that reveals object ID, max APDU size, segmentation support, and vendor ID (which identifies the building automation manufacturer).
**Why:** Industrial protocol coverage. Before this masscan had zero BACnet support.
#### 4. Added EtherNet/IP UDP probe
**Where:** Lines 506–516 (new entry).
**Port:** 44818 UDP.
**What:** 24-byte CIP List Identity packet. Response reveals vendor, product code, revision, serial number, device name, and the device's IP — extremely rich Allen-Bradley / Rockwell device fingerprint.
**Why:** Same reason as BACnet — industrial reconnaissance.
---
### `src/proto-ssl.c` — TLS ClientHello
#### 1. Removed `const` from both templates
**Where:** Lines 1098, 1129 (template declarations).
**Old:** `static const char ssl_hello_template[] = …`
**New:** `char ssl_hello_template[] = …` (file-scope, writable, externally visible via forward declarations).
**Why:** The `client_random` field has to be overwritten by `ssl_init()` at startup. `static const` couldn't be modified.
#### 2. `ssl_init()` randomizes `client_random` at startup
**Where:** Lines 1062–1086.
**What:** First call to `ssl_init()` (one-time, gated by `static int randomized` flag) overwrites bytes 11..42 of both `ssl_hello_template` and `ssl_12_hello_template` with `rand()` bytes.
**Why:** The original masscan random `97 e5 60 50 c4 a5 4a e0 …` is the single most reliable masscan signature on the wire — it's literally the same 32 bytes on every TLS probe from every masscan ever. Now each scan run has a fresh random, and the static byte sequence is gone.
**Trade-off documented:** All probes in one scan run still share the same random. Per-probe randomization would require switching from a static `stream->hello` buffer to a `transmit_hello` callback. Deferred — defeats the static-bytes fingerprint, which is the actual public detector.
#### 3. Replaced `ssl_hello_template` with curl-shape ClientHello + ALPN
**Where:** Lines 1098–1163 (the template body).
**What:**
- **Version:** TLS 1.2 only (no `supported_versions` extension → servers don't negotiate TLS 1.3 → certs come back in cleartext).
- **Ciphers:** 21 modern TLS 1.2 ciphers, AEAD-first ordering (`ECDHE-*-AES256-GCM-SHA384` first, RSA fallbacks last, `EMPTY_RENEGOTIATION_INFO_SCSV` always).
- **Extensions:** `ec_point_formats`, `supported_groups` (x25519, secp256r1, secp384r1), `session_ticket`, `extended_master_secret`, `encrypt_then_mac`, `signature_algorithms` (13 algos), and the new **`application_layer_protocol_negotiation` extension advertising `h2, http/1.1`**.
**Why:** Original masscan TLS hello had unique cipher ordering + extension set that produced a masscan-specific JA3 hash. New version matches the JA3 of a curl 8.x request — passes most "block known scanners" lists.
**Why TLS 1.2 only, not Chrome 120:** If we advertised TLS 1.3 (via `supported_versions`), modern servers would pick TLS 1.3 and encrypt the certificate. Defeats the whole point of cert harvesting. The trade-off is being slightly less browser-like.
**Validation:** Cloudflare DNS (1.1.1.1:443) handshakes successfully, returns cipher 0xc02b, full cert chain + SAN list. Frantech scan: cert records went from 29 → 40 (+38%) with this hello vs. upstream.
---
### `src/proto-http.c` — HTTP probe
#### Replaced HTTP request line and headers
**Where:** Lines 386–401 (the `http_hello[]` static).
**Old:**
```
GET / HTTP/1.0\r\n
User-Agent: ivre-masscan/1.3 https://github.com/robertdavidgraham/\r\n
Accept: */*\r\n
\r\n
```
**New:** Full Chrome 120 (Linux) header set:
```
GET / HTTP/1.1\r\n
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8\r\n
Accept-Language: en-US,en;q=0.9\r\n
Accept-Encoding: gzip, deflate\r\n
Connection: close\r\n
\r\n
```
**Why:** The `ivre-masscan/1.3 https://github.com/robertdavidgraham/` User-Agent appearing in a web server log says "masscan" instantly. Chrome 120 with a full browser header set passes basic UA filters and looks like normal traffic in access logs. HTTP/1.0 → HTTP/1.1 because Chrome never sends 1.0.
---
### `src/stack-tcp-core.c` — Banner-grab TCP stack
#### Small-window kludge value changed
**Where:** Line 1167.
**Old:** `tcp_set_window(response->px, response->length, 600);`
**New:** `tcp_set_window(response->px, response->length, 32768);`
**Why:** The 600-byte window was a masscan-specific small-window kludge that combined with `MSS=1460` option in non-SYN segments to make the banner-grab signature trivially detectable. 32768 is a realistic Linux half-default.
---
### `src/proto-banner1.c` — Port → protocol bindings + new TCP hellos
This is the biggest single file change (+229 lines, no deletions). Everything is additive.
#### 1. New TCP hellos for application services
**Where:** Lines 487–521 (after `banner_ms_sql_s`).
| Protocol | Port | Probe sent | Yields |
| --- | --- | --- | --- |
| PostgreSQL | 5432 | 8-byte SSLRequest packet `\x00\x00\x00\x08\x04\xd2\x16\x2f` | `S`/`N` response confirming postgres; on `N` often errors with version |
| Cassandra | 9042, 9160 | CQL v3 STARTUP frame with `CQL_VERSION=3.0.0` | READY or AUTHENTICATE frame revealing protocol version |
#### 2. New TCP hellos for industrial / SCADA
**Where:** Lines 525–610.
| Protocol | Port | Probe | Yields |
| --- | --- | --- | --- |
| Siemens S7 | 102 | TPKT + COTP CR (rack 0, slot 0, dst TSAP = PG) | COTP CC with PLC presence |
| Modbus | 502 | MBAP + Read Device Identification (func 0x2B / 0x0E) | Vendor name, product code, revision |
| OPC UA | 4840 | HEL message with 1 MB receive buffer | ACK message with endpoint protocol version |
| DNP3 | 20000 | Link layer request, dst=0, src=1 | DNP3 link header revealing addresses |
| EtherNet/IP | 44818 | CIP List Identity command | Vendor, product, revision, serial, device name |
#### 3. Mass port → protocol registrations
**Where:** Lines 666–784 (inside `banner1_create()`).
Added bindings for ~70 ports:
- **HTTP** on 81, 280, 591, 631, 2000, 3000, 3001, 3128, 4000, 4040, 4567, 5000, 5601, 5984, 6080, 7001, 7070, 7474, 7547, 8000, 8001, 8008, 8009, 8086, 8088, 8090, 8123, 8161, 8200, 8222, 8400, 8500, 8545, 8800, 8888, 8983, 9000, 9043, 9080, 9090, 9091, 9100, 9200, 9300, 9418, 9981, 10000, 10255, 15672, 18083, 26657, 50070, 50075
- **TLS** on 4443, 5044, 5223, 5986, 6443, 7443, 8243, 8333, 8443, 8834, 8843, 9443, 10250, 10443
- **Redis** on 16379, 26379 (cluster bus + Sentinel)
- **memcached** on 11212 (alt)
- **etcd HTTP** on 2379, 2380 (overrides upstream Docker registration for these)
- **PostgreSQL** on 5432
- **Cassandra** on 9042, 9160
- **Industrial:** 102, 502, 4840, 20000, 44818
**Why:** Stock masscan only had ~30 baked-in port→protocol bindings. Every additional binding means a probe gets sent on that port without the user passing a CLI flag — much higher banner capture rate on modern app servers (Grafana, Kibana, Prometheus, k8s, Docker, Elasticsearch, RabbitMQ, etc.) and on industrial systems.
---
### `src/main-conf.c` — Auto-load nmap-payloads
#### Auto-detect installed nmap and load its UDP payload database
**Where:** Lines 3174–3208.
**What:** If `--nmap-payloads` was not passed (or was passed as empty), check three standard install paths in order:
1. `/usr/share/nmap/nmap-payloads`
2. `/usr/local/share/nmap/nmap-payloads`
3. `/opt/nmap/share/nmap/nmap-payloads`
If any exists, log `[+] auto-loading nmap-payloads from <path>` and use it as if `--nmap-payloads <path>` had been passed.
**Why:** nmap's `nmap-payloads` file ships ~300 UDP probes for protocols masscan doesn't have built-in. Loading it adds full coverage of those services for free, without changing the user's CLI. (Note: modern nmap on some distros — Ubuntu 24.04 included — compiles the payloads into the binary instead of shipping the file, so the auto-load is a no-op there. Still works on Alpine and older systems.)
---
## What the fork does **not** change
These are deliberately deferred:
- **No SNI extension in TLS hello.** The `server_name` extension would significantly improve cert harvesting from CDN-fronted hosts, but per-target SNI requires switching from `static char hello[]` to a `transmit_hello` callback. ~half-day of work; not done yet.
- **No Chrome 120 ClientHello with GREASE.** The curl-shape hello defeats the "is this masscan?" detection, which was the goal. Full Chrome JA3 mimicry only helps if you specifically need to evade Cloudflare WAF / Imperva / Akamai bot blockers.
- **No per-connection TCP timestamp simulation.** The Linux SYN template includes a Timestamp option with zero placeholders. Real Linux uses a tcp_clock millisecond counter and echoes the peer's TSval. Adding this requires per-TCB state — not free.
- **No multi-threaded banner-grab.** The userland TCP stack in `stack-tcp-core.c` is single-threaded by design. At very high banner-grab rates (theoretically) the receive thread saturates. Multi-threading it is real work and we haven't proved it's actually the bottleneck in practice.
- **CLI unchanged.** No new flags. The OS-mimic chosen at startup is random (50% Linux, 50% Windows) and there's no `--mimic-os <linux|windows>` flag. Add one if you want determinism.
---
## Validation done
| Test | How | Result |
| --- | --- | --- |
| Build cleanly | `make` with `-Wall -O2` | ✅ no warnings beyond upstream's |
| Real-world scan | Frantech /9 (72k hosts), 8 ports, `--banners`, `--rate 3000` | ✅ 34,897 opens found, ~309 banners completed, TLS cert chains parsed from Let's Encrypt |
| Cloudflare DNS TLS handshake | `masscan 1.1.1.1 -p443 --banners` | ✅ TLS/1.2 cipher 0xc02b, full SAN list (`cloudflare-dns.com, *.cloudflare-dns.com`) |
| Fingerprint detection (own tool) | Run `masscanned.py` against 500-packet capture (310 SYNs + 41 PSH-ACKs from real scan) | ✅ 0 hits across all 9 detector categories |
| SYN shape check | Inspect outbound SYN with scapy | ✅ TTL=128, win=64240, DF set, dataofs=8, options match Windows 10 byte-for-byte |
The fingerprint detector ([`masscanned.py`](../masscan/research/masscanned.py)) is the same one used in our research to identify masscan packets — it knows every public masscan signature. On 500 packets from this build, zero signatures fire.
---
## Open issues
Documented honestly because they exist:
1. **Banner-grab completion rate at high `--rate` is low.** On 35k open ports we got ~309 banners. That's either (a) most of those ports aren't running services that respond to our default hellos, (b) masscan's single-threaded userland stack saturates somewhere below 5–10k banner-grabs/sec, or (c) `--wait 10` isn't enough for the tail of slow targets. Unverified — we haven't measured the actual saturation point. Workaround: two-pass scanning (SYN-only first, targeted banner-grab pass at lower rate on the open set).
2. **One occasional `UNHANDLED EVENT` warning** during banner-grab: `[-] <ip>:443: FIN-WAIT-1-SEND:SYNACK **** UNHANDLED EVENT ****`. Pre-existing upstream behavior — happens when a target sends SYN-ACK during connection teardown. Harmless.
3. **TLS 1.2-only ClientHello means TLS-1.3-mandatory servers don't return clear-text certs.** A small minority of modern sites (mostly Cloudflare with HTTP/3 priority). Workaround: send a follow-up probe with TLS 1.3 advertisement — but then certs are encrypted and you only get the JA3-S.
4. **Some hello-string probes overlap with masscan's built-in pattern matching.** When the user generates a config file from `nmap-service-probes` and includes it, ports may get an unexpected hello (e.g., port 80 gets `OPTIONS / RTSP/1.0` instead of `GET /`). Last-wins / first-wins semantics handled by the Python generator, but worth understanding.
---
## How to use
Identical to upstream masscan. Examples:
```sh
# Simple scan
sudo bin/masscan 10.0.0.0/8 -p80,443 --rate 100000
# With banners
sudo bin/masscan -iL targets.txt -p- --rate 50000 --banners \
--output-format ndjson --output-filename results.ndjson
# Distributed (32 shards)
sudo bin/masscan --shards 1/32 --seed 42 -iL all-v4.txt ...
sudo bin/masscan --shards 2/32 --seed 42 -iL all-v4.txt ...
# etc.
```
Which OS template got picked this run? Check the first outbound SYN's TTL:
- TTL = 64 → Linux template active
- TTL = 128 → Windows template active
---
## Building
Standard masscan build. From the repo root:
```sh
make # gcc, -Wall -O2, single-threaded build
make clean # nuke tmp/
```
Binary lands at `bin/masscan`. Drop-in replacement for `/usr/bin/masscan`.
Dependencies same as upstream: `libpcap-dev`, `gcc`, `make`. No new deps.
---
## Diff against upstream
```sh
# Quick stats
diff -r /home/acidvegas/masscan-src/src ./src | grep -c "^[<>]"
# Full diff
diff -r /home/acidvegas/masscan-src/src ./src
```

661
LICENSE Normal file
View File

@@ -0,0 +1,661 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.

138
Makefile Normal file
View File

@@ -0,0 +1,138 @@
# If Windows, then assume the compiler is `gcc` for the
# MinGW environment. I can't figure out how to tell if it's
# actually MingGW. FIXME TODO
ifeq ($(OS),Windows_NT)
CC = gcc
endif
# Try to figure out the default compiler. I dont know the best
# way to do this with `gmake`. If you have better ideas, please
# submit a pull request on github.
ifeq ($(CC),)
ifneq (, $(shell which clang))
CC = clang
else ifneq (, $(shell which gcc))
CC = gcc
else
CC = cc
endif
endif
PREFIX ?= /usr
BINDIR ?= $(PREFIX)/bin
SYS := $(shell $(CC) -dumpmachine)
GITVER := $(shell git describe --tags)
INSTALL_DATA := -pDm755
ifeq ($(GITVER),)
GITVER = "unknown"
endif
# LINUX
# The automated regression tests run on Linux, so this is the one
# environment where things likely will work -- as well as anything
# works on the bajillion of different Linux environments
ifneq (, $(findstring linux, $(SYS)))
ifneq (, $(findstring musl, $(SYS)))
LIBS =
else
LIBS = -lm -lrt -ldl -lpthread
endif
INCLUDES =
FLAGS2 =
endif
# MAC OS X
# I occassionally develope code on Mac OS X, but it's not part of
# my regularly regression-test environment. That means at any point
# in time, something might be minorly broken in Mac OS X.
ifneq (, $(findstring darwin, $(SYS)))
LIBS = -lm
INCLUDES = -I.
FLAGS2 =
INSTALL_DATA = -pm755
endif
# MinGW on Windows
# I develope on Visual Studio 2010, so that's the Windows environment
# that'll work. However, 'git' on Windows runs under MingGW, so one
# day I acccidentally typed 'make' instead of 'git, and felt compelled
# to then fix all the errors, so this kinda works now. It's not the
# intended environment, so it make break in the future.
ifneq (, $(findstring mingw, $(SYS)))
INCLUDES = -Ivs10/include
LIBS = -L vs10/lib -lIPHLPAPI -lWs2_32
#FLAGS2 = -march=i686
endif
# Cygwin
# I hate Cygwin, use Visual Studio or MingGW instead. I just put this
# second here for completeness, or in case I gate tired of hitting my
# head with a hammer and want to feel a different sort of pain.
ifneq (, $(findstring cygwin, $(SYS)))
INCLUDES = -I.
LIBS =
FLAGS2 =
endif
# OpenBSD
ifneq (, $(findstring openbsd, $(SYS)))
LIBS = -lm -lpthread
INCLUDES = -I.
FLAGS2 =
endif
# FreeBSD
ifneq (, $(findstring freebsd, $(SYS)))
LIBS = -lm -lpthread
INCLUDES = -I.
FLAGS2 =
endif
# NetBSD
ifneq (, $(findstring netbsd, $(SYS)))
LIBS = -lm -lpthread
INCLUDES = -I.
FLAGS2 =
endif
DEFINES =
CFLAGS = -g -ggdb $(FLAGS2) $(INCLUDES) $(DEFINES) -Wall -O2
.SUFFIXES: .c .cpp
all: bin/masscan
tmp/main-conf.o: src/main-conf.c src/*.h
$(CC) $(CFLAGS) -c $< -o $@ -DGIT=\"$(GITVER)\"
# just compile everything in the 'src' directory. Using this technique
# means that include file dependencies are broken, so sometimes when
# the program crashes unexpectedly, 'make clean' then 'make' fixes the
# problem that a .h file was out of date
tmp/%.o: src/%.c src/*.h
$(CC) $(CFLAGS) -c $< -o $@
SRC = $(sort $(wildcard src/*.c))
OBJ = $(addprefix tmp/, $(notdir $(addsuffix .o, $(basename $(SRC)))))
bin/masscan: $(OBJ)
$(CC) $(CFLAGS) -o $@ $(OBJ) $(LDFLAGS) $(LIBS)
clean:
rm -f tmp/*.o
rm -f bin/masscan
regress: bin/masscan
bin/masscan --selftest
test: regress
install: bin/masscan
install $(INSTALL_DATA) bin/masscan $(DESTDIR)$(BINDIR)/masscan
default: bin/masscan

617
README.md Normal file
View File

@@ -0,0 +1,617 @@
[![unittests](https://github.com/robertdavidgraham/masscan/actions/workflows/unittests.yml/badge.svg?branch=master)](https://github.com/robertdavidgraham/masscan/actions/workflows/unittests.yml/?branch=master)
# MASSCAN: Mass IP port scanner
This is an Internet-scale port scanner. It can scan the entire Internet
in under 5 minutes, transmitting 10 million packets per second,
from a single machine.
Its usage (parameters, output) is similar to `nmap`, the most famous port scanner.
When in doubt, try one of those features -- features that support widespread
scanning of many machines are supported, while in-depth scanning of single
machines aren't.
Internally, it uses asynchronous transmission, similar to port scanners
like `scanrand`, `unicornscan`, and `ZMap`. It's more flexible, allowing
arbitrary port and address ranges.
NOTE: masscan uses its own **ad hoc TCP/IP stack**. Anything other than
simple port scans may cause conflict with the local TCP/IP stack. This means you
need to use either the `--src-ip` option to run from a different IP address, or
use `--src-port` to configure which source ports masscan uses, then also
configure the internal firewall (like `pf` or `iptables`) to firewall those ports
from the rest of the operating system.
This tool is free, but consider contributing money to its development:
Bitcoin wallet address: 1MASSCANaHUiyTtR3bJ2sLGuMw5kDBaj4T
# Building
On Debian/Ubuntu, it goes something like the following. It doesn't
really have any dependencies other than a C compiler (such as `gcc`
or `clang`).
sudo apt-get --assume-yes install git make gcc
git clone https://github.com/robertdavidgraham/masscan
cd masscan
make
This puts the program in the `masscan/bin` subdirectory.
To install it (on Linux) run:
make install
The source consists of a lot of small files, so building goes a lot faster
by using the multi-threaded build. This requires more than 2gigs on a
Raspberry Pi (and breaks), so you might use a smaller number, like `-j4` rather than
all possible threads.
make -j
While Linux is the primary target platform, the code runs well on many other
systems (Windows, macOS, etc.). Here's some additional build info:
* Windows w/ Visual Studio: use the VS10 project
* Windows w/ MinGW: just type `make`
* Windows w/ cygwin: won't work
* Mac OS X /w XCode: use the XCode4 project
* Mac OS X /w cmdline: just type `make`
* FreeBSD: type `gmake`
* other: try just compiling all the files together, `cc src/*.c -o bin/masscan`
On macOS, the x86 binaries seem to work just as fast under ARM emulation.
# Usage
Usage is similar to `nmap`. To scan a network segment for some ports:
# masscan -p80,8000-8100 10.0.0.0/8 2603:3001:2d00:da00::/112
This will:
* scan the `10.x.x.x` subnet, and `2603:3001:2d00:da00::x` subnets
* scans port 80 and the range 8000 to 8100, or 102 ports total, on both subnets
* print output to `<stdout>` that can be redirected to a file
To see the complete list of options, use the `--echo` feature. This
dumps the current configuration and exits. This output can be used as input back
into the program:
# masscan -p80,8000-8100 10.0.0.0/8 2603:3001:2d00:da00::/112 --echo > xxx.conf
# masscan -c xxx.conf --rate 1000
## Banner checking
Masscan can do more than just detect whether ports are open. It can also
complete the TCP connection and interaction with the application at that
port in order to grab simple "banner" information.
Masscan supports banner checking on the following protocols:
* FTP
* HTTP
* IMAP4
* memcached
* POP3
* SMTP
* SSH
* SSL
* SMBv1
* SMBv2
* Telnet
* RDP
* VNC
The problem with this is that masscan contains its own TCP/IP stack
separate from the system you run it on. When the local system receives
a SYN-ACK from the probed target, it responds with a RST packet that kills
the connection before masscan can grab the banner.
The easiest way to prevent this is to assign masscan a separate IP
address. This would look like one of the following examples:
# masscan 10.0.0.0/8 -p80 --banners --source-ip 192.168.1.200
# masscan 2a00:1450:4007:810::/112 -p80 --banners --source-ip 2603:3001:2d00:da00:91d7:b54:b498:859d
The address you choose has to be on the local subnet and not otherwise
be used by another system. Masscan will warn you that you've made a
mistake, but you might've messed up the other machine's communications
for several minutes, so be careful.
In some cases, such as WiFi, this isn't possible. In those cases, you can
firewall the port that masscan uses. This prevents the local TCP/IP stack
from seeing the packet, but masscan still sees it since it bypasses the
local stack. For Linux, this would look like:
# iptables -A INPUT -p tcp --dport 61000 -j DROP
# masscan 10.0.0.0/8 -p80 --banners --source-port 61000
You probably want to pick ports that don't conflict with ports Linux might otherwise
choose for source-ports. You can see the range Linux uses, and reconfigure
that range, by looking in the file:
/proc/sys/net/ipv4/ip_local_port_range
On the latest version of Kali Linux (2018-August), that range is 32768 to 60999, so
you should choose ports either below 32768 or 61000 and above.
Setting an `iptables` rule only lasts until the next reboot. You need to lookup how to
save the configuration depending upon your distro, such as using `iptables-save`
and/or `iptables-persistent`.
On Mac OS X and BSD, there are similar steps. To find out the ranges to avoid,
use a command like the following:
# sysctl net.inet.ip.portrange.first net.inet.ip.portrange.last
On FreeBSD and older MacOS, use an `ipfw` command:
# sudo ipfw add 1 deny tcp from any to any 40000 in
# masscan 10.0.0.0/8 -p80 --banners --source-port 40000
On newer MacOS and OpenBSD, use the `pf` packet-filter utility.
Edit the file `/etc/pf.conf` to add a line like the following:
block in proto tcp from any to any port 40000:40015
Then to enable the firewall, run the command:
# pfctl -E
If the firewall is already running, then either reboot or reload the rules
with the following command:
# pfctl -f /etc/pf.conf
Windows doesn't respond with RST packets, so neither of these techniques
are necessary. However, masscan is still designed to work best using its
own IP address, so you should run that way when possible, even when it is
not strictly necessary.
The same thing is needed for other checks, such as the `--heartbleed` check,
which is just a form of banner checking.
## How to scan the entire Internet
While useful for smaller, internal networks, the program is really designed
with the entire Internet in mind. It might look something like this:
# masscan 0.0.0.0/0 -p0-65535
Scanning the entire Internet is bad. For one thing, parts of the Internet react
badly to being scanned. For another thing, some sites track scans and add you
to a ban list, which will get you firewalled from useful parts of the Internet.
Therefore, you want to exclude a lot of ranges. To blacklist or exclude ranges,
you want to use the following syntax:
# masscan 0.0.0.0/0 -p0-65535 --excludefile exclude.txt
This just prints the results to the command-line. You probably want them
saved to a file instead. Therefore, you want something like:
# masscan 0.0.0.0/0 -p0-65535 -oX scan.xml
This saves the results in an XML file, allowing you to easily dump the
results in a database or something.
But, this only goes at the default rate of 100 packets/second, which will
take forever to scan the Internet. You need to speed it up as so:
# masscan 0.0.0.0/0 -p0-65535 --max-rate 100000
This increases the rate to 100,000 packets/second, which will scan the
entire Internet (minus excludes) in about 10 hours per port (or 655,360 hours
if scanning all ports).
The thing to notice about this command-line is that these are all `nmap`
compatible options. In addition, "invisible" options compatible with `nmap`
are also set for you: `-sS -Pn -n --randomize-hosts --send-eth`. Likewise,
the format of the XML file is inspired by `nmap`. There are, of course, a
lot of differences, because the *asynchronous* nature of the program
leads to a fundamentally different approach to the problem.
The above command-line is a bit cumbersome. Instead of putting everything
on the command-line, it can be stored in a file instead. The above settings
would look like this:
# My Scan
rate = 100000.00
output-format = xml
output-status = all
output-filename = scan.xml
ports = 0-65535
range = 0.0.0.0-255.255.255.255
excludefile = exclude.txt
To use this configuration file, use the `-c`:
# masscan -c myscan.conf
This also makes things easier when you repeat a scan.
By default, masscan first loads the configuration file
`/etc/masscan/masscan.conf`. Any later configuration parameters override what's
in this default configuration file. That's where I put my "excludefile"
parameter so that I don't ever forget it. It just works automatically.
## Getting output
By default, masscan produces fairly large text files, but it's easy
to convert them into any other format. There are five supported output formats:
1. xml: Just use the parameter `-oX <filename>`.
Or, use the parameters `--output-format xml` and `--output-filename <filename>`.
2. binary: This is the masscan builtin format. It produces much smaller files so that
when I scan the Internet my disk doesn't fill up. They need to be parsed,
though. The command-line option `--readscan` will read binary scan files.
Using `--readscan` with the `-oX` option will produce an XML version of the
results file.
3. grepable: This is an implementation of the Nmap -oG
output that can be easily parsed by command-line tools. Just use the
parameter `-oG <filename>`. Or, use the parameters `--output-format grepable` and
`--output-filename <filename>`.
4. json: This saves the results in JSON format. Just use the
parameter `-oJ <filename>`. Or, use the parameters `--output-format json` and
`--output-filename <filename>`.
5. list: This is a simple list with one host and port pair
per line. Just use the parameter `-oL <filename>`. Or, use the parameters
`--output-format list` and `--output-filename <filename>`. The format is:
```
<port state> <protocol> <port number> <IP address> <POSIX timestamp>
open tcp 80 XXX.XXX.XXX.XXX 1390380064
```
## Comparison with Nmap
Where reasonable, every effort has been taken to make the program familiar
to `nmap` users, even though it's fundamentally different. Masscan is tuned
for wide range scanning of a lot of machines, whereas nmap is designed for
intensive scanning of a single machine or a small range.
Two important differences are:
* no default ports to scan, you must specify `-p <ports>`
* target hosts are IP addresses or simple ranges, not DNS names, nor
the funky subnet ranges `nmap` can use (like `10.0.0-255.0-255`).
You can think of `masscan` as having the following settings permanently
enabled:
* `-sS`: this does SYN scan only (currently, will change in the future)
* `-Pn`: doesn't ping hosts first, which is fundamental to the async operation
* `-n`: no DNS resolution happens
* `--randomize-hosts`: scan completely randomized, always, you can't change this
* `--send-eth`: sends using raw `libpcap`
If you want a list of additional `nmap` compatible settings, use the following
command:
# masscan --nmap
## Transmit rate (IMPORTANT!!)
This program spews out packets very fast. On Windows, or from VMs,
it can do 300,000 packets/second. On Linux (no virtualization) it'll
do 1.6 million packets-per-second. That's fast enough to melt most networks.
Note that it'll only melt your own network. It randomizes the target
IP addresses so that it shouldn't overwhelm any distant network.
By default, the rate is set to 100 packets/second. To increase the rate to
a million use something like `--rate 1000000`.
When scanning the IPv4 Internet, you'll be scanning lots of subnets,
so even though there's a high rate of packets going out, each
target subnet will receive a small rate of incoming packets.
However, with IPv6 scanning, you'll tend to focus on a single
target subnet with billions of addresses. Thus, your default
behavior will overwhelm the target network. Networks often
crash under the load that masscan can generate.
# Design
This section describes the major design issues of the program.
## Code Layout
The file `main.c` contains the `main()` function, as you'd expect. It also
contains the `transmit_thread()` and `receive_thread()` functions. These
functions have been deliberately flattened and heavily commented so that you
can read the design of the program simply by stepping line-by-line through
each of these.
## Asynchronous
This is an *asynchronous* design. In other words, it is to `nmap` what
the `nginx` web-server is to `Apache`. It has separate transmit and receive
threads that are largely independent from each other. It's the same sort of
design found in `scanrand`, `unicornscan`, and `ZMap`.
Because it's asynchronous, it runs as fast as the underlying packet transmit
allows.
## Randomization
A key difference between Masscan and other scanners is the way it randomizes
targets.
The fundamental principle is to have a single index variable that starts at
zero and is incremented by one for every probe. In C code, this is expressed
as:
for (i = 0; i < range; i++) {
scan(i);
}
We have to translate the index into an IP address. Let's say that you want to
scan all "private" IP addresses. That would be the table of ranges like:
192.168.0.0/16
10.0.0.0/8
172.16.0.0/12
In this example, the first 64k indexes are appended to 192.168.x.x to form
the target address. Then, the next 16-million are appended to 10.x.x.x.
The remaining indexes in the range are applied to 172.16.x.x.
In this example, we only have three ranges. When scanning the entire Internet,
we have in practice more than 100 ranges. That's because you have to blacklist
or exclude a lot of sub-ranges. This chops up the desired range into hundreds
of smaller ranges.
This leads to one of the slowest parts of the code. We transmit 10 million
packets per second and have to convert an index variable to an IP address
for each and every probe. We solve this by doing a "binary search" in a small
amount of memory. At this packet rate, cache efficiencies start to dominate
over algorithm efficiencies. There are a lot of more efficient techniques in
theory, but they all require so much memory as to be slower in practice.
We call the function that translates from an index into an IP address
the `pick()` function. In use, it looks like:
for (i = 0; i < range; i++) {
ip = pick(addresses, i);
scan(ip);
}
Masscan supports not only IP address ranges, but also port ranges. This means
we need to pick from the index variable both an IP address and a port. This
is fairly straightforward:
range = ip_count * port_count;
for (i = 0; i < range; i++) {
ip = pick(addresses, i / port_count);
port = pick(ports, i % port_count);
scan(ip, port);
}
This leads to another expensive part of the code. The division/modulus
instructions are around 90 clock cycles, or 30 nanoseconds, on x86 CPUs. When
transmitting at a rate of 10 million packets/second, we have only
100 nanoseconds per packet. I see no way to optimize this any better. Luckily,
though, two such operations can be executed simultaneously, so doing two
of these, as shown above, is no more expensive than doing one.
There are actually some easy optimizations for the above performance problems,
but they all rely upon `i++`, the fact that the index variable increases one
by one through the scan. Actually, we need to randomize this variable. We
need to randomize the order of IP addresses that we scan or we'll blast the
heck out of target networks that aren't built for this level of speed. We
need to spread our traffic evenly over the target.
The way we randomize is simply by encrypting the index variable. By definition,
encryption is random and creates a 1-to-1 mapping between the original index
variable and the output. This means that while we linearly go through the
range, the output IP addresses are completely random. In code, this looks like:
range = ip_count * port_count;
for (i = 0; i < range; i++) {
x = encrypt(i);
ip = pick(addresses, x / port_count);
port = pick(ports, x % port_count);
scan(ip, port);
}
This also has a major cost. Since the range is an unpredictable size instead
of a nice even power of 2, we can't use cheap binary techniques like
AND (&) and XOR (^). Instead, we have to use expensive operations like
MODULUS (%). In my current benchmarks, it's taking 40 nanoseconds to
encrypt the variable.
This architecture allows for lots of cool features. For example, it supports
"shards". You can set up 5 machines each doing a fifth of the scan or
`range / shard_count`. Shards can be multiple machines, or simply multiple
network adapters on the same machine, or even (if you want) multiple IP
source addresses on the same network adapter.
Or, you can use a 'seed' or 'key' to the encryption function, so that you get
a different order each time you scan, like `x = encrypt(seed, i)`.
We can also pause the scan by exiting out of the program, and simply
remembering the current value of `i`, and restart it later. I do that a lot
during development. I see something going wrong with my Internet scan, so
I hit <ctrl-c> to stop the scan, then restart it after I've fixed the bug.
Another feature is retransmits/retries. Packets sometimes get dropped on the
Internet, so you can send two packets back-to-back. However, something that
drops one packet may drop the immediately following packet. Therefore, you
want to send the copy about 1 second apart. This is simple. We already have
a 'rate' variable, which is the number of packets-per-second rate we are
transmitting at, so the retransmit function is simply to use `i + rate`
as the index. One of these days I'm going to do a study of the Internet,
and differentiate "back-to-back", "1 second", "10 second", and "1 minute"
retransmits this way in order to see if there is any difference in what
gets dropped.
## C10 Scalability
The asynchronous technique is known as a solution to the "c10k problem".
Masscan is designed for the next level of scalability, the "C10M problem".
The C10M solution is to bypass the kernel. There are three primary kernel
bypasses in Masscan:
* custom network driver
* user-mode TCP stack
* user-mode synchronization
Masscan can use the PF_RING DNA driver. This driver DMAs packets directly
from user-mode memory to the network driver with zero kernel involvement.
That allows software, even with a slow CPU, to transmit packets at the maximum
rate the hardware allows. If you put 8 10-gbps network cards in a computer,
this means it could transmit at 100-million packets/second.
Masscan has its own built-in TCP stack for grabbing banners from TCP
connections. This means it can easily support 10 million concurrent TCP
connections, assuming of course that the computer has enough memory.
Masscan has no "mutex". Modern mutexes (aka. futexes) are mostly user-mode,
but they have two problems. The first problem is that they cause cache-lines
to bounce quickly back-and-forth between CPUs. The second is that when there
is contention, they'll do a system call into the kernel, which kills
performance. A mutex on the fast path of a program severely limits scalability.
Instead, Masscan uses "rings" to synchronize things, such as when the
user-mode TCP stack in the receive thread needs to transmit a packet without
interfering with the transmit thread.
## Portability
The code runs well on Linux, Windows, and Mac OS X. All the important bits are
in standard C (C90). Therefore, it compiles on Visual Studio with Microsoft's
compiler, the Clang/LLVM compiler on Mac OS X, and GCC on Linux.
Windows and Macs aren't tuned for packet transmit, and get only about 300,000
packets-per-second, whereas Linux can do 1,500,000 packets/second. That's
probably faster than you want anyway.
## Safe code
A bounty is offered for vulnerabilities, see the VULNINFO.md file for more
information.
This project uses safe functions like `safe_strcpy()` instead of unsafe functions
like `strcpy()`.
This project has automated unit regression tests (`make regress`).
## Compatibility
A lot of effort has gone into making the input/output look like `nmap`, which
everyone who does port scans is (or should be) familiar with.
## IPv6 and IPv4 coexistence
Masscan supports IPv6, but there is no special mode, both are supported
at the same time. (There is no `-6` option -- it's always available).
In any example you see of masscan usage,
simply put an IPv6 address where you see an IPv4 address. You can include
IPv4 and IPv6 addresses simultaneously in the same scan. Output includes
the appropriate address at the same location, with no special marking.
Just remember that IPv6 address space is really big. You probably don't want to scan
for big ranges, except maybe the first 64k addresses of a subnet that were assigned
via DHCPv6.
Instead, you'll probably want to scan large lists of addresses stored
in a file (`--include-file filename.txt`) that you got from other sources.
Like everywhere else, this file can contain lists of both IPv4 and IPv6 addresses.
The test file I use contains 8 million addresses. Files of that size need a couple
extra seconds to be read on startup (masscan sorts the addresses and removes
duplicates before scanning).
Remember that masscan contains its own network stack. Thus, the local machine
you run masscan from does not need to be IPv6 enabled -- though the local
network needs to be able to route IPv6 packets.
## PF_RING
To get beyond 2 million packets/second, you need an Intel 10-gbps Ethernet
adapter and a special driver known as ["PF_RING ZC" from ntop](https://www.ntop.org/installation-guide-for-pf_ring/). Masscan doesn't need to be rebuilt in order to use PF_RING. To use PF_RING,
you need to build the following components:
* `libpfring.so` (installed in /usr/lib/libpfring.so)
* `pf_ring.ko` (their kernel driver)
* `ixgbe.ko` (their version of the Intel 10-gbps Ethernet driver)
You don't need to build their version of `libpcap.so`.
When Masscan detects that an adapter is named something like `zc:enp1s0` instead
of something like `enp1s0`, it'll automatically switch to PF_RING ZC mode.
A more detail discussion can be found in **PoC||GTFO 0x15**.
## Regression testing
The project contains a built-in unit test:
$ make test
bin/masscan --selftest
selftest: success!
This tests a lot of tricky bits of the code. You should do this after building.
## Performance testing
To test performance, run something like the following to a throw-away address,
to avoid overloading your local router:
$ bin/masscan 0.0.0.0/4 -p80 --rate 100000000 --router-mac 66-55-44-33-22-11
The bogus `--router-mac` keeps packets on the local network segments so that
they won't go out to the Internet.
You can also test in "offline" mode, which is how fast the program runs
without the transmit overhead:
$ bin/masscan 0.0.0.0/4 -p80 --rate 100000000 --offline
This second benchmark shows roughly how fast the program would run if it were
using PF_RING, which has near zero overhead.
By the way, the randomization algorithm makes heavy use of "integer arithmetic",
a chronically slow operation on CPUs. Modern CPUs have doubled the speed
at which they perform this calculation, making `masscan` much faster.
# Authors
This tool created by Robert Graham:
email: robert_david_graham@yahoo.com
twitter: @ErrataRob
# License
Copyright (c) 2013 Robert David Graham
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.

69
VULNINFO.md Normal file
View File

@@ -0,0 +1,69 @@
# Vulnerability Information and Policy
This document contains information about robustness of this project against
hacker attacks. It describes known vulnerabilities that have been found
in previous versions, and describes policies how vulnerabilities are handled.
## Security contact
robert_david_graham@yahoo.com
@ErrataRob on twitter
## Known vulnerabilities and advisories
none
## Bounty
I'm offering $100, payable in cash or Bitcoin, for security vulnerabilities.
This is primarily for remote vulnerabilities, such as the ability of a target
to buffer-overflow the scanner, or even cause it to crash.
But I'd consider other vulnerabilities as well. Does Kali ship this with suid
and there's a preload bug? That's not really a vuln in this code, but if it's
something I could fix, I'd consider paying a bounty for it.
## Disclosure policy
If you've got a vuln, just announce it. Please send info to the contact above
as well, please.
I'll probably get around to fixing it within a month or so. This really isn't
heavily used software, so I'm lax on this.
## Threats
The primary threat is from hostile targets on the Internet sending back
responses in order to:
* exploit a buffer-overflow vulnerability
* spoof packets trying to give fraudulent scan results (mitigated with our
SYN cookies)
* flood packets trying to overload bandwidth/storage
* bad data, such as corrupting banners or DNS names trying to exploit
downstream consumers with bad html or script tags.
The secondary threat is from use of the program. For example, when a bad
parameter is entered on the command-line, the program spits it back out
in a helpful error message. This is fine for a command-line program that
should run as `root` anyway, but if somebody tries to make it into a
scriptable service, this becomes a potential vulnerability.
## Safe code policy
Unsafe functions like `strcpy()` are banned.
The code contains an automated regression test by running with the
`--regress` option. However, currently the regression only tests
a small percentage of the code.

4
bin/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
*
!.gitignore

BIN
data/afl-http.pcap Normal file
View File

Binary file not shown.

513
data/exclude.conf Normal file
View File

@@ -0,0 +1,513 @@
# http://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml
# http://tools.ietf.org/html/rfc5735
# "This" network
0.0.0.0/8
# Private networks
10.0.0.0/8
# Carrier-grade NAT - RFC 6598
100.64.0.0/10
# Host loopback
127.0.0.0/8
# Link local
169.254.0.0/16
# Private networks
172.16.0.0/12
# IETF Protocol Assignments
192.0.0.0/24
# DS-Lite
192.0.0.0/29
# NAT64
192.0.0.170/32
# DNS64
192.0.0.171/32
# Documentation (TEST-NET-1)
192.0.2.0/24
# 6to4 Relay Anycast
192.88.99.0/24
# Private networks
192.168.0.0/16
# Benchmarking
198.18.0.0/15
# Documentation (TEST-NET-2)
198.51.100.0/24
# Documentation (TEST-NET-3)
203.0.113.0/24
# Reserved
240.0.0.0/4
# Limited Broadcast
255.255.255.255/32
#Received: from elbmasnwh002.us-ct-eb01.gdeb.com ([153.11.13.41]
# helo=ebsmtp.gdeb.com) by mx1.gd-ms.com with esmtp (Exim 4.76) (envelope-from
# <bmandes@gdeb.com>) id 1VS55c-0004qL-0F for support@erratasec.com; Fri, 04
# Oct 2013 09:06:40 -0400
#To: <support@erratasec.com>
#CC: <ebsoc@gdeb.com>
#Subject: Scanning and Probing our network
#From: Robert Mandes <bmandes@gdeb.com>
#Date: Fri, 4 Oct 2013 09:06:36 -0400
#
#Stop scanning and probing our network, 153.11.0.0/16. We are a defense
#contractor and report to Federal law enforcement authorities when scans
#and probes are directed at our network. I assume you don't want to be
#part of that report. Please permanently remove our network range from
#your current and future research.
#
#Thank you
#
#Robert Mandes
#Information Security Officer
#General Dynamics
#Electric Boat
#
#C 860-625-0605
#P 860-433-1553
153.11.0.0/16
#Date: Mon, 7 Oct 2013 17:25:41 -0700
#Subject: Re: please stop the attack to our router
#From: Di Li <di@egihosting.com>
#
#Make sure you stop the scan immediately, that's not OK for any company or
#organization scan our network at all.
#
#If you fail to do that we will block whole traffic from ASN 10439, and we
#will file a police report after that.
#
#Let me know when you stop, since we still receive the attack from you, and
#by the way your scan are not going anywhere, it's was dropped from our edge
#since the first 5 scan
#
#Oct 7 17:17:32:I:SNMP: Auth. failure, intruder IP: 209.126.230.72
#...
#Oct 7 16:55:27:I:SNMP: Auth. failure, intruder IP: 209.126.230.72
#
#Di
4.53.201.0/24
5.152.179.0/24
8.12.162.0-8.12.164.255
8.14.84.0/22
8.14.145.0-8.14.147.255
8.17.250.0-8.17.252.255
23.27.0.0/16
23.231.128.0/17
37.72.172.0/23
38.72.200.0/22
50.93.192.0-50.93.197.255
50.115.128.0/20
50.117.0.0/17
50.118.128.0/17
63.141.222.0/24
64.62.253.0/24
64.92.96.0/19
64.145.79.0/24
64.145.82.0/23
64.158.146.0/23
65.49.24.0/24
65.49.93.0/24
65.162.192.0/22
66.79.160.0/19
66.160.191.0/24
68.68.96.0/20
69.46.64.0/19
69.176.80.0/20
72.13.80.0/20
72.52.76.0/24
74.82.43.0/24
74.82.160.0/19
74.114.88.0/22
74.115.0.0/24
74.115.2.0/24
74.115.4.0/24
74.122.100.0/22
75.127.0.0/24
103.251.91.0/24
108.171.32.0/24
108.171.42.0/24
108.171.52.0/24
108.171.62.0/24
118.193.78.0/23
130.93.16.0/23
136.0.0.0/16
142.111.0.0/16
142.252.0.0/16
146.82.55.93
149.54.136.0/21
149.54.152.0/21
166.88.0.0/16
172.252.0.0/16
173.245.64.0/19
173.245.194.0/23
173.245.220.0/22
173.252.192.0/18
178.18.16.0/22
178.18.26.0-178.18.29.255
183.182.22.0/24
192.92.114.0/24
192.155.160.0/19
192.177.0.0/16
192.186.0.0/18
192.249.64.0/20
192.250.240.0/20
194.110.214.0/24
198.12.120.0-198.12.122.255
198.144.240.0/20
199.33.120.0/24
199.33.124.0/22
199.48.147.0/24
199.68.196.0/22
199.127.240.0/21
199.187.168.0/22
199.188.238.0/23
199.255.208.0/24
203.12.6.0/24
204.13.64.0/21
204.16.192.0/21
204.19.238.0/24
204.74.208.0/20
205.159.189.0/24
205.164.0.0/18
205.209.128.0/18
206.108.52.0/23
206.165.4.0/24
208.77.40.0/21
208.80.4.0/22
208.123.223.0/24
209.51.185.0/24
209.54.48.0/20
209.107.192.0/23
209.107.210.0/24
209.107.212.0/24
211.156.110.0/23
216.83.33.0-216.83.49.255
216.83.51.0-216.83.63.255
216.151.183.0/24
216.151.190.0/23
216.172.128.0/19
216.185.36.0/24
216.218.233.0/24
216.224.112.0/20
#Received: from [194.77.40.242] (HELO samba.agouros.de)
# for abuse@erratasec.com; Sat, 12 Oct 2013 09:55:35 -0500
#Received: from rumba.agouros.de (rumba-internal [192.168.8.1]) by
# samba.agouros.de (Postfix) with ESMTPS id 9055FBAD1D for
# <abuse@erratasec.com>; Sat, 12 Oct 2013 16:55:32 +0200 (CEST)
#Received: from rumba.agouros.de (localhost [127.0.0.1]) by rumba.agouros.de
# (Postfix) with ESMTP id 7B5DD206099 for <abuse@erratasec.com>; Sat, 12 Oct
# 2013 16:55:32 +0200 (CEST)
#Received: from localhost.localdomain (localhost [127.0.0.1]) by
# rumba.agouros.de (Postfix) with ESMTP id 5FBC420601D for
# <abuse@erratasec.com>; Sat, 12 Oct 2013 16:55:32 +0200 (CEST)
#To: <abuse@erratasec.com>
#Subject: Loginattempts from Your net
#Message-ID: <20131012145532.5FBC420601D@rumba.agouros.de>
#Date: Sat, 12 Oct 2013 16:55:32 +0200
#From: <elwood@agouros.de>
#
#The address 209.126.230.72 from Your network tried to log in to
#our network using Port 22 (1)/tcp. Below You will find a listing of the dates and
#times the incidents occured as well as the attacked IP-Addresses.
#This is a matter of concern for us and continued tries might result in
#legal action. If the machine was victim to a hack take it offline, repair
#the damage and use better protection next time.
#The times included are in Central European (Summer) Time.
#Date Sourceip port destips
#
#07.10.2013 22:34:40 CEST 209.126.230.72 22 194.77.40.242 (1)
#08.10.2013 01:44:15 CEST 209.126.230.72 22 194.77.40.246 (1)
#
#Regards,
#Konstantin Agouros
194.77.40.242
194.77.40.246
#Received: from [165.160.9.58] (HELO mx2.cscinfo.com)
#X-Virus-Scanned: amavisd-new at cscinfo.com
#Received: from mx2.cscinfo.com ([127.0.0.1]) by localhost
# (plmail02.wil.csc.local [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id
# GGQ7EiQaK2P0 for <protodev@erratasec.com>; Wed, 30 Oct 2013 09:26:00 -0400
# (EDT)
#Received: from casarray.cscinfo.com (pwmailch02.cscinfo.com [172.20.53.94]) by
# mx2.cscinfo.com (Postfix) with ESMTPS id 4BA5E58170 for
# <protodev@erratasec.com>; Wed, 30 Oct 2013 09:26:00 -0400 (EDT)
#Received: from PWMAILM02.cscinfo.com ([169.254.7.52]) by
# PWMAILCH02.cscinfo.com ([172.20.53.94]) with mapi id 14.02.0247.003; Wed, 30
# Oct 2013 09:26:00 -0400
#From: "Derksen, Bill" <bderksen@cscinfo.com>
#Subject: Unauthorized Scanning
#Date: Wed, 30 Oct 2013 13:25:59 +0000
#Message-ID: <1F80316A0C861F40A9A88F18465F138E01EF885F@PWMAILM02.cscinfo.com>
#x-originating-ip: [172.31.252.72]
#
#We have detected unauthorized activity from your systems on our public netw=
#ork. Please suspend scanning of our networks immediately.
#
#Our network block is 165.160/16
#
#Further scanning will result in reports of unauthorized activity being file=
#d with law enforcement agencies.
#
#Corporation Service Company
#
#
#
#________________________________
#
#NOTICE: This e-mail and any attachments is intended only for use by the add=
#ressee(s) named herein and may contain legally privileged, proprietary or c=
#onfidential information. If you are not the intended recipient of this e-ma=
#il, you are hereby notified that any dissemination, distribution or copying=
# of this email, and any attachments thereto, is strictly prohibited. If you=
# receive this email in error please immediately notify me via reply email o=
#r at (800) 927-9800 and permanently delete the original copy and any copy o=
#f any e-mail, and any printout.
165.160.0.0/16
#******************************
#Greetings from the IT Security Team at Utah State University.
#
#We have detected network activity that might be suspicious or
#malicious. We think it might be sourced from your network. We
#include IP Addresses as well as description, log snippets, and
#other useful information.
#
#Please review this information or forward to the responsible person.
129.123.0.0/16
144.39.0.0/16
204.113.91.0/24
#On Friday, November 17th 2017 starting at 03:39 EST (UTC-5:00), part of the
#Physics Network at McGill University (132.206.9.0/24, 132.206.123.0/24
#and/or 132.206.125.0/24) was scanned from xxx.xxx.xxx.xxx (see syslog
#snippet below). The scan targetted the domain service (port 53/udp). We
#consider this scan to be an attempt to unlawfully access or abuse our
#network (intentionally or as a result of virus or worm activity).
132.206.9.0/24
132.206.123.0/24
132.206.125.0/24
#
# Add DOD + US Military, often not a great idea to scan military ranges.
# If you desire, you can comment these ranges out.
#
6.0.0.0/8
7.0.0.0/8
11.0.0.0/8
21.0.0.0/8
22.0.0.0/8
26.0.0.0/8
28.0.0.0/8
29.0.0.0/8
30.0.0.0/8
33.0.0.0/8
55.0.0.0/8
205.0.0.0/8
214.0.0.0/8
215.0.0.0/8
#******************************
#Janet is a UK research and education network!
#Please DO NOT scan, you been warned!
31.25.0.0/23
31.25.2.0/23
31.25.4.0/22
37.72.112.0/21
46.254.200.0/21
81.87.0.0/16
85.12.64.0/18
89.207.208.0/21
92.245.224.0/19
128.16.0.0/16
128.40.0.0/16
128.41.0.0/18
128.86.0.0/16
128.232.0.0/16
128.240.0.0/16
128.243.0.0/16
129.11.0.0/16
129.12.0.0/16
129.31.0.0/16
129.67.0.0/16
129.169.0.0/16
129.215.0.0/16
129.234.0.0/16
130.88.0.0/16
130.159.0.0/16
130.209.0.0/16
130.246.0.0/16
131.111.0.0/16
131.227.0.0/16
131.231.0.0/16
131.251.0.0/16
134.36.0.0/16
134.83.0.0/16
134.151.0.0/16
134.219.0.0/16
134.220.0.0/16
134.225.0.0/16
136.148.0.0/16
136.156.0.0/16
137.44.0.0/16
137.50.0.0/16
137.73.0.0/16
137.108.0.0/16
137.195.0.0/16
137.222.0.0/16
137.253.0.0/16
138.38.0.0/16
138.40.0.0/16
138.250.0.0/15
138.253.0.0/16
139.133.0.0/16
139.153.0.0/16
139.166.0.0/16
139.184.0.0/16
139.222.0.0/16
140.97.0.0/16
141.163.0.0/16
141.170.64.0/19
141.170.96.0/22
141.170.100.0/23
141.241.0.0/16
143.52.0.0/15
143.117.0.0/16
143.167.0.0/16
143.210.0.0/16
143.234.0.0/16
144.32.0.0/16
144.82.0.0/16
144.124.0.0/16
144.173.0.0/16
146.87.0.0/16
146.97.0.0/16
146.169.0.0/16
146.176.0.0/16
146.179.0.0/16
146.191.0.0/16
146.227.0.0/16
147.143.0.0/16
147.188.0.0/16
147.197.0.0/16
148.79.0.0/16
148.88.0.0/16
148.197.0.0/16
149.155.0.0/16
149.170.0.0/16
150.204.0.0/16
152.71.0.0/16
152.78.0.0/16
152.105.0.0/16
155.198.0.0/16
155.245.0.0/16
157.140.0.0/16
157.228.0.0/16
158.94.0.0/16
158.125.0.0/16
158.143.0.0/16
158.223.0.0/16
159.86.128.0/18
159.92.0.0/16
160.5.0.0/16
160.9.0.0/16
161.73.0.0/16
161.74.0.0/16
161.76.0.0/16
161.112.0.0/16
163.1.0.0/16
163.119.0.0/16
163.160.0.0/16
163.167.0.0/16
164.11.0.0/16
185.83.168.0/22
192.12.72.0/24
192.18.195.0/24
192.35.172.0/24
192.41.104.0/21
192.41.112.0/20
192.41.128.0/22
192.68.153.0/24
192.76.6.0/23
192.76.8.0/21
192.76.16.0/20
192.76.32.0/22
192.82.153.0/24
192.84.5.0/24
192.84.75.0/24
192.84.76.0/22
192.84.80.0/22
192.84.212.0/24
192.88.9.0/24
192.88.10.0/24
192.94.235.0/24
192.100.78.0/24
192.100.154.0/24
192.107.168.0/24
192.108.120.0/24
192.124.46.0/24
192.133.244.0/24
192.149.111.0/24
192.150.180.0/22
192.150.184.0/24
192.153.213.0/24
192.156.162.0/24
192.160.194.0/24
192.171.128.0/18
192.171.192.0/21
192.173.1.0/24
192.173.2.0/23
192.173.4.0/24
192.173.128.0/21
192.188.157.0/24
192.188.158.0/24
192.190.201.0/24
192.190.202.0/24
192.195.42.0/23
192.195.105.0/24
192.195.116.0/23
192.195.118.0/24
193.32.22.0/24
193.37.225.0/24
193.37.240.0/21
193.38.143.0/24
193.39.80.0/21
193.39.172.0/22
193.39.212.0/24
193.60.0.0/14
193.107.116.0/22
193.130.15.0/24
193.133.28.0/23
193.138.86.0/24
194.32.32.0/20
194.35.93.0/24
194.35.186.0/24
194.35.192.0/19
194.35.241.0/24
194.36.1.0/24
194.36.2.0/23
194.36.121.0/24
194.36.152.0/21
194.60.218.0/24
194.66.0.0/16
194.80.0.0/14
194.187.32.0/22
195.194.0.0/15
212.121.0.0/19
212.121.192.0/19
212.219.0.0/16

BIN
data/flush_all.pcap Normal file
View File

Binary file not shown.

4
debian/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
files
masscan
masscan.debhelper.log
masscan.substvars

6
debian/README.Debian vendored Normal file
View File

@@ -0,0 +1,6 @@
masscan for Debian
------------------
Initial debianisation
-- Vladimir Vitkov <vvitkov@sdl.com> Fri, 24 Jan 2014 11:03:38 +0200

18
debian/changelog vendored Normal file
View File

@@ -0,0 +1,18 @@
masscan (1.0.3-46+gbbadb7b-1) precise; urgency=low
* Rebuild from current git
* Use propper git hash versioning
-- Vladimir Vitkov (Packaging Key) <vvitkov@sdl.com> Tue, 10 Jun 2014 12:11:04 +0300
masscan (1.0.1+git20140124-1) unstable; urgency=low
* Rebuild from current git
-- Vladimir Vitkov <vvitkov@sdl.com> Fri, 24 Jan 2014 14:37:30 +0200
masscan (1.0.1+git20140106-1) unstable; urgency=low
* Initial release
-- Vladimir Vitkov <vvitkov@sdl.com> Fri, 24 Jan 2014 11:03:38 +0200

1
debian/compat vendored Normal file
View File

@@ -0,0 +1 @@
8

24
debian/control vendored Normal file
View File

@@ -0,0 +1,24 @@
Source: masscan
Section: net
Priority: extra
Maintainer: Vladimir Vitkov <vvitkov@sdl.com>
Build-Depends: debhelper (>= 8.0.0), libpcap-dev
Standards-Version: 3.9.3
Homepage: https://github.com/robertdavidgraham/masscan
#Vcs-Git: https://github.com/robertdavidgraham/masscan.git
#Vcs-Browser: http://git.debian.org/?p=collab-maint/masscan.git;a=summary
Package: masscan
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}
Description: Mass IP port scanner
This is the fastest Internet port scanner. It can scan the
entire Internet in under 6 minutes, transmitting 10 million
packets per second.
.
It produces results similar to nmap, the most famous port
scanner. Internally, it operates more like scanrand,
unicornscan, and ZMap, using asynchronous transmission.
The major difference is that it's faster than these other
scanners. In addition, it's more flexible, allowing
arbitrary address ranges and port ranges.

45
debian/copyright vendored Normal file
View File

@@ -0,0 +1,45 @@
Format: http://dep.debian.net/deps/dep5
Upstream-Name: masscan
Source: https://github.com/robertdavidgraham/masscan
Files: *
Copyright: 2013 Robert David Graham <robert_david_graham@yahoo.com>
License: AGPL
This program, "masscan", is not completely free software. It may not be
used by the United States Department of Defense (DoD) or National Security
Agency (NSA), or by agents acting on their behalf, such as contractors,
sub-contractors, and so on. These entitities must contact me to acquire
a different license.
.
Barring the above exception, you can use, redistribute, and/or modify
this code under the terms of the GNU Affero General Public License
version 3.
.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Files: debian/*
Copyright: 2014 Vladimir Vitkov <vvitkov@sdl.com>
License: GPL-3.0+
License: GPL-3.0+
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
.
This package is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
.
On Debian systems, the complete text of the GNU General
Public License version 3 can be found in "/usr/share/common-licenses/GPL-3".

1
debian/masscan.dirs vendored Normal file
View File

@@ -0,0 +1 @@
usr/bin

4
debian/masscan.docs vendored Normal file
View File

@@ -0,0 +1,4 @@
README.md
VULNINFO.md
doc/algorithm.js
doc/bot.html

1
debian/masscan.install vendored Normal file
View File

@@ -0,0 +1 @@
bin/masscan usr/bin/

1
debian/masscan.manpages vendored Normal file
View File

@@ -0,0 +1 @@
doc/masscan.8

16
debian/rules vendored Executable file
View File

@@ -0,0 +1,16 @@
#!/usr/bin/make -f
# -*- makefile -*-
# Sample debian/rules that uses debhelper.
# This file was originally written by Joey Hess and Craig Small.
# As a special exception, when this file is copied by dh-make into a
# dh-make output file, you may use that output file without restriction.
# This special exception was added by Craig Small in version 0.37 of dh-make.
# Uncomment this to turn on verbose mode.
#export DH_VERBOSE=1
%:
dh $@
override_dh_auto_test:
$(MAKE) regress

1
debian/source/format vendored Normal file
View File

@@ -0,0 +1 @@
3.0 (quilt)

4
debian/watch vendored Normal file
View File

@@ -0,0 +1,4 @@
version=3
# use the github release pages
https://github.com/robertdavidgraham/masscan/releases .*/(\d\S*)\.(?:tgz|tbz|txz|(?:tar\.(?:gz|bz2|xz)))

426
doc/algorithm.js Normal file
View File

@@ -0,0 +1,426 @@
/*
This is an implementation of the core Masscan scanning algorithm
in JavaScript/NodeJS. The core scanning algorithm is what makes
Masscan unique from other scanners, so it's worth highlighting
separately in a sample program.
REVIEW OF SCANNERS
The most famous port-scanner is "nmap". However, it is a
"host-at-a-time" scanner, and struggles at scanning large networks.
Masscan is an asynchronous, probe-at-a-time scanner. It spews out
probes to different ports, without caring if two probes happen to
be send to the same host. If the user wants a list of all ports
open on a single host, they have to post-process the masscan output
themselves, because masscan doesn't do it.
There are other asynchronous port-scanners, like scanrand, unicornscan,
and zmap. However, they have limitations in the way they do randomization
of their scans. They have limitations on the ranges of addresses and
ports that they'll accept, try to store an individual memory record
for everything scanned, or only partly randomize their scans.
THE WAY MASSCAN WORKS
Masscan first stores the targets as a "list of ranges". IP address
ranges are stored in one structure, and port ranges are stored
in another structure.
Then, a single index variable is used to enumerate the set of all
IP:port combinations. The scan works by simply incrementing the
index variable from 0 to the total number of probes (the 'range').
Then, before the enumeration step, the index is permuted into another
random index within the same range, in a 1-to-1 mapping. In other
words, the algorithm is theoretically reversable: given the output
of the permutation function, we can obtain the original index.
EXAMPLE
This program can be run like the following:
node patent.js 10.0.0.0-10.0.0.5 192.168.0.0/31 80,U:161
10.0.0.0-10.0.0.5
192.168.0.0-192.168.0.1
0.0.0.80-0.0.0.80
0.1.0.161-0.1.0.161
--> 10.0.0.4 udp:161
--> 10.0.0.0 udp:161
--> 10.0.0.1 udp:161
--> 10.0.0.4 tcp:80
--> 192.168.0.1 tcp:80
--> 10.0.0.0 tcp:80
--> 10.0.0.2 udp:161
--> 10.0.0.5 udp:161
--> 192.168.0.0 tcp:80
--> 192.168.0.0 udp:161
--> 10.0.0.1 tcp:80
--> 10.0.0.3 udp:161
--> 10.0.0.2 tcp:80
--> 10.0.0.5 tcp:80
--> 192.168.0.1 udp:161
--> 10.0.0.3 tcp:80
What you see first is the target ranges being echoed back that it scans,
first the IP address ranges, followed by the port ranges. The port ranges
are in weird decimal-dot notation because they share the same code
as for IPv4 addresses.
Then we see the randomized output, where individual probes are sent to a
random IP address and port.
TransmitThread
All the majic happens in the "TransmitThread()" function near the bottom
of this file.
We first see how the index variable 'i' is incremented from 0 to the
total number of packets that will be sent. We then see how first this
index is permuted to 'xXx', then this variable is separated into
one index for the IP address and another index for the port. Then,
those indexes are used to enumerate one of the IP addresses and
one of the ports.
Blackrock
This is the permutation function. It implements an encryption algorithm
based on DES (Data Encryption Standard). However, the use of real DES
would impose a restricting on the range that it be an even power of 2.
In the above example, with 14 total probes, this doesn't apply.
Therefore, we have to change binary operators like XOR with their
non-binary equivelents.
The upshot is that we first initialize Blackrock with the range (and
a seed/key), and then shuffle the index. The process is stateless,
meaning that any time we shuffle the number '5' we always get the
same result, regardless of what has happened before.
Targets, RangeList, Range
A Range is just a begin/end of an integer. We use this both for
IPv4 addresses (which are just 32-bit integers) and ports
(which are 16 bit integers).
A RangeList is just an array of Ranges. In Masscan, this object
sorts and combines ranges, making sure there are no duplicates,
but that isn't used in this example.
The RangeList object shows how an index can enumerate the
individual addresses/ports. This is down by walking the list
and subtracting from the index the size of each range, until
we reach a range that is larger than the index.
The Targets object just holds both the IP and port lists.
*/
function Range(begin, end) {
if (typeof begin == 'undefined' && typeof end == 'undefined') {
this.begin = 0xFFFFFFFF;
this.end = 0;
} else if (typeof end == 'undefined') {
this.begin = begin;
this.end = begin;
} else {
this.begin = begin;
this.end = end;
}
this.toString = function () {
return ((this.begin >> 24) & 0xFF)
+ "." + ((this.begin >> 16) & 0xFF)
+ "." + ((this.begin >> 8) & 0xFF)
+ "." + ((this.begin >> 0) & 0xFF)
+ "-" + ((this.end >> 24) & 0xFF)
+ "." + ((this.end >> 16) & 0xFF)
+ "." + ((this.end >> 8) & 0xFF)
+ "." + ((this.end >> 0) & 0xFF);
}
this.count = function () {
return this.end - this.begin + 1;
}
this.pick = function (index) {
return this.begin + index;
}
return this;
}
function RangeList() {
this.list = [];
this.total_count = 0;
this.push = function (range) {
this.list.push(range);
this.total_count += range.count();
}
this.count = function () {
return this.total_count;
}
this.pick = function (index) {
for (var i in this.list) {
var item = this.list[i];
if (index < item.count())
return item.pick(index);
else
index -= item.count();
}
return null;
}
}
function Targets() {
this.ports = new RangeList();
this.ips = new RangeList();
this.parse_ip = function (text) {
var x = text.split(".");
var result = 0;
result |= parseInt(x[0]) << 24;
result |= parseInt(x[1]) << 16;
result |= parseInt(x[2]) << 8;
result |= parseInt(x[3]) << 0;
return result;
}
this.parse_ports = function (arg) {
var offset = 0;
if (arg.indexOf(":") !== -1) {
var x = arg.split(":");
if (x[0] == "U")
offset = 65536;
else if (x[0] == "S")
offset = 65536 * 2;
arg = x[1];
}
var target;
if (arg.indexOf("-") !== -1) {
var x = arg.split("-");
target = new Range(parseInt(x[0]), parseInt(x[1]));
} else
target = new Range(parseInt(arg));
target.begin += offset;
target.end += offset;
this.ports.push(target);
}
this.parse_args = function (argv) {
for (var i in argv) {
var arg = argv[i];
if (arg.indexOf(",") !== -1) {
var x = arg.split(",");
for (var j in x)
this.parse_ports(x[j]);
} else if (arg.indexOf("/") !== -1) {
var x = arg.split("/");
var address = this.parse_ip(x[0]);
var prefix = parseInt(x[1]);
var mask = 0xFFFFFFFF << (32 - prefix);
address = address & mask;
var target = new Range(address, address | ~mask);
this.ips.push(target);
} else if (arg.indexOf("-") !== -1) {
var x = arg.split("-");
var begin = this.parse_ip(x[0]);
var end = this.parse_ip(x[1]);
var target = new Range(begin, end);
this.ips.push(target);
} else if (arg.indexOf(".") !== -1) {
var target = new Range(this.parse_ip(arg));
this.ips.push(target);
} else {
this.parse_ports(arg);
}
}
}
this.print = function () {
var i;
for (i in this.ips.list) {
console.log(this.ips.list[i].toString());
}
for (i in this.ports.list) {
console.log(this.ports.list[i].toString());
}
}
return this;
}
function Blackrock(range, seed) {
var split = Math.floor(Math.sqrt(range * 1.0));
this.rounds = 3;
this.seed = seed;
this.range = range;
this.a = split - 1;
this.b = split + 1;
while (this.a * this.b <= range)
this.b++;
/** Inner permutation function */
this.F = function (j, R, seed) {
var primes = [961752031, 982324657, 15485843, 961752031];
R = ((R << (R & 0x4)) + R + seed);
return Math.abs((((primes[j] * R + 25) ^ R) + j));
}
/** Outer feistal construction */
this.fe = function (r, a, b, m, seed) {
var L, R;
var j;
var tmp;
L = m % a;
R = Math.floor(m / a);
for (j = 1; j <= r; j++) {
if (j & 1) {
tmp = (L + this.F(j, R, seed)) % a;
} else {
tmp = (L + this.F(j, R, seed)) % b;
}
L = R;
R = tmp;
}
if (r & 1) {
return a * L + R;
} else {
return a * R + L;
}
}
/** Outer reverse feistal construction */
this.unfe = function (r, a, b, m, seed) {
var L, R;
var j;
var tmp;
if (r & 1) {
R = m % a;
L = Math.floor(m / a);
} else {
L = m % a;
R = Math.floor(m / a);
}
for (j = r; j >= 1; j--) {
if (j & 1) {
tmp = this.F(j, L, seed);
if (tmp > R) {
tmp = (tmp - R);
tmp = a - (tmp % a);
if (tmp == a)
tmp = 0;
} else {
tmp = (R - tmp);
tmp %= a;
}
} else {
tmp = this.F(j, L, seed);
if (tmp > R) {
tmp = (tmp - R);
tmp = b - (tmp % b);
if (tmp == b)
tmp = 0;
} else {
tmp = (R - tmp);
tmp %= b;
}
}
R = L;
L = tmp;
}
return a * R + L;
}
this.shuffle = function (m) {
var c;
c = this.fe(this.rounds, this.a, this.b, m, this.seed);
while (c >= this.range)
c = this.fe(this.rounds, this.a, this.b, c, this.seed);
return c;
}
this.unshuffle = function (m) {
var c;
c = unfe(this.rounds, this.a, this.b, m, this.seed);
while (c >= this.range)
c = unfe(this.rounds, this.a, this.b, c, this.seed);
return c;
}
return this;
}
function TransmitThread(targets, transmit, seed) {
var range = targets.ips.count() * targets.ports.count();
var b = Blackrock(range, seed);
for (var i = 0; i < range; i++) {
var xXx = b.shuffle(i);
var ip_index = Math.floor(xXx / targets.ports.count());
var port_index = Math.floor(xXx % targets.ports.count());
var ip = targets.ips.pick(ip_index);
var port = targets.ports.pick(port_index);
transmit(ip, port);
}
}
function Transmit2Thread(targets, transmit, seed, start, stop, increment) {
var range = targets.ips.count() * targets.ports.count();
var b = Blackrock(range, seed);
for (var i = start; i < range && i < stop; i += increment) {
var xXx = b.shuffle(i);
var ip_index = Math.floor(xXx / targets.ports.count());
var port_index = Math.floor(xXx % targets.ports.count());
var ip = targets.ips.pick(ip_index);
var port = targets.ports.pick(port_index);
transmit(ip, port);
}
}
function transmit(ip, port) {
var proto = "tcp";
if (port > 65536 * 2) {
proto = "sctp";
port -= 65536 * 2;
} else if (port > 65536) {
proto = "udp";
port -= 65536;
}
var ipstring = ((ip >> 24) & 0xFF)
+ "." + ((ip >> 16) & 0xFF)
+ "." + ((ip >> 8) & 0xFF)
+ "." + ((ip >> 0) & 0xFF)
console.log("--> " + ipstring + " " + proto + ":" + port);
}
var targets = new Targets();
targets.parse_args(process.argv.splice(2));
targets.print();
TransmitThread(targets, transmit, 42);

17
doc/bot.html Normal file
View File

@@ -0,0 +1,17 @@
<html>
<head>
<title>Masscan/1.0 - fast port scanner</title>
</head>
<body>
<h1>Masscan/1.0 - fast port scanner</h1>
<p>This tool is not a web spider, but a port scanner. It'll make only one request to a website, usually for the root / webpage. It then records the <b>Server:</b> field from the HTTP header, the <b>&lt;title&gt;</b> from the page contents, and possibly a few other interesting fields.</p>
<p>This does not follow links, it doesn't scan your web pages, but the ports on your machine.</p>
<p>The source code for this tool is at <a href="https://github.com/robertdavidgraham/masscan/">https://github.com/robertdavidgraham/masscan/</a>. This is an open source project, so that this means it's not me (Robert Graham) who is using this tool to scan your website, but likely somebody else. I can't speak for their intentions, but this tool is more useful at doing surveys of the Internet than trying to hack in (tools like 'nmap' or 'nessus' are more often used for that).</p>
</body>
</html>

29
doc/faq/FAQ0001-slow.md Normal file
View File

@@ -0,0 +1,29 @@
# Why is it not as fast as I expect?
## Question
Why is scanning speed only around 100,000 packets-per-second instead of a million packets-per-second?
## Answer
I don't know.
If you have the latest Linux distro on the latest hardware, you can sometime
see scanning speeds of 1 million packets-per-second, even when virtualized.
However, sometimes you also see only 100,000 packets-per-second.
I've spent a lot of time trying to diagnose this situation and cannot
figure out what's going on. The box I use in a colo does 500,000 packets-per-second.
A relatively slow machine in my home lab does 1.2 million packets-per-second.
The speed is determined by the operating system. The amount of CPU used by `masscan`
itself is insignificant.
My theory is various configuration options within the operating system that can make
packet transmission very slow. Simple features that would not otherwise impact network
stacks that run at lower rates become really important at high rates.
One way around this is to install `PF_RING` and dedicate a network adapter to packet
transmission completely bypassing the operating system. In that case, packet transmission
rates can reach 15 million packets-per-second.

31
doc/faq/FAQ0002-drops.md Normal file
View File

@@ -0,0 +1,31 @@
# Why are many results missing that I expect?
# Question
When I do a scan, results are missing that I know are there.
They show up when I repeat the scan, but then others are missing.
The faster I scan, the more results are missing.
# Answer
Network infrastructure does not like high rates of small packets.
Even though they can handle high **bit-rates** then cannot handle
high **packet-rates**.
This is what makes `masscan` so unique. It transmits packets at rates
far higher than other things can cope with. It often crashes networks.
Therefore, the faster you transmit packets, the more it overloads network
equipment, causing the packets to be dropped, causing probes to fail.
As the issue #546 below indicates, they are experiencing this at very low
rates of less than 10,000 packets-per-second. That seems excessively low.
I assume the actual reason is because of policy enforcement limiting traffic
rates rather than overloading network equipment.
# Issues
- [#546 fast scan get result](https://github.com/robertdavidgraham/masscan/issues/546)

25
doc/faq/FAQ0003-excludelist.md Normal file
View File

@@ -0,0 +1,25 @@
# How can I add my IP address to an exclude list so that people stop scanning me?
# Question
I hate everyone probing me all the time and want them to stop.
How can I add my IP address ranges to an exclude list?
# Answer
You can't.
First of all, nobody is going to pay attention to a sample exclude list
within this project. Sure, I can add IP addresses to the list, but that
won't help you.
Second, there's no way I can confirm who you are. So I can't simply
add to an exclude list just because you ask.
Thirdly, it'll just make you more of a target, as smart hackers know to
use the exclude-list as one of their first include-lists, as it marks
people who have something to hide.
Fourthly, and most importantly, it's Wrong Think on how to manage your
network.

28
doc/faq/FAQ0004-serverlogs.md Normal file
View File

@@ -0,0 +1,28 @@
# Why is masscan in my server logs?
## Question
Some example questions:
* Why is `masscan` appearing in my server logs?
* Why are you scanning me?
* Why is my server trying to connect to this github repo?
## Answer
When `masscan` connections to a webserver, it puts a link
back to this repo in the `User-Agent` field.
Since lots of people run Internet-wide scans using this tool,
and an Internet wide scan hits every public web server, you'll
see this appear in your web server logs several times a day.
It's the **end-to-end** principle of the Internet. Having a public
webserver on the Internet means that anybody can and will try to
connect to the web server.
It's nothing necessarily malicious. Lots of people run Internet-wide
scans to gather information about the state of the Internet. Of course,
some are indeed malicious, scanning to find vulnerabilities. However,
even when malicious, they probably aren't targetting you in particular,
but are instead scanning everybody.

9
doc/faq/README.md Normal file
View File

@@ -0,0 +1,9 @@
# FAQs (frequently asked questions)
This directory contains some documents discussing frequently asked
questions
- 1 - [Why is it not as fast as I expect?](FAQ0001-slow.md)
- 2 - [Why are many results missing that I expect?](FAQ0002-drops.md)
- 3 - [How can I add my IPs to an official exclude list, to get people to stop scanning me?](FAQ0003-excludelist.md)
- 4 - [Why is this in my server logs?](FAQ0004-serverlogs.md)

151
doc/howto-afl.md Normal file
View File

@@ -0,0 +1,151 @@
Using afl fuzzer against masscan
================================
AFL (American-Fuzzy-Lop) is an automated *fuzzer*. It takes existing
input to a program, then morphs that input in order to test new
code paths through the code. It's extremely successful at finding
bugs as well as *vulnerabilities*.
There are two inputs to *masscan*. One is the files it reads, which
come in various formats. The second is input from the network, in
response to network probes, which consist of various network protocols.
For AFL, there is also the issue of how *masscan* crashes. It tries to
print a backtrace. This causes AFL to falsely mark this as a "hang"
rather than a "crash". To fix this, run *masscan* with the *--nobacktrace*
option.
## Fuzzing file formats ##
The *masscan* program reads the following files. You can set the fuzzer
at each one of these to fuzz how it parses the contents.
*-c <filename.conf>*
*Masscan* can read its configuration either from the command-line or from a file.
To create a file, run *masscan* as normal, then hit <ctrl-c>. This will save all
it's settings, even default values, into a file. It's a good starting point for
fuzzing.
*--readscan <filename.mass>*
One of the possible outputs of *masscan* is in a proprietary binary format. You
can then run *masscan* to convert to any other output format.
In other words, you can run masscan to output XML like:
masscan <blah blah blah> -oX scan.xml
Or, in a two step process:
masscan <blah blah blah> -ob scan.mass
masscan --readscan scan.mass -oX scan.xml
*--exclude-file <filename.ranges>*
*Masscan* can scan large ragnes, like *0.0.0.0/0* (the entire Internet). You may
want to exclude specific addresses or ranges. These are configured in the
"exclude file".
You can also read ranges using *--include-file <filename.ranges>* or
*-iL <filename.ranges>*, but as far as fuzzing, I'm pretty sure it'll
be the same results.
*--hello-file[<port>] <filename>*
This file is read, then dumped blindly across a TCP connection, in order
to say "hello" to a service. Since there's no parsing here, I'm not sure
you'll find anything fuzzing this.
*--nmap-payloads <filename>*
This file specifies the default payloads for UDP. It's in the same file
format as for *nmap*. There's some juicy parsing here that may lead
to bugs.
*--pcap-payloads <filename.pcap>*
This is the same as *--nmap-payloads*, but reads the UDP payloads from
a *libpcap* file.
## Fuzzing network protocols ##
*Masscan* parses several network protocols. It also must reassemble
fragmented responses over TCP for any application protocol. Remember:
*masscan* has it's own stack, so must parse everything that comes
over the network.
AFL has no ability to read from the network at this time. Moreover,
even then it wouldn't work easily, since *masscan* has a network stack
rather than just an application layer to deal with.
The trick is to first use *masscan* on a target that responds back
on a protocol, then save just the response side of the TCP connection
to a file. Then, when running *masscan* under AFL, read in that file
using the option *--adapter file:<filename.pcap>*. Then, and this is
critical, you must hard code all the TCP stack values to match those
of the original connection.
I generated the file *masscan/data/afl-http.pcap* as an example file
to read for fuzzing the parsing of HTTP. The command-line parameters
to use are:
bin/masscan --nobacktrace --adapter file:data/afl-http.pcap --source-ip 10.20.30.200 --source-port 6000 --source-mac 00-11-22-33-44-55 --router-mac c0-c1-c0-a0-9b-9d --seed 0 --banners -p80 74.125.196.147 --nostatus
The explanation are:
*--nobacktrace*: so that AFL correctly marks crashes as crashes
and not as hangs.
*--adapter file:*: This option normally specifies the adapter,
like *eth0* or *en1*. By putting the *file:* prefix on an adapter name,
it'll use a file (in *libpcap* format) to use instead. In this case,
transmits are dropped, and any packets are read from a file.
*--source-ip*, *--source-port*, *--source-mac*, *--router-mac*:
These are the hard-coded TCP/IP stack settings that must match the packets
in the file.
*--seed*: This must match the randomization seed in the packet file. Since
everything else is hardcoded, I think the only thing this will control
will be the sequence number of the TCP connection.
*--banners*: this tell the scanner to not simply find open ports, but also
establish a TCP connection and interact with the protocol, and report on
the results.
*-p<port>*: The destination IP address to connect to.
*<ip-address>*: The IP address to connect to
This should produce an output like the following. If you get the
banner back, then you know you've successfully done everything
correctly. Conversely, if you set *--seed 1*, then it won't work,
because it'll reject responses that match the wrong seed.
Starting masscan 1.0.3 (http://bit.ly/14GZzcT) at 2016-06-06 05:19:03 GMT
-- forced options: -sS -Pn -n --randomize-hosts -v --send-eth
Initiating SYN Stealth Scan
Scanning 1 hosts [1 port/host]
Discovered open port 80/tcp on 74.125.196.147
Banner on port 80/tcp on 74.125.196.147: [title] Google
Banner on port 80/tcp on 74.125.196.147: [http] HTTP/1.0 200 OK\x0d\x0a...
...
(Additional output is truncated -- you get the idea).
The problem with this is that *masscan* will take about 10 seconds before it
produces the results. When it establishes a connection, it waits a few seconds
for the other side to transmit, then sends it's "hello", then waits many
seconds for all output to be received. I don't know if this messes AFL up,
whether I need to add additional options to truncate any waiting.

363
doc/masscan.8 Normal file
View File

@@ -0,0 +1,363 @@
.\" generated with Ronn/v0.7.3
.\" http://github.com/rtomayko/ronn/tree/0.7.3
.
.TH "MASSCAN" "8" "January 2014" "" ""
.
.SH "NAME"
\fBmasscan\fR \- Fast scan of the Internet
.
.SH "SYNOPSIS"
masscan [\fIoptions\fR] [<IP|RANGE>... \-p \fIPORT\fR[,\fIPORT\fR...]]
.
.SH "DESCRIPTION"
\fBmasscan\fR is an Internet\-scale port scanner, useful for large scale surveys of the Internet, or of internal networks\. While the default transmit rate is only 100 packets/second, it can optional go as fast as 25 million packets/second, a rate sufficient to scan the Internet in 3 minutes for one port\.
.
.SH "OPTIONS"
.
.IP "\(bu" 4
\fB<IP|RANGE>\fR: anything on the command\-line not prefixed with a \'\-\' is assumed to be an IP address or range\. There are three valid formats\. The first is a single IPv4 address like "192\.168\.0\.1"\. The second is a range like "10\.0\.0\.1\-10\.0\.0\.100"\. The third is a CIDR address, like "0\.0\.0\.0/0"\. At least one target must be specified\. Multiple targets can be specified\. This can be specified as multiple options separated by space, or can be separated by a comma as a single option, such as \fB10\.0\.0\.0/8,192\.168\.0\.1\fR\.
.
.IP "\(bu" 4
\fB\-\-range <IP|RANGE>\fR: the same as target range spec described above, except as a named parameter instead of an unnamed one\.
.
.IP "\(bu" 4
\fB\-p PORT[,PORT...]\fR, \fB\-\-ports PORT[,PORT...]\fR: specifies the port(s) to be scanned\. A single port can be specified, like \fB\-p80\fR\. A range of ports can be specified, like \fB\-p 20\-25\fR\. A list of ports/ranges can be specified, like \fB\-p80,20\-25\fR\. UDP ports can also be specified, like \fB\-\-ports U:161,U:1024\-1100\fR\.
.
.IP "\(bu" 4
\fB\-\-banners\fR: specifies that banners should be grabbed, like HTTP server versions, HTML title fields, and so forth\. Only a few protocols are supported\.
.
.IP "\(bu" 4
\fB\-\-rate RATE\fR: specifies the desired rate for transmitting packets\. This can be very small numbers, like \fB0\.1\fR for transmitting packets at rates of one every 10 seconds, for very large numbers like 10000000, which attempts to transmit at 10 million packets/second\. In my experience, Windows and can do 250 thousand packets per second, and latest versions of Linux can do 2\.5 million packets per second\. The PF_RING driver is needed to get to 25 million packets/second\.
.
.IP "\(bu" 4
\fB\-c FILE\fR, \fB\-\-conf FILE\fR: reads in a configuration file\. The format of the configuration file is described below\.
.
.IP "\(bu" 4
\fB\-\-resume FILE\fR: the same as \fB\-\-conf\fR, except that a few options are automatically set, such as \fB\-\-append\-output\fR\. The format of the configuration file is described below\.
.
.IP "\(bu" 4
\fB\-\-echo\fR: don\'t run, but instead dump the current configuration to a file\. This file can then be used with the \fB\-c\fR option\. The format of this output is described below under \'CONFIGURATION FILE\'\.
.
.IP "\(bu" 4
\fB\-e IFNAME\fR, \fB\-\-adapter IFNAME\fR: use the named raw network interface, such as "eth0" or "dna1"\. If not specified, the first network interface found with a default gateway will be used\.
.
.IP "\(bu" 4
\fB\-\-adapter\-ip IP\fR: send packets using this IP address\. If not specified, then the first IP address bound to the network interface will be used\. Instead of a single IP address, a range may be specified\. NOTE: The size of the range must be an even power of 2, such as 1, 2, 4, 8, 16, 1024 etc\. addresses\.
.
.IP "\(bu" 4
\fB\-\-adapter\-port PORT\fR: send packets using this port number as the source\. If not specified, a random port will be chosen in the range 40000 through 60000\. This port should be filtered by the host firewall (like iptables) to prevent the host network stack from interfering with arriving packets\. Instead of a single port, a range can be specified, like \fB40000\-40003\fR\. NOTE: The size of the range must be an even power of 2, such as the example above that has a total of 4 addresses\.
.
.IP "\(bu" 4
\fB\-\-adapter\-mac MAC\fR: send packets using this as the source MAC address\. If not specified, then the first MAC address bound to the network interface will be used\.
.
.IP "\(bu" 4
\fB\-\-router\-mac MAC\fR: send packets to this MAC address as the destination\. If not specified, then the gateway address of the network interface will be ARPed\.
.
.IP "\(bu" 4
\fB\-\-ping\fR: indicates that the scan should include an ICMP echo request\. This may be included with TCP and UDP scanning\.
.
.IP "\(bu" 4
\fB\-\-exclude <IP|RANGE>\fR: blacklist an IP address or range, preventing it from being scanned\. This overrides any target specification, guaranteeing that this address/range won\'t be scanned\. This has the same format as the normal target specification\.
.
.IP "\(bu" 4
\fB\-\-excludefile FILE\fR: reads in a list of exclude ranges, in the same target format described above\. These ranges override any targets, preventing them from being scanned\.
.
.IP "\(bu" 4
\fB\-\-append\-output\fR: causes output to append to the file, rather than overwriting the file\.
.
.IP "\(bu" 4
\fB\-\-iflist\fR: list the available network interfaces, and then exits\.
.
.IP "\(bu" 4
\fB\-\-retries NUM\fR: the number of retries to send, at 1 second intervals\. Note that since this scanner is stateless, retries are sent regardless if replies have already been received\.
.
.IP "\(bu" 4
\fB\-\-nmap\fR: print help about nmap\-compatibility alternatives for these options\.
.
.IP "\(bu" 4
\fB\-\-pcap\-payloads FILE\fR: read packets from a libpcap file containing packets and extract the UDP payloads, and associate those payloads with the destination port\. These payloads will then be used when sending UDP packets with the matching destination port\. Only one payload will be remembered per port\. Similar to \fB\-\-nmap\-payloads\fR\.
.
.IP "\(bu" 4
\fB\-\-nmap\-payloads FILE\fR: read in a file in the same format as the nmap file \fBnmap\-payloads\fR\. This contains UDP payload, so that we can send useful UDP packets instead of empty ones\. Similar to \fB\-\-pcap\-payloads\fR\.
.
.IP "\(bu" 4
\fB\-\-http\-user\-agent USER_AGENT\fR: replaces the existing user\-agent field with the indicated value when doing HTTP requests\.
.
.IP "\(bu" 4
\fB\-\-open\-only\fR: report only open ports, not closed ports\.
.
.IP "\(bu" 4
\fB\-\-pcap FILE\fR: saves received packets (but not transmitted packets) to the libpcap\-format file\.
.
.IP "\(bu" 4
\fB\-\-packet\-trace\fR: prints a summary of those packets sent and received\. This is useful at low rates, like a few packets per second, but will overwhelm the terminal at high rates\.
.
.IP "\(bu" 4
\fB\-\-pfring\fR: force the use of the PF_RING driver\. The program will exit if PF_RING DNA drvers are not available\.
.
.IP "\(bu" 4
\fB\-\-resume\-index INDEX\fR: the point in the scan at when it was paused\.
.
.IP "\(bu" 4
\fB\-\-resume\-count NUM\fR: the maximum number of probes to send before exiting\. This is useful with the \fB\-\-resume\-index\fR to chop up a scan and split it among multiple instances, though the \fB\-\-shards\fR option might be better\.
.
.IP "\(bu" 4
\fB\-\-shards X/Y\fR: splits the scan among instances\. \fBx\fR is the id for this scan, while \fBy\fR is the total number of instances\. For example, \fB\-\-shards 1/2\fR tells an instance to send every other packet, starting with index 0\. Likewise, \fB\-\-shards 2/2\fR sends every other packet, but starting with index 1, so that it doesn\'t overlap with the first example\.
.
.IP "\(bu" 4
\fB\-\-rotate TIME\fR: rotates the output file, renaming it with the current timestamp, moving it to a separate directory\. The time is specified in number of seconds, like "3600" for an hour\. Or, units of time can be specified, such as "hourly", or "6hours", or "10min"\. Times are aligned on an even boundary, so if "daily" is specified, then the file will be rotated every day at midnight\.
.
.IP "\(bu" 4
\fB\-\-rotate\-offset TIME\fR: an offset in the time\. This is to accommodate timezones\.
.
.IP "\(bu" 4
\fB\-\-rotate\-dir DIR\fR: when rotating the file, this specifies which directory to move the file to\. A useful directory is \fB/var/log/masscan\fR\.
.
.IP "\(bu" 4
\fB\-\-seed INT\fR: an integer that seeds the random number generator\. Using a different seed will cause packets to be sent in a different random order\. Instead of an integer, the string \fBtime\fR can be specified, which seeds using the local timestamp, automatically generating a differnet random order of scans\. If no seed specified, \fBtime\fR is the default\.
.
.IP "\(bu" 4
\fB\-\-regress\fR: run a regression test, returns \'0\' on success and \'1\' on failure\.
.
.IP "\(bu" 4
\fB\-\-ttl NUM\fR: specifies the TTL of outgoing packets, defaults to 255\.
.
.IP "\(bu" 4
\fB\-\-wait SECONDS\fR: specifies the number of seconds after transmit is done to wait for receiving packets before exiting the program\. The default is 10 seconds\. The string \fBforever\fR can be specified to never terminate\.
.
.IP "\(bu" 4
\fB\-\-offline\fR: don\'t actually transmit packets\. This is useful with a low rate and \fB\-\-packet\-trace\fR to look at what packets might\'ve been transmitted\. Or, it\'s useful with \fB\-\-rate 100000000\fR in order to benchmark how fast transmit would work (assuming a zero\-overhead driver)\. PF_RING is about 20% slower than the benchmark result from offline mode\.
.
.IP "\(bu" 4
\fB\-sL\fR: this doesn\'t do a scan, but instead creates a list of random addresses\. This is useful for importing into other tools\. The options \fB\-\-shard\fR, \fB\-\-resume\-index\fR, and \fB\-\-resume\-count\fR can be useful with this feature\.
.
.IP "\(bu" 4
\fB\-\-interactive\fR: show the results in realtime on the console\. It has no effect if used with \-\-output\-format or \-\-output\-filename\.
.
.IP "\(bu" 4
\fB\-\-output\-format FMT\fR: indicates the format of the output file, which can be \fBxml\fR, \fBbinary\fR, \fBgrepable\fR, \fBlist\fR, or \fBJSON\fR\. The option \fB\-\-output\-filename\fR must be specified\.
.
.IP "\(bu" 4
\fB\-\-output\-filename FILE\fR: the file which to save results to\. If the parameter \fB\-\-output\-format\fR is not specified, then the default of \fBxml\fR will be used\.
.
.IP "\(bu" 4
\fB\-oB FILE\fR: sets the output format to binary and saves the output in the given filename\. This is equivelent to using the \fB\-\-output\-format\fR and \fB\-\-output\-filename\fR parameters\. The option \fB\-\-readscan\fR can then be used to read the binary file\. Binary files are much smaller than their XML equivelents, but require a separate step to convert back into XML or another readable format\.
.
.IP "\(bu" 4
\fB\-oX FILE\fR: sets the output format to XML and saves the output in the given filename\. This is equivelent to using the \fB\-\-output\-format xml\fR and \fB\-\-output\-filename\fR parameters\.
.
.IP "\(bu" 4
\fB\-oG FILE\fR: sets the output format to grepable and saves the output in the given filename\. This is equivelent to using the \-\-output\-format grepable and \-\-output\-filename parameters\.
.
.IP "\(bu" 4
\fB\-oJ FILE\fR: sets the output format to JSON and saves the output in the given filename\. This is equivelent to using the \-\-output\-format json and \-\-output\-filename parameters\.
.
.IP "\(bu" 4
\fB\-oL FILE\fR: sets the output format to a simple list format and saves the output in the given filename\. This is equivelent to using the \-\-output\-format list and \-\-output\-filename parameters\.
.
.IP "\(bu" 4
\fB\-\-readscan FILE\fR: reads the files created by the \fB\-oB\fR option from a scan, then outputs them in one of the other formats, depending on command\-line parameters\. In other words, it can take the binary version of the output and convert it to an XML or JSON format\.
.
.IP "" 0
.
.SH "CONFIGURATION FILE FORMAT"
The configuration file uses the same parameter names as on the commandline, but without the \fB\-\-\fR prefix, and with an \fB=\fR sign between the name and the value\. An example configuration file might be:
.
.IP "" 4
.
.nf
# targets
range = 10\.0\.0\.0/8,192\.168\.0\.0/16
range = 172\.16\.0\.0/14
ports = 20\-25,80,U:53
ping = true
# adapter
adapter = eth0
adapter\-ip = 192\.168\.0\.1
router\-mac = 66\-55\-44\-33\-22\-11
# other
exclude\-file = /etc/masscan/exludes\.txt
.
.fi
.
.IP "" 0
.
.P
By default, the program will read default configuration from the file \fB/etc/masscan/masscan\.conf\fR\. This is useful for system\-specific settings, such as the \fB\-\-adapter\-xxx\fR options\. This is also useful for excluded IP addresses, so that you can scan the entire Internet, while skipping dangerous addresses, like those owned by the DoD, and not make an accidental mistake\.
.
.SH "CONTROL\-C BEHAVIOR"
When the user presses \fIctrl\-c\fR, the scan will stop, and the current state of the scan will be saved in the file \'paused\.conf\'\. The scan can be resumed with the \fB\-\-resume\fR option:
.
.IP "" 4
.
.nf
# masscan \-\-resume paused\.conf
.
.fi
.
.IP "" 0
.
.P
The program will not exit immediately, but will wait a default of 10 seconds to receive results from the Internet and save the results before exiting completely\. This time can be changed with the \fB\-\-wait\fR option\.
.
.SH "SIMPLE EXAMPLES"
The following example scans all private networks for webservers, and prints all open ports that were found\.
.
.IP "" 4
.
.nf
# masscan 10\.0\.0\.0/8 192\.168\.0\.0/16 172\.16\.0\.0/12 \-p80 \-\-open\-only
.
.fi
.
.IP "" 0
.
.P
The following example scans the entire Internet for DNS servers, grabbing their versions, then saves the results in an XML file\.
.
.IP "" 4
.
.nf
# masscan 0\.0\.0\.0/0 \-\-excludefile no\-dod\.txt \-pU:53 \-\-banners \-\-output\-filename dns\.xml
.
.fi
.
.IP "" 0
.
.P
You should be able to import the XML into databases and such\.
.
.P
The following example reads a binary scan results file called bin\-test\.scan and prints results to console\.
.
.IP "" 4
.
.nf
# masscan \-\-readscan bin\-test\.scan
.
.fi
.
.IP "" 0
.
.P
The following example reads a binary scan results file called bin\-test\.scan and creates an XML output file called bin\-test\.xml\.
.
.IP "" 4
.
.nf
# masscan \-\-readscan bin\-test\.scan \-oX bin\-test\.xml
.
.fi
.
.IP "" 0
.
.SH "ADVANCED EXAMPLES"
Let\'s say that you want to scan the entire Internet and spread the scan across three machines\. Masscan would be launched on all three machines using the following command\-lines:
.
.IP "" 4
.
.nf
# masscan 0\.0\.0\.0/0 \-p0\-65535 \-\-shard 1/3
# masscan 0\.0\.0\.0/0 \-p0\-65535 \-\-shard 2/3
# masscan 0\.0\.0\.0/0 \-p0\-65535 \-\-shard 3/3
.
.fi
.
.IP "" 0
.
.P
An alternative is with the "resume" feature\. A scan has an internal index that goes from zero to the number of ports times then number of IP addresses\. The following example shows splitting up a scan into chunks of a 1000 items each:
.
.IP "" 4
.
.nf
# masscan 0\.0\.0\.0/0 \-p0\-65535 \-\-resume\-index 0 \-\-resume\-count 1000
# masscan 0\.0\.0\.0/0 \-p0\-65535 \-\-resume\-index 1000 \-\-resume\-count 1000
# masscan 0\.0\.0\.0/0 \-p0\-65535 \-\-resume\-index 2000 \-\-resume\-count 1000
# masscan 0\.0\.0\.0/0 \-p0\-65535 \-\-resume\-index 3000 \-\-resume\-count 1000
.
.fi
.
.IP "" 0
.
.P
A script can use this to split smaller tasks across many other machines, such as Amazon EC2 instances\. As each instance completes a job, the script might send a request to a central coordinating server for more work\.
.
.SH "SPURIOUS RESETS"
When scanning TCP using the default IP address of your adapter, the built\-in stack will generate RST packets\. This will prevent banner grabbing\. There are are two ways to solve this\. The first way is to create a firewall rule to block that port from being seen by the stack\. How this works is dependent on the operating system, but on Linux this looks something like:
.
.IP "" 4
.
.nf
# iptables \-A INPUT \-p tcp \-i eth0 \-\-dport 61234 \-j DROP
.
.fi
.
.IP "" 0
.
.P
Then, when scanning, that same port must be used as the source:
.
.IP "" 4
.
.nf
# masscan 10\.0\.0\.0/8 \-p80 \-\-banners \-\-adapter\-port 61234
.
.fi
.
.IP "" 0
.
.P
An alternative is to "spoof" a different IP address\. This IP address must be within the range of the local network, but must not otherwise be in use by either your own computer or another computer on the network\. An example of this would look like:
.
.IP "" 4
.
.nf
# masscan 10\.0\.0\.0/8 \-p80 \-\-banners \-\-adapter\-ip 192\.168\.1\.101
.
.fi
.
.IP "" 0
.
.P
Setting your source IP address this way is the preferred way of running this scanner\.
.
.SH "ABUSE COMPLAINTS"
This scanner is designed for large\-scale surveys, of either an organization, or of the Internet as a whole\. This scanning will be noticed by those monitoring their logs, which will generate complaints\.
.
.P
If you are scanning your own organization, this may lead to you being fired\. Never scan outside your local subnet without getting permission from your boss, with a clear written declaration of why you are scanning\.
.
.P
The same applies to scanning the Internet from your employer\. This is another good way to get fired, as your IT department gets flooded with complaints as to why your organization is hacking them\.
.
.P
When scanning on your own, such as your home Internet or ISP, this will likely cause them to cancel your account due to the abuse complaints\.
.
.P
One solution is to work with your ISP, to be clear about precisely what we are doing, to prove to them that we are researching the Internet, not "hacking" it\. We have our ISP send the abuse complaints directly to us\. For anyone that asks, we add them to our "\-\-excludefile", blacklisting them so that we won\'t scan them again\. While interacting with such people, some instead add us to their whitelist, so that their firewalls won\'t log us anymore (they\'ll still block us, of course, they just won\'t log that fact to avoid filling up their logs with our scans)\.
.
.P
Ultimately, I don\'t know if it\'s possible to completely solve this problem\. Despite the Internet being a public, end\-to\-end network, you are still "guilty until proven innocent" when you do a scan\.
.
.SH "COMPATIBILITY"
While not listed in this document, a lot of parameters compatible with \fBnmap\fR will also work\.
.
.SH "SEE ALSO"
nmap(8), pcap(3)
.
.SH "AUTHORS"
This tool was written by Robert Graham\. The source code is available at https://github\.com/robertdavidgraham/masscan\.

486
doc/masscan.8.markdown Normal file
View File

@@ -0,0 +1,486 @@
masscan(8) -- Fast scan of the Internet
=======================================
## SYNOPSIS
masscan \[options\] \[<IP|ranges>... -p PORT\[,PORT...\]\]
## DESCRIPTION
**masscan** is an Internet-scale port scanner, useful for large scale surveys
of the Internet, or of internal networks. While the default transmit rate
is only 100 packets/second, it can optionally go as fast as 25 million
packets/second, a rate sufficient to scan the Internet in 3 minutes for
one port.
## OPTIONS
* `<IP|RANGE>`: anything on the command-line not prefixed with a '-' is
assumed to be an IP address or range. There are three valid formats.
The first is a single IP address like `192.168.0.1` or `2001:db8::1`. The second
is a range like `10.0.0.1-10.0.0.100`. The third is a CIDR address,
like `0.0.0.0/0` or `2001:db8::/90`. At least one target must be specified. Multiple
targets can be specified. This can be specified as multiple options
separated by space, or can be separated by a comma as a single option,
such as `10.0.0.0/8,192.168.0.1,2001:db8::1`.
* `--range <IP|RANGE>`: the same as target range spec described above,
except as a named parameter instead of an unnamed one.
* `-p PORT[,PORT..]`, `--ports PORT[,PORT...]`: specifies the port(s) to be scanned. A
single port can be specified, like `-p80`. A range of ports can be
specified, like `-p 20-25`. A list of ports/ranges can be specified, like
`-p80,20-25`. UDP ports can also be specified, like
`--ports U:161,U:1024-1100`.
* `--banners`: specifies that banners should be grabbed after establishing
a TCP connection. Protocols supported include HTTP, FTP, IMAP4, memcached,
POP3, SMTP, SSH, SSL, SMB, Telnet, RDP, and VNC. Note that banner will be
grabbed on standard port of the service only.
You may change this behavior with `--hello-string` or `--hello-file` options.
* `--rate RATE`: specifies the desired rate for transmitting
packets. This can be very small numbers, like `0.1` for transmitting
packets at rates of one every 10 seconds, for very large numbers like
10000000, which attempts to transmit at 10 million packets/second. In my
experience, Windows and can do 250 thousand packets per second, and latest
versions of Linux can do 2.5 million packets per second. The PF_RING driver
is needed to get to 25 million packets/second.
* `-c FILE`, `--conf FILE`: reads in a configuration file.
If not specified, then will read from `/etc/masscan/masscan.conf` by default.
The format is described below under 'CONFIGURATION FILE'.
* `--resume FILE`: the same as `--conf`, except that a few options
are automatically set, such as `--append-output`. The format of the
configuration file is described below. The purpose is to resume a scan
saved in `paused.conf` that was interupted with [ctrl-c].
* `--echo`: don't run, but instead dump the current configuration to a file.
This file can then be used with the `-c` option. The format of this
output is described below under 'CONFIGURATION FILE'.
* `-e IFNAME`, `--adapter IFNAME`: use the named raw network interface,
such as "eth0" or "dna1". If not specified, the first network interface
found with a default gateway will be used.
* `--adapter-ip IP`, `--source-ip IP`: send packets using this IP address. If not
specified, then the first IP address bound to the network interface
will be used. Instead of a single IP address, a range may be specified.
NOTE: The size of the range must be an even power of 2, such as 1, 2, 4,
8, 16, 1024 etc. addresses.
* `--adapter-port PORT`: send packets using this port number as the
source. If not specified, a random port will be chosen in the range 40000
through 60000. This port should be filtered by the host firewall (like
iptables) to prevent the host network stack from interfering with arriving
packets. Instead of a single port, a range can be specified, like
`40000-40003`. NOTE: The size of the range must be an even power of 2,
such as the example above that has a total of 4 addresses.
* `--adapter-mac MAC`: send packets using this as the source MAC
address. If not specified, then the first MAC address bound to the network
interface will be used.
* `--adapter-vlan VLANID`: send packets using this 802.1q VLAN ID
* `--router-mac MAC`: send packets to this MAC address as the
destination. If not specified, then the gateway address of the network
interface will be ARPed.
* `--ping`: indicates that the scan should include an ICMP echo request.
This may be included with TCP and UDP scanning.
* `--exclude <IP|RANGE>`: blacklist an IP address or range, preventing it
from being scanned. This overrides any target specification, guaranteeing
that this address/range won't be scanned. This has the same format
as the normal target specification.
* `--excludefile FILE`: reads in a list of exclude ranges, in the same
target format described above. These ranges override any targets,
preventing them from being scanned.
* `-iL FILE`, `--includefile FILE`: reads in a list of ranges to scan, in the same
target format described above for IP addresses and ranges. This file can contain
millions of addresses and ranges.
* `--append-output`: causes output to append to the file, rather than
overwriting the file. Useful for when resumeing scans (see `--resume`).
* `--iflist`: list the available network interfaces, and then exits. The
`-e IFNAME` can then be used with one of the listed adapters.
* `--retries <num>`: the number of retries to send, at 1 second intervals. Note
that since this scanner is stateless, retries are sent regardless if
replies have already been received.
* `--nmap`: print help about nmap-compatibility alternatives for these
options.
* `--pcap-payloads FILE`: read packets from a libpcap file containing packets
and extract the UDP payloads, and associate those payloads with the
destination port. These payloads will then be used when sending UDP
packets with the matching destination port. Only one payload will
be remembered per port. Similar to `--nmap-payloads`.
* `--nmap-payloads FILE`: read in a file in the same format as
the nmap file `nmap-payloads`. This contains UDP payload, so that we
can send useful UDP packets instead of empty ones. Similar to
`--pcap-payloads`.
* `--http-* HEADER`: replaces the existing field in the HTTP header
with a new one. Fields that can be replaced are `--http-method`, `--http-url`,
`--http-version`,`--http-host`, and `--http-user-agent`.
Example: `--http-user-agent Keurig K575 Coffee Maker`. See also `--http-field` and `--http-cookie`.
* `--http-field NAME:VALUE`: replaces the existing HTTP header field,
or inserts a new one if the field doesn't exist, given as a `name:value` pair.
Cannot be used to replace the fields in the request-line (method, url, version).
Example: `--http-field Accept:image/gif`.
* `--http-field-remove NAME`: removes the first field from the header that matches
(may be needed multiple times for fields like `Cookie` that can exist multiple times)
* `--http-cookie VALUE`: adds a `Cookie:` field to the HTTP header, even
if other cookie fields exist. The other `--http-*` options replace existing
fields in the HTTP header, this one adds more even if some already exist.
* `--http-payload STR`: adds a payload string after the header; this will
automatically add a `--http-field Content-Length:LEN` field to match the length of the string,
but the user will have to add their own `--http-field Content-Type:TYPE` field to match
the string. Presumably, the user will also change the method to something like
`--http-method POST`. Common conntent types would be `application/x-www-form-urlencoded`,
`application/json`, or `text/xml`.
* `--show [open|closed]`: tells which port status to display, such
as 'open' for those ports that respond with a SYN-ACK on TCP, or
'closed' for those ports that repsond with RST. The default is
only to display 'open' ports.
* `--noshow [open|closed]`: disables a port status to display, such
as to no longer display 'open' ports.
* `--pcap FILE`: saves received packets (but not transmitted
packets) to the libpcap-format file.
* `--packet-trace`: prints a summary of those packets sent and received.
This is useful at low rates, like a few packets per second, but will
overwhelm the terminal at high rates.
* `--pfring`: force the use of the PF_RING driver. The program will exit
if PF_RING DNA drvers are not available.
* `--resume-index INDEX`: the point in the scan at when it was paused.
* `--resume-count NUM`: the maximum number of probes to send before exiting.
This is useful with the `--resume-index` to chop up a scan and split
it among multiple instances, though the `--shards` option might be
better.
* `--shards X/Y`: splits the scan among instances. `x` is the id
for this scan, while `y` is the total number of instances. For example,
`--shards 1/2` tells an instance to send every other packet, starting
with index 0. Likewise, `--shards 2/2` sends every other packet, but
starting with index 1, so that it doesn't overlap with the first example.
* `--rotate TIME`: rotates the output file, renaming it with the
current timestamp, moving it to a separate directory. The time is
specified in number of seconds, like "3600" for an hour. Or, units
of time can be specified, such as "hourly", or "6hours", or "10min".
Times are aligned on an even boundary, so if "daily" is specified,
then the file will be rotated every day at midnight.
* `--rotate-offset TIME`: an offset in the time. This is to accomodate
timezones.
* `--rotate-size SIZE`: rotates the output file when it exceeds the
given size. Typical suffixes can be applied (k,m,g,t) for kilo, mega,
giga, tera.
* `--rotate-dir DIR`: when rotating the file, this specifies which
directory to move the file to. A useful directory is `/var/log/masscan`.
* `--seed INT`: an integer that seeds the random number generator.
Using a different seed will cause packets to be sent in a different
random order. Instead of an integer, the string `time` can be specified,
which seeds using the local timestamp, automatically generating a
different random order of scans. If no seed specified, `time` is the
default.
* `--regress`: run a regression test, returns '0' on success and '1' on
failure.
* `--ttl NUM`: specifies the TTL of outgoing packets, defaults to 255.
* `--wait SECONDS`: specifies the number of seconds after transmit is
done to wait for receiving packets before exiting the program. The default
is 10 seconds. The string `forever` can be specified to never terminate.
* `--offline`: don't actually transmit packets. This is useful with
a low rate and `--packet-trace` to look at what packets might've been
transmitted. Or, it's useful with `--rate 100000000` in order to
benchmark how fast transmit would work (assuming a zero-overhead
driver). PF_RING is about 20% slower than the benchmark result from
offline mode.
* `-sL`: this doesn't do a scan, but instead creates a list of random
addresses. This is useful for importing into other tools. The options
`--shard`, `--resume-index`, and `--resume-count` can be useful with
this feature.
* `--interactive`: show the results in realtime on the console. It has
no effect if used with --output-format or --output-filename.
* `--output-format FMT`: indicates the format of the output file, which
can be `xml`, `binary`, `grepable`, `list`, or `JSON`. The
option `--output-filename` must be specified.
* `--output-filename FILE`: the file which to save results to. If
the parameter `--output-format` is not specified, then the default of
`xml` will be used.
* `-oB FILE`: sets the output format to binary and saves the output in
the given filename. This is equivalent to using the `--output-format` and
`--output-filename` parameters. The option `--readscan` can then be used to
read the binary file. Binary files are mush smaller than their XML
equivalents, but require a separate step to convert back into XML or
another readable format.
* `-oX FILE`: sets the output format to XML and saves the output in the
given filename. This is equivalent to using the `--output-format xml` and
`--output-filename` parameters.
* `-oG FILE`: sets the output format to grepable and saves the output
in the given filename. This is equivalent to using the --output-format grepable
and --output-filename parameters.
* `-oJ FILE`: sets the output format to JSON and saves the output in
the given filename. This is equivalent to using the --output-format json
and --output-filename parameters.
* `-oL FILE`: sets the output format to a simple list format and saves
the output in the given filename. This is equivalent to using
the --output-format list and --output-filename parameters.
* `--readscan FILE`: reads the files created by the `-oB` option
from a scan, then outputs them in one of the other formats, depending
on command-line parameters. In other words, it can take the binary
version of the output and convert it to an XML or JSON format. When this option
is given, defaults from `/etc/masscan/masscan.conf` will not be read.
* `--connection-timeout SECS`: when doing banner checks, this specifies the
maximum number of seconds that a TCP connection can be held open. The default
is 30 seconds. Increase this time if banners are incomplete. For example,
we have to increase the timeout when downloading all the SSL certs from
the Internet, because some sites take that long to deliver all the certs
in the chain. However, beware that when this is set to a large value, it'll
consume a lot of memory on fast scans. While the code may handle millions of
open TCP connections, you may not have enough memory for that.
* `--hello-file[PORT] FILE`: send the contents of the file once the
TCP connection has been established with the given port. Requires that
`--banners` also be set. Heuristics will be performed on the reponse in
an attempt to discover what protocol, so HTTP responses will be parsed
differently than other protocols.
* `--hello-string[PORT] BASE64`: same as `--hello-file` except that the
contents of the BASE64 encoded string are decoded, then used as the hello
string that greets the server.
* `--capture TYPE` or `--nocapture TYPE`: when doing banners (`--banner`), this
determines what to capture from the banners. By default, only the TITLE field from
HTML documents is captured, to get the entire document, use `--capture html`.
By default, the entire certificate from SSL is captured, to disable this, use
`--nocapture cert`. Currently, only the values `html` and `cert` are currently
supported for this option, but many more will be added in the future.
## CONFIGURATION FILE FORMAT
The configuration file uses the same parameter names as on the
commandline, but without the `--` prefix, and with an `=` sign
between the name and the value. An example configuration file
might be:
# targets
range = 10.0.0.0/8,192.168.0.0/16
range = 172.16.0.0/12
ports = 20-25,80,U:53
ping = true
# adapter
adapter = eth0
adapter-ip = 192.168.0.1
router-mac = 66-55-44-33-22-11
# other
exclude-file = /etc/masscan/exludes.txt
By default, the program will read default configuration from the file
`/etc/masscan/masscan.conf`. This is useful for system-specific settings,
such as the `--adapter-xxx` options. This is also useful for
excluded IP addresses, so that you can scan the entire Internet,
while skipping dangerous addresses, like those owned by the DoD,
and not make an accidental mistake.
## CONTROL-C BEHAVIOR
When the user presses <ctrl-c>, the scan will stop, and the current
state of the scan will be saved in the file 'paused.conf'. The scan
can be resumed with the `--resume` option:
# masscan --resume paused.conf
The program will not exit immediately, but will wait a default of 10
seconds to receive results from the Internet and save the results before
exiting completely. This time can be changed with the `--wait` option.
## USER-MODE STACK
Masscan has a user-mode TCP/IP stack that co-exists with the operating-system's
stack. Normally, this works fine but sometimes can cause problems, especially
with the `--banners` option that establishes a TCP/IP connection. In some
cases, all the stack's parameters will have to be specified separately:
--adapter-port PORT
--adapter-ip IP
--adapter-mac MAC
--adapter-vlan VLANID
--router-mac MAC
If the user-mode stack shares the same IP address as the operating-system,
then the kernel will send RST packets during a scan. This can cause
unnecessary traffic during a simple port scan, and will terminate TCP
connections when doing a `--banners` scan. To prevent, this, the built-in
firewall should be used to filter the source ports. On Linux, this can be done
by doing something like:
# iptables -A INPUT -i eth0 -p tcp --dport 44444 -j DROP
This will prevent the Linux kernel from processing incoming packets to port
44444, but `masscan` will still see the packets. Set the maching parameter
of `--adapter-port 44444` to force `masscan` to use that port instead of
a random port.
## SIMPLE EXAMPLES
The following example scans all private networks for webservers, and prints
all open ports that were found.
# masscan 10.0.0.0/8 192.168.0.0/16 172.16.0.0/12 -p80 --open-only
The following example scans the entire Internet for DNS servers, grabbing
their versions, then saves the results in an XML file.
# masscan 0.0.0.0/0 --excludefile no-dod.txt -pU:53 --banners --output-filename dns.xml
You should be able to import the XML into databases and such.
The following example reads a binary scan results file called bin-test.scan and prints
results to console.
# masscan --readscan bin-test.scan
The following example reads a binary scan results file called bin-test.scan and creates
an XML output file called bin-test.xml.
# masscan --readscan bin-test.scan -oX bin-test.xml
## ADVANCED EXAMPLES
Let's say that you want to scan the entire Internet and spread the scan
across three machines. Masscan would be launched on all three machines
using the following command-lines:
# masscan 0.0.0.0/0 -p0-65535 --shard 1/3
# masscan 0.0.0.0/0 -p0-65535 --shard 2/3
# masscan 0.0.0.0/0 -p0-65535 --shard 3/3
An alternative is with the "resume" feature. A scan has an internal index that
goes from zero to the number of ports times the number of IP addresses. The
following example shows splitting up a scan into chunks of a 1000 items each:
# masscan 0.0.0.0/0 -p0-65535 --resume-index 0 --resume-count 1000
# masscan 0.0.0.0/0 -p0-65535 --resume-index 1000 --resume-count 1000
# masscan 0.0.0.0/0 -p0-65535 --resume-index 2000 --resume-count 1000
# masscan 0.0.0.0/0 -p0-65535 --resume-index 3000 --resume-count 1000
A script can use this to split smaller tasks across many other machines,
such as Amazon EC2 instances. As each instance completes a job, the
script might send a request to a central coordinating server for more
work.
## SPURIOUS RESETS
When scanning TCP using the default IP address of your adapter, the built-in
stack will generate RST packets. This will prevent banner grabbing. There are
are two ways to solve this. The first way is to create a firewall rule
to block that port from being seen by the stack. How this works is dependent
on the operating system, but on Linux this looks something like:
# iptables -A INPUT -p tcp -i eth0 --dport 61234 -j DROP
Then, when scanning, that same port must be used as the source:
# masscan 10.0.0.0/8 -p80 --banners --adapter-port 61234
An alternative is to "spoof" a different IP address. This IP address must be
within the range of the local network, but must not otherwise be in use by
either your own computer or another computer on the network. An example of this
would look like:
# masscan 10.0.0.0/8 -p80 --banners --adapter-ip 192.168.1.101
Setting your source IP address this way is the preferred way of running this
scanner.
## ABUSE COMPLAINTS
This scanner is designed for large-scale surveys, of either an organization,
or of the Internet as a whole. This scanning will be noticed by those
monitoring their logs, which will generate complaints.
If you are scanning your own organization, this may lead to you being fired.
Never scan outside your local subnet without getting permission from your boss,
with a clear written declaration of why you are scanning.
The same applies to scanning the Internet from your employer. This is another
good way to get fired, as your IT department gets flooded with complaints as
to why your organization is hacking them.
When scanning on your own, such as your home Internet or ISP, this will likely
cause them to cancel your account due to the abuse complaints.
One solution is to work with your ISP, to be clear about precisely what we are
doing, to prove to them that we are researching the Internet, not "hacking" it.
We have our ISP send the abuse complaints directly to us. For anyone that asks,
we add them to our "--excludefile", blacklisting them so that we won't scan
them again. While interacting with such people, some instead add us to their
whitelist, so that their firewalls won't log us anymore (they'll still block
us, of course, they just won't log that fact to avoid filling up their logs
with our scans).
Ultimately, I don't know if it's possible to completely solve this problem.
Despite the Internet being a public, end-to-end network, you are still
"guilty until proven innocent" when you do a scan.
## COMPATIBILITY
While not listed in this document, a lot of parameters compatible with
`nmap` will also work. It runs on macOS, Linux, and Windows. It's compiled
in fairly portable C language. It supports IPv4 and IPv6.
## SEE ALSO
nmap(8), pcap(3)
## AUTHORS
This tool was written by Robert Graham. The source code is available at
https://github.com/robertdavidgraham/masscan.

225
src/crypto-base64.c Normal file
View File

@@ -0,0 +1,225 @@
#include "crypto-base64.h"
#include <ctype.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <time.h>
/*****************************************************************************
*****************************************************************************/
size_t
base64_encode(void *vdst, size_t sizeof_dst,
const void *vsrc, size_t sizeof_src)
{
static const char *b64 =
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"abcdefghijklmnopqrstuvwxyz"
"0123456789"
"+/";
size_t i = 0;
size_t d = 0;
unsigned char *dst = (unsigned char *)vdst;
const unsigned char *src = (const unsigned char *)vsrc;
/* encode every 3 bytes of source into 4 bytes of destination text */
while (i + 3 <= sizeof_src) {
unsigned n;
/* make sure there is enough space */
if (d + 4 > sizeof_dst)
return d;
/* convert the chars */
n = src[i]<<16 | src[i+1]<<8 | src[i+2];
dst[d+0] = b64[ (n>>18) & 0x3F ];
dst[d+1] = b64[ (n>>12) & 0x3F ];
dst[d+2] = b64[ (n>> 6) & 0x3F ];
dst[d+3] = b64[ (n>> 0) & 0x3F ];
i += 3;
d += 4;
}
/* If the source text isn't an even multiple of 3 characters, then we'll
* have to append a '=' or '==' to the output to compensate */
if (i + 2 <= sizeof_src && d + 4 <= sizeof_dst) {
unsigned n = src[i]<<16 | src[i+1]<<8;
dst[d+0] = b64[ (n>>18) & 0x3F ];
dst[d+1] = b64[ (n>>12) & 0x3F ];
dst[d+2] = b64[ (n>> 6) & 0x3F ];
dst[d+3] = '=';
d += 4;
} else if (i + 1 <= sizeof_src && d + 4 <= sizeof_dst) {
unsigned n = src[i]<<16;
dst[d+0] = b64[ (n>>18) & 0x3F ];
dst[d+1] = b64[ (n>>12) & 0x3F ];
dst[d+2] = '=';
dst[d+3] = '=';
d += 4;
}
return d;
}
/*****************************************************************************
*****************************************************************************/
size_t
base64_decode(void *vdst, size_t sizeof_dst,
const void *vsrc, size_t sizeof_src)
{
static const unsigned char rstr[] = {
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 62, 0xFF, 0xFF, 0xFF, 63,
52, 53, 54, 55, 56, 57, 58, 59,
60, 61, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0, 1, 2, 3, 4, 5, 6,
7, 8, 9, 10, 11, 12, 13, 14,
15, 16, 17, 18, 19, 20, 21, 22,
23, 24, 25, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 26, 27, 28, 29, 30, 31, 32,
33, 34, 35, 36, 37, 38, 39, 40,
41, 42, 43, 44, 45, 46, 47, 48,
49, 50, 51, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
};
size_t i = 0;
size_t d = 0;
unsigned char *dst = (unsigned char *)vdst;
const unsigned char *src = (const unsigned char *)vsrc;
while (i < sizeof_src) {
unsigned b;
unsigned c=0;
/* byte#1 */
while (i<sizeof_src && (c = rstr[src[i]]) > 64)
i++;
if (src[i] == '=' || i++ >= sizeof_src)
break;
b = (c << 2) & 0xfc;
while (i<sizeof_src && (c = rstr[src[i]]) > 64)
i++;
if (src[i] == '=' || i++ >= sizeof_src)
break;
b |= (c>>4) & 0x03;
if (d<sizeof_dst)
dst[d++] = (unsigned char)b;
if (i>=sizeof_src)
break;
/* byte#2 */
b = (c<<4) & 0xF0;
while (i<sizeof_src && src[i] != '=' && (c = rstr[src[i]]) > 64)
;
if (src[i] == '=' || i++ >= sizeof_src)
break;
b |= (c>>2) & 0x0F;
if (d<sizeof_dst)
dst[d++] = (unsigned char)b;
if (i>=sizeof_src)
break;
/* byte#3*/
b = (c<<6) & 0xC0;
while (i<sizeof_src && src[i] != '=' && (c = rstr[src[i]]) > 64)
;
if (src[i] == '=' || i++ >= sizeof_src)
break;
b |= c;
if (d<sizeof_dst)
dst[d++] = (unsigned char)b;
if (i>=sizeof_src)
break;
}
if (d<sizeof_dst)
dst[d] = '\0';
return d;
}
/*****************************************************************************
* Provide my own rand() simply to avoid static-analysis warning me that
* 'rand()' is unrandom, when in fact we want the non-random properties of
* rand() for regression testing.
*****************************************************************************/
static unsigned
r_rand(unsigned *seed)
{
static const unsigned a = 214013;
static const unsigned c = 2531011;
*seed = (*seed) * a + c;
return (*seed)>>16 & 0x7fff;
}
/*****************************************************************************
*****************************************************************************/
int
base64_selftest(void)
{
char buf[100];
char buf2[100];
char buf3[100];
size_t buf_len;
size_t buf2_len;
unsigned i;
unsigned seed = (unsigned)time(0);
buf_len = base64_encode(buf, sizeof(buf), "hello", 5);
buf2_len = base64_decode(buf2, sizeof(buf2), buf, buf_len);
if (buf2_len != 5 && memcmp(buf2, "hello", 5) != 0) {
fprintf(stderr, "base64: selftest failed\n");
return 1;
}
/*
* Generate a bunch of random strings, encode them, then decode them,
* making sure the final result matches the original string
*/
for (i=0; i<100; i++) {
unsigned j;
size_t buf3_len;
/* create a string of random bytes */
buf_len = r_rand(&seed) % 50;
for (j=0; j<buf_len; j++) {
buf[j] = (char)r_rand(&seed);
}
/* encode it */
buf2_len = base64_encode(buf2, sizeof(buf2), buf, buf_len);
/* decode it back again */
buf3_len = base64_decode(buf3, sizeof(buf3), buf2, buf2_len);
/* now make sure result equals original */
if (buf3_len != buf_len && memcmp(buf3, buf, buf_len) != 0) {
fprintf(stderr, "base64: selftest failed\n");
return 1;
}
}
return 0;
}

10
src/crypto-base64.h Normal file
View File

@@ -0,0 +1,10 @@
#ifndef CRYPTO_BASE64_H
#define CRYPTO_BASE64_H
#include <stdio.h>
size_t base64_decode(void *dst, size_t sizeof_dst, const void *src, size_t sizeof_src);
size_t base64_encode(void *dst, size_t sizeof_dst, const void *src, size_t sizeof_src);
int base64_selftest(void);
#endif

420
src/crypto-blackrock.c Normal file
View File

@@ -0,0 +1,420 @@
/*
BlackRock cipher
(h/t Marsh Ray @marshray for this idea)
This is a randomization/reshuffling function based on a crypto
"Feistel network" as described in the paper:
'Ciphers with Arbitrary Finite Domains'
by John Black and Phillip Rogaway
http://www.cs.ucdavis.edu/~rogaway/papers/subset.pdf
This is a crypto-like construction that encrypts an arbitrary sized
range. Given a number in the range [0..9999], it'll produce a mapping
to a distinct different number in the same range (and back again).
In other words, it randomizes the order of numbers in a sequence.
For example, it can be used to randomize the sequence [0..9]:
0 -> 6
1 -> 4
2 -> 8
3 -> 1
4 -> 9
5 -> 3
6 -> 0
7 -> 5
8 -> 2
9 -> 7
As you can see on the right hand side, the numbers are in random
order, and they don't repeat.
This is create for port scanning. We can take an index variable
and increment it during a scan, then use this function to
randomize it, yet be assured that we've probed every IP and port
within the range.
The cryptographic strength of this construction depends upon the
number of rounds, and the exact nature of the inner "READ()" function.
Because it's a Feistel network, that "READ()" function can be almost
anything.
We don't care about cryptographic strength, just speed, so we are
using a trivial READ() function.
This is a class of "format-preserving encryption". There are
probably better constructions than what I'm using.
*/
#include "crypto-blackrock.h"
#include "pixie-timer.h"
#include "util-malloc.h"
#include <stdint.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <ctype.h>
#include <time.h>
#if defined(_MSC_VER)
#define inline _inline
#endif
/***************************************************************************
* It's an s-box. You gotta have an s-box
***************************************************************************/
const unsigned char sbox[256] = {
0x91, 0x58, 0xb3, 0x31, 0x6c, 0x33, 0xda, 0x88,
0x57, 0xdd, 0x8c, 0xf2, 0x29, 0x5a, 0x08, 0x9f,
0x49, 0x34, 0xce, 0x99, 0x9e, 0xbf, 0x0f, 0x81,
0xd4, 0x2f, 0x92, 0x3f, 0x95, 0xf5, 0x23, 0x00,
0x0d, 0x3e, 0xa8, 0x90, 0x98, 0xdd, 0x20, 0x00,
0x03, 0x69, 0x0a, 0xca, 0xba, 0x12, 0x08, 0x41,
0x6e, 0xb9, 0x86, 0xe4, 0x50, 0xf0, 0x84, 0xe2,
0xb3, 0xb3, 0xc8, 0xb5, 0xb2, 0x2d, 0x18, 0x70,
0x0a, 0xd7, 0x92, 0x90, 0x9e, 0x1e, 0x0c, 0x1f,
0x08, 0xe8, 0x06, 0xfd, 0x85, 0x2f, 0xaa, 0x5d,
0xcf, 0xf9, 0xe3, 0x55, 0xb9, 0xfe, 0xa6, 0x7f,
0x44, 0x3b, 0x4a, 0x4f, 0xc9, 0x2f, 0xd2, 0xd3,
0x8e, 0xdc, 0xae, 0xba, 0x4f, 0x02, 0xb4, 0x76,
0xba, 0x64, 0x2d, 0x07, 0x9e, 0x08, 0xec, 0xbd,
0x52, 0x29, 0x07, 0xbb, 0x9f, 0xb5, 0x58, 0x6f,
0x07, 0x55, 0xb0, 0x34, 0x74, 0x9f, 0x05, 0xb2,
0xdf, 0xa9, 0xc6, 0x2a, 0xa3, 0x5d, 0xff, 0x10,
0x40, 0xb3, 0xb7, 0xb4, 0x63, 0x6e, 0xf4, 0x3e,
0xee, 0xf6, 0x49, 0x52, 0xe3, 0x11, 0xb3, 0xf1,
0xfb, 0x60, 0x48, 0xa1, 0xa4, 0x19, 0x7a, 0x2e,
0x90, 0x28, 0x90, 0x8d, 0x5e, 0x8c, 0x8c, 0xc4,
0xf2, 0x4a, 0xf6, 0xb2, 0x19, 0x83, 0xea, 0xed,
0x6d, 0xba, 0xfe, 0xd8, 0xb6, 0xa3, 0x5a, 0xb4,
0x48, 0xfa, 0xbe, 0x5c, 0x69, 0xac, 0x3c, 0x8f,
0x63, 0xaf, 0xa4, 0x42, 0x25, 0x50, 0xab, 0x65,
0x80, 0x65, 0xb9, 0xfb, 0xc7, 0xf2, 0x2d, 0x5c,
0xe3, 0x4c, 0xa4, 0xa6, 0x8e, 0x07, 0x9c, 0xeb,
0x41, 0x93, 0x65, 0x44, 0x4a, 0x86, 0xc1, 0xf6,
0x2c, 0x97, 0xfd, 0xf4, 0x6c, 0xdc, 0xe1, 0xe0,
0x28, 0xd9, 0x89, 0x7b, 0x09, 0xe2, 0xa0, 0x38,
0x74, 0x4a, 0xa6, 0x5e, 0xd2, 0xe2, 0x4d, 0xf3,
0xf4, 0xc6, 0xbc, 0xa2, 0x51, 0x58, 0xe8, 0xae,
};
/***************************************************************************
***************************************************************************/
void
blackrock_init(struct BlackRock *br, uint64_t range, uint64_t seed, unsigned rounds)
{
double foo = sqrt(range * 1.0);
/* This algorithm gets very non-random at small numbers, so I'm going
* to try to fix some constants here to make it work. It doesn't have
* to be good, since it's kinda pointless having ranges this small */
switch (range) {
case 0:
br->a = 0;
br->b = 0;
break;
case 1:
br->a = 1;
br->b = 1;
break;
case 2:
br->a = 1;
br->b = 2;
break;
case 3:
br->a = 2;
br->b = 2;
break;
case 4:
case 5:
case 6:
br->a = 2;
br->b = 3;
break;
case 7:
case 8:
br->a = 3;
br->b = 3;
break;
default:
br->range = range;
br->a = (uint64_t)(foo - 2);
br->b = (uint64_t)(foo + 3);
break;
}
while (br->a * br->b <= range)
br->b++;
br->rounds = rounds;
br->seed = seed;
br->range = range;
}
/***************************************************************************
* The inner round/mixer function. In DES, it's a series of S-box lookups,
* which
***************************************************************************/
static inline uint64_t
READ(uint64_t r, uint64_t R, uint64_t seed)
{
uint64_t r0, r1, r2, r3;
#define GETBYTE(R,n) ((((R)>>(n*8))^seed^r)&0xFF)
R ^= (seed << r) ^ (seed >> (64 - r));
r0 = sbox[GETBYTE(R,0)]<< 0 | sbox[GETBYTE(R,1)]<< 8;
r1 = (sbox[GETBYTE(R,2)]<<16UL | sbox[GETBYTE(R,3)]<<24UL)&0x0ffffFFFFUL;
r2 = sbox[GETBYTE(R,4)]<< 0 | sbox[GETBYTE(R,5)]<< 8;
r3 = (sbox[GETBYTE(R,6)]<<16UL | sbox[GETBYTE(R,7)]<<24UL)&0x0ffffFFFFUL;
R = r0 ^ r1 ^ r2<<23UL ^ r3<<33UL;
return R;
}
/***************************************************************************
*
* NOTE:
* the names in this function are cryptic in order to match as closely
* as possible the pseudocode in the following paper:
* http://www.cs.ucdavis.edu/~rogaway/papers/subset.pdf
* Read that paper in order to understand this code.
***************************************************************************/
static inline uint64_t
ENCRYPT(unsigned r, uint64_t a, uint64_t b, uint64_t m, uint64_t seed)
{
uint64_t L, R;
unsigned j;
uint64_t tmp;
L = m % a;
R = m / a;
for (j=1; j<=r; j++) {
if (j & 1) {
tmp = (L + READ(j, R, seed)) % a;
} else {
tmp = (L + READ(j, R, seed)) % b;
}
L = R;
R = tmp;
}
if (r & 1) {
return a * L + R;
} else {
return a * R + L;
}
}
/***************************************************************************
***************************************************************************/
static inline uint64_t
UNENCRYPT(unsigned r, uint64_t a, uint64_t b, uint64_t m, uint64_t seed)
{
uint64_t L, R;
unsigned j;
uint64_t tmp;
if (r & 1) {
R = m % a;
L = m / a;
} else {
L = m % a;
R = m / a;
}
for (j=r; j>=1; j--) {
if (j & 1) {
tmp = READ(j, L, seed);
if (tmp > R) {
tmp = (tmp - R);
tmp = a - (tmp%a);
if (tmp == a)
tmp = 0;
} else {
tmp = (R - tmp);
tmp %= a;
}
} else {
tmp = READ(j, L, seed);
if (tmp > R) {
tmp = (tmp - R);
tmp = b - (tmp%b);
if (tmp == b)
tmp = 0;
} else {
tmp = (R - tmp);
tmp %= b;
}
}
R = L;
L = tmp;
}
return a * R + L;
}
/***************************************************************************
***************************************************************************/
uint64_t
blackrock_shuffle(const struct BlackRock *br, uint64_t m)
{
uint64_t c;
c = ENCRYPT(br->rounds, br->a, br->b, m, br->seed);
while (c >= br->range)
c = ENCRYPT(br->rounds, br->a, br->b, c, br->seed);
return c;
}
/***************************************************************************
***************************************************************************/
uint64_t
blackrock_unshuffle(const struct BlackRock *br, uint64_t m)
{
uint64_t c;
c = UNENCRYPT(br->rounds, br->a, br->b, m, br->seed);
while (c >= br->range)
c = UNENCRYPT(br->rounds, br->a, br->b, c, br->seed);
return c;
}
/***************************************************************************
* This function called only during selftest/regression-test.
***************************************************************************/
static unsigned
blackrock_verify(struct BlackRock *br, uint64_t max)
{
unsigned char *list;
uint64_t i;
unsigned is_success = 1;
uint64_t range = br->range;
/* Allocate a list of 1-byte counters */
list = CALLOC(1, (size_t)((range<max)?range:max));
/* For all numbers in the range, verify increment the counter for
* the output. */
for (i=0; i<range; i++) {
uint64_t x = blackrock_shuffle(br, i);
if (x < max)
list[x]++;
}
/* Now check the output to make sure that every counter is set exactly
* to the value of '1'. */
for (i=0; i<max && i<range; i++) {
if (list[i] != 1)
is_success = 0;
}
free(list);
return is_success;
}
/***************************************************************************
***************************************************************************/
void
blackrock_benchmark(unsigned rounds)
{
struct BlackRock br;
uint64_t range = 0x012356789123ULL;
uint64_t i;
uint64_t result = 0;
uint64_t start, stop;
static const uint64_t ITERATIONS = 5000000ULL;
printf("-- blackrock-1 -- \n");
printf("rounds = %u\n", rounds);
blackrock_init(&br, range, 1, rounds);
/*
* Time the algorithm
*/
start = pixie_nanotime();
for (i=0; i<ITERATIONS; i++) {
result += blackrock_shuffle(&br, i);
}
stop = pixie_nanotime();
/*
* Print the results
*/
if (result) {
double elapsed = ((double)(stop - start))/(1000000000.0);
double rate = ITERATIONS/elapsed;
rate /= 1000000.0;
printf("iterations/second = %5.3f-million\n", rate);
}
printf("\n");
}
/***************************************************************************
***************************************************************************/
int
blackrock_selftest(void)
{
uint64_t i;
uint64_t range;
/* @marshray
* Basic test of decryption. I take the index, encrypt it, then decrypt it,
* which means I should get the original index back again. Only, it's not
* working. The decryption fails. The reason it's failing is obvious -- I'm
* just not seeing it though. The error is probably in the 'UNENCRYPT()'
* function above.
*/
{
struct BlackRock br;
blackrock_init(&br, 1000, 0, 4);
for (i=0; i<10; i++) {
uint64_t result, result2;
result = blackrock_shuffle(&br, i);
result2 = blackrock_unshuffle(&br, result);
if (i != result2)
return 1; /*fail*/
}
}
range = 3015 * 3;
for (i=0; i<5; i++) {
struct BlackRock br;
int is_success;
range += 10 + i;
range *= 2;
blackrock_init(&br, range, time(0), 4);
is_success = blackrock_verify(&br, range);
if (!is_success) {
fprintf(stderr, "BLACKROCK: randomization failed\n");
return 1; /*fail*/
}
}
return 0; /*success*/
}

79
src/crypto-blackrock.h Normal file
View File

@@ -0,0 +1,79 @@
#ifndef RAND_BLACKROCK_H
#define RAND_BLACKROCK_H
#include <stdint.h>
struct BlackRock {
uint64_t range;
uint64_t a;
uint64_t b;
uint64_t seed;
unsigned rounds;
uint64_t a_bits;
uint64_t a_mask;
uint64_t b_bits;
uint64_t b_mask;
};
/**
* Initializes a structure for shuffling numbers within
* a range.
*
* @param range
* The size of the range of numbers needing to be
* shuffled/randomized.
*/
void
blackrock_init(struct BlackRock *br, uint64_t range, uint64_t seed, unsigned rounds);
void
blackrock2_init(struct BlackRock *br, uint64_t range, uint64_t seed, unsigned rounds);
/**
* Given a number within a range, produce a different number with
* the same range. There is a 1-to-1 mapping between the two,
* so when linearly incrementing through the range, the output
* of this function won't repeat. In other words, encrypt the index variable.
* @param br
* The randomization parameters created with 'blackrock_init()'
* @param index
* An input within the specified range. We call it an 'index' variable
* because that's how we intend to use this function, shuffling a
* monotonically increasing index variable, but in truth, any sort
* of integer can be used. This must be within the 'range' specified
* during the call to blackrock_init(), or the results are undefined.
* @return
* A one-to-one matching index that's in the same range.
*/
uint64_t
blackrock_shuffle(const struct BlackRock *br, uint64_t index);
uint64_t
blackrock2_shuffle(const struct BlackRock *br, uint64_t index);
/**
* The reverse of the shuffle function above: given the shuffled/encrypted
* integer, return the original index value before the shuffling/encryption.
*/
uint64_t
blackrock_unshuffle(const struct BlackRock *br, uint64_t m);
uint64_t
blackrock2_unshuffle(const struct BlackRock *br, uint64_t m);
/**
* Do a regression test.
* @return
* 0 of the regression test succeeds or non-zero if it fails
*/
int
blackrock_selftest(void);
int
blackrock2_selftest(void);
/**
* Do a benchmark of this module regression test.
*/
void
blackrock_benchmark(unsigned rounds);
void
blackrock2_benchmark(unsigned rounds);
#endif

573
src/crypto-blackrock2.c Normal file
View File

@@ -0,0 +1,573 @@
#include "crypto-blackrock.h"
#include "pixie-timer.h"
#include "unusedparm.h"
#include "util-malloc.h"
#include "util-safefunc.h"
#include <stdint.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <ctype.h>
#include <time.h>
#if defined(_MSC_VER)
#define inline _inline
#endif
/*
* Expanded DES S-boxes
*/
static const uint32_t SB1[64] =
{
0x01010400, 0x00000000, 0x00010000, 0x01010404,
0x01010004, 0x00010404, 0x00000004, 0x00010000,
0x00000400, 0x01010400, 0x01010404, 0x00000400,
0x01000404, 0x01010004, 0x01000000, 0x00000004,
0x00000404, 0x01000400, 0x01000400, 0x00010400,
0x00010400, 0x01010000, 0x01010000, 0x01000404,
0x00010004, 0x01000004, 0x01000004, 0x00010004,
0x00000000, 0x00000404, 0x00010404, 0x01000000,
0x00010000, 0x01010404, 0x00000004, 0x01010000,
0x01010400, 0x01000000, 0x01000000, 0x00000400,
0x01010004, 0x00010000, 0x00010400, 0x01000004,
0x00000400, 0x00000004, 0x01000404, 0x00010404,
0x01010404, 0x00010004, 0x01010000, 0x01000404,
0x01000004, 0x00000404, 0x00010404, 0x01010400,
0x00000404, 0x01000400, 0x01000400, 0x00000000,
0x00010004, 0x00010400, 0x00000000, 0x01010004
};
static const uint32_t SB2[64] =
{
0x80108020, 0x80008000, 0x00008000, 0x00108020,
0x00100000, 0x00000020, 0x80100020, 0x80008020,
0x80000020, 0x80108020, 0x80108000, 0x80000000,
0x80008000, 0x00100000, 0x00000020, 0x80100020,
0x00108000, 0x00100020, 0x80008020, 0x00000000,
0x80000000, 0x00008000, 0x00108020, 0x80100000,
0x00100020, 0x80000020, 0x00000000, 0x00108000,
0x00008020, 0x80108000, 0x80100000, 0x00008020,
0x00000000, 0x00108020, 0x80100020, 0x00100000,
0x80008020, 0x80100000, 0x80108000, 0x00008000,
0x80100000, 0x80008000, 0x00000020, 0x80108020,
0x00108020, 0x00000020, 0x00008000, 0x80000000,
0x00008020, 0x80108000, 0x00100000, 0x80000020,
0x00100020, 0x80008020, 0x80000020, 0x00100020,
0x00108000, 0x00000000, 0x80008000, 0x00008020,
0x80000000, 0x80100020, 0x80108020, 0x00108000
};
static const uint32_t SB3[64] =
{
0x00000208, 0x08020200, 0x00000000, 0x08020008,
0x08000200, 0x00000000, 0x00020208, 0x08000200,
0x00020008, 0x08000008, 0x08000008, 0x00020000,
0x08020208, 0x00020008, 0x08020000, 0x00000208,
0x08000000, 0x00000008, 0x08020200, 0x00000200,
0x00020200, 0x08020000, 0x08020008, 0x00020208,
0x08000208, 0x00020200, 0x00020000, 0x08000208,
0x00000008, 0x08020208, 0x00000200, 0x08000000,
0x08020200, 0x08000000, 0x00020008, 0x00000208,
0x00020000, 0x08020200, 0x08000200, 0x00000000,
0x00000200, 0x00020008, 0x08020208, 0x08000200,
0x08000008, 0x00000200, 0x00000000, 0x08020008,
0x08000208, 0x00020000, 0x08000000, 0x08020208,
0x00000008, 0x00020208, 0x00020200, 0x08000008,
0x08020000, 0x08000208, 0x00000208, 0x08020000,
0x00020208, 0x00000008, 0x08020008, 0x00020200
};
static const uint32_t SB4[64] =
{
0x00802001, 0x00002081, 0x00002081, 0x00000080,
0x00802080, 0x00800081, 0x00800001, 0x00002001,
0x00000000, 0x00802000, 0x00802000, 0x00802081,
0x00000081, 0x00000000, 0x00800080, 0x00800001,
0x00000001, 0x00002000, 0x00800000, 0x00802001,
0x00000080, 0x00800000, 0x00002001, 0x00002080,
0x00800081, 0x00000001, 0x00002080, 0x00800080,
0x00002000, 0x00802080, 0x00802081, 0x00000081,
0x00800080, 0x00800001, 0x00802000, 0x00802081,
0x00000081, 0x00000000, 0x00000000, 0x00802000,
0x00002080, 0x00800080, 0x00800081, 0x00000001,
0x00802001, 0x00002081, 0x00002081, 0x00000080,
0x00802081, 0x00000081, 0x00000001, 0x00002000,
0x00800001, 0x00002001, 0x00802080, 0x00800081,
0x00002001, 0x00002080, 0x00800000, 0x00802001,
0x00000080, 0x00800000, 0x00002000, 0x00802080
};
static const uint32_t SB5[64] =
{
0x00000100, 0x02080100, 0x02080000, 0x42000100,
0x00080000, 0x00000100, 0x40000000, 0x02080000,
0x40080100, 0x00080000, 0x02000100, 0x40080100,
0x42000100, 0x42080000, 0x00080100, 0x40000000,
0x02000000, 0x40080000, 0x40080000, 0x00000000,
0x40000100, 0x42080100, 0x42080100, 0x02000100,
0x42080000, 0x40000100, 0x00000000, 0x42000000,
0x02080100, 0x02000000, 0x42000000, 0x00080100,
0x00080000, 0x42000100, 0x00000100, 0x02000000,
0x40000000, 0x02080000, 0x42000100, 0x40080100,
0x02000100, 0x40000000, 0x42080000, 0x02080100,
0x40080100, 0x00000100, 0x02000000, 0x42080000,
0x42080100, 0x00080100, 0x42000000, 0x42080100,
0x02080000, 0x00000000, 0x40080000, 0x42000000,
0x00080100, 0x02000100, 0x40000100, 0x00080000,
0x00000000, 0x40080000, 0x02080100, 0x40000100
};
static const uint32_t SB6[64] =
{
0x20000010, 0x20400000, 0x00004000, 0x20404010,
0x20400000, 0x00000010, 0x20404010, 0x00400000,
0x20004000, 0x00404010, 0x00400000, 0x20000010,
0x00400010, 0x20004000, 0x20000000, 0x00004010,
0x00000000, 0x00400010, 0x20004010, 0x00004000,
0x00404000, 0x20004010, 0x00000010, 0x20400010,
0x20400010, 0x00000000, 0x00404010, 0x20404000,
0x00004010, 0x00404000, 0x20404000, 0x20000000,
0x20004000, 0x00000010, 0x20400010, 0x00404000,
0x20404010, 0x00400000, 0x00004010, 0x20000010,
0x00400000, 0x20004000, 0x20000000, 0x00004010,
0x20000010, 0x20404010, 0x00404000, 0x20400000,
0x00404010, 0x20404000, 0x00000000, 0x20400010,
0x00000010, 0x00004000, 0x20400000, 0x00404010,
0x00004000, 0x00400010, 0x20004010, 0x00000000,
0x20404000, 0x20000000, 0x00400010, 0x20004010
};
static const uint32_t SB7[64] =
{
0x00200000, 0x04200002, 0x04000802, 0x00000000,
0x00000800, 0x04000802, 0x00200802, 0x04200800,
0x04200802, 0x00200000, 0x00000000, 0x04000002,
0x00000002, 0x04000000, 0x04200002, 0x00000802,
0x04000800, 0x00200802, 0x00200002, 0x04000800,
0x04000002, 0x04200000, 0x04200800, 0x00200002,
0x04200000, 0x00000800, 0x00000802, 0x04200802,
0x00200800, 0x00000002, 0x04000000, 0x00200800,
0x04000000, 0x00200800, 0x00200000, 0x04000802,
0x04000802, 0x04200002, 0x04200002, 0x00000002,
0x00200002, 0x04000000, 0x04000800, 0x00200000,
0x04200800, 0x00000802, 0x00200802, 0x04200800,
0x00000802, 0x04000002, 0x04200802, 0x04200000,
0x00200800, 0x00000000, 0x00000002, 0x04200802,
0x00000000, 0x00200802, 0x04200000, 0x00000800,
0x04000002, 0x04000800, 0x00000800, 0x00200002
};
static const uint32_t SB8[64] =
{
0x10001040, 0x00001000, 0x00040000, 0x10041040,
0x10000000, 0x10001040, 0x00000040, 0x10000000,
0x00040040, 0x10040000, 0x10041040, 0x00041000,
0x10041000, 0x00041040, 0x00001000, 0x00000040,
0x10040000, 0x10000040, 0x10001000, 0x00001040,
0x00041000, 0x00040040, 0x10040040, 0x10041000,
0x00001040, 0x00000000, 0x00000000, 0x10040040,
0x10000040, 0x10001000, 0x00041040, 0x00040000,
0x00041040, 0x00040000, 0x10041000, 0x00001000,
0x00000040, 0x10040040, 0x00001000, 0x00041040,
0x10001000, 0x00000040, 0x10000040, 0x10040000,
0x10040040, 0x10000000, 0x00040000, 0x10001040,
0x00000000, 0x10041040, 0x00040040, 0x10000040,
0x10040000, 0x10001000, 0x10001040, 0x00000000,
0x10041040, 0x00041000, 0x00041000, 0x00001040,
0x00001040, 0x00040040, 0x10000000, 0x10041000
};
/***************************************************************************
* It's an s-box. You gotta have an s-box
***************************************************************************/
const unsigned char sbox2[] = {
0x91, 0x58, 0xb3, 0x31, 0x6c, 0x33, 0xda, 0x88,
0x57, 0xdd, 0x8c, 0xf2, 0x29, 0x5a, 0x08, 0x9f,
0x49, 0x34, 0xce, 0x99, 0x9e, 0xbf, 0x0f, 0x81,
0xd4, 0x2f, 0x92, 0x3f, 0x95, 0xf5, 0x23, 0x00,
0x0d, 0x3e, 0xa8, 0x90, 0x98, 0xdd, 0x20, 0x00,
0x03, 0x69, 0x0a, 0xca, 0xba, 0x12, 0x08, 0x41,
0x6e, 0xb9, 0x86, 0xe4, 0x50, 0xf0, 0x84, 0xe2,
0xb3, 0xb3, 0xc8, 0xb5, 0xb2, 0x2d, 0x18, 0x70,
0x0a, 0xd7, 0x92, 0x90, 0x9e, 0x1e, 0x0c, 0x1f,
0x08, 0xe8, 0x06, 0xfd, 0x85, 0x2f, 0xaa, 0x5d,
0xcf, 0xf9, 0xe3, 0x55, 0xb9, 0xfe, 0xa6, 0x7f,
0x44, 0x3b, 0x4a, 0x4f, 0xc9, 0x2f, 0xd2, 0xd3,
0x8e, 0xdc, 0xae, 0xba, 0x4f, 0x02, 0xb4, 0x76,
0xba, 0x64, 0x2d, 0x07, 0x9e, 0x08, 0xec, 0xbd,
0x52, 0x29, 0x07, 0xbb, 0x9f, 0xb5, 0x58, 0x6f,
0x07, 0x55, 0xb0, 0x34, 0x74, 0x9f, 0x05, 0xb2,
0xdf, 0xa9, 0xc6, 0x2a, 0xa3, 0x5d, 0xff, 0x10,
0x40, 0xb3, 0xb7, 0xb4, 0x63, 0x6e, 0xf4, 0x3e,
0xee, 0xf6, 0x49, 0x52, 0xe3, 0x11, 0xb3, 0xf1,
0xfb, 0x60, 0x48, 0xa1, 0xa4, 0x19, 0x7a, 0x2e,
0x90, 0x28, 0x90, 0x8d, 0x5e, 0x8c, 0x8c, 0xc4,
0xf2, 0x4a, 0xf6, 0xb2, 0x19, 0x83, 0xea, 0xed,
0x6d, 0xba, 0xfe, 0xd8, 0xb6, 0xa3, 0x5a, 0xb4,
0x48, 0xfa, 0xbe, 0x5c, 0x69, 0xac, 0x3c, 0x8f,
0x63, 0xaf, 0xa4, 0x42, 0x25, 0x50, 0xab, 0x65,
0x80, 0x65, 0xb9, 0xfb, 0xc7, 0xf2, 0x2d, 0x5c,
0xe3, 0x4c, 0xa4, 0xa6, 0x8e, 0x07, 0x9c, 0xeb,
0x41, 0x93, 0x65, 0x44, 0x4a, 0x86, 0xc1, 0xf6,
0x2c, 0x97, 0xfd, 0xf4, 0x6c, 0xdc, 0xe1, 0xe0,
0x28, 0xd9, 0x89, 0x7b, 0x09, 0xe2, 0xa0, 0x38,
0x74, 0x4a, 0xa6, 0x5e, 0xd2, 0xe2, 0x4d, 0xf3,
0xf4, 0xc6, 0xbc, 0xa2, 0x51, 0x58, 0xe8, 0xae,
0x91, 0x58, 0xb3, 0x31, 0x6c, 0x33, 0xda, 0x88,
};
/****************************************************************************
* Given a number, figure out the nearest power-of-two (16,32,64,128,etc.)
* that can hold that number. We do this so that we can convert multiplies
* into shifts.
****************************************************************************/
static uint64_t
next_power_of_two(uint64_t num)
{
uint64_t power_of_two = 1;
num++;
while ((uint64_t)(1ULL << power_of_two) < num)
power_of_two++;
return (1ULL << power_of_two);
}
static uint64_t
bit_count(uint64_t num)
{
uint64_t bits = 0;
while ((num >> bits) > 1)
bits++;
return bits;
}
/***************************************************************************
***************************************************************************/
void
blackrock2_init(struct BlackRock *br, uint64_t range, uint64_t seed, unsigned rounds)
{
uint64_t a;
uint64_t b;
a = next_power_of_two(
(uint64_t)sqrt(range * 1.0)
);
b = next_power_of_two(range/a);
//printf("a=%llu b=%llu seed = 0x%llu\n", a, b, seed);
br->range = range;
br->a = a;
br->a_bits = bit_count(br->a);
br->a_mask = br->a - 1ULL;
br->b = b;
br->b_bits = bit_count(br->b);
br->b_mask = br->b - 1ULL;
//printf("a: 0x%llx / %llu\n", br->a_mask, br->a_bits);
//printf("b: 0x%llx / %llu\n", br->b_mask, br->b_bits);
br->rounds = rounds;
br->seed = seed;
br->range = range;
}
/***************************************************************************
* The inner round/mixer function. In DES, it's a series of S-box lookups,
* which
***************************************************************************/
static inline uint64_t
ROUND(uint64_t r, uint64_t R, uint64_t seed)
{
#define GETBYTE(R,n) ((uint64_t)(((((R)>>(n*8ULL)))&0xFFULL)))
#if 0
uint64_t r0, r1, r2, r3;
#endif
uint64_t T, Y;
T = R ^ ((seed>>r) | (seed<<(64-r)));
if (r & 1) {
Y = SB8[ (T ) & 0x3F ] ^ \
SB6[ (T >> 8) & 0x3F ] ^ \
SB4[ (T >> 16) & 0x3F ] ^ \
SB2[ (T >> 24) & 0x3F ]; \
} else {
Y = SB7[ (T ) & 0x3F ] ^ \
SB5[ (T >> 8) & 0x3F ] ^ \
SB3[ (T >> 16) & 0x3F ] ^ \
SB1[ (T >> 24) & 0x3F ];
}
return Y;
#if 0
r0 = sbox2[GETBYTE(R,0)]<< 6 | sbox2[GETBYTE(R,1)]<< 0;
r1 = sbox2[GETBYTE(R,2)]<< 6 | sbox2[GETBYTE(R,5)]<< 0;
r2 = sbox2[GETBYTE(R,4)]<< 6 | sbox2[GETBYTE(R,5)]<< 0;
r3 = sbox2[GETBYTE(R,6)]<< 6 | sbox2[GETBYTE(R,7)]<< 0;
R = r0 ^ (r1<<12) * (r2 << 24) ^ (r3 << 36) * r;
return R;
/*return((uint64_t)sbox2[GETBYTE(R,7ULL)]<< 0ULL)
| ((uint64_t)sbox2[GETBYTE(R,6ULL)]<< 8ULL)
| ((uint64_t)sbox2[GETBYTE(R,5ULL)]<<16ULL)
| ((uint64_t)sbox2[GETBYTE(R,4ULL)]<<24ULL)
| ((uint64_t)sbox2[GETBYTE(R,3ULL)]<<32ULL)
| ((uint64_t)sbox2[GETBYTE(R,2ULL)]<<40ULL)
| ((uint64_t)sbox2[GETBYTE(R,1ULL)]<<48ULL)
| ((uint64_t)sbox2[GETBYTE(R,0ULL)]<<56ULL)
;*/
return R;
#endif
}
/***************************************************************************
***************************************************************************/
static inline uint64_t
ENCRYPT(unsigned r, uint64_t a_bits, uint64_t a_mask, uint64_t b_bits, uint64_t b_mask, uint64_t m, uint64_t seed)
{
uint64_t L, R;
unsigned j = 1;
uint64_t tmp;
UNUSEDPARM(b_bits);
L = m & a_mask;
R = m >> a_bits;
for (j=1; j<=r; j++) {
tmp = (L + ROUND(j, R, seed)) & a_mask;
L = R;
R = tmp;
j++;
tmp = (L + ROUND(j, R, seed)) & b_mask;
L = R;
R = tmp;
}
if ((j-1) & 1) {
return (L << (a_bits)) + R;
} else {
return (R << (a_bits)) + L;
}
}
static inline uint64_t
DECRYPT(unsigned r, uint64_t a, uint64_t b, uint64_t m, uint64_t seed)
{
uint64_t L, R;
unsigned j;
uint64_t tmp;
if (r & 1) {
R = m % a;
L = m / a;
} else {
L = m % a;
R = m / a;
}
for (j=r; j>=1; j--) {
if (j & 1) {
tmp = ROUND(j, L, seed);
if (tmp > R) {
tmp = (tmp - R);
tmp = a - (tmp%a);
if (tmp == a)
tmp = 0;
} else {
tmp = (R - tmp);
tmp %= a;
}
} else {
tmp = ROUND(j, L, seed);
if (tmp > R) {
tmp = (tmp - R);
tmp = b - (tmp%b);
if (tmp == b)
tmp = 0;
} else {
tmp = (R - tmp);
tmp %= b;
}
}
R = L;
L = tmp;
}
return a * R + L;
}
/***************************************************************************
***************************************************************************/
uint64_t
blackrock2_shuffle(const struct BlackRock *br, uint64_t m)
{
uint64_t c;
c = ENCRYPT(br->rounds, br->a_bits, br->a_mask, br->b_bits, br->b_mask, m, br->seed);
while (c >= br->range)
c = ENCRYPT(br->rounds, br->a_bits, br->a_mask, br->b_bits, br->b_mask, c, br->seed);
return c;
}
/***************************************************************************
***************************************************************************/
uint64_t
blackrock2_unshuffle(const struct BlackRock *br, uint64_t m)
{
uint64_t c;
c = DECRYPT(br->rounds, br->a, br->b, m, br->seed);
while (c >= br->range)
c = DECRYPT(br->rounds, br->a, br->b, c, br->seed);
return c;
}
/***************************************************************************
* This function called only during selftest/regression-test.
***************************************************************************/
static unsigned
verify(struct BlackRock *br, uint64_t max)
{
unsigned char *list;
uint64_t i;
unsigned is_success = 1;
uint64_t range = br->range;
/* Allocate a list of 1-byte counters */
list = CALLOC(1, (size_t)((range<max)?range:max));
/* For all numbers in the range, verify increment the counter for
* the output. */
for (i=0; i<range; i++) {
uint64_t x = blackrock2_shuffle(br, i);
if (x < max)
list[x]++;
}
/* Now check the output to make sure that every counter is set exactly
* to the value of '1'. */
for (i=0; i<max && i<range; i++) {
if (list[i] != 1)
is_success = 0;
}
free(list);
return is_success;
}
/***************************************************************************
* Benchmarks the crypto function.
***************************************************************************/
void
blackrock2_benchmark(unsigned rounds)
{
struct BlackRock br;
uint64_t range = 0x010356789123ULL;
uint64_t i;
uint64_t result = 0;
uint64_t start, stop;
static const uint64_t ITERATIONS = 5000000ULL;
printf("-- blackrock-2 -- \n");
printf("rounds = %u\n", rounds);
blackrock2_init(&br, range, 1, rounds);
/*printf("range = 0x%10" PRIx64 "\n", range);
printf("rangex= 0x%10" PRIx64 "\n", br.a*br.b);
printf(" a = 0x%10" PRIx64 "\n", br.a);
printf(" b = 0x%10" PRIx64 "\n", br.b);*/
/*
* Time the algorithm
*/
start = pixie_nanotime();
for (i=0; i<ITERATIONS; i++) {
result += blackrock2_shuffle(&br, i);
}
stop = pixie_nanotime();
/*
* Print the results
*/
if (result) {
double elapsed = ((double)(stop - start))/(1000000000.0);
double rate = ITERATIONS/elapsed;
rate /= 1000000.0;
printf("iterations/second = %5.3f-million\n", rate);
}
printf("\n");
}
/***************************************************************************
***************************************************************************/
int
blackrock2_selftest(void)
{
uint64_t i;
int is_success = 0;
uint64_t range;
/* @marshray
* Basic test of decryption. I take the index, encrypt it, then decrypt it,
* which means I should get the original index back again. Only, it's not
* working. The decryption fails. The reason it's failing is obvious -- I'm
* just not seeing it though. The error is probably in the 'unfe()'
* function above.
*/
{
struct BlackRock br;
uint64_t result, result2;
blackrock2_init(&br, 1000, 0, 6);
for (i=0; i<10; i++) {
result = blackrock2_shuffle(&br, i);
result2 = blackrock2_unshuffle(&br, result);
if (i != result2)
return 1; /*fail*/
}
}
range = 3015 * 3;
for (i=0; i<5; i++) {
struct BlackRock br;
range += 11 + i;
range *= 1 + i;
blackrock2_init(&br, range, time(0), 6);
is_success = verify(&br, range);
if (!is_success) {
fprintf(stderr, "BLACKROCK: randomization failed\n");
return 1; /*fail*/
}
}
return 0; /*success*/
}

390
src/crypto-lcg.c Normal file
View File

@@ -0,0 +1,390 @@
/*
This is a "linear-congruent-generator", a type of random number
generator.
*/
#include "crypto-lcg.h"
#include "crypto-primegen.h" /* DJB's prime factoring code */
#include "util-safefunc.h"
#include "util-malloc.h"
#include <math.h> /* for 'sqrt()', may need -lm for gcc */
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
/**
* A 64 bit number can't have more than 16 prime factors. The first factors
* are:
* 2*3*5*7*11*13*17*19*23*29*31*37*41*43*47*53 = 0xC443F2F861D29C3A
* 0123456789abcdef
* We zero termiante this list, so we are going to reserve 20 slots.
*/
typedef uint64_t PRIMEFACTORS[20];
/****************************************************************************
* Break down the number into prime factors using DJB's sieve code, which
* is about 5 to 10 times faster than the Sieve of Eratosthenes.
*
* @param number
* The integer that we are factoring. It can be any value up to 64 bits
* in size.
* @param factors
* The list of all the prime factors, zero terminated.
* @param non_factors
* A list of smallest numbers that aren't prime factors. We return
* this because we are going to use prime non-factors for finding
* interesting numbers.
****************************************************************************/
static unsigned
sieve_prime_factors(uint64_t number, PRIMEFACTORS factors,
PRIMEFACTORS non_factors, double *elapsed)
{
primegen pg;
clock_t start;
clock_t stop;
uint64_t prime;
uint64_t max;
unsigned factor_count = 0;
unsigned non_factor_count = 0;
/*
* We only need to sieve up to the square-root of the target number. Only
* one prime factor can be bigger than the square root, so once we find
* all the other primes, the square root is the only one left.
* Note: you have to link to the 'm' math library for some gcc platforms.
*/
max = (uint64_t)sqrt(number + 1.0);
/*
* Init the DJB primegen library.
*/
primegen_init(&pg);
/*
* Enumerate all the primes starting with 2
*/
start = clock();
for (;;) {
/* Sieve the next prime */
prime = primegen_next(&pg);
/* If we've reached the square root, then that's as far as we need
* to go */
if (prime > max)
break;
/* If this prime is not a factor (evenly divisible with no remainder)
* then loop back and get the next prime */
if ((number % prime) != 0) {
if (non_factor_count < 12)
non_factors[non_factor_count++] = prime;
continue;
}
/* Else we've found a prime factor, so add this to the list of primes */
factors[factor_count++] = prime;
/* At the end, we may have one prime factor left that's bigger than the
* sqrt. Therefore, as we go along, divide the original number
* (possibly several times) by the prime factor so that this large
* remaining factor will be the only one left */
while ((number % prime) == 0)
number /= prime;
/* exit early if we've found all prime factors. comment out this
* code if you want to benchmark it */
if (number == 1 && non_factor_count > 10)
break;
}
/*
* See if there is one last prime that's bigger than the square root.
* Note: This is the only number that can be larger than 32-bits in the
* way this code is written.
*/
if (number != 1)
factors[factor_count++] = number;
/*
* Zero terminate the results.
*/
factors[factor_count] = 0;
non_factors[non_factor_count] = 0;
/*
* Since prime factorization takes a long time, especially on slow
* CPUs, we benchmark it to keep track of performance.
*/
stop = clock();
if (elapsed)
*elapsed = ((double)stop - (double)start)/(double)CLOCKS_PER_SEC;
/* should always be at least 1, because if the number itself is prime,
* then that's it's only prime factor */
return factor_count;
}
/****************************************************************************
* Do a pseudo-random 1-to-1 translation of a number within a range to
* another number in that range.
*
* The constants 'a' and 'c' must be chosen to match the LCG algorithm
* to fit 'm' (range).
*
* This the same as the function 'rand()', except all the constants and
* seeds are specified as parameters.
*
* @param index
* The index within the range that we are randomizing.
* @param a
* The 'multiplier' of the LCG algorithm.
* @param c
* The 'increment' of the LCG algorithm.
* @param range
* The 'modulus' of the LCG algorithm.
****************************************************************************/
uint64_t
lcg_rand(uint64_t index, uint64_t a, uint64_t c, uint64_t range)
{
return (index * a + c) % range;
}
/****************************************************************************
* Verify the LCG algorithm. You shouldn't do this for large ranges,
* because we'll run out of memory. Therefore, this algorithm allocates
* a buffer only up to a smaller range. We still have to traverse the
* entire range of numbers, but we only need store values for a smaller
* range. If 10% of the range checks out, then there's a good chance
* it applies to the other 90% as well.
*
* This works by counting the results of rand(), which should be produced
* exactly once.
****************************************************************************/
static unsigned
lcg_verify(uint64_t a, uint64_t c, uint64_t range, uint64_t max)
{
unsigned char *list;
uint64_t i;
unsigned is_success = 1;
/* Allocate a list of 1-byte counters */
list = CALLOC(1, (size_t)((range<max)?range:max));
/* For all numbers in the range, verify increment the counter for the
* the output. */
for (i=0; i<range; i++) {
uint64_t x = lcg_rand(i, a, c, range);
if (x < max)
list[x]++;
}
/* Now check the output to make sure that every counter is set exactly
* to the value of '1'. */
for (i=0; i<max && i<range; i++) {
if (list[i] != 1)
is_success = 0;
}
free(list);
return is_success;
}
/****************************************************************************
* Count the number of digits in a number so that we can pretty-print a
* bunch of numbers in nice columns.
****************************************************************************/
static unsigned
count_digits(uint64_t num)
{
unsigned result = 0;
while (num) {
result++;
num /= 10;
}
return result;
}
/****************************************************************************
* Tell whether the number has any prime factors in common with the list
* of factors. In other words, if it's not coprime with the other number.
* @param c
* The number we want to see has common factors with the other number.
* @param factors
* The factors from the other number
* @return
* !is_coprime(c, factors)
****************************************************************************/
static uint64_t
has_factors_in_common(uint64_t c, PRIMEFACTORS factors)
{
unsigned i;
for (i=0; factors[i]; i++) {
if ((c % factors[i]) == 0)
return factors[i]; /* found a common factor */
}
return 0; /* no factors in common */
}
/****************************************************************************
* Given a range, calculate some possible constants for the LCG algorithm
* for randomizing the order of the array.
* @parm m
* The range for which we'll be finding random numbers. If we are
* looking for random numbers between [0..100), this number will
* be 100.
* @parm a
* The LCG 'a' constant that will be the result of this function.
* @param c
* The LCG 'c' constant that will be the result of this function. This
* should be set to 0 on the input to this function, or a suggested
* value.
****************************************************************************/
void
lcg_calculate_constants(uint64_t m, uint64_t *out_a, uint64_t *inout_c, int is_debug)
{
uint64_t a;
uint64_t c = *inout_c;
double elapsed = 0.0; /* Benchmark of 'sieve' algorithm */
PRIMEFACTORS factors; /* List of prime factors of 'm' */
PRIMEFACTORS non_factors;
unsigned i;
/*
* Find all the prime factors of the number. This step can take several
* seconds for 48 bit numbers, which is why we benchmark how long it
* takes.
*/
sieve_prime_factors(m, factors, non_factors, &elapsed);
/*
* Calculate the 'a-1' constant. It must share all the prime factors
* with the range, and if the range is a multiple of 4, must also
* be a multiple of 4
*/
if (factors[0] == m) {
/* this number has no prime factors, so we can choose anything.
* Therefore, we are going to pick something at random */
unsigned j;
a = 1;
for (j=0; non_factors[j] && j < 5; j++)
a *= non_factors[j];
} else {
//unsigned j;
a = 1;
for (i=0; factors[i]; i++)
a = a * factors[i];
if ((m % 4) == 0)
a *= 2;
/*for (j=0; j<0 && non_factors[j]; j++)
a *= non_factors[j];*/
}
a += 1;
/*
* Calculate the 'c' constant. It must have no prime factors in
* common with the range.
*/
if (c == 0)
c = 2531011 ; /* something random */
while (has_factors_in_common(c, factors))
c++;
if (is_debug) {
/*
* print the results
*/
//printf("sizeof(int) = %" PRIu64 "-bits\n", (uint64_t)(sizeof(size_t)*8));
printf("elapsed = %5.3f-seconds\n", elapsed);
printf("factors = ");
for (i=0; factors[i]; i++)
printf("%" PRIu64 " ", factors[i]);
printf("%s\n", factors[0]?"":"(none)");
printf("m = %-24" PRIu64 " (0x%" PRIx64 ")\n", m, m);
printf("a = %-24" PRIu64 " (0x%" PRIx64 ")\n", a, a);
printf("c = %-24" PRIu64 " (0x%" PRIx64 ")\n", c, c);
printf("c%%m = %-24" PRIu64 " (0x%" PRIx64 ")\n", c%m, c%m);
printf("a%%m = %-24" PRIu64 " (0x%" PRIx64 ")\n", a%m, a%m);
if (m < 1000000000) {
if (lcg_verify(a, c+1, m, 280))
printf("verify = success\n");
else
printf("verify = failure\n");
} else {
printf("verify = too big to check\n");
}
/*
* Print some first numbers. We use these to visually inspect whether
* the results are random or not.
*/
{
unsigned count = 0;
uint64_t x = 0;
unsigned digits = count_digits(m);
for (i=0; i<100 && i < m; i++) {
x = lcg_rand(x, a, c, m);
count += printf("%*" PRIu64 " ", digits, x);
if (count >= 70) {
count = 0;
printf("\n");
}
}
printf("\n");
}
}
*out_a = a;
*inout_c = c;
}
/***************************************************************************
***************************************************************************/
int
lcg_selftest(void)
{
unsigned i;
int is_success = 0;
uint64_t m, a, c;
m = 3015 * 3;
for (i=0; i<5; i++) {
a = 0;
c = 0;
m += 10 + i;
lcg_calculate_constants(m, &a, &c, 0);
is_success = lcg_verify(a, c, m, m);
if (!is_success) {
fprintf(stderr, "LCG: randomization failed\n");
return 1; /*fail*/
}
}
return 0; /*success*/
}

20
src/crypto-lcg.h Normal file
View File

@@ -0,0 +1,20 @@
#ifndef RAND_LCG_H
#define RAND_LCG_H
#include <stdint.h>
void
lcg_calculate_constants(uint64_t m, uint64_t *out_a, uint64_t *inout_c, int is_debug);
uint64_t
lcg_rand(uint64_t index, uint64_t a, uint64_t c, uint64_t range);
/**
* Performs a regression test on this module.
* @return
* 0 on success, or a positive integer on failure
*/
int
lcg_selftest(void);
#endif

716
src/crypto-primegen.c Normal file
View File

@@ -0,0 +1,716 @@
/*
This is DJB's code for calculating primes, with a few modifications,
such as making it work with Microsoft's compiler on Windows, and
getting rid of warnings.
*/
#include "crypto-primegen.h"
/*
B is 32 times X.
Total memory use for one generator is 2B bytes = 64X bytes.
Covers primes in an interval of length 1920X.
Working set size for one generator is B bits = 4X bytes.
Speedup by a factor of 2 or 3 for L1 cache instead of L2 cache.
Slowdown by a factor of roughly n for primes past (nB)^2.
Possible choices of X:
2002 to fit inside an 8K L1 cache (e.g., Pentium).
4004 to fit inside a 16K L1 cache (e.g., Pentium II).
64064 to fit inside a 256K L2 cache.
There are various word-size limits on X; 1000000 should still be okay.
*/
#define B32 PRIMEGEN_WORDS
#define B (PRIMEGEN_WORDS * 32)
#ifdef _MSC_VER
#pragma warning(disable:4244)
#endif
static const uint32_t two[32] = {
0x00000001 , 0x00000002 , 0x00000004 , 0x00000008
, 0x00000010 , 0x00000020 , 0x00000040 , 0x00000080
, 0x00000100 , 0x00000200 , 0x00000400 , 0x00000800
, 0x00001000 , 0x00002000 , 0x00004000 , 0x00008000
, 0x00010000 , 0x00020000 , 0x00040000 , 0x00080000
, 0x00100000 , 0x00200000 , 0x00400000 , 0x00800000
, 0x01000000 , 0x02000000 , 0x04000000 , 0x08000000
, 0x10000000 , 0x20000000 , 0x40000000 , 0x80000000
} ;
static void clear(register uint32_t (*buf)[B32])
{
register int i;
register int j;
for (j = 0;j < 16;++j) {
for (i = 0;i < B32;++i)
(*buf)[i] = (uint32_t)~0;
++buf;
}
}
static void doit4(register uint32_t *a,register long x,register long y,int64_t start)
{
long i0;
long y0;
register long i;
register uint32_t data;
register uint32_t pos;
register uint32_t bits;
x += x; x += 15;
y += 15;
start += 1000000000;
while (start < 0) { start += x; x += 30; }
start -= 1000000000;
i = start;
while (i < B) { i += x; x += 30; }
for (;;) {
x -= 30;
if (x <= 15) return;
i -= x;
while (i < 0) { i += y; y += 30; }
i0 = i; y0 = y;
while (i < B) {
pos = (uint32_t)i; data = (uint32_t)i;
pos >>= 5; data &= 31;
i += y; y += 30;
bits = a[pos]; data = two[data];
bits ^= data;
a[pos] = bits;
}
i = i0; y = y0;
}
}
static void doit6(register uint32_t *a,register long x,register long y,int64_t start)
{
long i0;
long y0;
register long i;
register uint32_t data;
register uint32_t pos;
register uint32_t bits;
x += 5;
y += 15;
start += 1000000000;
while (start < 0) { start += x; x += 10; }
start -= 1000000000;
i = start;
while (i < B) { i += x; x += 10; }
for (;;) {
x -= 10;
if (x <= 5) return;
i -= x;
while (i < 0) { i += y; y += 30; }
i0 = i; y0 = y;
while (i < B) {
pos = (uint32_t)i; data = (uint32_t)i;
pos >>= 5; data &= 31;
i += y; y += 30;
bits = a[pos]; data = two[data];
bits ^= data;
a[pos] = bits;
}
i = i0; y = y0;
}
}
static void doit12(register uint32_t *a,register long x,register long y,int64_t start)
{
register long i;
register uint32_t data;
register uint32_t bits;
x += 5;
start += 1000000000;
while (start < 0) { start += x; x += 10; }
start -= 1000000000;
i = start;
while (i < 0) { i += x; x += 10; }
y += 15;
x += 10;
for (;;) {
long i0;
long y0;
while (i >= B) {
if (x <= y) return;
i -= y;
y += 30;
}
i0 = i;
y0 = y;
while ((i >= 0) && (y < x)) {
register uint32_t pos;
pos = (uint32_t)i; data = (uint32_t)i;
pos >>= 5; data &= 31;
i -= y;
y += 30;
bits = a[pos]; data = two[data];
bits ^= data;
a[pos] = bits;
}
i = i0;
y = y0;
i += x - 10;
x += 10;
}
}
static const int deltainverse[60] = {
-1,B32 * 0,-1,-1,-1,-1,-1,B32 * 1,-1,-1,-1,B32 * 2,-1,B32 * 3,-1
,-1,-1,B32 * 4,-1,B32 * 5,-1,-1,-1,B32 * 6,-1,-1,-1,-1,-1,B32 * 7
,-1,B32 * 8,-1,-1,-1,-1,-1,B32 * 9,-1,-1,-1,B32 * 10,-1,B32 * 11,-1
,-1,-1,B32 * 12,-1,B32 * 13,-1,-1,-1,B32 * 14,-1,-1,-1,-1,-1,B32 * 15
} ;
static void squarefree1big(uint32_t (*buf)[B32],uint64_t base,uint32_t q,uint64_t qq)
{
uint64_t i;
uint32_t pos;
int n;
uint64_t bound = base + 60 * B;
while (qq < bound) {
if (bound < 2000000000)
i = qq - (((uint32_t) base) % ((uint32_t) qq));
else
i = qq - (base % qq);
if (!(i & 1)) i += qq;
if (i < B * 60) {
pos = (uint32_t)i;
n = deltainverse[pos % 60];
if (n >= 0) {
pos /= 60;
(*buf)[n + (pos >> 5)] |= two[pos & 31];
}
}
qq += q; q += 1800;
}
}
static void squarefree1(register uint32_t (*buf)[B32],uint64_t L,uint32_t q)
{
uint32_t qq;
register uint32_t qqhigh;
uint32_t i;
register uint32_t ilow;
register uint32_t ihigh;
register int n;
uint64_t base;
base = 60 * L;
qq = q * q;
q = 60 * q + 900;
while (qq < B * 60) {
if (base < 2000000000)
i = qq - (((uint32_t) base) % qq);
else
i = qq - (base % qq);
if (!(i & 1)) i += qq;
if (i < B * 60) {
qqhigh = qq / 60;
ilow = i % 60; ihigh = i / 60;
qqhigh += qqhigh;
while (ihigh < B) {
n = deltainverse[ilow];
if (n >= 0)
(*buf)[n + (ihigh >> 5)] |= two[ihigh & 31];
ilow += 2; ihigh += qqhigh;
if (ilow >= 60) { ilow -= 60; ihigh += 1; }
}
}
qq += q; q += 1800;
}
squarefree1big(buf,base,q,qq);
}
static void squarefree49big(uint32_t (*buf)[B32],uint64_t base,uint32_t q,uint64_t qq)
{
uint64_t i;
uint32_t pos;
int n;
uint64_t bound = base + 60 * B;
while (qq < bound) {
if (bound < 2000000000)
i = qq - (((uint32_t) base) % ((uint32_t) qq));
else
i = qq - (base % qq);
if (!(i & 1)) i += qq;
if (i < B * 60) {
pos = (uint32_t)i;
n = deltainverse[pos % 60];
if (n >= 0) {
pos /= 60;
(*buf)[n + (pos >> 5)] |= two[pos & 31];
}
}
qq += q; q += 1800;
}
}
static void squarefree49(register uint32_t (*buf)[B32],uint64_t L,uint32_t q)
{
uint32_t qq;
register uint32_t qqhigh;
uint32_t i;
register uint32_t ilow;
register uint32_t ihigh;
register int n;
uint64_t base = 60 * L;
qq = q * q;
q = 60 * q + 900;
while (qq < B * 60) {
if (base < 2000000000)
i = qq - (((uint32_t) base) % qq);
else
i = qq - (base % qq);
if (!(i & 1)) i += qq;
if (i < B * 60) {
qqhigh = qq / 60;
ilow = i % 60; ihigh = i / 60;
qqhigh += qqhigh;
qqhigh += 1;
while (ihigh < B) {
n = deltainverse[ilow];
if (n >= 0)
(*buf)[n + (ihigh >> 5)] |= two[ihigh & 31];
ilow += 38; ihigh += qqhigh;
if (ilow >= 60) { ilow -= 60; ihigh += 1; }
}
}
qq += q; q += 1800;
}
squarefree49big(buf,base,q,qq);
}
/* squares of primes >= 7, < 240 */
uint32_t qqtab[49] = {
49,121,169,289,361,529,841,961,1369,1681
,1849,2209,2809,3481,3721,4489,5041,5329,6241,6889
,7921,9409,10201,10609,11449,11881,12769,16129,17161,18769
,19321,22201,22801,24649,26569,27889,29929,32041,32761,36481
,37249,38809,39601,44521,49729,51529,52441,54289,57121
} ;
/* (qq * 11 + 1) / 60 or (qq * 59 + 1) / 60 */
uint32_t qq60tab[49] = {
9,119,31,53,355,97,827,945,251,1653
,339,405,515,3423,3659,823,4957,977,6137
,1263,7789,1725,10031,1945,2099,11683,2341,2957
,16875,3441,18999,21831,22421,4519,4871,5113,5487
,31507,32215,35873,6829,7115,38941,43779,9117,9447,51567,9953,56169
} ;
static void squarefreetiny(register uint32_t *a,uint32_t *Lmodqq,int d)
{
int j;
for (j = 0;j < 49;++j) {
register uint32_t k;
register uint32_t qq;
qq = qqtab[j];
k = qq - 1 - ((Lmodqq[j] + qq60tab[j] * d - 1) % qq);
while (k < B) {
register uint32_t pos;
register uint32_t data;
register uint32_t bits;
pos = k;
data = k;
pos >>= 5;
data &= 31;
k += qq;
bits = a[pos];
data = two[data];
bits |= data;
a[pos] = bits;
}
}
}
typedef struct { char index; char f; char g; char k; } todo;
static const todo for4[] = {
{0,2,15,4} , {0,3,5,1} , {0,3,25,11} , {0,5,9,3}
, {0,5,21,9} , {0,7,15,7} , {0,8,15,8} , {0,10,9,8}
, {0,10,21,14} , {0,12,5,10} , {0,12,25,20} , {0,13,15,15}
, {0,15,1,15} , {0,15,11,17} , {0,15,19,21} , {0,15,29,29}
, {3,1,3,0} , {3,1,27,12} , {3,4,3,1} , {3,4,27,13}
, {3,6,7,3} , {3,6,13,5} , {3,6,17,7} , {3,6,23,11}
, {3,9,7,6} , {3,9,13,8} , {3,9,17,10} , {3,9,23,14}
, {3,11,3,8} , {3,11,27,20} , {3,14,3,13} , {3,14,27,25}
, {4,2,1,0} , {4,2,11,2} , {4,2,19,6} , {4,2,29,14}
, {4,7,1,3} , {4,7,11,5} , {4,7,19,9} , {4,7,29,17}
, {4,8,1,4} , {4,8,11,6} , {4,8,19,10} , {4,8,29,18}
, {4,13,1,11} , {4,13,11,13} , {4,13,19,17} , {4,13,29,25}
, {7,1,5,0} , {7,1,25,10} , {7,4,5,1} , {7,4,25,11}
, {7,5,7,2} , {7,5,13,4} , {7,5,17,6} , {7,5,23,10}
, {7,10,7,7} , {7,10,13,9} , {7,10,17,11} , {7,10,23,15}
, {7,11,5,8} , {7,11,25,18} , {7,14,5,13} , {7,14,25,23}
, {9,2,9,1} , {9,2,21,7} , {9,3,1,0} , {9,3,11,2}
, {9,3,19,6} , {9,3,29,14} , {9,7,9,4} , {9,7,21,10}
, {9,8,9,5} , {9,8,21,11} , {9,12,1,9} , {9,12,11,11}
, {9,12,19,15} , {9,12,29,23} , {9,13,9,12} , {9,13,21,18}
, {10,2,5,0} , {10,2,25,10} , {10,5,1,1} , {10,5,11,3}
, {10,5,19,7} , {10,5,29,15} , {10,7,5,3} , {10,7,25,13}
, {10,8,5,4} , {10,8,25,14} , {10,10,1,6} , {10,10,11,8}
, {10,10,19,12} , {10,10,29,20} , {10,13,5,11} , {10,13,25,21}
, {13,1,15,3} , {13,4,15,4} , {13,5,3,1} , {13,5,27,13}
, {13,6,5,2} , {13,6,25,12} , {13,9,5,5} , {13,9,25,15}
, {13,10,3,6} , {13,10,27,18} , {13,11,15,11} , {13,14,15,16}
, {13,15,7,15} , {13,15,13,17} , {13,15,17,19} , {13,15,23,23}
, {14,1,7,0} , {14,1,13,2} , {14,1,17,4} , {14,1,23,8}
, {14,4,7,1} , {14,4,13,3} , {14,4,17,5} , {14,4,23,9}
, {14,11,7,8} , {14,11,13,10} , {14,11,17,12} , {14,11,23,16}
, {14,14,7,13} , {14,14,13,15} , {14,14,17,17} , {14,14,23,21}
} ;
static const todo for6[] = {
{1,1,2,0} , {1,1,8,1} , {1,1,22,8} , {1,1,28,13}
, {1,3,10,2} , {1,3,20,7} , {1,7,10,4} , {1,7,20,9}
, {1,9,2,4} , {1,9,8,5} , {1,9,22,12} , {1,9,28,17}
, {5,1,4,0} , {5,1,14,3} , {5,1,16,4} , {5,1,26,11}
, {5,5,2,1} , {5,5,8,2} , {5,5,22,9} , {5,5,28,14}
, {5,9,4,4} , {5,9,14,7} , {5,9,16,8} , {5,9,26,15}
, {8,3,2,0} , {8,3,8,1} , {8,3,22,8} , {8,3,28,13}
, {8,5,4,1} , {8,5,14,4} , {8,5,16,5} , {8,5,26,12}
, {8,7,2,2} , {8,7,8,3} , {8,7,22,10} , {8,7,28,15}
, {11,1,10,1} , {11,1,20,6} , {11,3,4,0} , {11,3,14,3}
, {11,3,16,4} , {11,3,26,11} , {11,7,4,2} , {11,7,14,5}
, {11,7,16,6} , {11,7,26,13} , {11,9,10,5} , {11,9,20,10}
} ;
static const todo for12[] = {
{2,2,1,0} , {2,2,11,-2} , {2,2,19,-6} , {2,2,29,-14}
, {2,3,4,0} , {2,3,14,-3} , {2,3,16,-4} , {2,3,26,-11}
, {2,5,2,1} , {2,5,8,0} , {2,5,22,-7} , {2,5,28,-12}
, {2,7,4,2} , {2,7,14,-1} , {2,7,16,-2} , {2,7,26,-9}
, {2,8,1,3} , {2,8,11,1} , {2,8,19,-3} , {2,8,29,-11}
, {2,10,7,4} , {2,10,13,2} , {2,10,17,0} , {2,10,23,-4}
, {6,1,10,-2} , {6,1,20,-7} , {6,2,7,-1} , {6,2,13,-3}
, {6,2,17,-5} , {6,2,23,-9} , {6,3,2,0} , {6,3,8,-1}
, {6,3,22,-8} , {6,3,28,-13} , {6,4,5,0} , {6,4,25,-10}
, {6,6,5,1} , {6,6,25,-9} , {6,7,2,2} , {6,7,8,1}
, {6,7,22,-6} , {6,7,28,-11} , {6,8,7,2} , {6,8,13,0}
, {6,8,17,-2} , {6,8,23,-6} , {6,9,10,2} , {6,9,20,-3}
, {12,1,4,-1} , {12,1,14,-4} , {12,1,16,-5} , {12,1,26,-12}
, {12,2,5,-1} , {12,2,25,-11} , {12,3,10,-2} , {12,3,20,-7}
, {12,4,1,0} , {12,4,11,-2} , {12,4,19,-6} , {12,4,29,-14}
, {12,6,1,1} , {12,6,11,-1} , {12,6,19,-5} , {12,6,29,-13}
, {12,7,10,0} , {12,7,20,-5} , {12,8,5,2} , {12,8,25,-8}
, {12,9,4,3} , {12,9,14,0} , {12,9,16,-1} , {12,9,26,-8}
, {15,1,2,-1} , {15,1,8,-2} , {15,1,22,-9} , {15,1,28,-14}
, {15,4,7,-1} , {15,4,13,-3} , {15,4,17,-5} , {15,4,23,-9}
, {15,5,4,0} , {15,5,14,-3} , {15,5,16,-4} , {15,5,26,-11}
, {15,6,7,0} , {15,6,13,-2} , {15,6,17,-4} , {15,6,23,-8}
, {15,9,2,3} , {15,9,8,2} , {15,9,22,-5} , {15,9,28,-10}
, {15,10,1,4} , {15,10,11,2} , {15,10,19,-2} , {15,10,29,-10}
} ;
void primegen_sieve(primegen *pg)
{
uint32_t (*buf)[B32];
uint64_t L;
int i;
uint32_t Lmodqq[49];
buf = pg->buf;
L = pg->L;
if (L > 2000000000)
for (i = 0;i < 49;++i)
Lmodqq[i] = L % qqtab[i];
else
for (i = 0;i < 49;++i)
Lmodqq[i] = ((uint32_t) L) % qqtab[i];
clear(buf);
for (i = 0;i < 16;++i)
doit4(buf[0],for4[i].f,for4[i].g,(int64_t) for4[i].k - L);
squarefreetiny(buf[0],Lmodqq,1);
for (;i < 32;++i)
doit4(buf[3],for4[i].f,for4[i].g,(int64_t) for4[i].k - L);
squarefreetiny(buf[3],Lmodqq,13);
for (;i < 48;++i)
doit4(buf[4],for4[i].f,for4[i].g,(int64_t) for4[i].k - L);
squarefreetiny(buf[4],Lmodqq,17);
for (;i < 64;++i)
doit4(buf[7],for4[i].f,for4[i].g,(int64_t) for4[i].k - L);
squarefreetiny(buf[7],Lmodqq,29);
for (;i < 80;++i)
doit4(buf[9],for4[i].f,for4[i].g,(int64_t) for4[i].k - L);
squarefreetiny(buf[9],Lmodqq,37);
for (;i < 96;++i)
doit4(buf[10],for4[i].f,for4[i].g,(int64_t) for4[i].k - L);
squarefreetiny(buf[10],Lmodqq,41);
for (;i < 112;++i)
doit4(buf[13],for4[i].f,for4[i].g,(int64_t) for4[i].k - L);
squarefreetiny(buf[13],Lmodqq,49);
for (;i < 128;++i)
doit4(buf[14],for4[i].f,for4[i].g,(int64_t) for4[i].k - L);
squarefreetiny(buf[14],Lmodqq,53);
for (i = 0;i < 12;++i)
doit6(buf[1],for6[i].f,for6[i].g,(int64_t) for6[i].k - L);
squarefreetiny(buf[1],Lmodqq,7);
for (;i < 24;++i)
doit6(buf[5],for6[i].f,for6[i].g,(int64_t) for6[i].k - L);
squarefreetiny(buf[5],Lmodqq,19);
for (;i < 36;++i)
doit6(buf[8],for6[i].f,for6[i].g,(int64_t) for6[i].k - L);
squarefreetiny(buf[8],Lmodqq,31);
for (;i < 48;++i)
doit6(buf[11],for6[i].f,for6[i].g,(int64_t) for6[i].k - L);
squarefreetiny(buf[11],Lmodqq,43);
for (i = 0;i < 24;++i)
doit12(buf[2],for12[i].f,for12[i].g,(int64_t) for12[i].k - L);
squarefreetiny(buf[2],Lmodqq,11);
for (;i < 48;++i)
doit12(buf[6],for12[i].f,for12[i].g,(int64_t) for12[i].k - L);
squarefreetiny(buf[6],Lmodqq,23);
for (;i < 72;++i)
doit12(buf[12],for12[i].f,for12[i].g,(int64_t) for12[i].k - L);
squarefreetiny(buf[12],Lmodqq,47);
for (;i < 96;++i)
doit12(buf[15],for12[i].f,for12[i].g,(int64_t) for12[i].k - L);
squarefreetiny(buf[15],Lmodqq,59);
squarefree49(buf,L,247);
squarefree49(buf,L,253);
squarefree49(buf,L,257);
squarefree49(buf,L,263);
squarefree1(buf,L,241);
squarefree1(buf,L,251);
squarefree1(buf,L,259);
squarefree1(buf,L,269);
}
void primegen_fill(primegen *pg)
{
int i;
uint32_t mask;
uint32_t bits0, bits1, bits2, bits3, bits4, bits5, bits6, bits7;
uint32_t bits8, bits9, bits10, bits11, bits12, bits13, bits14, bits15;
uint64_t base;
i = pg->pos;
if (i == B32) {
primegen_sieve(pg);
pg->L += B;
i = 0;
}
pg->pos = i + 1;
bits0 = ~pg->buf[0][i];
bits1 = ~pg->buf[1][i];
bits2 = ~pg->buf[2][i];
bits3 = ~pg->buf[3][i];
bits4 = ~pg->buf[4][i];
bits5 = ~pg->buf[5][i];
bits6 = ~pg->buf[6][i];
bits7 = ~pg->buf[7][i];
bits8 = ~pg->buf[8][i];
bits9 = ~pg->buf[9][i];
bits10 = ~pg->buf[10][i];
bits11 = ~pg->buf[11][i];
bits12 = ~pg->buf[12][i];
bits13 = ~pg->buf[13][i];
bits14 = ~pg->buf[14][i];
bits15 = ~pg->buf[15][i];
base = pg->base + 1920;
pg->base = base;
pg->num = 0;
for (mask = 0x80000000;mask;mask >>= 1) {
base -= 60;
if (bits15 & mask) pg->p[pg->num++] = base + 59;
if (bits14 & mask) pg->p[pg->num++] = base + 53;
if (bits13 & mask) pg->p[pg->num++] = base + 49;
if (bits12 & mask) pg->p[pg->num++] = base + 47;
if (bits11 & mask) pg->p[pg->num++] = base + 43;
if (bits10 & mask) pg->p[pg->num++] = base + 41;
if (bits9 & mask) pg->p[pg->num++] = base + 37;
if (bits8 & mask) pg->p[pg->num++] = base + 31;
if (bits7 & mask) pg->p[pg->num++] = base + 29;
if (bits6 & mask) pg->p[pg->num++] = base + 23;
if (bits5 & mask) pg->p[pg->num++] = base + 19;
if (bits4 & mask) pg->p[pg->num++] = base + 17;
if (bits3 & mask) pg->p[pg->num++] = base + 13;
if (bits2 & mask) pg->p[pg->num++] = base + 11;
if (bits1 & mask) pg->p[pg->num++] = base + 7;
if (bits0 & mask) pg->p[pg->num++] = base + 1;
}
}
uint64_t primegen_next(primegen *pg)
{
while (!pg->num)
primegen_fill(pg);
return pg->p[--pg->num];
}
uint64_t primegen_peek(primegen *pg)
{
while (!pg->num)
primegen_fill(pg);
return pg->p[pg->num - 1];
}
void primegen_init(primegen *pg)
{
pg->L = 1;
pg->base = 60;
pg->pos = PRIMEGEN_WORDS;
pg->p[0] = 59;
pg->p[1] = 53;
pg->p[2] = 47;
pg->p[3] = 43;
pg->p[4] = 41;
pg->p[5] = 37;
pg->p[6] = 31;
pg->p[7] = 29;
pg->p[8] = 23;
pg->p[9] = 19;
pg->p[10] = 17;
pg->p[11] = 13;
pg->p[12] = 11;
pg->p[13] = 7;
pg->p[14] = 5;
pg->p[15] = 3;
pg->p[16] = 2;
pg->num = 17;
}
static const unsigned long pop[256] = {
0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5
,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6
,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6
,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7
,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6
,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7
,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7
,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,4,5,5,6,5,6,6,7,5,6,6,7,6,7,7,8
};
uint64_t primegen_count(primegen *pg,uint64_t to)
{
uint64_t count = 0;
for (;;) {
register int pos;
register int j;
register uint32_t bits;
register uint32_t smallcount;
while (pg->num) {
if (pg->p[pg->num - 1] >= to) return count;
++count;
--pg->num;
}
smallcount = 0;
pos = pg->pos;
while ((pos < B32) && (pg->base + 1920 < to)) {
for (j = 0;j < 16;++j) {
bits = ~pg->buf[j][pos];
smallcount += pop[bits & 255]; bits >>= 8;
smallcount += pop[bits & 255]; bits >>= 8;
smallcount += pop[bits & 255]; bits >>= 8;
smallcount += pop[bits & 255];
}
pg->base += 1920;
++pos;
}
pg->pos = pos;
count += smallcount;
if (pos == B32)
while (pg->base + B * 60 < to) {
primegen_sieve(pg);
pg->L += B;
smallcount = 0;
for (j = 0;j < 16;++j)
for (pos = 0;pos < B32;++pos) {
bits = ~pg->buf[j][pos];
smallcount += pop[bits & 255]; bits >>= 8;
smallcount += pop[bits & 255]; bits >>= 8;
smallcount += pop[bits & 255]; bits >>= 8;
smallcount += pop[bits & 255];
}
count += smallcount;
pg->base += B * 60;
}
primegen_fill(pg);
}
}
void primegen_skipto(primegen *pg,uint64_t to)
{
for (;;) {
int pos;
while (pg->num) {
if (pg->p[pg->num - 1] >= to) return;
--pg->num;
}
pos = pg->pos;
while ((pos < B32) && (pg->base + 1920 < to)) {
pg->base += 1920;
++pos;
}
pg->pos = pos;
if (pos == B32)
while (pg->base + B * 60 < to) {
pg->L += B;
pg->base += B * 60;
}
primegen_fill(pg);
}
}

44
src/crypto-primegen.h Normal file
View File

@@ -0,0 +1,44 @@
#ifndef PRIMEGEN_H
#define PRIMEGEN_H
#include <stdint.h>
/**
* This is B/32: the number of 32-bit words of space used in the primegen
* inner loop. This should fit into the CPU's level-1 cache.
*
* 2048 works well on a Pentium-100.
* 3600 works well on a Pentium II-350
* 4004 works well on an UltraSPARC-I/167
*
* 2012-nov (Rob): This code was written 15 years ago. Processor caches
* haven't really gotten any larger. A number like 8008 works slightly
* better on an Ivy Bridge CPU, but works noticeably worse on an Atom
* or ARM processor. The value 4004 seems to be a good compromise for
* all these processors. In any case, modern CPUs will automatically
* prefetch the buffers anyway, significantly lessoning the impact of
* having a poor number defined here. I tried 16016, but it crashed, and
* I don't know why, but I don't care because I'm not going to use such a
* large size.
*/
#define PRIMEGEN_WORDS 4004
typedef struct {
uint32_t buf[16][PRIMEGEN_WORDS];
uint64_t p[512]; /* p[num-1] ... p[0], in that order */
int num;
int pos; /* next entry to use in buf; WORDS to restart */
uint64_t base;
uint64_t L;
} primegen;
extern void primegen_sieve(primegen *);
extern void primegen_fill(primegen *);
extern void primegen_init(primegen *);
extern uint64_t primegen_next(primegen *);
extern uint64_t primegen_peek(primegen *);
extern uint64_t primegen_count(primegen *,uint64_t to);
extern void primegen_skipto(primegen *,uint64_t to);
#endif

252
src/crypto-siphash24.c Normal file
View File

@@ -0,0 +1,252 @@
/*
SipHash reference C implementation
Written in 2012 by
Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
Daniel J. Bernstein <djb@cr.yp.to>
To the extent possible under law, the author(s) have dedicated all copyright
and related and neighboring rights to this software to the public domain
worldwide. This software is distributed without any warranty.
You should have received a copy of the CC0 Public Domain Dedication along with
this software. If not, see <http://creativecommons.org/publicdomain/zero/1.0/>.
*/
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include "crypto-siphash24.h"
typedef uint64_t u64;
typedef uint32_t u32;
typedef uint8_t u8;
#define ROTL(x,b) (u64)( ((x) << (b)) | ( (x) >> (64 - (b))) )
#define U32TO8_LE(p, v) \
(p)[0] = (u8)((v) ); (p)[1] = (u8)((v) >> 8); \
(p)[2] = (u8)((v) >> 16); (p)[3] = (u8)((v) >> 24);
#define U64TO8_LE(p, v) \
U32TO8_LE((p), (u32)((v) )); \
U32TO8_LE((p) + 4, (u32)((v) >> 32));
#define U8TO64_LE(p) \
(((u64)((p)[0]) ) | \
((u64)((p)[1]) << 8) | \
((u64)((p)[2]) << 16) | \
((u64)((p)[3]) << 24) | \
((u64)((p)[4]) << 32) | \
((u64)((p)[5]) << 40) | \
((u64)((p)[6]) << 48) | \
((u64)((p)[7]) << 56))
#define SIPROUND \
{ \
v0 += v1; v1=ROTL(v1,13); v1 ^= v0; v0=ROTL(v0,32); \
v2 += v3; v3=ROTL(v3,16); v3 ^= v2; \
v0 += v3; v3=ROTL(v3,21); v3 ^= v0; \
v2 += v1; v1=ROTL(v1,17); v1 ^= v2; v2=ROTL(v2,32); \
}
/* SipHash-2-4 */
static int crypto_auth( unsigned char *out, const unsigned char *in, unsigned long long inlen, const unsigned char *k )
{
/* "somepseudorandomlygeneratedbytes" */
u64 v0 = 0x736f6d6570736575ULL;
u64 v1 = 0x646f72616e646f6dULL;
u64 v2 = 0x6c7967656e657261ULL;
u64 v3 = 0x7465646279746573ULL;
u64 b;
u64 k0 = U8TO64_LE( k );
u64 k1 = U8TO64_LE( k + 8 );
const u8 *end = in + inlen - ( inlen % sizeof( u64 ) );
const int left = inlen & 7;
b = ( ( u64 )inlen ) << 56;
v3 ^= k1;
v2 ^= k0;
v1 ^= k1;
v0 ^= k0;
for ( ; in != end; in += 8 )
{
u64 m;
m = U8TO64_LE( in );
#ifdef xxxDEBUG
printf( "(%3d) v0 %08x %08x\n", ( int )inlen, ( u32 )( v0 >> 32 ), ( u32 )v0 );
printf( "(%3d) v1 %08x %08x\n", ( int )inlen, ( u32 )( v1 >> 32 ), ( u32 )v1 );
printf( "(%3d) v2 %08x %08x\n", ( int )inlen, ( u32 )( v2 >> 32 ), ( u32 )v2 );
printf( "(%3d) v3 %08x %08x\n", ( int )inlen, ( u32 )( v3 >> 32 ), ( u32 )v3 );
printf( "(%3d) compress %08x %08x\n", ( int )inlen, ( u32 )( m >> 32 ), ( u32 )m );
#endif
v3 ^= m;
SIPROUND;
SIPROUND;
v0 ^= m;
}
switch( left )
{
case 7: b |= ( ( u64 )in[ 6] ) << 48;
case 6: b |= ( ( u64 )in[ 5] ) << 40;
case 5: b |= ( ( u64 )in[ 4] ) << 32;
case 4: b |= ( ( u64 )in[ 3] ) << 24;
case 3: b |= ( ( u64 )in[ 2] ) << 16;
case 2: b |= ( ( u64 )in[ 1] ) << 8;
case 1: b |= ( ( u64 )in[ 0] ); break;
case 0: break;
}
#ifdef xxxDEBUG
printf( "(%3d) v0 %08x %08x\n", ( int )inlen, ( u32 )( v0 >> 32 ), ( u32 )v0 );
printf( "(%3d) v1 %08x %08x\n", ( int )inlen, ( u32 )( v1 >> 32 ), ( u32 )v1 );
printf( "(%3d) v2 %08x %08x\n", ( int )inlen, ( u32 )( v2 >> 32 ), ( u32 )v2 );
printf( "(%3d) v3 %08x %08x\n", ( int )inlen, ( u32 )( v3 >> 32 ), ( u32 )v3 );
printf( "(%3d) padding %08x %08x\n", ( int )inlen, ( u32 )( b >> 32 ), ( u32 )b );
#endif
v3 ^= b;
SIPROUND;
SIPROUND;
v0 ^= b;
#ifdef xxxDEBUG
printf( "(%3d) v0 %08x %08x\n", ( int )inlen, ( u32 )( v0 >> 32 ), ( u32 )v0 );
printf( "(%3d) v1 %08x %08x\n", ( int )inlen, ( u32 )( v1 >> 32 ), ( u32 )v1 );
printf( "(%3d) v2 %08x %08x\n", ( int )inlen, ( u32 )( v2 >> 32 ), ( u32 )v2 );
printf( "(%3d) v3 %08x %08x\n", ( int )inlen, ( u32 )( v3 >> 32 ), ( u32 )v3 );
#endif
v2 ^= 0xff;
SIPROUND;
SIPROUND;
SIPROUND;
SIPROUND;
b = v0 ^ v1 ^ v2 ^ v3;
U64TO8_LE( out, b );
return 0;
}
uint64_t
siphash24(const void *in, size_t inlen, const uint64_t key[2])
{
uint64_t result;
crypto_auth((unsigned char*)&result,
(const unsigned char *)in,
inlen,
(const unsigned char *)&key[0]);
return result;
}
/*
SipHash-2-4 output with
k = 00 01 02 ...
and
in = (empty string)
in = 00 (1 byte)
in = 00 01 (2 bytes)
in = 00 01 02 (3 bytes)
...
in = 00 01 02 ... 3e (63 bytes)
*/
static u8 vectors[64][8] =
{
{ 0x31, 0x0e, 0x0e, 0xdd, 0x47, 0xdb, 0x6f, 0x72, },
{ 0xfd, 0x67, 0xdc, 0x93, 0xc5, 0x39, 0xf8, 0x74, },
{ 0x5a, 0x4f, 0xa9, 0xd9, 0x09, 0x80, 0x6c, 0x0d, },
{ 0x2d, 0x7e, 0xfb, 0xd7, 0x96, 0x66, 0x67, 0x85, },
{ 0xb7, 0x87, 0x71, 0x27, 0xe0, 0x94, 0x27, 0xcf, },
{ 0x8d, 0xa6, 0x99, 0xcd, 0x64, 0x55, 0x76, 0x18, },
{ 0xce, 0xe3, 0xfe, 0x58, 0x6e, 0x46, 0xc9, 0xcb, },
{ 0x37, 0xd1, 0x01, 0x8b, 0xf5, 0x00, 0x02, 0xab, },
{ 0x62, 0x24, 0x93, 0x9a, 0x79, 0xf5, 0xf5, 0x93, },
{ 0xb0, 0xe4, 0xa9, 0x0b, 0xdf, 0x82, 0x00, 0x9e, },
{ 0xf3, 0xb9, 0xdd, 0x94, 0xc5, 0xbb, 0x5d, 0x7a, },
{ 0xa7, 0xad, 0x6b, 0x22, 0x46, 0x2f, 0xb3, 0xf4, },
{ 0xfb, 0xe5, 0x0e, 0x86, 0xbc, 0x8f, 0x1e, 0x75, },
{ 0x90, 0x3d, 0x84, 0xc0, 0x27, 0x56, 0xea, 0x14, },
{ 0xee, 0xf2, 0x7a, 0x8e, 0x90, 0xca, 0x23, 0xf7, },
{ 0xe5, 0x45, 0xbe, 0x49, 0x61, 0xca, 0x29, 0xa1, },
{ 0xdb, 0x9b, 0xc2, 0x57, 0x7f, 0xcc, 0x2a, 0x3f, },
{ 0x94, 0x47, 0xbe, 0x2c, 0xf5, 0xe9, 0x9a, 0x69, },
{ 0x9c, 0xd3, 0x8d, 0x96, 0xf0, 0xb3, 0xc1, 0x4b, },
{ 0xbd, 0x61, 0x79, 0xa7, 0x1d, 0xc9, 0x6d, 0xbb, },
{ 0x98, 0xee, 0xa2, 0x1a, 0xf2, 0x5c, 0xd6, 0xbe, },
{ 0xc7, 0x67, 0x3b, 0x2e, 0xb0, 0xcb, 0xf2, 0xd0, },
{ 0x88, 0x3e, 0xa3, 0xe3, 0x95, 0x67, 0x53, 0x93, },
{ 0xc8, 0xce, 0x5c, 0xcd, 0x8c, 0x03, 0x0c, 0xa8, },
{ 0x94, 0xaf, 0x49, 0xf6, 0xc6, 0x50, 0xad, 0xb8, },
{ 0xea, 0xb8, 0x85, 0x8a, 0xde, 0x92, 0xe1, 0xbc, },
{ 0xf3, 0x15, 0xbb, 0x5b, 0xb8, 0x35, 0xd8, 0x17, },
{ 0xad, 0xcf, 0x6b, 0x07, 0x63, 0x61, 0x2e, 0x2f, },
{ 0xa5, 0xc9, 0x1d, 0xa7, 0xac, 0xaa, 0x4d, 0xde, },
{ 0x71, 0x65, 0x95, 0x87, 0x66, 0x50, 0xa2, 0xa6, },
{ 0x28, 0xef, 0x49, 0x5c, 0x53, 0xa3, 0x87, 0xad, },
{ 0x42, 0xc3, 0x41, 0xd8, 0xfa, 0x92, 0xd8, 0x32, },
{ 0xce, 0x7c, 0xf2, 0x72, 0x2f, 0x51, 0x27, 0x71, },
{ 0xe3, 0x78, 0x59, 0xf9, 0x46, 0x23, 0xf3, 0xa7, },
{ 0x38, 0x12, 0x05, 0xbb, 0x1a, 0xb0, 0xe0, 0x12, },
{ 0xae, 0x97, 0xa1, 0x0f, 0xd4, 0x34, 0xe0, 0x15, },
{ 0xb4, 0xa3, 0x15, 0x08, 0xbe, 0xff, 0x4d, 0x31, },
{ 0x81, 0x39, 0x62, 0x29, 0xf0, 0x90, 0x79, 0x02, },
{ 0x4d, 0x0c, 0xf4, 0x9e, 0xe5, 0xd4, 0xdc, 0xca, },
{ 0x5c, 0x73, 0x33, 0x6a, 0x76, 0xd8, 0xbf, 0x9a, },
{ 0xd0, 0xa7, 0x04, 0x53, 0x6b, 0xa9, 0x3e, 0x0e, },
{ 0x92, 0x59, 0x58, 0xfc, 0xd6, 0x42, 0x0c, 0xad, },
{ 0xa9, 0x15, 0xc2, 0x9b, 0xc8, 0x06, 0x73, 0x18, },
{ 0x95, 0x2b, 0x79, 0xf3, 0xbc, 0x0a, 0xa6, 0xd4, },
{ 0xf2, 0x1d, 0xf2, 0xe4, 0x1d, 0x45, 0x35, 0xf9, },
{ 0x87, 0x57, 0x75, 0x19, 0x04, 0x8f, 0x53, 0xa9, },
{ 0x10, 0xa5, 0x6c, 0xf5, 0xdf, 0xcd, 0x9a, 0xdb, },
{ 0xeb, 0x75, 0x09, 0x5c, 0xcd, 0x98, 0x6c, 0xd0, },
{ 0x51, 0xa9, 0xcb, 0x9e, 0xcb, 0xa3, 0x12, 0xe6, },
{ 0x96, 0xaf, 0xad, 0xfc, 0x2c, 0xe6, 0x66, 0xc7, },
{ 0x72, 0xfe, 0x52, 0x97, 0x5a, 0x43, 0x64, 0xee, },
{ 0x5a, 0x16, 0x45, 0xb2, 0x76, 0xd5, 0x92, 0xa1, },
{ 0xb2, 0x74, 0xcb, 0x8e, 0xbf, 0x87, 0x87, 0x0a, },
{ 0x6f, 0x9b, 0xb4, 0x20, 0x3d, 0xe7, 0xb3, 0x81, },
{ 0xea, 0xec, 0xb2, 0xa3, 0x0b, 0x22, 0xa8, 0x7f, },
{ 0x99, 0x24, 0xa4, 0x3c, 0xc1, 0x31, 0x57, 0x24, },
{ 0xbd, 0x83, 0x8d, 0x3a, 0xaf, 0xbf, 0x8d, 0xb7, },
{ 0x0b, 0x1a, 0x2a, 0x32, 0x65, 0xd5, 0x1a, 0xea, },
{ 0x13, 0x50, 0x79, 0xa3, 0x23, 0x1c, 0xe6, 0x60, },
{ 0x93, 0x2b, 0x28, 0x46, 0xe4, 0xd7, 0x06, 0x66, },
{ 0xe1, 0x91, 0x5f, 0x5c, 0xb1, 0xec, 0xa4, 0x6c, },
{ 0xf3, 0x25, 0x96, 0x5c, 0xa1, 0x6d, 0x62, 0x9f, },
{ 0x57, 0x5f, 0xf2, 0x8e, 0x60, 0x38, 0x1b, 0xe5, },
{ 0x72, 0x45, 0x06, 0xeb, 0x4c, 0x32, 0x8a, 0x95, }
};
static int
test_vectors()
{
#define MAXLEN 64
u8 in[MAXLEN], out[8], k[16];
int i;
int ok = 1;
for( i = 0; i < 16; ++i ) k[i] = (u8)i;
for( i = 0; i < MAXLEN; ++i )
{
in[i] = (u8)i;
crypto_auth( out, in, i, k );
if ( memcmp( out, vectors[i], 8 ) )
{
printf( "test vector failed for %d bytes\n", i );
ok = 0;
}
}
return ok;
}
int
siphash24_selftest(void)
{
if (test_vectors())
return 0;
else
return 1;
}

17
src/crypto-siphash24.h Normal file
View File

@@ -0,0 +1,17 @@
#ifndef CRYPTO_SIPHASH24_H
#define CRYPTO_SIPHASH24_H
#include <stdint.h>
uint64_t
siphash24(const void *in, size_t inlen, const uint64_t key[2]);
/**
* Regression-test this module.
* @return
* 0 on success, a positive integer otherwise.
*/
int
siphash24_selftest(void);
#endif

176
src/event-timeout.c Normal file
View File

@@ -0,0 +1,176 @@
/*
Event timeout
This is for the user-mode TCP stack. We need to mark timeouts in the
future when we'll re-visit a connection/tcb. For example, when we
send a packet, we need to resend it in the future in case we don't
get a response.
This design creates a large "ring" of timeouts, and then cycles
again and again through the ring. This is a fairly high granularity,
just has hundreds, thousands, or 10 thousand entries per second.
(I keep adjusting the granularity up and down). Not that at any
slot in the ring, there may be entries from the far future.
NOTE: a big feature of this system is that the structure that tracks
the timeout is actually held within the TCB structure. In other
words, each TCB can have one-and-only-one timeout.
NOTE: a recurring bug is that the TCP code removes a TCB from the
timeout ring and forgets to put it back somewhere else. Since the
TCB is cleaned up on a timeout, such TCBs never get cleaned up,
leading to a memory leak. I keep fixing this bug, then changing the
code and causing the bug to come back again.
*/
#include "event-timeout.h"
#include "util-logger.h"
#include "util-malloc.h"
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
/***************************************************************************
* The timeout system is a circular ring. We move an index around the
* ring. At each slot in the ring is a linked-list of all entries at
* that time index. Because the ring can wrap, not everything at a given
* entry will be the same timestamp. Therefore, when doing the timeout
* logic at a slot, we have to doublecheck the actual timestamp, and skip
* those things that are further in the future.
***************************************************************************/
struct Timeouts {
/**
* This index is a monotonically increasing number, modulus the mask.
* Every time we check timeouts, we simply move it forward in time.
*/
uint64_t current_index;
/**
* Counts the number of outstanding timeouts. Adding a timeout increments
* this number, and removing a timeout decrements this number. The
* program shouldn't exit until this number is zero.
*/
uint64_t outstanding_count;
/**
* The number of slots is a power-of-2, so the mask is just this
* number minus 1
*/
unsigned mask;
/**
* The ring of entries.
*/
struct TimeoutEntry *slots[1024*1024];
};
/***************************************************************************
***************************************************************************/
struct Timeouts *
timeouts_create(uint64_t timestamp)
{
struct Timeouts *timeouts;
/*
* Allocate memory and initialize it to zero
*/
timeouts = CALLOC(1, sizeof(*timeouts));
/*
* We just mask off the low order bits to determine wrap. I'm using
* a variable here because one of these days I'm going to make
* the size of the ring dynamically adjustable depending upon
* the speed of the scan.
*/
timeouts->mask = sizeof(timeouts->slots)/sizeof(timeouts->slots[0]) - 1;
/*
* Set the index to the current time. Note that this timestamp is
* the 'time_t' value multiplied by the number of ticks-per-second,
* where 'ticks' is something I've defined for scanning. Right now
* I hard-code in the size of the ticks, but eventually they'll be
* dynamically resized depending upon the speed of the scan.
*/
timeouts->current_index = timestamp;
return timeouts;
}
/***************************************************************************
* This inserts the timeout entry into the appropriate place in the
* timeout ring.
***************************************************************************/
void
timeouts_add(struct Timeouts *timeouts, struct TimeoutEntry *entry,
size_t offset, uint64_t timestamp)
{
unsigned index;
/* Unlink from wherever the entry came from */
if (entry->timestamp)
timeouts->outstanding_count--;
timeout_unlink(entry);
if (entry->prev) {
LOG(1, "***CHANGE %d-seconds\n",
(int)((timestamp-entry->timestamp)/TICKS_PER_SECOND));
}
/* Initialize the new entry */
entry->timestamp = timestamp;
entry->offset = (unsigned)offset;
/* Link it into it's new location */
index = timestamp & timeouts->mask;
entry->next = timeouts->slots[index];
timeouts->slots[index] = entry;
entry->prev = &timeouts->slots[index];
if (entry->next)
entry->next->prev = &entry->next;
timeouts->outstanding_count++;
}
/***************************************************************************
* Remove the next event that it older than the specified timestamp
***************************************************************************/
void *
timeouts_remove(struct Timeouts *timeouts, uint64_t timestamp)
{
struct TimeoutEntry *entry = NULL;
/* Search until we find one */
while (timeouts->current_index <= timestamp) {
/* Start at the current slot */
entry = timeouts->slots[timeouts->current_index & timeouts->mask];
/* enumerate through the linked list until we find a used slot */
while (entry && entry->timestamp > timestamp)
entry = entry->next;
if (entry)
break;
/* found nothing at this slot, so move to next slot */
timeouts->current_index++;
}
if (entry == NULL) {
/* we've caught up to the current time, and there's nothing
* left to timeout, so return NULL */
return NULL;
}
/* unlink this entry from the timeout system */
timeouts--;
timeout_unlink(entry);
/* return a pointer to the structure holding this entry */
return ((char*)entry) - entry->offset;
}

132
src/event-timeout.h Normal file
View File

@@ -0,0 +1,132 @@
#ifndef EVENT_TIMEOUT_H
#define EVENT_TIMEOUT_H
#include <stdint.h>
#include <stdio.h>
#include <stddef.h> /* offsetof*/
#include "util-bool.h" /* <stdbool.h> */
#if defined(_MSC_VER)
#undef inline
#define inline _inline
#endif
struct Timeouts;
/***************************************************************************
***************************************************************************/
struct TimeoutEntry {
/**
* In units of 1/16384 of a second. We use power-of-two units here
* to make the "modulus" operation a simple binary "and".
* See the TICKS_FROM_TV() macro for getting the timestamp from
* the current time.
*/
uint64_t timestamp;
/** we build a doubly-linked list */
struct TimeoutEntry *next;
struct TimeoutEntry **prev;
/** The timeout entry is never allocated by itself, but instead
* lives inside another data structure. This stores the value of
* 'offsetof()', so given a pointer to this structure, we can find
* the original structure that contains it */
unsigned offset;
};
/***************************************************************************
***************************************************************************/
static inline bool
timeout_is_unlinked(const struct TimeoutEntry *entry) {
if (entry->prev == 0 || entry->next == 0)
return true;
else
return false;
}
/***************************************************************************
***************************************************************************/
static inline void
timeout_unlink(struct TimeoutEntry *entry)
{
if (entry->prev == 0 && entry->next == 0)
return;
*(entry->prev) = entry->next;
if (entry->next)
entry->next->prev = entry->prev;
entry->next = 0;
entry->prev = 0;
entry->timestamp = 0;
}
/***************************************************************************
***************************************************************************/
static inline void
timeout_init(struct TimeoutEntry *entry)
{
entry->next = 0;
entry->prev = 0;
}
/**
* Create a timeout subsystem.
* @param timestamp_now
* The current timestamp indicating "now" when the thing starts.
* This should be 'time(0) * TICKS_PER_SECOND'.
*/
struct Timeouts *
timeouts_create(uint64_t timestamp_now);
/**
* Insert the timeout 'entry' into the future location in the timeout
* ring, as determined by the timestamp.
* @param timeouts
* A ring of timeouts, with each slot corresponding to a specific
* time in the future.
* @param entry
* The entry that we are going to insert into the ring. If it's
* already in the ring, it'll be removed from the old location
* first before inserting into the new location.
* @param offset
* The 'entry' field above is part of an existing structure. This
* tells the offset_of() from the beginning of that structure.
* In other words, this tells us the pointer to the object that
* that is the subject of the timeout.
* @param timestamp_expires
* When this timeout will expire. This is in terms of internal
* ticks, which in units of TICKS_PER_SECOND.
*/
void
timeouts_add(struct Timeouts *timeouts, struct TimeoutEntry *entry,
size_t offset, uint64_t timestamp_expires);
/**
* Remove an object from the timestamp system that is older than than
* the specified timestamp. This function must be called repeatedly
* until it returns NULL to remove all the objects that are older
* than the given timestamp.
* @param timeouts
* A ring of timeouts. We'll walk the ring until we've caught
* up with the current time.
* @param timestamp_now
* Usually, this timestmap will be "now", the current time,
* and anything older than this will be aged out.
* @return
* an object older than the specified timestamp, or NULL
* if there are no more objects to be found
*/
void *
timeouts_remove(struct Timeouts *timeouts, uint64_t timestamp_now);
/*
* This macros convert a normal "timeval" structure into the timestamp
* that we use for timeouts. The timeval structure probably will come
* from the packets that we are capturing.
*/
#define TICKS_PER_SECOND (16384ULL)
#define TICKS_FROM_SECS(secs) ((secs)*16384ULL)
#define TICKS_FROM_USECS(usecs) ((usecs)/16384ULL)
#define TICKS_FROM_TV(secs,usecs) (TICKS_FROM_SECS(secs)+TICKS_FROM_USECS(usecs))
#endif

691
src/in-binary.c Normal file
View File

@@ -0,0 +1,691 @@
/*
Read in the binary file produced by "out-binary.c". This allows you to
translate the "binary" format into any of the other output formats.
*/
#include "massip-addr.h"
#include "in-binary.h"
#include "masscan.h"
#include "masscan-app.h"
#include "masscan-status.h"
#include "main-globals.h"
#include "output.h"
#include "util-safefunc.h"
#include "in-filter.h"
#include "in-report.h"
#include "util-malloc.h"
#include "util-logger.h"
#include <stdlib.h>
#include <assert.h>
#ifdef _MSC_VER
#pragma warning(disable:4996)
#endif
static const size_t BUF_MAX = 1024*1024;
struct MasscanRecord {
unsigned timestamp;
ipaddress ip;
unsigned char ip_proto;
unsigned short port;
unsigned char reason;
unsigned char ttl;
unsigned char mac[6];
enum ApplicationProtocol app_proto;
};
/***************************************************************************
***************************************************************************/
static void
parse_status(struct Output *out,
enum PortStatus status, /* open/closed */
const unsigned char *buf, size_t buf_length)
{
struct MasscanRecord record;
if (buf_length < 12)
return;
/* parse record */
record.timestamp = buf[0]<<24 | buf[1]<<16 | buf[2]<<8 | buf[3];
record.ip.ipv4 = buf[4]<<24 | buf[5]<<16 | buf[6]<<8 | buf[7];
record.ip.version = 4;
record.port = buf[8]<<8 | buf[9];
record.reason = buf[10];
record.ttl = buf[11];
/* if ARP, then there will be a MAC address */
if (record.ip.ipv4 == 0 && buf_length >= 12+6)
memcpy(record.mac, buf+12, 6);
else
memset(record.mac, 0, 6);
if (out->when_scan_started == 0)
out->when_scan_started = record.timestamp;
switch (record.port) {
case 53:
case 123:
case 137:
case 161:
record.ip_proto = 17;
break;
case 36422:
case 36412:
case 2905:
record.ip_proto = 132;
break;
default:
record.ip_proto = 6;
break;
}
/*
* Now report the result
*/
output_report_status(out,
record.timestamp,
status,
record.ip,
record.ip_proto,
record.port,
record.reason,
record.ttl,
record.mac);
}
/***************************************************************************
***************************************************************************/
static void
parse_status2(struct Output *out,
enum PortStatus status, /* open/closed */
const unsigned char *buf, size_t buf_length,
struct MassIP *filter)
{
struct MasscanRecord record;
if (buf_length < 13)
return;
/* parse record */
record.timestamp = buf[0]<<24 | buf[1]<<16 | buf[2]<<8 | buf[3];
record.ip.ipv4 = buf[4]<<24 | buf[5]<<16 | buf[6]<<8 | buf[7];
record.ip.version = 4;
record.ip_proto = buf[8];
record.port = buf[9]<<8 | buf[10];
record.reason = buf[11];
record.ttl = buf[12];
/* if ARP, then there will be a MAC address */
if (record.ip.ipv4 == 0 && buf_length >= 13+6)
memcpy(record.mac, buf+13, 6);
else
memset(record.mac, 0, 6);
if (out->when_scan_started == 0)
out->when_scan_started = record.timestamp;
/* Filter for known IP/ports, if specified on command-line */
if (filter && filter->count_ipv4s) {
if (!massip_has_ip(filter, record.ip))
return;
}
if (filter && filter->count_ports) {
if (!massip_has_port(filter, record.port))
return;
}
/*
* Now report the result
*/
output_report_status(out,
record.timestamp,
status,
record.ip,
record.ip_proto,
record.port,
record.reason,
record.ttl,
record.mac);
}
static unsigned char
_get_byte(const unsigned char *buf, size_t length, size_t *offset)
{
unsigned char result;
if (*offset < length) {
result = buf[*offset];
} else {
result = 0xFF;
}
(*offset)++;
return result;
}
static unsigned
_get_integer(const unsigned char *buf, size_t length, size_t *r_offset)
{
unsigned result;
size_t offset = *r_offset;
(*r_offset) += 4;
if (offset + 4 <= length) {
result = buf[offset+0]<<24
| buf[offset+1]<<16
| buf[offset+2]<<8
| buf[offset+3]<<0;
} else {
result = 0xFFFFFFFF;
}
return result;
}
static unsigned short
_get_short(const unsigned char *buf, size_t length, size_t *r_offset)
{
unsigned short result;
size_t offset = *r_offset;
(*r_offset) += 2;
if (offset + 2 <= length) {
result = buf[offset+0]<<8
| buf[offset+1]<<0;
} else {
result = 0xFFFF;
}
return result;
}
static unsigned long long
_get_long(const unsigned char *buf, size_t length, size_t *r_offset)
{
unsigned long long result;
size_t offset = *r_offset;
(*r_offset) += 8;
if (offset + 8 <= length) {
result =
(unsigned long long)buf[offset+0]<<56ULL
| (unsigned long long)buf[offset+1]<<48ULL
| (unsigned long long)buf[offset+2]<<40ULL
| (unsigned long long)buf[offset+3]<<32ULL
| (unsigned long long)buf[offset+4]<<24ULL
| (unsigned long long)buf[offset+5]<<16ULL
| (unsigned long long)buf[offset+6]<<8ULL
| (unsigned long long)buf[offset+7]<<0ULL;
} else {
result = 0xFFFFFFFFffffffffULL;
}
return result;
}
/***************************************************************************
***************************************************************************/
static void
parse_status6(struct Output *out,
enum PortStatus status, /* open/closed */
const unsigned char *buf, size_t length,
struct MassIP *filter)
{
struct MasscanRecord record;
size_t offset = 0;
/* parse record */
record.timestamp = _get_integer(buf, length, &offset);
record.ip_proto = _get_byte(buf, length, &offset);
record.port = _get_short(buf, length, &offset);
record.reason = _get_byte(buf, length, &offset);
record.ttl = _get_byte(buf, length, &offset);
record.ip.version= _get_byte(buf, length, &offset);
if (record.ip.version != 6) {
fprintf(stderr, "[-] corrupt record\n");
return;
}
record.ip.ipv6.hi = _get_long(buf, length, &offset);
record.ip.ipv6.lo = _get_long(buf, length, &offset);
if (out->when_scan_started == 0)
out->when_scan_started = record.timestamp;
/* Filter for known IP/ports, if specified on command-line */
if (filter && filter->count_ipv4s) {
if (!massip_has_ip(filter, record.ip))
return;
}
if (filter && filter->count_ports) {
if (!massip_has_port(filter, record.port))
return;
}
/*
* Now report the result
*/
output_report_status(out,
record.timestamp,
status,
record.ip,
record.ip_proto,
record.port,
record.reason,
record.ttl,
record.mac);
}
/***************************************************************************
***************************************************************************/
static void
parse_banner6(struct Output *out, unsigned char *buf, size_t length,
const struct MassIP *filter,
const struct RangeList *btypes)
{
struct MasscanRecord record;
size_t offset = 0;
/*
* Parse the parts that are common to most records
*/
record.timestamp = _get_integer(buf, length, &offset);
record.ip_proto = _get_byte(buf, length, &offset);
record.port = _get_short(buf, length, &offset);
record.app_proto = _get_short(buf, length, &offset);
record.ttl = _get_byte(buf, length, &offset);
record.ip.version= _get_byte(buf, length, &offset);
if (record.ip.version != 6) {
fprintf(stderr, "[-] corrupt record\n");
return;
}
record.ip.ipv6.hi = _get_long(buf, length, &offset);
record.ip.ipv6.lo = _get_long(buf, length, &offset);
if (out->when_scan_started == 0)
out->when_scan_started = record.timestamp;
/*
* Filter out records if requested
*/
if (!readscan_filter_pass(record.ip, record.port, record.app_proto,
filter, btypes))
return;
/*
* Now print the output
*/
if (offset > length)
return;
output_report_banner(
out,
record.timestamp,
record.ip,
record.ip_proto, /* TCP=6, UDP=17 */
record.port,
record.app_proto, /* HTTP, SSL, SNMP, etc. */
record.ttl, /* ttl */
buf+offset, (unsigned)(length-offset)
);
}
/***************************************************************************
* [OBSOLETE]
* This parses an old version of the banner record. I've still got files
* hanging around with this version, so I'm keeping it in the code for
* now, but eventually I'll get rid of it.
***************************************************************************/
static void
parse_banner3(struct Output *out, unsigned char *buf, size_t buf_length)
{
struct MasscanRecord record;
/*
* Parse the parts that are common to most records
*/
record.timestamp = buf[0]<<24 | buf[1]<<16 | buf[2]<<8 | buf[3];
record.ip.ipv4 = buf[4]<<24 | buf[5]<<16 | buf[6]<<8 | buf[7];
record.ip.version = 4;
record.port = buf[8]<<8 | buf[9];
record.app_proto = buf[10]<<8 | buf[11];
if (out->when_scan_started == 0)
out->when_scan_started = record.timestamp;
/*
* Now print the output
*/
output_report_banner(
out,
record.timestamp,
record.ip,
6, /* this is always TCP */
record.port,
record.app_proto,
0, /* ttl */
buf+12, (unsigned)buf_length-12
);
}
/***************************************************************************
* Parse the BANNER record, extracting the timestamp, IP address, and port
* number. We also convert the banner string into a safer form.
***************************************************************************/
static void
parse_banner4(struct Output *out, unsigned char *buf, size_t buf_length)
{
struct MasscanRecord record;
if (buf_length < 13)
return;
/*
* Parse the parts that are common to most records
*/
record.timestamp = buf[0]<<24 | buf[1]<<16 | buf[2]<<8 | buf[3];
record.ip.ipv4 = buf[4]<<24 | buf[5]<<16 | buf[6]<<8 | buf[7];
record.ip.version = 4;
record.ip_proto = buf[8];
record.port = buf[9]<<8 | buf[10];
record.app_proto = buf[11]<<8 | buf[12];
if (out->when_scan_started == 0)
out->when_scan_started = record.timestamp;
/*
* Now print the output
*/
output_report_banner(
out,
record.timestamp,
record.ip,
record.ip_proto, /* TCP=6, UDP=17 */
record.port,
record.app_proto, /* HTTP, SSL, SNMP, etc. */
0, /* ttl */
buf+13, (unsigned)buf_length-13
);
}
/***************************************************************************
***************************************************************************/
static void
parse_banner9(struct Output *out, unsigned char *buf, size_t buf_length,
const struct MassIP *filter,
const struct RangeList *btypes)
{
struct MasscanRecord record;
unsigned char *data = buf+14;
size_t data_length = buf_length-14;
if (buf_length < 14)
return;
/*
* Parse the parts that are common to most records
*/
record.timestamp = buf[0]<<24 | buf[1]<<16 | buf[2]<<8 | buf[3];
record.ip.ipv4 = buf[4]<<24 | buf[5]<<16 | buf[6]<<8 | buf[7];
record.ip.version = 4;
record.ip_proto = buf[8];
record.port = buf[9]<<8 | buf[10];
record.app_proto = buf[11]<<8 | buf[12];
record.ttl = buf[13];
if (out->when_scan_started == 0)
out->when_scan_started = record.timestamp;
/*
* KLUDGE: when doing SSL stuff, add a IP:name pair to a database
* so we can annotate [VULN] strings with this information
*/
//readscan_report(record.ip, record.app_proto, &data, &data_length);
/*
* Filter out records if requested
*/
if (!readscan_filter_pass(record.ip, record.port, record.app_proto,
filter, btypes))
return;
/*
* Now print the output
*/
output_report_banner(
out,
record.timestamp,
record.ip,
record.ip_proto, /* TCP=6, UDP=17 */
record.port,
record.app_proto, /* HTTP, SSL, SNMP, etc. */
record.ttl, /* ttl */
data, (unsigned)data_length
);
}
/***************************************************************************
* Read in the file, one record at a time.
***************************************************************************/
static uint64_t
_binaryfile_parse(struct Output *out, const char *filename,
struct MassIP *filter,
const struct RangeList *btypes)
{
FILE *fp = 0;
unsigned char *buf = 0;
size_t bytes_read;
uint64_t total_records = 0;
/* Allocate a buffer of up to one megabyte per record */
buf = MALLOC(BUF_MAX);
/* Open the file */
fp = fopen(filename, "rb");
if (fp == NULL) {
fprintf(stderr, "[-] FAIL: --readscan\n");
fprintf(stderr, "[-] %s: %s\n", filename, strerror(errno));
goto end;
}
LOG(0, "[+] --readscan %s\n", filename);
if (feof(fp)) {
LOG(0, "[-] %s: file is empty\n", filename);
goto end;
}
/* first record is pseudo-record */
bytes_read = fread(buf, 1, 'a'+2, fp);
if (bytes_read < 'a'+2) {
LOG(0, "[-] %s: %s\n", filename, strerror(errno));
goto end;
}
/* Make sure it's got the format string */
if (memcmp(buf, "masscan/1.1", 11) != 0) {
LOG(0,
"[-] %s: unknown file format (expected \"masscan/1.1\")\n",
filename);
goto end;
}
/*
* Look for start time
*/
if (buf[11] == '.' && strtoul((char*)buf+12,0,0) >= 2) {
unsigned i;
/* move to next field */
for (i=0; i<'a' && buf[i] && buf[i] != '\n'; i++)
;
i++;
if (buf[i] == 's')
i++;
if (buf[i] == ':')
i++;
/* extract timestamp */
if (i < 'a')
out->when_scan_started = strtoul((char*)buf+i,0,0);
}
/* Now read all records */
for (;;) {
unsigned type;
unsigned length;
/* [TYPE]
* This is one or more bytes indicating the type of type of the
* record
*/
bytes_read = fread(buf, 1, 1, fp);
if (bytes_read != 1)
break;
type = buf[0] & 0x7F;
while (buf[0] & 0x80) {
bytes_read = fread(buf, 1, 1, fp);
if (bytes_read != 1)
break;
type = (type << 7) | (buf[0] & 0x7F);
}
/* [LENGTH]
* Is one byte for lengths smaller than 127 bytes, or two
* bytes for lengths up to 16384.
*/
bytes_read = fread(buf, 1, 1, fp);
if (bytes_read != 1)
break;
length = buf[0] & 0x7F;
while (buf[0] & 0x80) {
bytes_read = fread(buf, 1, 1, fp);
if (bytes_read != 1)
break;
length = (length << 7) | (buf[0] & 0x7F);
}
if (length > BUF_MAX) {
LOG(0, "[-] file corrupt\n");
goto end;
}
/* get the remainder of the record */
bytes_read = fread(buf, 1, length, fp);
if (bytes_read < length)
break; /* eof */
/* Depending on record type, do something different */
switch (type) {
case 1: /* STATUS: open */
if (!btypes->count)
parse_status(out, PortStatus_Open, buf, bytes_read);
break;
case 2: /* STATUS: closed */
if (!btypes->count)
parse_status(out, PortStatus_Closed, buf, bytes_read);
break;
case 3: /* BANNER */
parse_banner3(out, buf, bytes_read);
break;
case 4:
if (fread(buf+bytes_read,1,1,fp) != 1) {
LOG(0, "[-] read() error\n");
exit(1);
}
bytes_read++;
parse_banner4(out, buf, bytes_read);
break;
case 5:
parse_banner4(out, buf, bytes_read);
<