acidvegas/masscan
1
Fork 0
Code Issues Pull Requests Releases Activity

more tweaks

Browse Source
This commit is contained in:
Dionysus
2026-06-20 03:43:10 +00:00
parent 41130a5093
commit 06f23cc4c9
17 changed files with 1110 additions and 126 deletions
Download Patch File Download Diff File Expand all files Collapse all files

192
FORK.md
View File

@@ -9,25 +9,39 @@ This is a modified fork of [robertdavidgraham/masscan](https://github.com/robert
1. Remove every wire-level fingerprint that lets a passive observer say "this is masscan." All known signatures (TLS hardcoded random, ICMP `0x08..0x37` payload, UDP `masscan-test` literals, IP-ID XOR formula, default SYN template, banner-grab anomalies) have been replaced with values that look like a normal Linux or Windows host doing TLS 1.2 cert harvesting.
2. Greatly expand the set of services masscan can banner-grab out of the box — adds PostgreSQL, Cassandra, Modbus, Siemens S7, OPC UA, DNP3, EtherNet/IP, BACnet, plus 50+ HTTP/HTTPS bindings on alt ports (Grafana, Kibana, Prometheus, k8s, Docker, Elasticsearch, etc.).
3. Auto-discover and load nmap's `nmap-payloads` file if installed, giving UDP probe coverage on hundreds more ports without a CLI flag.
4. Decode TLS certs into structured fields in NDJSON instead of a base64 DER blob. Adds source IP / source port to every output record so scans with `--source-port <range>` show which port actually fired.
CLI is **100% backward-compatible** with upstream masscan. Drop in `bin/masscan` and existing config files / commands keep working. The OS-mimicry chosen at startup is silently picked between Linux (50%) and Windows (50%); no flag changes that.
**Build dep change:** Linux/macOS builds now link `-lcrypto` (OpenSSL libcrypto) for the X.509 → JSON decoder used by the ndjson output. Install `libssl-dev` (Debian/Ubuntu) or `openssl-devel` (RHEL) before `make`.
---
## File-by-file changes
Diff stats vs. upstream:
Diff stats vs. upstream (approximate, includes all rounds of fork edits):
| File | Lines added | Lines removed |
| --- | --- | --- |
| `src/main-conf.c` | +26 | -4 |
| `src/proto-banner1.c` | +229 | -0 |
| `src/proto-http.c` | +15 | -9 |
| `src/proto-ssl.c` | +84 | -29 |
| `src/stack-tcp-core.c` | +3 | -1 |
| `src/templ-payloads.c` | +57 | -10 |
| `src/templ-pkt.c` | +152 | -41 |
| **Total** | **+566** | **-94** |
| File | Lines added | Lines removed | Notes |
| --- | --- | --- | --- |
| `src/main-conf.c` | +26 | -4 | nmap-payloads auto-load |
| `src/proto-banner1.c` | +229 | -0 | new TCP hellos, alt-port bindings |
| `src/proto-http.c` | +90 | -9 | Chrome UA + headers + `Host:` builder |
| `src/stack-tcp-api.h` | +18 | 0 | target ip/port accessors |
| `src/proto-ssl.c` | +110 | -30 | TLS hello + `srand()` + chain break |
| `src/stack-tcp-core.c` | +40 | -1 | small-window fix + src plumbing + target accessors |
| `src/templ-payloads.c` | +57 | -10 | strip masscan literals, BACnet, EtherNet/IP |
| `src/templ-pkt.c` | +160 | -41 | OS mimicry + `srand()` |
| `src/main.c` | +2 | 0 | src plumbing |
| `src/proto-udp.c` | +6 | 0 | src plumbing |
| `src/proto-icmp.c` | +3 | 0 | src plumbing |
| `src/proto-sctp.c` | +3 | 0 | src plumbing |
| `src/output.h` | +20 | 0 | side-channel + flush_target |
| `src/output.c` | +20 | 0 | output_flush_target() |
| `src/out-ndjson.c` | +325 | -68 | aggregation + cert object |
| `src/proto-x509-openssl.c` | +357 | 0 | new — OpenSSL cert decoder |
| `src/proto-x509-openssl.h` | +19 | 0 | new |
| `Makefile` | +2 | -2 | link -lcrypto |
| **Total** | **~1360** | **~165** | |
---
@@ -43,7 +57,7 @@ This is the file that owns the byte-level templates masscan copies and rewrites
- The original `default_tcp_template[]` (TTL=255, win=1024, DF clear, options = MSS=1460 only — every one of those is a known masscan tell) was replaced with two static templates:
- `linux_tcp_template[]` — TTL=64, win=64240, DF set, options = `MSS=1460, SAckOK, Timestamp(0/0), NOP, WScale=7` (full Linux 5.x/6.x SYN, 40-byte TCP header)
- `windows_tcp_template[]` — TTL=128, win=64240, DF set, options = `MSS=1460, NOP, WScale=8, NOP, NOP, SAckOK` (Windows 10/11 SYN, 32-byte TCP header)
- New function `mimicry_select_os()` picks one at startup using `time(NULL) ^ (getpid() << 16)` as the seed. Exported via `int g_masscan_mimic_os` (0=Linux, 1=Windows) so other code paths can match the chosen OS if needed.
- New function `mimicry_select_os()` picks one at startup using `time(NULL) ^ (getpid() << 16)` as the seed. **The same value also seeds libc `rand()` via `srand()`** — every other randomization in the fork (TLS client_random, DTLS client_random, IP-ID per packet) consumes `rand()`, and without this `srand()` they would all produce the same byte sequence across every run. Exported via `int g_masscan_mimic_os` (0=Linux, 1=Windows) so other code paths can match the chosen OS if needed.
- `default_tcp_template` is now a pointer set by `mimicry_select_os()`; `default_tcp_template_size` is the size of whichever was picked.
**Why:** The original masscan SYN signature (`win=1024 + TTL=255 + DF clear + opts=[MSS=1460]`) is detected by every "is this a scanner?" rule. Real Linux and Windows hosts never look like that.
@@ -170,12 +184,24 @@ This is the file that owns the byte-level templates masscan copies and rewrites
**Where:** Lines 1062–1086.
**What:** First call to `ssl_init()` (one-time, gated by `static int randomized` flag) overwrites bytes 11..42 of both `ssl_hello_template` and `ssl_12_hello_template` with `rand()` bytes.
**What:** First call to `ssl_init()` (one-time, gated by `static int randomized` flag) calls `srand(time ^ pid)` and then overwrites bytes 11..42 of both `ssl_hello_template` and `ssl_12_hello_template` with `rand()` bytes.
**Why:** The original masscan random `97 e5 60 50 c4 a5 4a e0 …` is the single most reliable masscan signature on the wire — it's literally the same 32 bytes on every TLS probe from every masscan ever. Now each scan run has a fresh random, and the static byte sequence is gone.
**The `srand()` call is load-bearing.** Without it, the C standard says `rand()` behaves as `srand(1)` — so every scan would produce the *same* constant client_random and we'd have just swapped upstream's fingerprint for a new equally-static one. The same seed is also reused by `mimicry_select_os()` and the IP-ID / DTLS random callsites elsewhere; gated by per-callsite once-flags so multiple `srand()` invocations are no-ops in practice.
**Trade-off documented:** All probes in one scan run still share the same random. Per-probe randomization would require switching from a static `stream->hello` buffer to a `transmit_hello` callback. Deferred — defeats the static-bytes fingerprint, which is the actual public detector.
#### 2b. Disabled the `banner_ssl_12` fallback chain
**Where:** Line ~1505 (`banner_ssl.next`).
**Old:** `&banner_ssl_12` — when the curl-shape TLS 1.2 hello closed without a response, masscan would reconnect and send a *second* hello from `ssl_12_hello_template`.
**New:** `0` — the second hello never gets sent.
**Why:** The bytes in `ssl_12_hello_template` (the original masscan TLS 1.3-aware hello) are unchanged from upstream apart from the 32 random bytes. Cipher list, key share, extensions are all the original masscan JA3. Letting the fallback fire would leak the upstream fingerprint on retry — exactly what the rewrite of `ssl_hello_template` was meant to kill.
#### 3. Replaced `ssl_hello_template` with curl-shape ClientHello + ALPN
**Where:** Lines 1098–1163 (the template body).
@@ -220,6 +246,23 @@ Connection: close\r\n
**Why:** The `ivre-masscan/1.3 https://github.com/robertdavidgraham/` User-Agent appearing in a web server log says "masscan" instantly. Chrome 120 with a full browser header set passes basic UA filters and looks like normal traffic in access logs. HTTP/1.0 → HTTP/1.1 because Chrome never sends 1.0.
#### Per-target `Host:` header (`http_transmit_hello` callback)
**Where:** New `http_transmit_hello()` function in `src/proto-http.c`. `banner_http` struct literal now sets the `transmit_hello` slot. Supporting accessors `tcpapi_get_target_ip_string()` and `tcpapi_get_target_port()` added to `src/stack-tcp-api.h` and implemented in `src/stack-tcp-core.c`.
**What:** Instead of sending the static `http_hello[]` blob, masscan now constructs the HTTP request per target at send time:
```
GET / HTTP/1.1\r\n
Host: <target_ip>[:<port>]\r\n
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36\r\n
...
```
IPv6 literals are bracketed per RFC 7230 §5.4 (`Host: [2001:db8::1]:8080`). Port is omitted when it's the default 80. If the accessor fails for any reason, falls back to sending the static `http_hello[]` so the probe still goes out.
**Why:** Sending HTTP/1.1 *without* a `Host:` header is illegal per RFC 7230. nginx default config, Apache, IIS, Cloudflare, AWS ALB, every modern load balancer responds with `400 Bad Request - Missing Host header` instead of the actual page. Upstream masscan and the previous fork iteration were leaving a large fraction of HTTP banner data on the floor as a result.
**Verified:** Test against `1.1.1.1:80`. Cloudflare returned `HTTP/1.1 301 Moved Permanently` with `Server: cloudflare` (the real edge response). Without the Host header Cloudflare returns a hard 400 with a specific "Missing Host header" body — confirming the new behavior is taking effect.
---
### `src/stack-tcp-core.c` — Banner-grab TCP stack
@@ -257,7 +300,7 @@ This is the biggest single file change (+229 lines, no deletions). Everything is
| --- | --- | --- | --- |
| Siemens S7 | 102 | TPKT + COTP CR (rack 0, slot 0, dst TSAP = PG) | COTP CC with PLC presence |
| Modbus | 502 | MBAP + Read Device Identification (func 0x2B / 0x0E) | Vendor name, product code, revision |
| OPC UA | 4840 | HEL message with 1 MB receive buffer | ACK message with endpoint protocol version |
| OPC UA | 4840 | HEL message with 1 MB receive buffer, endpoint URL `opc.tcp://localhost:4840` (length=24, message size=56 — corrected from initial 22/54 mismatch) | ACK message with endpoint protocol version |
| DNP3 | 20000 | Link layer request, dst=0, src=1 | DNP3 link header revealing addresses |
| EtherNet/IP | 44818 | CIP List Identity command | Vendor, product, revision, serial, device name |
@@ -293,5 +336,124 @@ Added bindings for ~70 ports:
If any exists, log `[+] auto-loading nmap-payloads from <path>` and use it as if `--nmap-payloads <path>` had been passed.
**Why:** nmap's `nmap-payloads` file ships ~300 UDP probes for protocols masscan doesn't have built-in. Loading it adds full coverage of those services for free, without changing the user's CLI. (Note: modern nmap on some distros — Ubuntu 24.04 included — compiles the payloads into the binary instead of shipping the file, so the auto-load is a no-op there. Still works on Alpine and older systems.)
**Why:** nmap's `nmap-payloads` file ships ~300 UDP probes for protocols masscan doesn't have built-in. Loading it adds full coverage of those services for free, without changing the user's CLI. (Note: modern nmap on some distros — Ubuntu 24.04 included — compiles the payloads into the binary instead of shipping the file, so the auto-load is a no-op there. Still works on Alpine and older systems. To activate on a modern Debian/Ubuntu box: drop a copy of `nmap-payloads` from any older nmap source release at `/usr/share/nmap/nmap-payloads`.)
---
### Output: `src/output.h`, `src/output.c`, `src/out-ndjson.c`, `src/proto-x509-openssl.[ch]`
This is the biggest semantic change in the fork: ndjson records are now self-contained and structured. Cert harvesting no longer emits a base64 DER blob — every cert field is parsed and rendered as JSON.
#### 1. Source IP + source port on every output record
**Where:** `src/output.h` (new fields on `struct Output`), `src/output.c`, `src/main.c` (TCP status), `src/stack-tcp-core.c` (TCP banner flush), `src/proto-udp.c`, `src/proto-icmp.c`, `src/proto-sctp.c`.
**What:** Two new fields on `struct Output`:
```c
ipaddress src_ip_me;
unsigned src_port_me;
```
Side-channel set by every callsite that knows the local side of the probe (TCB for TCP, `parsed->dst_ip`/`parsed->port_dst` for UDP/ICMP/SCTP) immediately before invoking `output_report_status` / `output_report_banner`. The ndjson backend reads them and emits `src_ip` / `src_port` keys in every record. Other backends ignore.
**Why:** With `--source-port 40000-50000` (or any range), the source port for a given probe is picked per packet from the range. Without this field, the operator can't tell which specific source actually fired and can't correlate records to e.g. specific outgoing flows or NAT entries. Critical for multi-source-port long-running 0.0.0.0/0 scans.
#### 2. `output_flush_target()` and the `flush_target` vtable slot
**Where:** `src/output.h`, `src/output.c`, called from `stack-tcp-core.c:banner_flush()`.
**What:** New optional callback `void (*flush_target)(struct Output *, FILE *);` on `struct OutputType`. Called by `banner_flush` after the TCB's banout list has been fully iterated. Tells aggregating backends "no more banners are coming for this target."
**Why:** ndjson needs to collect multiple banouts per target (`PROTO_SSL3` cipher line + N × `PROTO_X509_CERT` chain certs) into one combined record. Without an explicit end-of-target signal, ndjson would have to guess at flush boundaries by watching for (ip,port) changes. The hook is a no-op for every other backend (default NULL via C99 trailing-zero initialization in their struct literals — no edits needed to xml/json/text/grepable/binary/redis/etc.).
#### 3. OpenSSL-backed X.509 → JSON decoder
**Where:** New files `src/proto-x509-openssl.c` (~360 lines) and `src/proto-x509-openssl.h`. `Makefile` now links `-lcrypto` on Linux/macOS.
**What:** Single public function:
```c
int x509_json_write_cert(FILE *fp, const unsigned char *b64, size_t b64_len);
```
Decodes the base64 DER bytes masscan accumulates in a `PROTO_X509_CERT` / `PROTO_X509_CACERT` banout, hands them to `d2i_X509()`, and writes a JSON object directly to `fp` containing:
- `subject` (RFC 2253 DN)
- `issuer` (RFC 2253 DN)
- `not_before`, `not_after` (ISO 8601 UTC: `YYYY-MM-DDTHH:MM:SSZ`)
- `san` (array of strings — DNS / email / URI / IPv4 / IPv6 names from the SAN extension)
- `sig_algo` (long-name OID — e.g. `ecdsa-with-SHA384`, `sha256WithRSAEncryption`)
- `key_algo` (`rsa` / `ec` / `dsa` / `ed25519` / `ed448` / OID short name fallback)
- `key_bits`
- `key_curve` (only for EC keys — e.g. `prime256v1`, `secp384r1`)
- `serial` (lowercase hex, no separators)
- `version` (1, 2, or 3)
JSON string escaping is minimal — `"` and `\\` get backslash-escaped, control bytes get `\uXXXX`, everything else (including non-ASCII UTF-8 in SAN/subject) passes through.
**Why:** The previous output of `--capture cert` was a base64 DER blob — useful only as input to a separate `openssl x509 -inform DER -text` pass. The decoded object is operator-readable, queryable directly with `jq`, and gives you the data points anyone actually does cert harvesting for (validity window, SAN list, key strength, signature algorithm).
#### 4. Rewritten ndjson backend with per-target aggregation
**Where:** `src/out-ndjson.c` (full rewrite, ~325 new lines).
**What:**
- Backend keeps a per-`Output` `struct NdjsonState` (allocated in `open()`, freed in `close()`) holding any in-progress aggregation.
- TLS-related banouts (`PROTO_SSL3`, `PROTO_X509_CERT`, `PROTO_X509_CACERT`) are NOT emitted immediately. They accumulate into the state:
- `PROTO_SSL3` body is parsed for the `cipher:0xHHHH` and `TLS/1.X` prefix → goes into `cipher` and `tls_version` fields.
- `PROTO_X509_CERT` / `_CACERT` base64 bytes are stashed in a chain slot (max 6 certs per target).
- When `flush_target` fires (or another non-matching banner arrives), the leaf cert is decoded via `x509_json_write_cert`, all chain certs go into a `chain[]` array of decoded objects, and one combined ndjson line is emitted.
- Non-TLS banners (HTTP/SSH/Telnet/etc.) bypass aggregation and emit one record each, as before — but always with the new `src_ip` / `src_port` fields and a structured `data.service` + `data.banner` shape.
**Record shapes:**
Status:
```json
{"ip":"1.1.1.1","port":443,"proto":"tcp","src_ip":"10.0.0.5","src_port":50001,
"timestamp":"1781925682","rec_type":"status",
"data":{"status":"open","reason":"syn-ack","ttl":55}}
```
TLS banner (one record carrying cipher + version + leaf cert + entire chain):
```json
{"ip":"1.1.1.1","port":443,"proto":"tcp","src_ip":"10.0.0.5","src_port":50001,
"timestamp":"1781925686","rec_type":"banner",
"data":{
"service":"ssl",
"cipher":"0xc02b","tls_version":"TLS/1.2",
"cert":{
"subject":"CN=cloudflare-dns.com,O=Cloudflare\\, Inc.,L=San Francisco,...",
"issuer":"CN=SSL.com SSL Intermediate CA ECC R2,O=SSL Corp,...",
"not_before":"2025-12-31T19:20:01Z","not_after":"2026-12-21T19:20:01Z",
"san":["cloudflare-dns.com","*.cloudflare-dns.com","1.1.1.1",
"2606:4700:4700:0000:0000:0000:0000:1111","one.one.one.one"],
"sig_algo":"ecdsa-with-SHA384","key_algo":"ec","key_bits":256,
"key_curve":"prime256v1",
"serial":"4ed03304c46b87a8c2eb5569db9eba0c","version":3
},
"chain":[{"subject":"...","issuer":"...","not_before":"...","not_after":"...",
"san":[],"sig_algo":"...","key_algo":"ec","key_bits":384,
"key_curve":"secp384r1","serial":"...","version":3}]
}}
```
Non-TLS banner:
```json
{"ip":"1.2.3.4","port":80,"proto":"tcp","src_ip":"10.0.0.5","src_port":47891,
"timestamp":"...","rec_type":"banner",
"data":{"service":"http","banner":"HTTP/1.1 200 OK\\r\\nServer: nginx/1.18.0..."}}
```
**Notes:**
- The base64 DER blob is **suppressed entirely from ndjson** even when the user passes `--capture cert`. The decoded object replaces it. If you need the raw DER for some other tooling, use a different output format (xml/json/text) — those are unchanged.
- Chain depth is capped at 6 certs per target (rare for real servers to send more; protects memory if a server returns a runaway chain).
- The aggregation is per-`Output` (which is per-thread in masscan), so multi-thread receive paths don't share state.
**Verified:** Tested against `1.1.1.1:443` with `--source-port 50000-50003`. Confirmed: `src_port` in the emitted record matched the actual port used, leaf cert + intermediate CA both decoded, SAN list complete, ECDSA key bits and curve name present, validity window in ISO 8601.
---
### `Makefile`
#### Added `-lcrypto` to the link line
**Where:** Linux block (`musl` + glibc) and macOS block.
**Why:** The new `src/proto-x509-openssl.c` uses OpenSSL's `d2i_X509`, `X509_get_*`, `EVP_PKEY_*`, etc. Build dep: `libssl-dev` (Debian/Ubuntu) or `openssl-devel` (RHEL). Runtime dep is just `libcrypto.so.3`, present on every modern distro.

10
Makefile
View File

@@ -34,12 +34,12 @@ endif
# works on the bajillion of different Linux environments
ifneq (, $(findstring linux, $(SYS)))
ifneq (, $(findstring musl, $(SYS)))
LIBS =
LIBS = -lcrypto
else
LIBS = -lm -lrt -ldl -lpthread
LIBS = -lm -lrt -ldl -lpthread -lcrypto
endif
INCLUDES =
FLAGS2 =
FLAGS2 =
endif
# MAC OS X
@@ -47,9 +47,9 @@ endif
# my regularly regression-test environment. That means at any point
# in time, something might be minorly broken in Mac OS X.
ifneq (, $(findstring darwin, $(SYS)))
LIBS = -lm
LIBS = -lm -lcrypto
INCLUDES = -I.
FLAGS2 =
FLAGS2 =
INSTALL_DATA = -pm755
endif

2
src/main.c
View File

@@ -1074,6 +1074,8 @@ receive_thread(void *v)
/*
* This is where we do the output
*/
out->src_ip_me = ip_me;
out->src_port_me = port_me;
output_report_status(
out,
global_now,

438
src/out-ndjson.c
View File

@@ -1,65 +1,74 @@
// masscan - Developed by acidvegas in C (https://github.com/acidvegas)
// src/out-ndjson.c
#include "output.h"
#include "masscan-app.h"
#include "masscan-status.h"
#include "util-safefunc.h"
#include "util-malloc.h"
#include "proto-x509-openssl.h"
#include <ctype.h>
#include <stdlib.h>
#include <string.h>
/****************************************************************************
****************************************************************************/
static void
ndjson_out_open(struct Output *out, FILE *fp)
/*
* Per-target aggregation. The TCP stack delivers banners one at a time via
* banner_flush() in stack-tcp-core.c. For a TLS scan, the same target yields
* multiple banouts (PROTO_SSL3 for cipher/version, PROTO_X509_CERT for the
* leaf cert, PROTO_X509_CACERT for chain certs). We collect them here keyed
* by (ip, port) and emit one combined ndjson record when output_flush_target
* is called, or when a non-matching record arrives.
*
* Non-TLS banners (HTTP, SSH, etc.) bypass aggregation and emit immediately.
*/
#define NDJSON_CHAIN_MAX 6 /* most server chains are 1-3 certs */
#define NDJSON_BUF_MAX 16384
struct CertSlot {
/* base64 DER captured from the banout, kept until flush so we can hand
* it to the OpenSSL decoder once we know the record is complete. */
unsigned char *b64;
size_t b64_len;
int is_ca;
};
struct NdjsonState {
/* Whether a TLS aggregation is in progress. */
int has_pending;
/* Target identity for the pending record. */
ipaddress ip;
ipaddress src_ip;
unsigned port;
unsigned src_port;
unsigned ip_proto;
unsigned ttl;
time_t timestamp;
/* Cleartext fields parsed out of the PROTO_SSL3 banout text. */
char cipher[64];
char tls_version[16];
/* Certs queued for OpenSSL decoding at flush time. */
struct CertSlot chain[NDJSON_CHAIN_MAX];
unsigned chain_count;
};
static struct NdjsonState *
state_of(struct Output *out)
{
UNUSEDPARM(out);
UNUSEDPARM(fp);
return (struct NdjsonState *)out->backend_state;
}
/****************************************************************************
****************************************************************************/
static void
ndjson_out_close(struct Output *out, FILE *fp)
{
UNUSEDPARM(out);
UNUSEDPARM(fp);
}
//{ ip: "124.53.139.201", ports: [ {port: 443, proto: "tcp", status: "open", reason: "syn-ack", ttl: 48} ] }
/****************************************************************************
****************************************************************************/
static void
ndjson_out_status(struct Output *out, FILE *fp, time_t timestamp, int status,
ipaddress ip, unsigned ip_proto, unsigned port, unsigned reason, unsigned ttl)
{
char reason_buffer[128];
ipaddress_formatted_t fmt;
UNUSEDPARM(out);
fprintf(fp, "{");
fmt = ipaddress_fmt(ip);
fprintf(fp, "\"ip\":\"%s\",", fmt.string);
fprintf(fp, "\"timestamp\":\"%d\",\"port\":%u,\"proto\":\"%s\",\"rec_type\":\"status\",\"data\":{\"status\":\"%s\","
"\"reason\":\"%s\",\"ttl\":%u}",
(int) timestamp,
port,
name_from_ip_proto(ip_proto),
status_string(status),
reason_string(reason, reason_buffer, sizeof(reason_buffer)),
ttl
);
fprintf(fp, "}\n");
}
/*****************************************************************************
* Remove bad characters from the banner, especially new lines and HTML
* control codes.
*
* Keeping this here since we may need to change the behavior from what
* is done in the sister `normalize_json_string` function. It's unlikely
* but it's a small function and will save time later if needed. Could also
* set it up to base64 encode the banner payload.
* JSON-safe string emission. The upstream normalize_ndjson_string mirrored
* normalize_string but escaped a few extra chars; keep that behavior for
* banner bodies.
*****************************************************************************/
static const char *
normalize_ndjson_string(const unsigned char *px, size_t length,
@@ -68,7 +77,6 @@ normalize_ndjson_string(const unsigned char *px, size_t length,
size_t i=0;
size_t offset = 0;
for (i=0; i<length; i++) {
unsigned char c = px[i];
@@ -92,8 +100,203 @@ normalize_ndjson_string(const unsigned char *px, size_t length,
return buf;
}
/******************************************************************************
******************************************************************************/
/* Extract cipher suite hex from a PROTO_SSL3 banner. Upstream format is
* "cipher:0xHHHH" (no brackets), so we read up to the next space. */
static int
extract_cipher(const char *haystack, char *dst, size_t dst_len)
{
const char *p = strstr(haystack, "cipher:");
const char *end;
size_t n;
if (p == NULL) return 0;
p += sizeof("cipher:") - 1;
end = p;
while (*end && *end != ' ' && *end != '\t' &&
*end != ',' && *end != ';' && *end != ']') end++;
n = (size_t)(end - p);
if (n == 0) return 0;
if (n >= dst_len) n = dst_len - 1;
memcpy(dst, p, n);
dst[n] = 0;
return 1;
}
/* Extract TLS version label from the leading prefix of a PROTO_SSL3 banner.
* Upstream prefixes one of: "SSLv3 ", "TLS/1.0 ", "TLS/1.1 ", "TLS/1.2 ",
* "TLS/1.3 ", or "SSLver[major,minor] ". */
static int
extract_tls_version(const char *haystack, char *dst, size_t dst_len)
{
static const char *labels[] = {
"SSLv3", "TLS/1.0", "TLS/1.1", "TLS/1.2", "TLS/1.3", NULL
};
int i;
size_t n;
for (i = 0; labels[i]; i++) {
n = strlen(labels[i]);
if (strncmp(haystack, labels[i], n) == 0) {
if (n >= dst_len) n = dst_len - 1;
memcpy(dst, labels[i], n);
dst[n] = 0;
return 1;
}
}
return 0;
}
/*****************************************************************************
* Flush the pending aggregated TLS record. Emits a single ndjson line.
*****************************************************************************/
static void
ndjson_flush_pending(struct Output *out, FILE *fp)
{
struct NdjsonState *s = state_of(out);
ipaddress_formatted_t fmt;
ipaddress_formatted_t fmt_src;
unsigned i;
if (s == NULL || !s->has_pending)
return;
fmt = ipaddress_fmt(s->ip);
fputc('{', fp);
fprintf(fp, "\"ip\":\"%s\"", fmt.string);
fprintf(fp, ",\"port\":%u", s->port);
fprintf(fp, ",\"proto\":\"%s\"", s->ip_proto == 6 ? "tcp" : "udp");
/* Only emit src_ip if it was actually populated (non-zero). */
if (s->src_ip.version != 0) {
fmt_src = ipaddress_fmt(s->src_ip);
fprintf(fp, ",\"src_ip\":\"%s\"", fmt_src.string);
}
if (s->src_port != 0)
fprintf(fp, ",\"src_port\":%u", s->src_port);
fprintf(fp, ",\"timestamp\":\"%d\"", (int)s->timestamp);
fputs(",\"rec_type\":\"banner\"", fp);
fputs(",\"data\":{\"service\":\"ssl\"", fp);
if (s->cipher[0]) {
fputs(",\"cipher\":\"", fp);
fputs(s->cipher, fp);
fputc('"', fp);
}
if (s->tls_version[0]) {
fputs(",\"tls_version\":\"", fp);
fputs(s->tls_version, fp);
fputc('"', fp);
}
/* Decode and emit leaf cert (first non-CA, or first slot). */
{
int leaf_idx = -1;
for (i = 0; i < s->chain_count; i++) {
if (!s->chain[i].is_ca) { leaf_idx = (int)i; break; }
}
if (leaf_idx < 0 && s->chain_count > 0) leaf_idx = 0;
if (leaf_idx >= 0) {
fputs(",\"cert\":", fp);
x509_json_write_cert(fp,
s->chain[leaf_idx].b64,
s->chain[leaf_idx].b64_len);
}
}
/* Chain certs (CAs). */
{
int wrote = 0;
for (i = 0; i < s->chain_count; i++) {
if (!s->chain[i].is_ca) continue;
if (!wrote) { fputs(",\"chain\":[", fp); wrote = 1; }
else fputc(',', fp);
x509_json_write_cert(fp,
s->chain[i].b64,
s->chain[i].b64_len);
}
if (wrote) fputc(']', fp);
}
fputs("}}\n", fp);
/* Reset state. */
for (i = 0; i < s->chain_count; i++) {
free(s->chain[i].b64);
s->chain[i].b64 = NULL;
s->chain[i].b64_len = 0;
}
s->chain_count = 0;
s->cipher[0] = 0;
s->tls_version[0] = 0;
s->has_pending = 0;
s->src_ip.version = 0;
s->src_port = 0;
}
/*****************************************************************************
* Open/close
*****************************************************************************/
static void
ndjson_out_open(struct Output *out, FILE *fp)
{
UNUSEDPARM(fp);
if (out->backend_state == NULL)
out->backend_state = CALLOC(1, sizeof(struct NdjsonState));
}
static void
ndjson_out_close(struct Output *out, FILE *fp)
{
if (out->backend_state) {
ndjson_flush_pending(out, fp);
free(out->backend_state);
out->backend_state = NULL;
}
}
/*****************************************************************************
* status — open/closed port records. No TLS aggregation involved; emit
* directly. If a TLS aggregation was pending for a different target, flush
* it first.
*****************************************************************************/
static void
ndjson_out_status(struct Output *out, FILE *fp, time_t timestamp, int status,
ipaddress ip, unsigned ip_proto, unsigned port, unsigned reason, unsigned ttl)
{
char reason_buffer[128];
ipaddress_formatted_t fmt;
ipaddress_formatted_t fmt_src;
struct NdjsonState *s = state_of(out);
if (s && s->has_pending) ndjson_flush_pending(out, fp);
fmt = ipaddress_fmt(ip);
fputc('{', fp);
fprintf(fp, "\"ip\":\"%s\"", fmt.string);
fprintf(fp, ",\"port\":%u", port);
fprintf(fp, ",\"proto\":\"%s\"", name_from_ip_proto(ip_proto));
if (out->src_ip_me.version != 0) {
fmt_src = ipaddress_fmt(out->src_ip_me);
fprintf(fp, ",\"src_ip\":\"%s\"", fmt_src.string);
}
if (out->src_port_me != 0)
fprintf(fp, ",\"src_port\":%u", out->src_port_me);
fprintf(fp,
",\"timestamp\":\"%d\",\"rec_type\":\"status\""
",\"data\":{\"status\":\"%s\",\"reason\":\"%s\",\"ttl\":%u}}\n",
(int)timestamp,
status_string(status),
reason_string(reason, reason_buffer, sizeof(reason_buffer)),
ttl);
}
/*****************************************************************************
* banner — either aggregates (SSL3 / X509) or emits directly.
*****************************************************************************/
static void
ndjson_out_banner(struct Output *out, FILE *fp, time_t timestamp,
ipaddress ip, unsigned ip_proto, unsigned port,
@@ -101,57 +304,97 @@ ndjson_out_banner(struct Output *out, FILE *fp, time_t timestamp,
unsigned ttl,
const unsigned char *px, unsigned length)
{
char banner_buffer[65536];
struct NdjsonState *s = state_of(out);
ipaddress_formatted_t fmt;
ipaddress_formatted_t fmt_src;
char banner_buffer[MAX_BANNER_LENGTH];
UNUSEDPARM(ttl);
//UNUSEDPARM(timestamp);
fprintf(fp, "{");
/* If we have a pending TLS aggregation but this banner is for a
* different target, flush it first. */
if (s && s->has_pending) {
int same_target =
ipaddress_is_equal(s->ip, ip) &&
s->port == port &&
s->ip_proto == ip_proto;
if (!same_target)
ndjson_flush_pending(out, fp);
}
/* TLS-related banouts get aggregated. */
if (proto == PROTO_SSL3 ||
proto == PROTO_X509_CERT ||
proto == PROTO_X509_CACERT)
{
if (s == NULL) return;
if (!s->has_pending) {
s->has_pending = 1;
s->ip = ip;
s->port = port;
s->ip_proto = ip_proto;
s->ttl = ttl;
s->timestamp = timestamp;
s->src_ip = out->src_ip_me;
s->src_port = out->src_port_me;
}
if (proto == PROTO_SSL3) {
/* Upstream PROTO_SSL3 emits "TLS/1.2 cipher:0xc02b issuer[CN=R3,
* ...] subject[CN=foo.com]". We pull tls_version + cipher only
* — issuer/subject are already in the decoded cert. */
char tmp[MAX_BANNER_LENGTH];
size_t n = length < sizeof(tmp) - 1 ? length : sizeof(tmp) - 1;
memcpy(tmp, px, n);
tmp[n] = 0;
extract_tls_version(tmp, s->tls_version, sizeof(s->tls_version));
extract_cipher(tmp, s->cipher, sizeof(s->cipher));
} else {
/* Stash the base64 DER for OpenSSL decoding at flush time. */
if (s->chain_count < NDJSON_CHAIN_MAX && length > 0) {
struct CertSlot *slot = &s->chain[s->chain_count++];
slot->b64 = (unsigned char *)MALLOC(length);
memcpy(slot->b64, px, length);
slot->b64_len = length;
slot->is_ca = (proto == PROTO_X509_CACERT);
}
}
return;
}
/* Non-TLS banner: emit one record immediately. */
fmt = ipaddress_fmt(ip);
fprintf(fp, "\"ip\":\"%s\",", fmt.string);
fprintf(fp, "\"timestamp\":\"%d\",\"port\":%u,\"proto\":\"%s\",\"rec_type\":\"banner\",\"data\":{\"service_name\":\"%s\",\"banner\":\"%s\"}",
(int) timestamp,
port,
name_from_ip_proto(ip_proto),
fputc('{', fp);
fprintf(fp, "\"ip\":\"%s\"", fmt.string);
fprintf(fp, ",\"port\":%u", port);
fprintf(fp, ",\"proto\":\"%s\"", name_from_ip_proto(ip_proto));
if (out->src_ip_me.version != 0) {
fmt_src = ipaddress_fmt(out->src_ip_me);
fprintf(fp, ",\"src_ip\":\"%s\"", fmt_src.string);
}
if (out->src_port_me != 0)
fprintf(fp, ",\"src_port\":%u", out->src_port_me);
fprintf(fp,
",\"timestamp\":\"%d\",\"rec_type\":\"banner\""
",\"data\":{\"service\":\"%s\",\"banner\":\"%s\"}}\n",
(int)timestamp,
masscan_app_to_string(proto),
normalize_ndjson_string(px, length, banner_buffer, sizeof(banner_buffer))
);
// fprintf(fp, "\"timestamp\":\"%d\",\"ports\":[{\"port\":%u,\"proto\":\"%s\",\"service\":{\"name\":\"%s\",\"banner\":\"%s\"}}]",
// (int) timestamp,
// port,
// name_from_ip_proto(ip_proto),
// masscan_app_to_string(proto),
// normalize_ndjson_string(px, length, banner_buffer, sizeof(banner_buffer))
// );
fprintf(fp, "}\n");
UNUSEDPARM(out);
/* fprintf(fp, "<host endtime=\"%u\">"
"<address addr=\"%u.%u.%u.%u\" addrtype=\"%s\"/>"
"<ports>"
"<port protocol=\"%s\" portid=\"%u\">"
"<state state=\"open\" reason=\"%s\" reason_ttl=\"%u\" />"
"<service name=\"%s\" banner=\"%s\"></service>"
"</port>"
"</ports>"
"</host>"
"\r\n",
(unsigned)timestamp,
(ip>>24)&0xFF,
(ip>>16)&0xFF,
(ip>> 8)&0xFF,
(ip>> 0)&0xFF,
name_from_ip_version(ip.version),
name_from_ip_proto(ip_proto),
port,
reason, ttl,
masscan_app_to_string(proto),
normalize_string(px, length, banner_buffer, sizeof(banner_buffer))
);*/
normalize_ndjson_string(px, length,
banner_buffer, sizeof(banner_buffer)));
}
/*****************************************************************************
* flush_target — called from output_flush_target after banner_flush has
* emitted all banouts for one TCB.
*****************************************************************************/
static void
ndjson_out_flush_target(struct Output *out, FILE *fp)
{
ndjson_flush_pending(out, fp);
}
/****************************************************************************
****************************************************************************/
const struct OutputType ndjson_output = {
@@ -160,5 +403,6 @@ const struct OutputType ndjson_output = {
ndjson_out_open,
ndjson_out_close,
ndjson_out_status,
ndjson_out_banner
ndjson_out_banner,
ndjson_out_flush_target,
};

18
src/output.c
View File

@@ -956,6 +956,24 @@ output_report_banner(struct Output *out, time_t now,
}
/***************************************************************************
* Called by the TCP banner-flush path to signal "no more banners coming
* for this target". Backends that aggregate per-target state (ndjson) use
* this to flush a single combined record.
***************************************************************************/
void
output_flush_target(struct Output *out)
{
if (out == NULL || out->funcs == NULL || out->funcs->flush_target == NULL)
return;
if (out->fp == NULL)
return;
out->funcs->flush_target(out, out->fp);
if (out->is_output_flush)
fflush(out->fp);
}
/***************************************************************************
* Called on exit of the program to close/free everything
***************************************************************************/

25
src/output.h
View File

@@ -38,6 +38,11 @@ struct OutputType {
unsigned port, enum ApplicationProtocol proto,
unsigned ttl,
const unsigned char *px, unsigned length);
/* Optional. Called after the last banner for a target has been
* delivered (TCB closed, all banouts flushed). Lets backends that
* aggregate per-target state (currently only ndjson) emit a final
* combined record. May be NULL. */
void (*flush_target)(struct Output *out, FILE *fp);
};
/**
@@ -81,6 +86,19 @@ struct Output
char *directory;
} rotate;
/* Source IP+port for the next output_report_* call.
* Side-channel: callers (TCP banner flush, UDP banner parsers) set these
* before invoking output_report_banner / output_report_status, and ndjson
* reads them out. Other output backends ignore them. ip_me may be all-zero
* if the caller doesn't know it. */
ipaddress src_ip_me;
unsigned src_port_me;
/* Backend-private state. Currently used by ndjson to aggregate
* SSL3 cipher + X509 cert banouts per target into one combined record.
* Allocated by the backend's open() hook, freed by close(). */
void *backend_state;
unsigned is_banner:1; /* --banners */
unsigned is_banner_rawudp:1; /* --rawudp */
unsigned is_output_flush:1; /* --output-flush */
@@ -186,6 +204,13 @@ void output_report_banner(
unsigned ttl,
const unsigned char *px, unsigned length);
/* Tell the output backend "no more banners are coming for the current target
* (the ip/port already set via src_ip_me/src_port_me or the last reported ip)".
* Used by ndjson to flush a single combined record after all PROTO_SSL3 +
* PROTO_X509_CERT banouts for one TCB have been delivered. No-op for other
* backends. */
void output_flush_target(struct Output *output);
/**
* Regression tests this unit.
* @return

6
src/proto-banner1.c
View File

@@ -576,14 +576,14 @@ static const char
opcua_hello[] =
"HEL" /* Message type */
"F" /* Final chunk */
"\x36\x00\x00\x00" /* Message size = 54 */
"\x38\x00\x00\x00" /* Message size = 56 (8 hdr + 20 sizes + 4 len + 24 url) */
"\x00\x00\x00\x00" /* Protocol version */
"\x00\x00\x10\x00" /* Receive buffer size = 1MB */
"\x00\x00\x10\x00" /* Send buffer size */
"\x00\x00\x10\x00" /* Max message size */
"\x00\x00\x00\x00" /* Max chunk count = 0 (unlimited) */
"\x16\x00\x00\x00" /* Endpoint URL length = 22 */
"opc.tcp://localhost:4840"; /* 22 chars */
"\x18\x00\x00\x00" /* Endpoint URL length = 24 */
"opc.tcp://localhost:4840"; /* 24 chars */
struct ProtocolParserStream banner_opcua = {
"banner-opcua", 4840, opcua_hello, sizeof(opcua_hello) - 1, 0,
NULL, NULL, NULL,

71
src/proto-http.c
View File

@@ -402,6 +402,75 @@ http_hello[] = "GET / HTTP/1.1\r\n"
"\r\n";
/***************************************************************************
* Per-connection HTTP request builder. Constructs a GET / HTTP/1.1 request
* with a real Host: header containing the target IP (and port if non-80).
*
* Sending HTTP/1.1 without Host: is illegal per RFC 7230 and many servers
* (nginx default config, Apache, IIS, every modern LB) reply with 400 Bad
* Request — costing a large fraction of HTTP banner data on a real scan.
* Building the request per-target means we can no longer use the static
* stream->hello buffer, hence the transmit_hello callback.
***************************************************************************/
static void
http_transmit_hello(const struct Banner1 *banner1,
struct stack_handle_t *socket)
{
char ip_buf[64];
char host_buf[80];
char request[1024];
unsigned port;
int n;
UNUSEDPARM(banner1);
if (tcpapi_get_target_ip_string(socket, ip_buf, sizeof(ip_buf)) == 0) {
/* Couldn't resolve target IP — fall back to the static hello so
* we still send *something*. Won't carry Host:, but the
* connection-level send still works. */
tcpapi_send(socket, http_hello, sizeof(http_hello) - 1, TCP__static);
return;
}
port = tcpapi_get_target_port(socket);
/* Wrap IPv6 literals in brackets per RFC 7230 §5.4. Detect by ':' in
* the address string (dotted-quad IPv4 never contains one). */
if (strchr(ip_buf, ':')) {
if (port == 80 || port == 0)
snprintf(host_buf, sizeof(host_buf), "[%s]", ip_buf);
else
snprintf(host_buf, sizeof(host_buf), "[%s]:%u", ip_buf, port);
} else {
if (port == 80 || port == 0)
snprintf(host_buf, sizeof(host_buf), "%s", ip_buf);
else
snprintf(host_buf, sizeof(host_buf), "%s:%u", ip_buf, port);
}
n = snprintf(request, sizeof(request),
"GET / HTTP/1.1\r\n"
"Host: %s\r\n"
"User-Agent: Mozilla/5.0 (X11; Linux x86_64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/120.0.0.0 Safari/537.36\r\n"
"Accept: text/html,application/xhtml+xml,"
"application/xml;q=0.9,image/avif,image/webp,"
"image/apng,*/*;q=0.8\r\n"
"Accept-Language: en-US,en;q=0.9\r\n"
"Accept-Encoding: gzip, deflate\r\n"
"Connection: close\r\n"
"\r\n",
host_buf);
if (n <= 0 || (size_t)n >= sizeof(request)) {
tcpapi_send(socket, http_hello, sizeof(http_hello) - 1, TCP__static);
return;
}
tcpapi_send(socket, request, (size_t)n, TCP__copy);
}
/*****************************************************************************
*****************************************************************************/
void
@@ -954,5 +1023,7 @@ struct ProtocolParserStream banner_http = {
http_selftest,
http_init,
http_parse,
0, /* cleanup */
http_transmit_hello,/* per-target request builder with Host: header */
};

3
src/proto-icmp.c
View File

@@ -110,6 +110,9 @@ handle_icmp(struct Output *out, time_t timestamp,
if (!echo_reply_dedup)
echo_reply_dedup = dedup_create();
out->src_ip_me = ip_me;
out->src_port_me = 0;
seqno_me = px[parsed->transport_offset+4]<<24
| px[parsed->transport_offset+5]<<16
| px[parsed->transport_offset+6]<<8

3
src/proto-sctp.c
View File

@@ -155,6 +155,9 @@ handle_sctp(struct Output *out, time_t timestamp,
if (offset + 16 > length)
return;
out->src_ip_me = parsed->dst_ip;
out->src_port_me = parsed->port_dst;
switch (px[offset + 12]) {
case 2: /* init ACK */
output_report_status(

20
src/proto-ssl.c
View File

@@ -55,6 +55,14 @@
#include <string.h>
#include <ctype.h>
#include <assert.h>
#include <stdlib.h>
#include <time.h>
#ifdef _WIN32
#include <process.h>
#define getpid _getpid
#else
#include <unistd.h>
#endif
@@ -1076,6 +1084,11 @@ ssl_init(struct Banner1 *banner1)
if (!randomized) {
int i;
randomized = 1;
/* Seed libc rand() before consuming it. Without srand() the C
* standard says rand() behaves as srand(1), so the "random"
* client_random would be a CONSTANT byte string across every run
* — just a new masscan fingerprint replacing upstream's. */
srand((unsigned)time(NULL) ^ ((unsigned)getpid() << 16));
for (i = 0; i < 32; i++) {
ssl_hello_template[11 + i] = (char)rand();
ssl_12_hello_template[11 + i] = (char)rand();
@@ -1502,5 +1515,10 @@ struct ProtocolParserStream banner_ssl = {
ssl_parse_record,
0,
0,
&banner_ssl_12,
/* Fallback chain to banner_ssl_12 disabled: that template's bytes
* (cipher list + extensions + key share) are still the original
* upstream masscan ClientHello and produce the original JA3 on
* retry. With this set to NULL, only the curl-shape TLS 1.2 hello
* is ever transmitted. */
0,
};

9
src/proto-udp.c
View File

@@ -32,13 +32,15 @@ default_udp_parse(struct Output *out, time_t timestamp,
{
ipaddress ip_them = parsed->src_ip;
unsigned port_them = parsed->port_src;
UNUSEDPARM(entropy);
if (length > 64)
length = 64;
out->src_ip_me = parsed->dst_ip;
out->src_port_me = parsed->port_dst;
output_report_banner(
out, timestamp,
ip_them, 17, port_them,
@@ -60,6 +62,9 @@ handle_udp(struct Output *out, time_t timestamp,
unsigned port_them = parsed->port_src;
unsigned status = 0;
out->src_ip_me = parsed->dst_ip;
out->src_port_me = parsed->port_dst;
/* Report "open" status regardless */
output_report_status(
out,

357
src/proto-x509-openssl.c Normal file
View File

@@ -0,0 +1,357 @@
// masscan - Developed by acidvegas in C (https://github.com/acidvegas)
// src/proto-x509-openssl.c
/*
* X.509 certificate -> JSON via OpenSSL.
*
* Used by the ndjson output backend to render parsed cert fields in place of
* the raw base64 DER blob.
*
* Input is the same bytes masscan accumulates into PROTO_X509_CERT /
* PROTO_X509_CACERT banouts: ASCII base64 of the cert's DER encoding, with
* no whitespace and (per masscan's banout_finalize_base64) trailing '=' pad.
*
* Output is a JSON object written directly to a FILE*, no leading/trailing
* whitespace, containing the cert fields the caller cares about. The caller
* is responsible for surrounding it with whatever key it wants
* (e.g. "cert":{...}).
*/
#include "proto-x509-openssl.h"
#include <openssl/x509.h>
#include <openssl/x509v3.h>
#include <openssl/asn1.h>
#include <openssl/bio.h>
#include <openssl/bn.h>
#include <openssl/evp.h>
#include <openssl/objects.h>
#include <openssl/pem.h>
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
/* ------------------------------------------------------------------------ */
/* base64 decode — masscan emits standard base64 (no URL-safe, with padding) */
/* ------------------------------------------------------------------------ */
static int
b64_value(unsigned char c)
{
if (c >= 'A' && c <= 'Z') return c - 'A';
if (c >= 'a' && c <= 'z') return c - 'a' + 26;
if (c >= '0' && c <= '9') return c - '0' + 52;
if (c == '+') return 62;
if (c == '/') return 63;
return -1;
}
static size_t
b64_decode(const unsigned char *src, size_t src_len, unsigned char *dst, size_t dst_cap)
{
size_t i, dst_len = 0;
int v, accum = 0, bits = 0;
for (i = 0; i < src_len; i++) {
unsigned char c = src[i];
if (c == '=' || c == '\r' || c == '\n' || c == ' ' || c == '\t')
continue;
v = b64_value(c);
if (v < 0) continue;
accum = (accum << 6) | v;
bits += 6;
if (bits >= 8) {
bits -= 8;
if (dst_len >= dst_cap) return dst_len;
dst[dst_len++] = (unsigned char)((accum >> bits) & 0xFF);
}
}
return dst_len;
}
/* ------------------------------------------------------------------------ */
/* JSON string emission */
/* ------------------------------------------------------------------------ */
static void
json_write_escaped(FILE *fp, const char *s, size_t len)
{
size_t i;
for (i = 0; i < len; i++) {
unsigned char c = (unsigned char)s[i];
switch (c) {
case '"': fputs("\\\"", fp); break;
case '\\': fputs("\\\\", fp); break;
case '\b': fputs("\\b", fp); break;
case '\f': fputs("\\f", fp); break;
case '\n': fputs("\\n", fp); break;
case '\r': fputs("\\r", fp); break;
case '\t': fputs("\\t", fp); break;
default:
if (c < 0x20)
fprintf(fp, "\\u%04x", c);
else
fputc((int)c, fp);
}
}
}
static void
json_write_string(FILE *fp, const char *s)
{
fputc('"', fp);
if (s)
json_write_escaped(fp, s, strlen(s));
fputc('"', fp);
}
/* ------------------------------------------------------------------------ */
/* helpers to render X.509 fields */
/* ------------------------------------------------------------------------ */
/* X509_NAME -> RFC2253 string into bio buffer. Returns malloc'd buf the
* caller must free, or NULL. */
static char *
name_to_string(X509_NAME *name)
{
BIO *bio;
BUF_MEM *mem;
char *out;
if (name == NULL) return NULL;
bio = BIO_new(BIO_s_mem());
if (bio == NULL) return NULL;
if (X509_NAME_print_ex(bio, name, 0,
XN_FLAG_RFC2253 & ~ASN1_STRFLGS_ESC_MSB) < 0) {
BIO_free(bio);
return NULL;
}
BIO_get_mem_ptr(bio, &mem);
out = (char *)malloc(mem->length + 1);
if (out) {
memcpy(out, mem->data, mem->length);
out[mem->length] = 0;
}
BIO_free(bio);
return out;
}
/* ASN1_TIME -> "YYYY-MM-DDTHH:MM:SSZ" into buf. Returns 1 on success. */
static int
asn1_time_to_iso(const ASN1_TIME *t, char *buf, size_t buf_len)
{
struct tm tm;
if (t == NULL) return 0;
memset(&tm, 0, sizeof(tm));
if (ASN1_TIME_to_tm(t, &tm) != 1) return 0;
snprintf(buf, buf_len, "%04d-%02d-%02dT%02d:%02d:%02dZ",
tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday,
tm.tm_hour, tm.tm_min, tm.tm_sec);
return 1;
}
/* Write public key info into a {algo, bits[, curve]} JSON object. */
static void
write_pubkey(FILE *fp, X509 *cert)
{
EVP_PKEY *pk = X509_get_pubkey(cert);
int id;
const char *algo = "unknown";
int bits = 0;
fputs(",\"key_algo\":", fp);
if (pk == NULL) {
fputs("null,\"key_bits\":0", fp);
return;
}
id = EVP_PKEY_id(pk);
bits = EVP_PKEY_bits(pk);
switch (id) {
case EVP_PKEY_RSA: algo = "rsa"; break;
case EVP_PKEY_DSA: algo = "dsa"; break;
case EVP_PKEY_EC: algo = "ec"; break;
case EVP_PKEY_ED25519: algo = "ed25519"; break;
case EVP_PKEY_ED448: algo = "ed448"; break;
default: {
const char *n = OBJ_nid2sn(id);
if (n) algo = n;
break;
}
}
json_write_string(fp, algo);
fprintf(fp, ",\"key_bits\":%d", bits);
if (id == EVP_PKEY_EC) {
char curve_buf[80] = {0};
size_t curve_len = 0;
if (EVP_PKEY_get_group_name(pk, curve_buf, sizeof(curve_buf), &curve_len) == 1
&& curve_len > 0) {
fputs(",\"key_curve\":", fp);
json_write_string(fp, curve_buf);
}
}
EVP_PKEY_free(pk);
}
/* Write a "san" array from the SAN extension, if present. */
static void
write_san(FILE *fp, X509 *cert)
{
GENERAL_NAMES *gens;
int i, n, written = 0;
gens = X509_get_ext_d2i(cert, NID_subject_alt_name, NULL, NULL);
fputs(",\"san\":[", fp);
if (gens == NULL) {
fputc(']', fp);
return;
}
n = sk_GENERAL_NAME_num(gens);
for (i = 0; i < n; i++) {
GENERAL_NAME *g = sk_GENERAL_NAME_value(gens, i);
const char *val = NULL;
char ipbuf[64];
int val_len = 0;
if (g == NULL) continue;
if (g->type == GEN_DNS || g->type == GEN_EMAIL || g->type == GEN_URI) {
val = (const char *)ASN1_STRING_get0_data(g->d.ia5);
val_len = ASN1_STRING_length(g->d.ia5);
} else if (g->type == GEN_IPADD) {
int l = ASN1_STRING_length(g->d.iPAddress);
const unsigned char *p = ASN1_STRING_get0_data(g->d.iPAddress);
if (l == 4) {
snprintf(ipbuf, sizeof(ipbuf), "%u.%u.%u.%u",
p[0], p[1], p[2], p[3]);
val = ipbuf;
val_len = (int)strlen(ipbuf);
} else if (l == 16) {
snprintf(ipbuf, sizeof(ipbuf),
"%02x%02x:%02x%02x:%02x%02x:%02x%02x:"
"%02x%02x:%02x%02x:%02x%02x:%02x%02x",
p[0], p[1], p[2], p[3], p[4], p[5], p[6], p[7],
p[8], p[9], p[10], p[11], p[12], p[13], p[14], p[15]);
val = ipbuf;
val_len = (int)strlen(ipbuf);
}
}
if (val == NULL || val_len <= 0) continue;
if (written) fputc(',', fp);
fputc('"', fp);
json_write_escaped(fp, val, (size_t)val_len);
fputc('"', fp);
written++;
}
GENERAL_NAMES_free(gens);
fputc(']', fp);
}
/* Write the cert serial as hex (no separators, lower-case). */
static void
write_serial(FILE *fp, X509 *cert)
{
ASN1_INTEGER *ai;
BIGNUM *bn = NULL;
char *hex = NULL;
fputs(",\"serial\":", fp);
ai = X509_get_serialNumber(cert);
if (ai) bn = ASN1_INTEGER_to_BN(ai, NULL);
if (bn) hex = BN_bn2hex(bn);
if (hex) {
char *p;
for (p = hex; *p; p++)
if (*p >= 'A' && *p <= 'F') *p = (char)(*p + 32);
json_write_string(fp, hex);
OPENSSL_free(hex);
} else {
fputs("null", fp);
}
if (bn) BN_free(bn);
}
/* ------------------------------------------------------------------------ */
/* public entry point */
/* ------------------------------------------------------------------------ */
int
x509_json_write_cert(FILE *fp, const unsigned char *b64, size_t b64_len)
{
unsigned char *der;
const unsigned char *p;
size_t der_len;
X509 *cert;
char timebuf[40];
char *subj = NULL;
char *iss = NULL;
int sig_nid;
const char *sig_name;
/* Generous upper bound: base64 is 4 chars per 3 bytes, so DER <= b64*3/4. */
der_len = (b64_len / 4) * 3 + 4;
der = (unsigned char *)malloc(der_len);
if (der == NULL) return -1;
der_len = b64_decode(b64, b64_len, der, der_len);
if (der_len < 16) {
free(der);
fputs("null", fp);
return -1;
}
p = der;
cert = d2i_X509(NULL, &p, (long)der_len);
if (cert == NULL) {
free(der);
fputs("null", fp);
return -1;
}
fputc('{', fp);
subj = name_to_string(X509_get_subject_name(cert));
fputs("\"subject\":", fp);
if (subj) json_write_string(fp, subj); else fputs("null", fp);
iss = name_to_string(X509_get_issuer_name(cert));
fputs(",\"issuer\":", fp);
if (iss) json_write_string(fp, iss); else fputs("null", fp);
fputs(",\"not_before\":", fp);
if (asn1_time_to_iso(X509_get0_notBefore(cert), timebuf, sizeof(timebuf)))
json_write_string(fp, timebuf);
else
fputs("null", fp);
fputs(",\"not_after\":", fp);
if (asn1_time_to_iso(X509_get0_notAfter(cert), timebuf, sizeof(timebuf)))
json_write_string(fp, timebuf);
else
fputs("null", fp);
write_san(fp, cert);
sig_nid = X509_get_signature_nid(cert);
sig_name = OBJ_nid2ln(sig_nid);
if (sig_name == NULL) sig_name = OBJ_nid2sn(sig_nid);
fputs(",\"sig_algo\":", fp);
if (sig_name) json_write_string(fp, sig_name); else fputs("null", fp);
write_pubkey(fp, cert);
write_serial(fp, cert);
fprintf(fp, ",\"version\":%ld", X509_get_version(cert) + 1);
fputc('}', fp);
free(subj);
free(iss);
X509_free(cert);
free(der);
return 0;
}

19
src/proto-x509-openssl.h Normal file
View File

@@ -0,0 +1,19 @@
// masscan - Developed by acidvegas in C (https://github.com/acidvegas)
// src/proto-x509-openssl.h
#ifndef PROTO_X509_OPENSSL_H
#define PROTO_X509_OPENSSL_H
#include <stdio.h>
#include <stddef.h>
/* Decode a base64-encoded DER certificate (the bytes masscan stores in a
* PROTO_X509_CERT / PROTO_X509_CACERT banout) and write a JSON cert object
* directly to fp. The output is a single object with no surrounding
* whitespace, e.g. {"subject":"...","issuer":"...","not_before":"...", ...}.
*
* On parse failure, writes the literal `null` and returns non-zero.
* Returns 0 on success. */
int
x509_json_write_cert(FILE *fp, const unsigned char *b64, size_t b64_len);
#endif

16
src/stack-tcp-api.h
View File

@@ -76,5 +76,21 @@ int
tcpapi_close(struct stack_handle_t *socket);
/**
* Render the target (remote) IP of this connection into a caller-provided
* buffer as a printable string (dotted-quad for v4, RFC 5952 for v6).
* NUL-terminates. Returns the number of bytes written excluding the NUL,
* or 0 on error. Used by transmit_hello callbacks that need to inject
* the target IP into protocol-level headers (HTTP Host, TLS SNI). */
unsigned
tcpapi_get_target_ip_string(struct stack_handle_t *socket,
char *buf, size_t buf_len);
/**
* Return the target (remote) port of this connection. */
unsigned
tcpapi_get_target_port(struct stack_handle_t *socket);
#endif

40
src/stack-tcp-core.c
View File

@@ -803,10 +803,15 @@ banner_flush(struct stack_handle_t *socket)
struct TCP_Control_Block *tcb = socket->tcb;
struct BannerOutput *banout;
/* Go through and print all the banners. Some protocols have
/* Go through and print all the banners. Some protocols have
* multiple banners. For example, web servers have both
* HTTP and HTML banners, and SSL also has several
* HTTP and HTML banners, and SSL also has several
* X.509 certificate banners */
/* Stamp the source IP/port onto the output side-channel so ndjson can
* record which source the probe actually went out from (when
* --source-port is a range, this is the specific port that fired). */
tcpcon->out->src_ip_me = tcb->ip_me;
tcpcon->out->src_port_me = tcb->port_me;
for (banout = &tcb->banout; banout != NULL; banout = banout->next) {
if (banout->length && banout->protocol) {
tcpcon->report_banner(
@@ -821,6 +826,9 @@ banner_flush(struct stack_handle_t *socket)
banout->length);
}
}
/* Signal end-of-target so ndjson can flush any aggregated TLS state
* (SSL3 cipher + parsed X509 cert) as a single combined record. */
output_flush_target(tcpcon->out);
/*
* Free up all the banners.
@@ -1821,6 +1829,34 @@ tcpapi_close(struct stack_handle_t *socket) {
return 0;
}
unsigned
tcpapi_get_target_ip_string(struct stack_handle_t *socket,
char *buf, size_t buf_len)
{
struct TCP_Control_Block *tcb;
ipaddress_formatted_t fmt;
size_t n;
if (socket == NULL || socket->tcb == NULL || buf == NULL || buf_len == 0)
return 0;
tcb = (struct TCP_Control_Block *)socket->tcb;
fmt = ipaddress_fmt(tcb->ip_them);
n = strlen(fmt.string);
if (n >= buf_len) n = buf_len - 1;
memcpy(buf, fmt.string, n);
buf[n] = 0;
return (unsigned)n;
}
unsigned
tcpapi_get_target_port(struct stack_handle_t *socket)
{
struct TCP_Control_Block *tcb;
if (socket == NULL || socket->tcb == NULL) return 0;
tcb = (struct TCP_Control_Block *)socket->tcb;
return tcb->port_them;
}
static bool

7
src/templ-pkt.c
View File

@@ -126,8 +126,13 @@ mimicry_select_os(void)
if (picked) return;
picked = 1;
/* Seed from time+pid; we only need fingerprint variance, not crypto. */
/* Seed from time+pid; we only need fingerprint variance, not crypto.
* srand() also seeds the libc rand() used for TLS/DTLS client_random
* and IP-ID — without this they would emit a constant sequence across
* runs, giving every masscan invocation the SAME static TLS random
* (just a new fingerprint instead of upstream's). */
r = (unsigned)time(NULL) ^ ((unsigned)getpid() << 16);
srand(r);
g_masscan_mimic_os = r & 1;
if (g_masscan_mimic_os == 0) {