23 Commits

All Branches

Search

Author	SHA1	Message	Date
Dionysus	b15b3d8241	OCD about formatting again	2024-03-06 15:07:52 -05:00
Dionysus	fd617e3c9d	Certstream ingestion improved, still need to test stripping nulls and refactor the index mapping so we only store whats needed	2024-03-06 14:38:34 -05:00
Dionysus	90d6260b99	Updated massdns ingestion script with sentinal value checking and using the ip address as the document id	2024-03-06 14:33:21 -05:00
Dionysus	cba51ca2dd	Added a note about NSEC3 hash cracking for zone file data	2024-03-06 14:16:05 -05:00
Dionysus	654e4a8667	Zone file ingestion script now uses the same sentinal value as masscans ingestion, set document id as the domain name to allow updating records if they exist	2024-03-06 14:12:27 -05:00
Dionysus	84f124b23d	Masscan ingestion script updated to use ip:port as the document id to allow updating records that already exist. Added a sentinal value to trigger an EOF when using --watch with FIFO's	2024-03-06 13:26:45 -05:00
Dionysus	5a45be5c60	Updated cause I am OCD about spaces and formatting	2024-03-05 22:29:31 -05:00
Dionysus	598552d34f	Introduction paragraph descriving the project overview added, updated roadmap	2024-03-05 22:26:42 -05:00
Dionysus	a4b89e6e5a	Asyncronous refactorization pushed as main version 💯	2024-03-05 22:19:11 -05:00
Dionysus	ed547a27f4	Added a recursive JSON explorer to remove empty or nulled keys from a dictionary	2024-03-05 22:15:55 -05:00
Dionysus	4cf976aada	Asyncronous refactorization of the codebase is complete, testing & metrics and then it will be production ready	2024-03-05 21:40:34 -05:00
Dionysus	b6fb68ba3a	Asyncronous developed mon ERIS is complete, need to refactor ingestion helpers before pushing this as the main version	2024-03-05 16:47:11 -05:00
Dionysus	31287a99d2	Added certificate transparency monitoring via websockets for ingestion	2024-03-05 11:52:06 -05:00
Dionysus	be4938ee6e	Added certificate transparency monitoring via websockets for ingestion	2024-03-05 11:48:53 -05:00
Dionysus	c05c48f3fe	Started asyncronous implementation of bulk streaming data, altered ERIS defaults, etc	2024-03-04 17:44:09 -05:00
Dionysus	2ff7ddc869	Added an asyncronous sniffer patch in helpers	2024-02-02 13:21:59 -05:00
Dionysus	1ee71868ec	Ingestion agents are now modular, FIFO live ingestion added	2024-02-02 00:11:18 -05:00
Dionysus	c105db705d	Updated README, copied over consistencies across the ingestors, docstring updates to reflect on new arguments	2024-01-27 04:28:30 -05:00
Dionysus	88e0dbfea8	Added parallel bulk uploading, error handling, sniffing nodes for discovery, dynamic batch sizes, and more	2024-01-27 01:13:27 -05:00
Dionysus	c481edc883	Added parallel bulk uploading, error handling, sniffing nodes for discovery, dynamic batch sizes, and more	2024-01-27 01:13:11 -05:00
Dionysus	b14038a335	Updated mirrors	2024-01-20 21:37:27 -05:00
Dionysus	3ff233a991	Updated README, fixed issue using the wrong domain in records for zone file ingestion (woops)	2024-01-20 10:53:55 -05:00
Dionysus	14b6d1c88a	Initial commit	2024-01-20 02:04:50 -05:00