Commit Graph

19 Commits

Author SHA1 Message Date
Dionysus 654e4a8667
Zone file ingestion script now uses the same sentinal value as masscans ingestion, set document id as the domain name to allow updating records if they exist 2024-03-06 14:12:27 -05:00
Dionysus 84f124b23d
Masscan ingestion script updated to use ip:port as the document id to allow updating records that already exist. Added a sentinal value to trigger an EOF when using --watch with FIFO's 2024-03-06 13:26:45 -05:00
Dionysus 5a45be5c60
Updated cause I am OCD about spaces and formatting 2024-03-05 22:29:31 -05:00
Dionysus 598552d34f
Introduction paragraph descriving the project overview added, updated roadmap 2024-03-05 22:26:42 -05:00
Dionysus a4b89e6e5a
Asyncronous refactorization pushed as main version 💯 2024-03-05 22:19:11 -05:00
Dionysus ed547a27f4
Added a recursive JSON explorer to remove empty or nulled keys from a dictionary 2024-03-05 22:15:55 -05:00
Dionysus 4cf976aada
Asyncronous refactorization of the codebase is complete, testing & metrics and then it will be production ready 2024-03-05 21:40:34 -05:00
Dionysus b6fb68ba3a
Asyncronous developed mon ERIS is complete, need to refactor ingestion helpers before pushing this as the main version 2024-03-05 16:47:11 -05:00
Dionysus 31287a99d2
Added certificate transparency monitoring via websockets for ingestion 2024-03-05 11:52:06 -05:00
Dionysus be4938ee6e
Added certificate transparency monitoring via websockets for ingestion 2024-03-05 11:48:53 -05:00
Dionysus c05c48f3fe
Started asyncronous implementation of bulk streaming data, altered ERIS defaults, etc 2024-03-04 17:44:09 -05:00
Dionysus 2ff7ddc869
Added an asyncronous sniffer patch in helpers 2024-02-02 13:21:59 -05:00
Dionysus 1ee71868ec
Ingestion agents are now modular, FIFO live ingestion added 2024-02-02 00:11:18 -05:00
Dionysus c105db705d
Updated README, copied over consistencies across the ingestors, docstring updates to reflect on new arguments 2024-01-27 04:28:30 -05:00
Dionysus 88e0dbfea8
Added parallel bulk uploading, error handling, sniffing nodes for discovery, dynamic batch sizes, and more 2024-01-27 01:13:27 -05:00
Dionysus c481edc883
Added parallel bulk uploading, error handling, sniffing nodes for discovery, dynamic batch sizes, and more 2024-01-27 01:13:11 -05:00
Dionysus b14038a335
Updated mirrors 2024-01-20 21:37:27 -05:00
Dionysus 3ff233a991
Updated README, fixed issue using the wrong domain in records for zone file ingestion (woops) 2024-01-20 10:53:55 -05:00
Dionysus 14b6d1c88a
Initial commit 2024-01-20 02:04:50 -05:00