Commit Graph

16 Commits

Author SHA1 Message Date
de3878ef6b
Removed _id from certstream yield and renamed ingest_certs.py to ingest_certstream.py 2024-03-11 22:46:48 -04:00
681e3bd788
Fixed issue with ingest_certs and the ingestion function signature. Simple placeholder argument (un-used) added to maintain function uniformity 2024-03-07 23:33:20 -05:00
45f878285c
Testing function added to every ingestor to debug directly. No more --dry-run needed. 2024-03-07 23:31:30 -05:00
9c6beb71ce
MassDNS ingestion script now caches the previous record to support IP addresses that yield more than one PTR record (field turned into a list when +1). Records will now upsert so MassDNS can be streaming into ES 24/7= 2024-03-07 21:57:10 -05:00
5a45be5c60
Updated cause I am OCD about spaces and formatting 2024-03-05 22:29:31 -05:00
598552d34f
Introduction paragraph descriving the project overview added, updated roadmap 2024-03-05 22:26:42 -05:00
4cf976aada
Asyncronous refactorization of the codebase is complete, testing & metrics and then it will be production ready 2024-03-05 21:40:34 -05:00
b6fb68ba3a
Asyncronous developed mon ERIS is complete, need to refactor ingestion helpers before pushing this as the main version 2024-03-05 16:47:11 -05:00
31287a99d2
Added certificate transparency monitoring via websockets for ingestion 2024-03-05 11:52:06 -05:00
c05c48f3fe
Started asyncronous implementation of bulk streaming data, altered ERIS defaults, etc 2024-03-04 17:44:09 -05:00
1ee71868ec
Ingestion agents are now modular, FIFO live ingestion added 2024-02-02 00:11:18 -05:00
c105db705d
Updated README, copied over consistencies across the ingestors, docstring updates to reflect on new arguments 2024-01-27 04:28:30 -05:00
c481edc883
Added parallel bulk uploading, error handling, sniffing nodes for discovery, dynamic batch sizes, and more 2024-01-27 01:13:11 -05:00
b14038a335
Updated mirrors 2024-01-20 21:37:27 -05:00
3ff233a991
Updated README, fixed issue using the wrong domain in records for zone file ingestion (woops) 2024-01-20 10:53:55 -05:00
14b6d1c88a
Initial commit 2024-01-20 02:04:50 -05:00