|
|
da065c6889
|
Added a testing function for calling the ingest_massdns.py file directly for debugging
|
2024-03-07 22:55:48 -05:00 |
|
|
|
3c98f8f1f1
|
MassDNS ingestion script now caches the previous record to support IP addresses that yield more than one PTR record (field turned into a list when +1). Records will now upsert so MassDNS can be streaming into ES 24/7=
|
2024-03-07 21:57:44 -05:00 |
|
|
|
9c6beb71ce
|
MassDNS ingestion script now caches the previous record to support IP addresses that yield more than one PTR record (field turned into a list when +1). Records will now upsert so MassDNS can be streaming into ES 24/7=
|
2024-03-07 21:57:10 -05:00 |
|
|
|
b15b3d8241
|
OCD about formatting again
|
2024-03-06 15:07:52 -05:00 |
|
|
|
fd617e3c9d
|
Certstream ingestion improved, still need to test stripping nulls and refactor the index mapping so we only store whats needed
|
2024-03-06 14:38:34 -05:00 |
|
|
|
90d6260b99
|
Updated massdns ingestion script with sentinal value checking and using the ip address as the document id
|
2024-03-06 14:33:21 -05:00 |
|
|
|
cba51ca2dd
|
Added a note about NSEC3 hash cracking for zone file data
|
2024-03-06 14:16:05 -05:00 |
|
|
|
654e4a8667
|
Zone file ingestion script now uses the same sentinal value as masscans ingestion, set document id as the domain name to allow updating records if they exist
|
2024-03-06 14:12:27 -05:00 |
|
|
|
84f124b23d
|
Masscan ingestion script updated to use ip:port as the document id to allow updating records that already exist. Added a sentinal value to trigger an EOF when using --watch with FIFO's
|
2024-03-06 13:26:45 -05:00 |
|
|
|
5a45be5c60
|
Updated cause I am OCD about spaces and formatting
|
2024-03-05 22:29:31 -05:00 |
|
|
|
598552d34f
|
Introduction paragraph descriving the project overview added, updated roadmap
|
2024-03-05 22:26:42 -05:00 |
|
|
|
a4b89e6e5a
|
Asyncronous refactorization pushed as main version 💯
|
2024-03-05 22:19:11 -05:00 |
|
|
|
ed547a27f4
|
Added a recursive JSON explorer to remove empty or nulled keys from a dictionary
|
2024-03-05 22:15:55 -05:00 |
|
|
|
4cf976aada
|
Asyncronous refactorization of the codebase is complete, testing & metrics and then it will be production ready
|
2024-03-05 21:40:34 -05:00 |
|
|
|
b6fb68ba3a
|
Asyncronous developed mon ERIS is complete, need to refactor ingestion helpers before pushing this as the main version
|
2024-03-05 16:47:11 -05:00 |
|
|
|
31287a99d2
|
Added certificate transparency monitoring via websockets for ingestion
|
2024-03-05 11:52:06 -05:00 |
|
|
|
be4938ee6e
|
Added certificate transparency monitoring via websockets for ingestion
|
2024-03-05 11:48:53 -05:00 |
|
|
|
c05c48f3fe
|
Started asyncronous implementation of bulk streaming data, altered ERIS defaults, etc
|
2024-03-04 17:44:09 -05:00 |
|
|
|
2ff7ddc869
|
Added an asyncronous sniffer patch in helpers
|
2024-02-02 13:21:59 -05:00 |
|
|
|
1ee71868ec
|
Ingestion agents are now modular, FIFO live ingestion added
|
2024-02-02 00:11:18 -05:00 |
|
|
|
c105db705d
|
Updated README, copied over consistencies across the ingestors, docstring updates to reflect on new arguments
|
2024-01-27 04:28:30 -05:00 |
|
|
|
88e0dbfea8
|
Added parallel bulk uploading, error handling, sniffing nodes for discovery, dynamic batch sizes, and more
|
2024-01-27 01:13:27 -05:00 |
|
|
|
c481edc883
|
Added parallel bulk uploading, error handling, sniffing nodes for discovery, dynamic batch sizes, and more
|
2024-01-27 01:13:11 -05:00 |
|
|
|
b14038a335
|
Updated mirrors
|
2024-01-20 21:37:27 -05:00 |
|
|
|
3ff233a991
|
Updated README, fixed issue using the wrong domain in records for zone file ingestion (woops)
|
2024-01-20 10:53:55 -05:00 |
|
|
|
14b6d1c88a
|
Initial commit
|
2024-01-20 02:04:50 -05:00 |
|
