{"repo":"disclose/diosts","free":true,"listed":false,"github":"https://github.com/disclose/diosts","clone":"git clone https://github.com/disclose/diosts.git","description":"A Go scraper that validates security.txt files and outputs them in the disclose.io JSON format.","language":"Go","stars":23,"topics":["security-txt","scraper","golang","json"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"diosts Validate security.txt at Internet scale — a fast Go scraper for RFC 9116 that keeps the directory fresh. Part of the disclose.io Project — the open, vendor-neutral infrastructure for vulnerability disclosure. Browse the ecosystem → --- diosts The disclose.io security.txt scraper ( diosts ) takes a list of domains as the input, retrieves and validates the security.txt if available and outputs it in the disclose.io JSON format. Installation Prerequisites: - Go 1.22 or newer Option 1: Using go install (recommended) Option 2: From source Usage This will try and scrape the security.txt from the domains listed in domains.txt , with parallel threads (defaults to 8). Logging (with information on each of the domains in the input) will be written to diosts.log (because it's output to stderr ) and a JSON array of retrieved security.txt information in disclose.io format will be written to securitytxt.json . The -n or --non-compliant flag enables you to output the non-RFC-compliant security.txt files to a separate JSON file for further analysis and processing. For each input, the following URIs are tried, in order: 1. https:// /.well-known/security.txt 2. https:// /security.txt 3. http:// /.well-known/security.txt 4. http:// /security.txt Any non-fatal violations of the security.txt specification will be logged and tracked in the output. Supported Fields The tool supports all fields defined in RFC 9116 plus extensions: Field Required Description ------- ---------- ------------- Con","default_branch":null,"files":null,"tree":[],"storefront":"/r/disclose","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/disclose/diosts/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}