{"repo":"aielte-research/HackSynth","free":true,"listed":false,"github":"https://github.com/aielte-research/HackSynth","clone":"git clone https://github.com/aielte-research/HackSynth.git","description":"LLM Agent and Evaluation Framework for Autonomous Penetration Testing","language":"Python","stars":314,"topics":["ai","ctf","ctf-tools","cybersecurity","llms","penetration-testing","autonomous-pentesting"],"license":"AGPL-3.0","category":"security-tools","readme_excerpt":"HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testing The paper can be found on arXiv. Introduction We introduce HackSynth, a novel Large Language Model (LLM)-based agent capable of autonomous penetration testing. HackSynth's dual-module architecture includes a Planner and a Summarizer, which enable it to generate commands and process feedback iteratively. To benchmark HackSynth, we propose two new Capture The Flag (CTF)-based benchmark sets utilizing the popular platforms PicoCTF and OverTheWire. These benchmarks include two hundred challenges across diverse domains and difficulties, providing a standardized framework for evaluating LLM-based penetration testing agents. Using the repository - You will have to create a Hugging Face and a Neptune.ai account - Copy your API keys to the .env file, and set the desired CUDA devices, based on the .env example - Set up the PicoCTF benchmark - Set up the OverTheWire benchmark - Start the HackSynth Agent - Install the environment: - Start the benchmark with the following: The benchmark.json should be one of the generated benchmark solved.json files, or an equivalently structured file. The configuration files used by us for the measurements in the paper are also available in the configs folder. How to Cite If you use this code in your work or research, please cite the corresponding paper: Contributors - Lajos Muzsai (muzsailajos@protonmail.com) - David Imolai (david@imol.ai) - András Lukács (andras.lukacs@ttk","default_branch":null,"files":null,"tree":[],"storefront":"/r/aielte-research","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/aielte-research/HackSynth/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}