Hello, this is the official website for EB|
EBenchAttacker is a simple and easy-to-use tool that can evaluate the alignment ability of large language models(LLMs). You can easily integrate various open source or commercial models into EBenchAttacker, as long as you have the relevant access and computing resources. The development of the tool is supported by Beijing Advanced Innovation Center for Big Data and Brain Computing.
✨Update 2024.5.4: We have added experimental results for the LLaMA-3-8B-Instruct model~
EBenchAttacker
EBenchAttack has the following designs:
- ScenesEBenchAttacker considers scenarios including cybercrime, fraud, political sensitivity and so on. The design of the scene refers to some public terms of service of OpenAI, Meta, etc.
- EBench - Harmful Questions SetEBench is our specialized dataset designed for EBenchAttacker. It contains 1,000 harmful questions, 10 scenarios, each harmful question written in eight languages.
- Target ModelsEBenchAttacker integrated four open source models and three commercial models and can be easily expanded.
- Attack MethodsEBenchAttacker integrated Default Attack, GCG, PAIR and other attack methods to attack LLMs and collects experimental data.
- AnalysisAfter using EBenchAttacker to attack, we analyze the experimental results and present them.
By utilizing EBenchAttacker for the analysis of LLMs, we can gain a more comprehensive understanding of their security aspects. This enables us to refine the model in a manner that enhances its security and compliance. Additionally, our tool has the potential to inspire future research endeavors.