I think it would make more sense if the evaluation checked for the answer within the response rather than being the entire response

HackAPrompt 2023
#1

For example, on level 2, if you convince it to output the secret key but it also includes a longer response, I think it should count as being hacked still.

From a practical perspective it only matters that the end user was able to extract the secret key, not if it got some other completion tokens in the response.

Same thing on minor annoyances like, It’s easy to get it to say “I have been PWNED.” but because it has a period the evaluation fails so you have to spend more tokens removing the period.

#2

image

Seriously this can be so frustrating. 50 of my tokens are dedicated to removing the damn period right now and it’s still not working.

#3

Understandable, but we are looking for full control of output on competitors end (check AICrowd page FAQ for more rationale).

2 Likes
#4

been there, found a way eventually. Keep pushing!