How to ensure my agent only returns a single letter code? – Prompting

by


I am frustrated with my assistant on GPT-3.5-Turbo. I attached a 2 file JSON field: 1) Questions 2) Category Code. When a message is submitted, I want the assistant to return a code from this list {A, B, C, D, E, G, H}, with H reserved for nothing found. But instead, I always get a long sentence reply, and then with the letter code, which is usually correct. It works better in GPT4, but the costs are much higher for what is a simple lookup.

Is there a way for me to give better instructions to the assistant? I would even be happy if the long answer was outputted, provided that the code is encapsulated with {A}, instead of “A”. Do you think the instructions can be improved?

  1. Purpose: I am a dispatcher assistant using the data in the uploaded file “[t2b.json]” to categorize user prompts.

  2. The Data: The file contains two fields: “Prompts” and “Category Code”. The Category Code is a single letter from the set {A, B, C, D, E, F, G} while question are typical questions from users.

  3. Define the Task: My task is to match the user’s question to the corresponding “Prompts” field in the file. If a match is found, I should return only the single-letter category code from that entry.

  4. Handle No Match: If no exact match is found, I should return the letter “H” to indicate “Nothing Found”.

  5. Response Format: Please ensure my responses are limited to single letters from the set {A, B, C, D, E, F, G, H} and avoid any additional text or prompts. Enclose the Category Code in brackets {}. e.g., {A}, {B}, {C}, {D}, {E}. {F}. {G}, {H}

  6. Example Usage:

Prompt: What training and onboarding support does Acme Company offer to new subscribers? Response: {E}

Prompt: Tell me a joke. Response: {H}



1 Like

Might be a good case for a smaller fine-tuned model. You’d need to create a dataset, but it would likely not need many examples…

Before you go that far, maybe try giving it a one-shot or even two-shot example in the system prompt OR as a user/assistant pair.

You could potentially save a lot of money by converting this into local code that used embeddings to match prompts.

Embeddings would only need to be retrieved for a library of “examples”, each of which could be pre-classified with your codes. Then you would simply compare your runtime query vector with these examples via vector search and return the associated classification of the closest example.



2 Likes



Source link

Leave a Comment