Since we just integrated Anthropic’s models into PromptHub, I thought it would be good timing to put together a handbook of best practices and other information.
The focus will mostly be on Claude 2 given its superior performance.
What is Claude 2?
Claude 2 Anthropic’s most performant model, comparable to GPT-4 for OpenAI. Its most notable feature is its 100k context window. To put that in perspective, 'The Great Gatsby,' is 72k tokens long.
This makes Claude 2 a great option when your prompts need a lot of tokens. Maybe you're generating long chapters, maybe you're feeding in a lot of data.
You can try out Claude 2 in a chat window (like ChatGPT) at Claude.ai
Is Claude 2 free?
You can try out Claude 2 for free via a chat window like ChatGPT at claude.ai. Claude's API pricing is listed below. I also put together a quick comparison to GPT-4. Interestingly Claude 2 is much cheaper.
GPT-4 (OpenAI) costs per 1,000 tokens:
- Cost of Prompt: $0.03
- Cost of Completion: $0.06
Claude 2 (Anthropic) costs per 1,000 tokens:
- Cost of Prompt: $0.01102
- Cost of Completion: $0.03268
Here's an example breakdown between the 2 models with a prompt of 200 tokens and an output of 1,000 tokens.
Best practices for prompt engineering with Anthropic's models
Anthropic ran two experiments to evaluate prompting techniques for optimizing Claude’s ability to accurately recall specific information from large (Government) documents.
The documents were roughly 75,000 and 90,000 tokens long. They examined prompt performance across a few situtions:
- Adding examples from the document
- Adding unrelated examples
- Using a “scratchpad”
The results
Takeaways
- Relevant examples plus the scratchpad led to the best performance
- Instructions should be placed at the end of the prompt, after the long document. This increases the likelihood that the model remembers what its supposed to do.
- The scratchpad is helpful in helping the model recall relevant information to accomplish the task. This introduces some latency, but faster models like Claude 1.2 shouldn’t be affected too much.
- Generic examples don’t improve performance. Make sure your examples are contextually relevant.
Link to the full study
Here are a few other best practices, from Anthropic themselves (source):
- Communicate with Claude using natural language
- Give examples of your desired output as part of your prompt
- Use XML <tags> to focus its attention and divide sections
- Feed Claude the starting words of your desired output using the “Assistant:” notation
- Break down complex tasks step-by-step and ask Claude to recite the details of the task
A lot of prompting best practices are the same across models. For my info on best practices, check out out guide: 10 Best Practices for Prompt Engineering with Any Model
Anthropic Parameters
Anthropic shares many of the same parameters as OpenAI. We put together a whole run down on OpenAI's parameters, which you can check out here.
Max Tokens (integer)
Max tokens set the absolute limit on the number of tokens to generate.The model may stop before reaching this maximum.
Stop sequences (array of strings)
Sequences of text that will cause the model to stop generating text.
Temperature (number between 0-1)
The amount of randomness in the model's outputs. The higher the temperature, the more creative the response will be.
Top_p (number)
Top_p sets a threshold for token probability, allowing tokens to be selected as the next token in the sequence if their cumulative probability is below this threshold.
Don't edit Top_p and temperature at the same time.
Top_k (integer)
Only sample from the top K options for each subsequent token. Used to remove low probability responses.
You can also use metadata and stream to pass data with your request or to receive streamed responses via server-sent events when integrating Anthropic's models into your product.
Putting Claude 2’s Long Context Window to work with a PromptHub Form
Let’s say we operate an apartment services company in New York City. We help tenants and landlords with issues that may arise in their building (burst pipes, rats, etc). Part of our job is being knowledgable of the local laws to accurately determine liability when things go wrong.
Objective
We want to create and embed a PromptHub form, powered by Claude 2, onto our site as a lead magnet. This will allow our website visitors to input their property-related questions and get instant legal information regarding who is on the hook for repairs.
Claude 2 is perfect for this type of task because we will need to search through various, long, documents to find answers to specific questions.
Step 1: Gather information
First we will gather the necessary documentation, including local ordinances, building codes, local laws etc.
We’ll create a variable for each specific document ( e.g., {{Manhattan_Plumbing_Codes}} ) and simply copy and paste the entire text content into it.
Step 2: Write the prompt
Next, we’ll write a quick prompt that will power the form. The prompt will consider the user’s question and then read through the documentation to find the specific answer.
Step 3: Configure the Form and deploy!
Next, we'll configure the form by selecting which variables should be editable and adding instructions and our branding.
Once it is looking good, we can flip the switch to deploy it, grab the embed code and it’s ready to go.
Conclusion
Claude 2’s 100k context window opens up a ton of opportunities for companies and individuals using AI. There are some best practices to keep in mind, but if you’ve been working with OpenAI’s models, your prompts will stay pretty similar.
Whether you’re patching together dense legal documents, or leveraging your own data, the combination of Claude 2’s extensive context window and PromptHub’s user-friendly forms can enable you to build powerful AI applications, fast.
