Python LLM Package Cheatsheet

2024-06-20

Install

pipx install llm

CLI Basics

llm -m <model_name> -d <data_path> -o <output_path>
llm -m orca-mini-3b-gguf2-q4_0 '3 names for a pet pineapple'
1. Spiky Sammy
2. Fruitlet Foster
3. Sweetie Pie

Using GPT4All Models

llm install llm-gpt4all
llm -m orca-2-7b '3 names for a pet pineapple'
...
1. Pini
2. Pinny
3. Ananas

• 1GB RAM: all-MiniLM-L6-v2-f16, all-MiniLM-L6-v2, nomic-embed-text-v1
• 4GB RAM: orca-mini-3b-gguf2-q4_0, replit-code-v1_5-3b-newbpe-q4_0, Phi-3-mini-4k-instruct, starcoder-newbpe-q4_0
• 8GB RAM: mpt-7b-chat, orca-2-7b, rift-coder-v0-7b-q4_0, mpt-7b-chat-newbpe-q4_0, em_german_mistral_v01, mistral-7b-instruct-v0, ghost-7b-v0, Nous-Hermes-2-Mistral-7B-DPO, mistral-7b-openorca, gpt4all-falcon-newbpe-q4_0, Meta-Llama-3-8B-Instruct
• 16GB RAM: gpt4all-13b-snoozy-q4_0, wizardlm-13b-v1, orca-2-13b, nous-hermes-llama2-13b

Using Claude Models

llm install llm-claude-3
llm keys set claude
llm -m claude-3-haiku '3 names for a pet pineapple'
...
1. Piney
2. Prickles
3. Spiky

Ustng OpenAI Models

# No `llm install` needed for OpenAI models, it's pre-installed
llm keys set openai
llm -m 4o '3 names for a pet pineapple'

OpenAI Chat: 3.5, 3.5-16k, 4, 4-32k, gpt-4-1106-preview, gpt-4-0125-preview, gpt-4-turbo-2024-04-09, 4-turbo, 4o

OpenAI Completion: 3.5-instruct

Working With SQLite Chat Logs

llm logs path
datasette "$(llm logs path)"