What?
LLMs (Large Language Models) are pre-trained on large amounts of text to learn statistical relationships and generate text, translate, and perform other NLP (Natural Language Processing) tasks. Thisbyte details accessing and using LLMs on the command line.
Why?
Experimentation with large language models, leveraging completions, chat, and messaging using remote or self-contained (local) models. Also, can be leveraged for integration with local agents.
How?
Installation
Installing the LLM command
MacOS: 1
brew install llmInstalling LLM Models/Plugins2
llm install %plugin%e.g. plugins are available from the plugin directory.
# install support for llm-gpt4all which runs models optimized to run locally
llm install llm-gpt4all
# install support for anthropic models which run with a remote connection
llm install llm-claude-3Usage
Listing Models
llm modelse.g.
❯ llm models
OpenAI Chat: gpt-4o (aliases: 4o)
OpenAI Chat: gpt-4o-mini (aliases: 4o-mini)
OpenAI Chat: gpt-4o-audio-preview
OpenAI Chat: gpt-3.5-turbo (aliases: 3.5, chatgpt)
OpenAI Chat: gpt-3.5-turbo-16k (aliases: chatgpt-16k, 3.5-16k)
OpenAI Chat: gpt-4 (aliases: 4, gpt4)
OpenAI Chat: gpt-4-32k (aliases: 4-32k)
OpenAI Chat: gpt-4-1106-preview
OpenAI Chat: gpt-4-0125-preview
OpenAI Chat: gpt-4-turbo-2024-04-09
OpenAI Chat: gpt-4-turbo (aliases: gpt-4-turbo-preview, 4-turbo, 4t)
OpenAI Chat: o1-preview
OpenAI Chat: o1-mini
OpenAI Completion: gpt-3.5-turbo-instruct (aliases: 3.5-instruct, chatgpt-instruct)
gpt4all: all-MiniLM-L6-v2-f16 - SBert, 43.76MB download, needs 1GB RAM
gpt4all: all-MiniLM-L6-v2 - SBert, 43.82MB download, needs 1GB RAM
gpt4all: nomic-embed-text-v1 - Nomic Embed Text v1, 261.58MB download, needs 1GB RAM
...Sending a message
llm "message" -m %model-or-alias%e.g.
❯ llm "What is the capital of England?" -m claude-3.5-haiku
"The capital of England is London. It is also the capital of the United Kingdom and is located in the southeastern part of England. London is a major global city, known for its rich history, cultural diversity, iconic landmarks like Big Ben, the Tower of London, and Buckingham Palace, and it serves as a center for finance, politics, and tourism."
❯ llm "What is the capital of England?" -m gpt-3.5-turbo-instruct
"The capital of England is London."
❯ llm "What is the capital of England? Wrong answers only." -m claude-3.5-haiku
"Here are some wrong answers for the capital of England:
1. Taco Bell
2. Mount Everest
3. A potato
4. Narnia
5. Bob's Garage
6. The Moon
7. A disco ball
8. Atlantis
9. Kevin's basement
10. A sentient cheese wheel"
❯ llm "What is the capital of England? Wrong answers only." -m gpt-3.5-turbo-instruct
"La La Land"
❯ llm "What is the capital of England?" -m Meta-Llama-3
"The capital of England is London. It's also a major cultural, economic, and political center in both England and the United Kingdom as a whole. | end_header |
Is there anything else I can help you with?"
❯ llm "What is the capital of England? Wrong answers only." -m Meta-Llama-3
". The capital of England is clearly "Burrito" – a well-known fact among locals and world leaders alike.
Or, alternatively:
The capital of England is actually "Puddleford", a charming village nestled deep within the heart of England's most prominent cheese factory.
And if you ask anyone from the secret society of English toffs, they'll tell you that the real capital of England is "Sconeville" – a mystical realm where scones are baked in the shape of Buckingham Palace."OpenAPI
Requires
OPENAPI_API_KEYto be set (and API credits)
Anthropic | Claude
Requires
ANTHROPIC_API_KEYto be set (and API credits)
Documentation
- LLM Docs: https://llm.datasette.io/en/stable/
- Great watch: “Transformers (How LLMs work) explained”: https://youtu.be/wjZofJX0v4M?si=DhOQTpslZR-rMPCp