👋
AI has been all the rage for the last few years. It even got a mention at a local textile art festival.
But I haven't really experimented much with it that much. I've used code generation tools at work such as Github Copilot, AI summaries in Brave search, and AI to transcribe audio files . But I'm curious to see what else I can do.
So I spent a couple of hours on that today.
This isn't meant as a guide (there are better ways to do all this!), but I'll share what I did to get started.
I found Ollama - an open source tool to run various AI models locally.
Naturally, my first step was to put this inside docker.
docker run -ti -p 11434:11434 ubuntu
apt update
apt upgrade
apt install curl
curl -fsSL https://ollama.com/install.sh | sh
Now that I have a docker container set up I can run Ollama. Since it runs in the foreground, I also installed screen to have multiple terminals to play with ( screen quickstart ).
apt install screen
screen
In one terminal:
OLLAMA_HOST=0.0.0.0:11434 ollama serve
And in another terminal:
ollama run gemma:7b
Ollama has a library of models to pick from. I found gemma:7b works pretty well and requires just under 10GB of RAM.
Also note that I'm not running anything on the GPU, so performance will suffer. I'm not worried about that for now.
From there I have a couple of options. I can chat with the model in the second window - a text prompt is opened where I can type a prompt and get a response.
Alternatively, I can use Ollama's API to generate responses over HTTP (this is my favourite!). This is just begging me to use it in some scripts ✨
curl http://localhost:11434/api/generate -d '{
"model": "gemma:7b",
"stream":false,
"prompt":"What is the meaning of life?"
}'
Now that I have a basic MVP up and running, I'm thinking about where to go from here.
Maybe I'll use it to summarise a bunch of my voice memos or other notes. Maybe I'll make a game with it 🤔. Maybe I'll even ask it for more ideas on how to use it.
I'm not sure yet. I'm still playing.
One thing I won't do is have it write for me. It's tempting, right? Just give it an idea for a blog post, have it generate something, and publish it - would easily meet my consistency goals that way!
But that goes completely against the point of this website. I write because I enjoy writing, not because I'm trying to generate a certain amount of content. So having an AI write stuff in my name makes no sense.
That being said, I may include AI content on my website at one point or another, but it will be clearly labeled as such 🪧
See you tomorrow-ish 👋