Michael Limberger

Your First Vision Request

Ask the model to look

Time to put it together and actually ask a vision model to describe an image. We use curl, a command-line tool for making web requests.

The Ollama API

Ollama provides an HTTP API. You send a POST request to:

http://localhost:11434/api/generate

The request body is JSON containing:

Field	What it is
`model`	Which AI model to use
`prompt`	The question or instruction (text)
`images`	An array of Base64-encoded images
`stream`	Whether to stream the response. We use false

The JSON shape

Show me. A complete request.

{
  "model": "ministral",
  "prompt": "What do you see?",
  "images": ["base64_encoded_data_here..."],
  "stream": false
}

The images field is an array (a list) because you could theoretically send multiple images. For our purposes, we send one at a time.

Encode the image

First, save the encoded image to a shell variable:

Show me.

IMAGE_B64=$(base64 -i yourimage.png | perl -pe's~\s~~g')

You will not see any output, but the variable now contains the encoded image. Verify with:

echo ${#IMAGE_B64}

You should see a large number. If it is over 800,000, resize the image first with sips.

Send the request

Show me.

curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ministral",
    "prompt": "Describe what you see in this image.",
    "images": ["'"$IMAGE_B64"'"],
    "stream": false
  }'

Symbol map: -s is silent (no progress bar). http://localhost:11434 is Ollama's address. /api/generate is the endpoint for generating responses. -H sets a header, here telling the server we are sending JSON. -d is the data to send.

The tricky part is '"$IMAGE_B64"'. That is shell syntax to insert our variable into the JSON string. The sequence breaks out of the single-quoted JSON, inserts the variable, and goes back in.

Parse the response

The response comes back as JSON:

{
  "model": "ministral",
  "response": "The image shows a woman with long brown hair...",
  "done": true,
  "total_duration": 12345678
}

The part we care about is the response field. We extract it using jq, a command-line JSON query tool.

Show me.

curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ministral",
    "prompt": "Describe what you see in this image.",
    "images": ["'"$IMAGE_B64"'"],
    "stream": false
  }' | jq -r '.response'

jq -r '.response' extracts just the response text. The -r flag gives raw output without quotes.

If you do not have jq: on macOS, brew install jq. On Ubuntu, sudo apt install jq.

The whole sequence

Show me. Encode, then ask.

IMAGE_B64=$(base64 -i yourimage.png | perl -pe's~\s~~g')

curl -s http://localhost:11434/api/generate \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ministral",
    "prompt": "Describe what you see in this image in 2-3 sentences.",
    "images": ["'"$IMAGE_B64"'"],
    "stream": false
  }' | jq -r '.response'

Try different images. Notice how the model describes various subjects, styles, and compositions.