SSE-based Server and Client-Streamlit-App

Image recognition tool on top of the MCP protocol. This project is designed to provide a simple and efficient way to recognize images using a server-client architecture.

Why?

MCP server can now be some running process that agents (clients) connect to, use, and disconnect from whenever and wherever they want. In other words, an SSE-based server and clients can be decoupled processes (potentially even, on decoupled nodes). This is different and better fits "cloud-native" use-cases compared to the STDIO-based pattern where the client itself spawns the server as a subprocess.

Setup

Install the required packages and the MCP server and client:

# Installing Node.js on Linux
sudo apt install nodejs npm

# Installing Node.js on Mac
brew install nodejs npm

# Installing mcp
npm install -g mcp-framework
conda install -c conda-forge mcp 

# Cloning the repo
git clone https://github.com/DrBenjamin/imagerecog

Usage

Test bytes for an image to test on MCP Inspector or in VS Code Copilot Chat:

"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR4nGNgYAAAAAMAAWgmWQ0AAAAASUVORK5CYII="

Running the MCP server and client:

# Creating a conda environment using the environment.yml file
conda env create -f environment.yml

# Activating the conda environment
conda activate imagerecog

# 1. Running the MCP dev server
mcp dev src/server.py

# 2. Running the MCP server
python src/server.py

# 3. Running the Streamlit app
python -m streamlit run app.py

# or the run script
sudo chmod 755 run.sh
./run.sh
lsof -i :6274
lsof -i :8080
lsof -i :8501

Configuration

Change configuration and add the OpenAI API key in the .streamlit/st.secrets.toml file:

# LLM Provider
LLM_LOCAL = "False"  # `False` for local Ollama model, `True` for OpenAI API

# MCP API
[MCP]
MCP_URL = "http://127.0.0.1:8080"
MCP_SYSTEM_PROMPT = "<system prompt for image recognition>"
MCP_USER_PROMPT = "<user prompt for image recognition>"

# Ollama API
[OLLAMA]
OLLAMA_URL = "http://127.0.0.1:11434"
OLLAMA_MODEL = "<ollama model>" # e.g. llava or "llama3.2-vision"

# OpenAI API
[OPENAI]
OPENAI_API_KEY = "sk-..."
OPENAI_MODEL = "<model>" # e.g. "o4-mini" or "gpt-4.1" or "gpt-4o" or "gpt-4-turbo"

Ollama

To install und run the Ollama model, use the following command:

# Running the Ollama service
# Linux
systemctl start ollama
# or Mac
brew services start ollama

# Running the model
ollama run llama3.2-vision

# Sharing the models between Ollama and LM Studio
# https://smcleod.net/2024/03/llamalink-ollama-to-lm-studio-llm-model-linker/
go install github.com/sammcj/llamalink@latest
llamalink

Docker

To use the Docker for MCP hosting, use the following commands:

# Build the docker image
docker build -t <docker hub user name>/imagerecog .

# Login to Docker Hub
docker login

# Tagging the image (https://hub.docker.com/repositories/drbenjamin)
docker tag <docker hub user name>/imagerecog <docker hub user name>/imagerecog:latest

# Push the image to the registry
docker push <docker hub user name>/imagerecog:latest

Now the MCP docker can be added in VS Code or any other MCP client like Claude Desktop.