vizionStreamer/SOCKET_API.md

# VizionStreamer Socket Control API

VizionStreamer can be controlled via a Unix Domain Socket interface. This allows external applications to configure camera parameters and stream settings at runtime.

**Copyright (c) 2025 Maik Jurischka**
Licensed under CC BY-NC-SA 4.0 - https://creativecommons.org/licenses/by-nc-sa/4.0/

## Socket Connection

- **Socket Path**: `/tmp/vizion_control.sock`
- **Protocol**: Unix Domain Socket (SOCK_STREAM)
- **Message Format**: JSON

## Command Format

All commands follow this JSON structure:

```json
{
  "command": "command_name",
  "params": {
    "param1": "value1",
    "param2": "value2"
  }
}
```

## Response Format

All responses follow this JSON structure:

**Success Response:**
```json
{
  "status": "success",
  "message": "Optional success message"
}
```

**Error Response:**
```json
{
  "status": "error",
  "message": "Error description"
}
```

## Available Commands

### 1. Get Available Formats

Retrieve all supported video formats.

**Command:**
```json
{
  "command": "get_formats"
}
```

**Response:**
```json
{
  "status": "success",
  "formats": [
    {
      "width": 1920,
      "height": 1080,
      "framerate": 30,
      "format": "YUY2"
    },
    {
      "width": 1280,
      "height": 720,
      "framerate": 60,
      "format": "MJPG"
    }
  ]
}
```

**Supported Formats:** YUY2, UYVY, NV12, MJPG, BGR, RGB

---

### 2. Set Video Format

Change the video format (resolution, framerate, pixel format).

**Note:** Cannot be changed while streaming is active.

**Command:**
```json
{
  "command": "set_format",
  "params": {
    "width": "1920",
    "height": "1080",
    "framerate": "30",
    "format": "YUY2"
  }
}
```

**Response:**
```json
{
  "status": "success",
  "message": "Format set successfully"
}
```

---

### 3. Start Streaming

Start video streaming from the camera.

**Command:**
```json
{
  "command": "start_stream"
}
```

**Response:**
```json
{
  "status": "success",
  "message": "Streaming started"
}
```

---

### 4. Stop Streaming

Stop video streaming.

**Command:**
```json
{
  "command": "stop_stream"
}
```

**Response:**
```json
{
  "status": "success",
  "message": "Streaming stopped"
}
```

---

### 5. Set GStreamer Pipeline

Configure the GStreamer pipeline for video output. This determines where and how the video stream is processed/displayed.

**Note:** Cannot be changed while streaming is active.

**Command:**
```json
{
  "command": "set_pipeline",
  "params": {
    "pipeline": "videoconvert ! x264enc ! rtph264pay ! udpsink host=192.168.1.100 port=5000"
  }
}
```

**Response:**
```json
{
  "status": "success",
  "message": "Pipeline set successfully"
}
```

**Common Pipeline Examples:**

1. **Display locally:**
   ```
   videoconvert ! autovideosink
   ```

2. **Stream over UDP (H.264):**
   ```
   videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000
   ```

3. **Stream over RTSP (requires gst-rtsp-server):**
   ```
   videoconvert ! x264enc ! rtph264pay name=pay0
   ```

4. **Save to file:**
   ```
   videoconvert ! x264enc ! mp4mux ! filesink location=/tmp/output.mp4
   ```

5. **Stream over TCP:**
   ```
   videoconvert ! x264enc ! h264parse ! mpegtsmux ! tcpserversink host=0.0.0.0 port=5000
   ```

6. **MJPEG over HTTP:**
   ```
   videoconvert ! jpegenc ! multipartmux ! tcpserversink host=0.0.0.0 port=8080
   ```

---

### 6. Get Status

Get current streaming status and pipeline configuration.

**Command:**
```json
{
  "command": "get_status"
}
```

**Response:**
```json
{
  "status": "success",
  "streaming": true,
  "pipeline": "videoconvert ! autovideosink"
}
```

---

### 7. Set Exposure

Configure camera exposure settings.

**Command:**
```json
{
  "command": "set_exposure",
  "params": {
    "mode": "manual",
    "value": "100"
  }
}
```

**Parameters:**
- `mode`: "auto" or "manual"
- `value`: Exposure value (only used in manual mode)

**Response:**
```json
{
  "status": "success",
  "message": "Exposure set successfully"
}
```

---

### 8. Set White Balance

Configure white balance settings.

**Command:**
```json
{
  "command": "set_whitebalance",
  "params": {
    "mode": "auto",
    "temperature": "4500"
  }
}
```

**Parameters:**
- `mode`: "auto" or "manual"
- `temperature`: Color temperature in Kelvin (only used in manual mode)

**Response:**
```json
{
  "status": "success",
  "message": "White balance set successfully"
}
```

---

### 9. Set Brightness

Adjust camera brightness.

**Command:**
```json
{
  "command": "set_brightness",
  "params": {
    "value": "50"
  }
}
```

**Response:**
```json
{
  "status": "success",
  "message": "Brightness set successfully"
}
```

---

### 10. Set Contrast

Adjust camera contrast.

**Command:**
```json
{
  "command": "set_contrast",
  "params": {
    "value": "32"
  }
}
```

**Response:**
```json
{
  "status": "success",
  "message": "Contrast set successfully"
}
```

---

### 11. Set Saturation

Adjust color saturation.

**Command:**
```json
{
  "command": "set_saturation",
  "params": {
    "value": "64"
  }
}
```

**Response:**
```json
{
  "status": "success",
  "message": "Saturation set successfully"
}
```

---

### 12. Set Sharpness

Adjust image sharpness.

**Command:**
```json
{
  "command": "set_sharpness",
  "params": {
    "value": "3"
  }
}
```

**Response:**
```json
{
  "status": "success",
  "message": "Sharpness set successfully"
}
```

---

### 13. Set Gamma

Adjust gamma correction.

**Command:**
```json
{
  "command": "set_gamma",
  "params": {
    "value": "100"
  }
}
```

**Response:**
```json
{
  "status": "success",
  "message": "Gamma set successfully"
}
```

---

### 14. Set Gain

Adjust camera gain.

**Command:**
```json
{
  "command": "set_gain",
  "params": {
    "value": "0"
  }
}
```

**Response:**
```json
{
  "status": "success",
  "message": "Gain set successfully"
}
```

---

### 15. Set eHDR Mode

Enable or disable eHDR (Enhanced High Dynamic Range) mode.

**Note:** eHDR features are only available on specific camera models: VCI-AR0821/AR0822, VCS-AR0821/AR0822, VLS3-AR0821/AR0822, VLS-GM2-AR0821/AR0822, and TEVS-AR0821/AR0822.

**Command:**
```json
{
  "command": "set_ehdr_mode",
  "params": {
    "mode": "0"
  }
}
```

**Parameters:**
- `mode`: "0" to enable eHDR, "1" to disable eHDR

**Response:**
```json
{
  "status": "success",
  "message": "eHDR mode set successfully"
}
```

---

### 16. Set eHDR Exposure Minimum

Set the minimum number of exposure frames for eHDR.

**Command:**
```json
{
  "command": "set_ehdr_exposure_min",
  "params": {
    "value": "1"
  }
}
```

**Parameters:**
- `value`: Minimum exposure frames (range: 1-4, default: 1)

**Response:**
```json
{
  "status": "success",
  "message": "eHDR exposure min set successfully"
}
```

---

### 17. Set eHDR Exposure Maximum

Set the maximum number of exposure frames for eHDR.

**Command:**
```json
{
  "command": "set_ehdr_exposure_max",
  "params": {
    "value": "4"
  }
}
```

**Parameters:**
- `value`: Maximum exposure frames (range: 1-4, default: 4)

**Response:**
```json
{
  "status": "success",
  "message": "eHDR exposure max set successfully"
}
```

---

### 18. Set eHDR Ratio Minimum

Set the minimum exposure ratio for eHDR.

**Command:**
```json
{
  "command": "set_ehdr_ratio_min",
  "params": {
    "value": "12"
  }
}
```

**Parameters:**
- `value`: Minimum exposure ratio (range: 1-128, default: 12)

**Response:**
```json
{
  "status": "success",
  "message": "eHDR ratio min set successfully"
}
```

---

### 19. Set eHDR Ratio Maximum

Set the maximum exposure ratio for eHDR.

**Command:**
```json
{
  "command": "set_ehdr_ratio_max",
  "params": {
    "value": "24"
  }
}
```

**Parameters:**
- `value`: Maximum exposure ratio (range: 1-128, default: 24)

**Response:**
```json
{
  "status": "success",
  "message": "eHDR ratio max set successfully"
}
```

---

### 20. Get eHDR Status

Retrieve all current eHDR settings.

**Command:**
```json
{
  "command": "get_ehdr_status"
}
```

**Response:**
```json
{
  "status": "success",
  "ehdr_mode": 0,
  "exposure_min": 1,
  "exposure_max": 4,
  "ratio_min": 12,
  "ratio_max": 24
}
```

---

## Usage Examples

### Complete Workflow Example

```bash
# 1. Set GStreamer pipeline for UDP streaming
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 2. Set video format
echo '{"command":"set_format","params":{"width":"1920","height":"1080","framerate":"30","format":"YUY2"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 3. Configure camera settings
echo '{"command":"set_exposure","params":{"mode":"auto"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
echo '{"command":"set_brightness","params":{"value":"50"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 3a. (Optional) Configure eHDR settings (for compatible cameras)
echo '{"command":"set_ehdr_mode","params":{"mode":"0"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
echo '{"command":"set_ehdr_exposure_min","params":{"value":"1"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
echo '{"command":"set_ehdr_exposure_max","params":{"value":"4"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
echo '{"command":"set_ehdr_ratio_min","params":{"value":"12"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
echo '{"command":"set_ehdr_ratio_max","params":{"value":"24"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 4. Start streaming
echo '{"command":"start_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 5. Check status
echo '{"command":"get_status"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 6. Stop streaming when done
echo '{"command":"stop_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
```

### GStreamer Pipeline Examples

```bash
# Stream to local display
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! autovideosink"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Stream over UDP (H.264)
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Save to MP4 file
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc ! mp4mux ! filesink location=/tmp/output.mp4"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# MJPEG HTTP server
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! jpegenc ! multipartmux ! tcpserversink host=0.0.0.0 port=8080"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
```

### Using `socat`

```bash
# Get available formats
echo '{"command":"get_formats"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Set video format
echo '{"command":"set_format","params":{"width":"1920","height":"1080","framerate":"30","format":"YUY2"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Start streaming
echo '{"command":"start_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Set exposure to auto
echo '{"command":"set_exposure","params":{"mode":"auto"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Set brightness
echo '{"command":"set_brightness","params":{"value":"50"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Get status
echo '{"command":"get_status"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Stop streaming
echo '{"command":"stop_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
```

### eHDR Control Examples

```bash
# Enable eHDR mode
echo '{"command":"set_ehdr_mode","params":{"mode":"0"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Disable eHDR mode
echo '{"command":"set_ehdr_mode","params":{"mode":"1"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Configure eHDR exposure range
echo '{"command":"set_ehdr_exposure_min","params":{"value":"1"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
echo '{"command":"set_ehdr_exposure_max","params":{"value":"4"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Configure eHDR ratio range
echo '{"command":"set_ehdr_ratio_min","params":{"value":"12"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
echo '{"command":"set_ehdr_ratio_max","params":{"value":"24"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Get current eHDR settings
echo '{"command":"get_ehdr_status"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
```

### Using `nc` (netcat with Unix socket support)

```bash
echo '{"command":"get_formats"}' | nc -U /tmp/vizion_control.sock
```

### Using Python

```python
import socket
import json

def send_command(command, params=None):
    sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    sock.connect('/tmp/vizion_control.sock')

    cmd = {"command": command}
    if params:
        cmd["params"] = params

    sock.send(json.dumps(cmd).encode())
    response = sock.recv(4096).decode()
    sock.close()

    return json.loads(response)

# Examples
print(send_command("get_formats"))
print(send_command("set_format", {
    "width": "1920",
    "height": "1080",
    "framerate": "30",
    "format": "YUY2"
}))
print(send_command("set_exposure", {"mode": "auto"}))
print(send_command("start_stream"))

# eHDR control examples (for compatible cameras)
print(send_command("set_ehdr_mode", {"mode": "0"}))  # Enable eHDR
print(send_command("set_ehdr_exposure_min", {"value": "1"}))
print(send_command("set_ehdr_exposure_max", {"value": "4"}))
print(send_command("set_ehdr_ratio_min", {"value": "12"}))
print(send_command("set_ehdr_ratio_max", {"value": "24"}))
print(send_command("get_ehdr_status"))  # Get current eHDR settings
```

### Using C++

```cpp
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <string>
#include <iostream>

std::string sendCommand(const std::string& command) {
    int sock = socket(AF_UNIX, SOCK_STREAM, 0);

    struct sockaddr_un addr;
    memset(&addr, 0, sizeof(addr));
    addr.sun_family = AF_UNIX;
    strcpy(addr.sun_path, "/tmp/vizion_control.sock");

    connect(sock, (struct sockaddr*)&addr, sizeof(addr));
    send(sock, command.c_str(), command.length(), 0);

    char buffer[4096];
    int bytesRead = recv(sock, buffer, sizeof(buffer) - 1, 0);
    buffer[bytesRead] = '\0';

    close(sock);
    return std::string(buffer);
}

// Example usage
int main() {
    std::cout << sendCommand(R"({"command":"get_formats"})") << std::endl;
    std::cout << sendCommand(R"({"command":"set_brightness","params":{"value":"50"}})") << std::endl;

    // eHDR control examples (for compatible cameras)
    std::cout << sendCommand(R"({"command":"set_ehdr_mode","params":{"mode":"0"}})") << std::endl;
    std::cout << sendCommand(R"({"command":"set_ehdr_exposure_min","params":{"value":"1"}})") << std::endl;
    std::cout << sendCommand(R"({"command":"set_ehdr_exposure_max","params":{"value":"4"}})") << std::endl;
    std::cout << sendCommand(R"({"command":"get_ehdr_status"})") << std::endl;

    return 0;
}
```

## Parameter Value Ranges

The valid ranges for camera parameters depend on the specific camera model. You can query the camera capabilities through the VizionSDK API or experimentally determine valid ranges.

**Typical ranges (camera-dependent):**
- Brightness: 0-255
- Contrast: 0-255
- Saturation: 0-255
- Sharpness: 0-255
- Gamma: 72-500
- Gain: 0-100
- Exposure: 1-10000 (in auto mode, value is ignored)
- White Balance Temperature: 2800-6500 Kelvin

**eHDR ranges (for compatible cameras only):**
- eHDR Mode: 0 (enable) or 1 (disable)
- eHDR Exposure Min: 1-4 (default: 1)
- eHDR Exposure Max: 1-4 (default: 4)
- eHDR Ratio Min: 1-128 (default: 12)
- eHDR Ratio Max: 1-128 (default: 24)

**Compatible eHDR Camera Models:**
- VCI-AR0821/AR0822
- VCS-AR0821/AR0822
- VLS3-AR0821/AR0822
- VLS-GM2-AR0821/AR0822
- TEVS-AR0821/AR0822

## Error Handling

Always check the `status` field in the response:

```python
response = send_command("set_format", {...})
if response["status"] == "error":
    print(f"Command failed: {response['message']}")
else:
    print("Command successful")
```

## Thread Safety

The socket server handles one client connection at a time. Commands are processed sequentially with mutex protection to ensure thread safety with the camera operations.

## GStreamer Integration

VizionStreamer uses GStreamer for video processing and output. The captured frames from the VizionSDK camera are continuously fed into a GStreamer pipeline in a separate acquisition thread.

### How It Works

1. **Continuous Acquisition Loop**: A dedicated thread continuously captures frames from the camera using `VxGetImage()`
2. **Frame Buffering**: Captured frames are pushed into the GStreamer pipeline via `appsrc`
3. **Pipeline Processing**: GStreamer processes the frames according to the configured pipeline
4. **Output**: Frames are displayed, saved, or streamed based on the pipeline configuration

### Performance Monitoring

The acquisition loop prints FPS statistics every second:
```
FPS: 30 | Total frames: 1234 | Frame size: 4147200 bytes
```

### Receiving UDP Stream

If you configured a UDP streaming pipeline, receive it with:

```bash
# Using GStreamer
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H264 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! autovideosink

# Using FFplay
ffplay -fflags nobuffer -flags low_delay -framedrop udp://0.0.0.0:5000

# Using VLC
vlc udp://@:5000
```

### Receiving MJPEG HTTP Stream

If you configured an MJPEG HTTP server pipeline:

```bash
# View in browser
firefox http://192.168.1.100:8080

# Using FFplay
ffplay http://192.168.1.100:8080

# Using curl to save frames
curl http://192.168.1.100:8080 > stream.mjpg
```

## Notes

- The socket file is automatically created when VizionStreamer starts
- The socket file is removed when VizionStreamer exits cleanly
- Format and pipeline changes require streaming to be stopped first
- The acquisition loop runs continuously while streaming is active
- Some parameters may not be supported on all camera models
- Invalid parameter values will return an error response
- GStreamer pipeline errors will be reported when starting the stream
- Default pipeline: `videoconvert ! autovideosink` (display locally)
- eHDR features require compatible camera models (VCI/VCS/VLS3/VLS-GM2/TEVS-AR0821/AR0822)
- eHDR settings may be reset to defaults when the camera starts streaming (driver behavior)