gstreamerViewer/SOCKET_API.md

VizionStreamer Socket Control API

VizionStreamer can be controlled via a Unix Domain Socket interface. This allows external applications to configure camera parameters and stream settings at runtime.

Socket Connection

• Socket Path: /tmp/vizion_control.sock
• Protocol: Unix Domain Socket (SOCK_STREAM)
• Message Format: JSON

Command Format

All commands follow this JSON structure:

{
  "command": "command_name",
  "params": {
    "param1": "value1",
    "param2": "value2"
  }
}

Response Format

All responses follow this JSON structure:

Success Response:

{
  "status": "success",
  "message": "Optional success message"
}

Error Response:

{
  "status": "error",
  "message": "Error description"
}

Available Commands

1. Get Available Formats

Retrieve all supported video formats.

Command:

{
  "command": "get_formats"
}

Response:

{
  "status": "success",
  "formats": [
    {
      "width": 1920,
      "height": 1080,
      "framerate": 30,
      "format": "YUY2"
    },
    {
      "width": 1280,
      "height": 720,
      "framerate": 60,
      "format": "MJPG"
    }
  ]
}

Supported Formats: YUY2, UYVY, NV12, MJPG, BGR, RGB

---

2. Set Video Format

Change the video format (resolution, framerate, pixel format).

Note: Cannot be changed while streaming is active.

Command:

{
  "command": "set_format",
  "params": {
    "width": "1920",
    "height": "1080",
    "framerate": "30",
    "format": "YUY2"
  }
}

Response:

{
  "status": "success",
  "message": "Format set successfully"
}

---

3. Start Streaming

Start video streaming from the camera.

Command:

{
  "command": "start_stream"
}

Response:

{
  "status": "success",
  "message": "Streaming started"
}

---

4. Stop Streaming

Stop video streaming.

Command:

{
  "command": "stop_stream"
}

Response:

{
  "status": "success",
  "message": "Streaming stopped"
}

---

5. Set GStreamer Pipeline

Configure the GStreamer pipeline for video output. This determines where and how the video stream is processed/displayed.

Note: Cannot be changed while streaming is active.

Command:

{
  "command": "set_pipeline",
  "params": {
    "pipeline": "videoconvert ! x264enc ! rtph264pay ! udpsink host=192.168.1.100 port=5000"
  }
}

Response:

{
  "status": "success",
  "message": "Pipeline set successfully"
}

Common Pipeline Examples:

1. Display locally:

   videoconvert ! autovideosink
   

2. Stream over UDP (H.264):

   videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000
   

3. Stream over RTSP (requires gst-rtsp-server):

   videoconvert ! x264enc ! rtph264pay name=pay0
   

4. Save to file:

   videoconvert ! x264enc ! mp4mux ! filesink location=/tmp/output.mp4
   

5. Stream over TCP:

   videoconvert ! x264enc ! h264parse ! mpegtsmux ! tcpserversink host=0.0.0.0 port=5000
   

6. MJPEG over HTTP:

   videoconvert ! jpegenc ! multipartmux ! tcpserversink host=0.0.0.0 port=8080
   

---

6. Get Status

Get current streaming status and pipeline configuration.

Command:

{
  "command": "get_status"
}

Response:

{
  "status": "success",
  "streaming": true,
  "pipeline": "videoconvert ! autovideosink"
}

---

7. Set Exposure

Configure camera exposure settings.

Command:

{
  "command": "set_exposure",
  "params": {
    "mode": "manual",
    "value": "100"
  }
}

Parameters:

• mode: "auto" or "manual"
• value: Exposure value (only used in manual mode)

Response:

{
  "status": "success",
  "message": "Exposure set successfully"
}

---

8. Set White Balance

Configure white balance settings.

Command:

{
  "command": "set_whitebalance",
  "params": {
    "mode": "auto",
    "temperature": "4500"
  }
}

Parameters:

• mode: "auto" or "manual"
• temperature: Color temperature in Kelvin (only used in manual mode)

Response:

{
  "status": "success",
  "message": "White balance set successfully"
}

---

9. Set Brightness

Adjust camera brightness.

Command:

{
  "command": "set_brightness",
  "params": {
    "value": "50"
  }
}

Response:

{
  "status": "success",
  "message": "Brightness set successfully"
}

---

10. Set Contrast

Adjust camera contrast.

Command:

{
  "command": "set_contrast",
  "params": {
    "value": "32"
  }
}

Response:

{
  "status": "success",
  "message": "Contrast set successfully"
}

---

11. Set Saturation

Adjust color saturation.

Command:

{
  "command": "set_saturation",
  "params": {
    "value": "64"
  }
}

Response:

{
  "status": "success",
  "message": "Saturation set successfully"
}

---

12. Set Sharpness

Adjust image sharpness.

Command:

{
  "command": "set_sharpness",
  "params": {
    "value": "3"
  }
}

Response:

{
  "status": "success",
  "message": "Sharpness set successfully"
}

---

13. Set Gamma

Adjust gamma correction.

Command:

{
  "command": "set_gamma",
  "params": {
    "value": "100"
  }
}

Response:

{
  "status": "success",
  "message": "Gamma set successfully"
}

---

14. Set Gain

Adjust camera gain.

Command:

{
  "command": "set_gain",
  "params": {
    "value": "0"
  }
}

Response:

{
  "status": "success",
  "message": "Gain set successfully"
}

---

Usage Examples

Complete Workflow Example

# 1. Set GStreamer pipeline for UDP streaming
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 2. Set video format
echo '{"command":"set_format","params":{"width":"1920","height":"1080","framerate":"30","format":"YUY2"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 3. Configure camera settings
echo '{"command":"set_exposure","params":{"mode":"auto"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
echo '{"command":"set_brightness","params":{"value":"50"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 4. Start streaming
echo '{"command":"start_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 5. Check status
echo '{"command":"get_status"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# 6. Stop streaming when done
echo '{"command":"stop_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

GStreamer Pipeline Examples

# Stream to local display
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! autovideosink"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Stream over UDP (H.264)
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Save to MP4 file
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc ! mp4mux ! filesink location=/tmp/output.mp4"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# MJPEG HTTP server
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! jpegenc ! multipartmux ! tcpserversink host=0.0.0.0 port=8080"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

Using socat

# Get available formats
echo '{"command":"get_formats"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Set video format
echo '{"command":"set_format","params":{"width":"1920","height":"1080","framerate":"30","format":"YUY2"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Start streaming
echo '{"command":"start_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Set exposure to auto
echo '{"command":"set_exposure","params":{"mode":"auto"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Set brightness
echo '{"command":"set_brightness","params":{"value":"50"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Get status
echo '{"command":"get_status"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

# Stop streaming
echo '{"command":"stop_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock

Using nc (netcat with Unix socket support)

echo '{"command":"get_formats"}' | nc -U /tmp/vizion_control.sock

Using Python

import socket
import json

def send_command(command, params=None):
    sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    sock.connect('/tmp/vizion_control.sock')

    cmd = {"command": command}
    if params:
        cmd["params"] = params

    sock.send(json.dumps(cmd).encode())
    response = sock.recv(4096).decode()
    sock.close()

    return json.loads(response)

# Examples
print(send_command("get_formats"))
print(send_command("set_format", {
    "width": "1920",
    "height": "1080",
    "framerate": "30",
    "format": "YUY2"
}))
print(send_command("set_exposure", {"mode": "auto"}))
print(send_command("start_stream"))

Using C++

#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <string>
#include <iostream>

std::string sendCommand(const std::string& command) {
    int sock = socket(AF_UNIX, SOCK_STREAM, 0);

    struct sockaddr_un addr;
    memset(&addr, 0, sizeof(addr));
    addr.sun_family = AF_UNIX;
    strcpy(addr.sun_path, "/tmp/vizion_control.sock");

    connect(sock, (struct sockaddr*)&addr, sizeof(addr));
    send(sock, command.c_str(), command.length(), 0);

    char buffer[4096];
    int bytesRead = recv(sock, buffer, sizeof(buffer) - 1, 0);
    buffer[bytesRead] = '\0';

    close(sock);
    return std::string(buffer);
}

// Example usage
int main() {
    std::cout << sendCommand(R"({"command":"get_formats"})") << std::endl;
    std::cout << sendCommand(R"({"command":"set_brightness","params":{"value":"50"}})") << std::endl;
    return 0;
}

Parameter Value Ranges

The valid ranges for camera parameters depend on the specific camera model. You can query the camera capabilities through the VizionSDK API or experimentally determine valid ranges.

Typical ranges (camera-dependent):

• Brightness: 0-255
• Contrast: 0-255
• Saturation: 0-255
• Sharpness: 0-255
• Gamma: 72-500
• Gain: 0-100
• Exposure: 1-10000 (in auto mode, value is ignored)
• White Balance Temperature: 2800-6500 Kelvin

Error Handling

Always check the status field in the response:

response = send_command("set_format", {...})
if response["status"] == "error":
    print(f"Command failed: {response['message']}")
else:
    print("Command successful")

Thread Safety

The socket server handles one client connection at a time. Commands are processed sequentially with mutex protection to ensure thread safety with the camera operations.

GStreamer Integration

VizionStreamer uses GStreamer for video processing and output. The captured frames from the VizionSDK camera are continuously fed into a GStreamer pipeline in a separate acquisition thread.

How It Works

1. Continuous Acquisition Loop: A dedicated thread continuously captures frames from the camera using VxGetImage()
2. Frame Buffering: Captured frames are pushed into the GStreamer pipeline via appsrc
3. Pipeline Processing: GStreamer processes the frames according to the configured pipeline
4. Output: Frames are displayed, saved, or streamed based on the pipeline configuration

Performance Monitoring

The acquisition loop prints FPS statistics every second:

FPS: 30 | Total frames: 1234 | Frame size: 4147200 bytes

Receiving UDP Stream

If you configured a UDP streaming pipeline, receive it with:

# Using GStreamer
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H264 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! autovideosink

# Using FFplay
ffplay -fflags nobuffer -flags low_delay -framedrop udp://0.0.0.0:5000

# Using VLC
vlc udp://@:5000

Receiving MJPEG HTTP Stream

If you configured an MJPEG HTTP server pipeline:

# View in browser
firefox http://192.168.1.100:8080

# Using FFplay
ffplay http://192.168.1.100:8080

# Using curl to save frames
curl http://192.168.1.100:8080 > stream.mjpg

Notes

• The socket file is automatically created when VizionStreamer starts
• The socket file is removed when VizionStreamer exits cleanly
• Format and pipeline changes require streaming to be stopped first
• The acquisition loop runs continuously while streaming is active
• Some parameters may not be supported on all camera models
• Invalid parameter values will return an error response
• GStreamer pipeline errors will be reported when starting the stream
• Default pipeline: videoconvert ! autovideosink (display locally)