SkadiLabs/gstreamerViewer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
Maik Jurischka
2025-12-18 16:10:55 +01:00
commit 69e2f3ae1d
19 changed files with 2707 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

663
SOCKET_API.md Normal file
View File

@@ -0,0 +1,663 @@
# VizionStreamer Socket Control API
VizionStreamer can be controlled via a Unix Domain Socket interface. This allows external applications to configure camera parameters and stream settings at runtime.
## Socket Connection
- **Socket Path**: `/tmp/vizion_control.sock`
- **Protocol**: Unix Domain Socket (SOCK_STREAM)
- **Message Format**: JSON
## Command Format
All commands follow this JSON structure:
```json
{
"command": "command_name",
"params": {
"param1": "value1",
"param2": "value2"
}
}
```
## Response Format
All responses follow this JSON structure:
**Success Response:**
```json
{
"status": "success",
"message": "Optional success message"
}
```
**Error Response:**
```json
{
"status": "error",
"message": "Error description"
}
```
## Available Commands
### 1. Get Available Formats
Retrieve all supported video formats.
**Command:**
```json
{
"command": "get_formats"
}
```
**Response:**
```json
{
"status": "success",
"formats": [
{
"width": 1920,
"height": 1080,
"framerate": 30,
"format": "YUY2"
},
{
"width": 1280,
"height": 720,
"framerate": 60,
"format": "MJPG"
}
]
}
```
**Supported Formats:** YUY2, UYVY, NV12, MJPG, BGR, RGB
---
### 2. Set Video Format
Change the video format (resolution, framerate, pixel format).
**Note:** Cannot be changed while streaming is active.
**Command:**
```json
{
"command": "set_format",
"params": {
"width": "1920",
"height": "1080",
"framerate": "30",
"format": "YUY2"
}
}
```
**Response:**
```json
{
"status": "success",
"message": "Format set successfully"
}
```
---
### 3. Start Streaming
Start video streaming from the camera.
**Command:**
```json
{
"command": "start_stream"
}
```
**Response:**
```json
{
"status": "success",
"message": "Streaming started"
}
```
---
### 4. Stop Streaming
Stop video streaming.
**Command:**
```json
{
"command": "stop_stream"
}
```
**Response:**
```json
{
"status": "success",
"message": "Streaming stopped"
}
```
---
### 5. Set GStreamer Pipeline
Configure the GStreamer pipeline for video output. This determines where and how the video stream is processed/displayed.
**Note:** Cannot be changed while streaming is active.
**Command:**
```json
{
"command": "set_pipeline",
"params": {
"pipeline": "videoconvert ! x264enc ! rtph264pay ! udpsink host=192.168.1.100 port=5000"
}
}
```
**Response:**
```json
{
"status": "success",
"message": "Pipeline set successfully"
}
```
**Common Pipeline Examples:**
1. **Display locally:**
```
videoconvert ! autovideosink
```
2. **Stream over UDP (H.264):**
```
videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000
```
3. **Stream over RTSP (requires gst-rtsp-server):**
```
videoconvert ! x264enc ! rtph264pay name=pay0
```
4. **Save to file:**
```
videoconvert ! x264enc ! mp4mux ! filesink location=/tmp/output.mp4
```
5. **Stream over TCP:**
```
videoconvert ! x264enc ! h264parse ! mpegtsmux ! tcpserversink host=0.0.0.0 port=5000
```
6. **MJPEG over HTTP:**
```
videoconvert ! jpegenc ! multipartmux ! tcpserversink host=0.0.0.0 port=8080
```
---
### 6. Get Status
Get current streaming status and pipeline configuration.
**Command:**
```json
{
"command": "get_status"
}
```
**Response:**
```json
{
"status": "success",
"streaming": true,
"pipeline": "videoconvert ! autovideosink"
}
```
---
### 7. Set Exposure
Configure camera exposure settings.
**Command:**
```json
{
"command": "set_exposure",
"params": {
"mode": "manual",
"value": "100"
}
}
```
**Parameters:**
- `mode`: "auto" or "manual"
- `value`: Exposure value (only used in manual mode)
**Response:**
```json
{
"status": "success",
"message": "Exposure set successfully"
}
```
---
### 8. Set White Balance
Configure white balance settings.
**Command:**
```json
{
"command": "set_whitebalance",
"params": {
"mode": "auto",
"temperature": "4500"
}
}
```
**Parameters:**
- `mode`: "auto" or "manual"
- `temperature`: Color temperature in Kelvin (only used in manual mode)
**Response:**
```json
{
"status": "success",
"message": "White balance set successfully"
}
```
---
### 9. Set Brightness
Adjust camera brightness.
**Command:**
```json
{
"command": "set_brightness",
"params": {
"value": "50"
}
}
```
**Response:**
```json
{
"status": "success",
"message": "Brightness set successfully"
}
```
---
### 10. Set Contrast
Adjust camera contrast.
**Command:**
```json
{
"command": "set_contrast",
"params": {
"value": "32"
}
}
```
**Response:**
```json
{
"status": "success",
"message": "Contrast set successfully"
}
```
---
### 11. Set Saturation
Adjust color saturation.
**Command:**
```json
{
"command": "set_saturation",
"params": {
"value": "64"
}
}
```
**Response:**
```json
{
"status": "success",
"message": "Saturation set successfully"
}
```
---
### 12. Set Sharpness
Adjust image sharpness.
**Command:**
```json
{
"command": "set_sharpness",
"params": {
"value": "3"
}
}
```
**Response:**
```json
{
"status": "success",
"message": "Sharpness set successfully"
}
```
---
### 13. Set Gamma
Adjust gamma correction.
**Command:**
```json
{
"command": "set_gamma",
"params": {
"value": "100"
}
}
```
**Response:**
```json
{
"status": "success",
"message": "Gamma set successfully"
}
```
---
### 14. Set Gain
Adjust camera gain.
**Command:**
```json
{
"command": "set_gain",
"params": {
"value": "0"
}
}
```
**Response:**
```json
{
"status": "success",
"message": "Gain set successfully"
}
```
---
## Usage Examples
### Complete Workflow Example
```bash
# 1. Set GStreamer pipeline for UDP streaming
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# 2. Set video format
echo '{"command":"set_format","params":{"width":"1920","height":"1080","framerate":"30","format":"YUY2"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# 3. Configure camera settings
echo '{"command":"set_exposure","params":{"mode":"auto"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
echo '{"command":"set_brightness","params":{"value":"50"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# 4. Start streaming
echo '{"command":"start_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# 5. Check status
echo '{"command":"get_status"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# 6. Stop streaming when done
echo '{"command":"stop_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
```
### GStreamer Pipeline Examples
```bash
# Stream to local display
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! autovideosink"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# Stream over UDP (H.264)
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# Save to MP4 file
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc ! mp4mux ! filesink location=/tmp/output.mp4"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# MJPEG HTTP server
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! jpegenc ! multipartmux ! tcpserversink host=0.0.0.0 port=8080"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
```
### Using `socat`
```bash
# Get available formats
echo '{"command":"get_formats"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# Set video format
echo '{"command":"set_format","params":{"width":"1920","height":"1080","framerate":"30","format":"YUY2"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# Start streaming
echo '{"command":"start_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# Set exposure to auto
echo '{"command":"set_exposure","params":{"mode":"auto"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# Set brightness
echo '{"command":"set_brightness","params":{"value":"50"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# Get status
echo '{"command":"get_status"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# Stop streaming
echo '{"command":"stop_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
```
### Using `nc` (netcat with Unix socket support)
```bash
echo '{"command":"get_formats"}' | nc -U /tmp/vizion_control.sock
```
### Using Python
```python
import socket
import json
def send_command(command, params=None):
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.connect('/tmp/vizion_control.sock')
cmd = {"command": command}
if params:
cmd["params"] = params
sock.send(json.dumps(cmd).encode())
response = sock.recv(4096).decode()
sock.close()
return json.loads(response)
# Examples
print(send_command("get_formats"))
print(send_command("set_format", {
"width": "1920",
"height": "1080",
"framerate": "30",
"format": "YUY2"
}))
print(send_command("set_exposure", {"mode": "auto"}))
print(send_command("start_stream"))
```
### Using C++
```cpp
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <string>
#include <iostream>
std::string sendCommand(const std::string& command) {
int sock = socket(AF_UNIX, SOCK_STREAM, 0);
struct sockaddr_un addr;
memset(&addr, 0, sizeof(addr));
addr.sun_family = AF_UNIX;
strcpy(addr.sun_path, "/tmp/vizion_control.sock");
connect(sock, (struct sockaddr*)&addr, sizeof(addr));
send(sock, command.c_str(), command.length(), 0);
char buffer[4096];
int bytesRead = recv(sock, buffer, sizeof(buffer) - 1, 0);
buffer[bytesRead] = '\0';
close(sock);
return std::string(buffer);
}
// Example usage
int main() {
std::cout << sendCommand(R"({"command":"get_formats"})") << std::endl;
std::cout << sendCommand(R"({"command":"set_brightness","params":{"value":"50"}})") << std::endl;
return 0;
}
```
## Parameter Value Ranges
The valid ranges for camera parameters depend on the specific camera model. You can query the camera capabilities through the VizionSDK API or experimentally determine valid ranges.
**Typical ranges (camera-dependent):**
- Brightness: 0-255
- Contrast: 0-255
- Saturation: 0-255
- Sharpness: 0-255
- Gamma: 72-500
- Gain: 0-100
- Exposure: 1-10000 (in auto mode, value is ignored)
- White Balance Temperature: 2800-6500 Kelvin
## Error Handling
Always check the `status` field in the response:
```python
response = send_command("set_format", {...})
if response["status"] == "error":
print(f"Command failed: {response['message']}")
else:
print("Command successful")
```
## Thread Safety
The socket server handles one client connection at a time. Commands are processed sequentially with mutex protection to ensure thread safety with the camera operations.
## GStreamer Integration
VizionStreamer uses GStreamer for video processing and output. The captured frames from the VizionSDK camera are continuously fed into a GStreamer pipeline in a separate acquisition thread.
### How It Works
1. **Continuous Acquisition Loop**: A dedicated thread continuously captures frames from the camera using `VxGetImage()`
2. **Frame Buffering**: Captured frames are pushed into the GStreamer pipeline via `appsrc`
3. **Pipeline Processing**: GStreamer processes the frames according to the configured pipeline
4. **Output**: Frames are displayed, saved, or streamed based on the pipeline configuration
### Performance Monitoring
The acquisition loop prints FPS statistics every second:
```
FPS: 30 | Total frames: 1234 | Frame size: 4147200 bytes
```
### Receiving UDP Stream
If you configured a UDP streaming pipeline, receive it with:
```bash
# Using GStreamer
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H264 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! autovideosink
# Using FFplay
ffplay -fflags nobuffer -flags low_delay -framedrop udp://0.0.0.0:5000
# Using VLC
vlc udp://@:5000
```
### Receiving MJPEG HTTP Stream
If you configured an MJPEG HTTP server pipeline:
```bash
# View in browser
firefox http://192.168.1.100:8080
# Using FFplay
ffplay http://192.168.1.100:8080
# Using curl to save frames
curl http://192.168.1.100:8080 > stream.mjpg
```
## Notes
- The socket file is automatically created when VizionStreamer starts
- The socket file is removed when VizionStreamer exits cleanly
- Format and pipeline changes require streaming to be stopped first
- The acquisition loop runs continuously while streaming is active
- Some parameters may not be supported on all camera models
- Invalid parameter values will return an error response
- GStreamer pipeline errors will be reported when starting the stream
- Default pipeline: `videoconvert ! autovideosink` (display locally)