SkadiLabs/vizionStreamer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add GStreamer

Browse Source
This commit is contained in:
Maik Jurischka
2025-12-12 11:43:15 +01:00
parent fe0af4dc64
commit 212a9ec64c
9 changed files with 606 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

173
SOCKET_API.md
View File

@@ -151,9 +151,67 @@ Stop video streaming.
---
### 5. Get Status
### 5. Set GStreamer Pipeline
Get current streaming status.
Configure the GStreamer pipeline for video output. This determines where and how the video stream is processed/displayed.
**Note:** Cannot be changed while streaming is active.
**Command:**
```json
{
"command": "set_pipeline",
"params": {
"pipeline": "videoconvert ! x264enc ! rtph264pay ! udpsink host=192.168.1.100 port=5000"
}
}
```
**Response:**
```json
{
"status": "success",
"message": "Pipeline set successfully"
}
```
**Common Pipeline Examples:**
1. **Display locally:**
```
videoconvert ! autovideosink
```
2. **Stream over UDP (H.264):**
```
videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000
```
3. **Stream over RTSP (requires gst-rtsp-server):**
```
videoconvert ! x264enc ! rtph264pay name=pay0
```
4. **Save to file:**
```
videoconvert ! x264enc ! mp4mux ! filesink location=/tmp/output.mp4
```
5. **Stream over TCP:**
```
videoconvert ! x264enc ! h264parse ! mpegtsmux ! tcpserversink host=0.0.0.0 port=5000
```
6. **MJPEG over HTTP:**
```
videoconvert ! jpegenc ! multipartmux ! tcpserversink host=0.0.0.0 port=8080
```
---
### 6. Get Status
Get current streaming status and pipeline configuration.
**Command:**
```json
@@ -166,13 +224,14 @@ Get current streaming status.
```json
{
"status": "success",
"streaming": true
"streaming": true,
"pipeline": "videoconvert ! autovideosink"
}
```
---
### 6. Set Exposure
### 7. Set Exposure
Configure camera exposure settings.
@@ -201,7 +260,7 @@ Configure camera exposure settings.
---
### 7. Set White Balance
### 8. Set White Balance
Configure white balance settings.
@@ -230,7 +289,7 @@ Configure white balance settings.
---
### 8. Set Brightness
### 9. Set Brightness
Adjust camera brightness.
@@ -254,7 +313,7 @@ Adjust camera brightness.
---
### 9. Set Contrast
### 10. Set Contrast
Adjust camera contrast.
@@ -278,7 +337,7 @@ Adjust camera contrast.
---
### 10. Set Saturation
### 11. Set Saturation
Adjust color saturation.
@@ -302,7 +361,7 @@ Adjust color saturation.
---
### 11. Set Sharpness
### 12. Set Sharpness
Adjust image sharpness.
@@ -326,7 +385,7 @@ Adjust image sharpness.
---
### 12. Set Gamma
### 13. Set Gamma
Adjust gamma correction.
@@ -350,7 +409,7 @@ Adjust gamma correction.
---
### 13. Set Gain
### 14. Set Gain
Adjust camera gain.
@@ -376,6 +435,45 @@ Adjust camera gain.
## Usage Examples
### Complete Workflow Example
```bash
# 1. Set GStreamer pipeline for UDP streaming
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# 2. Set video format
echo '{"command":"set_format","params":{"width":"1920","height":"1080","framerate":"30","format":"YUY2"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# 3. Configure camera settings
echo '{"command":"set_exposure","params":{"mode":"auto"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
echo '{"command":"set_brightness","params":{"value":"50"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# 4. Start streaming
echo '{"command":"start_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# 5. Check status
echo '{"command":"get_status"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# 6. Stop streaming when done
echo '{"command":"stop_stream"}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
```
### GStreamer Pipeline Examples
```bash
# Stream to local display
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! autovideosink"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# Stream over UDP (H.264)
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.1.100 port=5000"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# Save to MP4 file
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! x264enc ! mp4mux ! filesink location=/tmp/output.mp4"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
# MJPEG HTTP server
echo '{"command":"set_pipeline","params":{"pipeline":"videoconvert ! jpegenc ! multipartmux ! tcpserversink host=0.0.0.0 port=8080"}}' | socat - UNIX-CONNECT:/tmp/vizion_control.sock
```
### Using `socat`
```bash
@@ -505,10 +603,61 @@ else:
The socket server handles one client connection at a time. Commands are processed sequentially with mutex protection to ensure thread safety with the camera operations.
## GStreamer Integration
VizionStreamer uses GStreamer for video processing and output. The captured frames from the VizionSDK camera are continuously fed into a GStreamer pipeline in a separate acquisition thread.
### How It Works
1. **Continuous Acquisition Loop**: A dedicated thread continuously captures frames from the camera using `VxGetImage()`
2. **Frame Buffering**: Captured frames are pushed into the GStreamer pipeline via `appsrc`
3. **Pipeline Processing**: GStreamer processes the frames according to the configured pipeline
4. **Output**: Frames are displayed, saved, or streamed based on the pipeline configuration
### Performance Monitoring
The acquisition loop prints FPS statistics every second:
```
FPS: 30 | Total frames: 1234 | Frame size: 4147200 bytes
```
### Receiving UDP Stream
If you configured a UDP streaming pipeline, receive it with:
```bash
# Using GStreamer
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=H264 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! autovideosink
# Using FFplay
ffplay -fflags nobuffer -flags low_delay -framedrop udp://0.0.0.0:5000
# Using VLC
vlc udp://@:5000
```
### Receiving MJPEG HTTP Stream
If you configured an MJPEG HTTP server pipeline:
```bash
# View in browser
firefox http://192.168.1.100:8080
# Using FFplay
ffplay http://192.168.1.100:8080
# Using curl to save frames
curl http://192.168.1.100:8080 > stream.mjpg
```
## Notes
- The socket file is automatically created when VizionStreamer starts
- The socket file is removed when VizionStreamer exits cleanly
- Format changes require streaming to be stopped first
- Format and pipeline changes require streaming to be stopped first
- The acquisition loop runs continuously while streaming is active
- Some parameters may not be supported on all camera models
- Invalid parameter values will return an error response
- GStreamer pipeline errors will be reported when starting the stream
- Default pipeline: `videoconvert ! autovideosink` (display locally)