GStreamer Viewer

A Qt6-based GUI application for controlling and viewing video streams from cameras via the VizionStreamer backend. This application provides real-time camera control, GStreamer pipeline configuration, and video display capabilities.

Features

• Video Streaming Control: Configure and start/stop camera streaming with GStreamer pipelines
• Real-time Video Display: View the streamed video in a separate window
• Camera Parameter Control: Adjust exposure, white balance, brightness, contrast, saturation, sharpness, gamma, and gain
• Pipeline Presets: Quick access to common pipeline configurations (MJPEG UDP, H.264 UDP, local display, etc.)
• Format Detection: Automatically fetch and select supported camera formats
• Quick Start: One-click auto-configuration and streaming
• Unix Socket Communication: Communicates with VizionStreamer backend via /tmp/vizion_control.sock

System Requirements

• Linux (tested on Arch Linux)
• Qt6
• GStreamer 1.0
• VizionStreamer backend (not included)

Installation

Arch Linux

Install the required packages using pacman:

sudo pacman -S qt6-base gstreamer gst-plugins-base gst-plugins-good gst-plugins-bad cmake base-devel

Package breakdown:

• qt6-base: Qt6 framework (Widgets, Network, Core modules)
• gstreamer: GStreamer multimedia framework
• gst-plugins-base: Base GStreamer plugins (videoconvert, etc.)
• gst-plugins-good: Good quality plugins (JPEG encoding/decoding, RTP, UDP)
• gst-plugins-bad: Additional plugins (optional, for more formats)
• cmake: Build system
• base-devel: C++ compiler and build tools

Debian/Ubuntu

Install the required packages using apt:

sudo apt update
sudo apt install qt6-base-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
                 gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-tools \
                 cmake build-essential

Package breakdown:

• qt6-base-dev: Qt6 development files
• libgstreamer1.0-dev: GStreamer development headers
• libgstreamer-plugins-base1.0-dev: Base plugins development files
• gstreamer1.0-plugins-good: Good quality plugins runtime
• gstreamer1.0-plugins-bad: Additional plugins (optional)
• gstreamer1.0-tools: GStreamer command-line tools (for debugging with gst-launch-1.0)
• cmake: Build system
• build-essential: C++ compiler and build tools

Optional: H.264 Support

For H.264 streaming (requires additional codec):

Arch Linux:

sudo pacman -S gst-libav

Debian/Ubuntu:

sudo apt install gstreamer1.0-libav

Building

Quick Build (using build script)

The easiest way to build the project:

cd /home/maik/project/gstreamerViewer
./build.sh

Manual Build

1. Clone or extract the project:

cd /home/maik/project/gstreamerViewer

2. Create a build directory:

mkdir -p build
cd build

3. Configure with CMake:

cmake ..

4. Build the project:

make -j$(nproc)

5. The executable will be located at:

./gstreamerViewer

Usage

Prerequisites

1. VizionStreamer Backend: Ensure the VizionStreamer backend is running and the Unix domain socket is available at /tmp/vizion_control.sock

2. Camera Connection: Connect your camera (e.g., VCI-AR0234-C) and ensure VizionStreamer has selected the correct camera device

Quick Start Workflow

The easiest way to start streaming:

1. Launch the application:

   # Using the run script (checks for VizionStreamer)
   ./run.sh
   
   # Or directly from build directory
   cd build && ./gstreamerViewer
   

2. Navigate to the "GStreamer Pipeline" tab

3. Click the "⚡ Quick Start (Auto Configure & Stream)" button

   • This automatically:
     • Sets the camera format (1280x720@30fps UYVY or best available)
     • Configures the MJPEG UDP streaming pipeline
     • Starts the stream

4. Switch to the "Video Viewer" tab

5. Ensure the source type is set to "UDP MJPEG Stream"

6. Click "Start Viewer" to display the video

Manual Configuration

For more control over the streaming setup:

Step 1: Configure Camera Format

1. Go to the "Camera Control" tab
2. Click "Get Available Formats" to fetch supported formats from the camera
3. Select your desired format from the dropdown
4. Click "Set Format"

Step 2: Configure Pipeline

1. Go to the "GStreamer Pipeline" tab
2. Select a pipeline preset from the dropdown, or enter a custom pipeline:
   • MJPEG UDP Stream: Best for raw formats (UYVY, YUY2), no additional plugins needed
   • UDP H.264 Stream: Requires gst-libav, better compression
   • Local Display: For testing (shows video on server side)
3. Click "Set Pipeline"

Step 3: Start Streaming

1. Click "Start Stream" in the GStreamer Pipeline tab
2. The status should show "Status: Streaming" with a green background

Step 4: View the Stream

1. Go to the "Video Viewer" tab
2. Select the appropriate source type (matches your pipeline):
   • UDP MJPEG Stream for MJPEG UDP pipeline
   • UDP H.264 Stream for H.264 UDP pipeline
3. Verify host/port settings (default: port 5000)
4. Click "Start Viewer"
5. Video will appear in a separate window

Camera Control

The "Camera Control" tab provides real-time adjustment of camera parameters:

• Exposure: Auto or Manual mode with adjustable value
• White Balance: Auto or Manual with temperature control (2800-6500K)
• Image Adjustments:
  • Brightness (0-255)
  • Contrast (0-255)
  • Saturation (0-255)
  • Sharpness (0-255)
  • Gamma (72-500)
  • Gain (0-100)

All slider changes are applied immediately to the camera.

Pipeline Presets Explained

MJPEG UDP Stream

videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000

• Best for: Raw formats (UYVY, YUY2, RGB)
• Pros: No additional plugins needed, reliable
• Cons: Lower compression than H.264

UDP H.264 Stream

videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=127.0.0.1 port=5000

• Best for: Higher compression, lower bandwidth
• Pros: Better compression
• Cons: Requires gst-libav plugin

Local Display

videoconvert ! autovideosink

• Best for: Testing camera without network streaming
• Shows: Video on the server machine

Troubleshooting

Issue: "Failed to start streaming"

Solution: Ensure the camera format is set before starting the stream:

1. Click "Get Available Formats" in Camera Control tab
2. Select a supported format
3. Click "Set Format"
4. Try starting the stream again

Issue: Video is black/not displaying

Possible causes:

1. Pipeline mismatch: Ensure the viewer source type matches the streaming pipeline
2. No UDP packets: Verify with tcpdump:

   sudo tcpdump -i lo udp port 5000 -nn
   

3. Wrong camera selected: Check VizionStreamer logs to ensure correct camera is active

Issue: "No element 'avdec_h264'"

Solution: Install the gst-libav plugin or use the MJPEG UDP pipeline instead

Arch Linux:

sudo pacman -S gst-libav

Debian/Ubuntu:

sudo apt install gstreamer1.0-libav

Issue: Connection error to VizionStreamer

Solution: Verify the backend is running:

ls -la /tmp/vizion_control.sock

If the socket doesn't exist, start VizionStreamer.

Test the connection:

./test_connection.sh

This script will verify the socket exists and test basic communication with VizionStreamer.

Issue: X11 BadWindow errors with video display

Note: The application uses autovideosink which opens a separate video window. This is intentional due to X11 limitations with embedded video overlays in Qt.

Debug: Test pipeline manually

Test if GStreamer can receive the stream:

gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec ! autovideosink

Supported Cameras

This application works with cameras supported by the VizionStreamer backend, including:

• VCI-AR0234-C (tested: UYVY at 1920x1200@60fps)
• Other V4L2-compatible cameras

Architecture

┌─────────────────────┐
│  gstreamerViewer    │
│  (Qt6 GUI)          │
└──────────┬──────────┘
           │ Unix Socket (/tmp/vizion_control.sock)
           │ JSON Commands
           ▼
┌─────────────────────┐
│  VizionStreamer     │
│  Backend            │
└──────────┬──────────┘
           │ V4L2 / VizionSDK
           ▼
┌─────────────────────┐
│  Camera Hardware    │
│  (VCI-AR0234-C)     │
└─────────────────────┘

Video Stream Flow:
Camera → VizionStreamer → GStreamer Pipeline → UDP/Local → VideoViewer

Socket API Commands

The application communicates with VizionStreamer using JSON commands. See SOCKET_API.md for full protocol documentation.

Example commands:

• get_formats: Retrieve available camera formats
• set_format: Set camera resolution, framerate, and pixel format
• set_pipeline: Configure GStreamer pipeline
• start_stream: Start camera streaming
• stop_stream: Stop camera streaming
• get_status: Query streaming status
• set_exposure, set_brightness, etc.: Camera parameter controls

License

[Specify your license here]

Credits

Built with:

• Qt6 Framework
• GStreamer Multimedia Framework
• VizionSDK

Support

For issues or questions:

1. Check the Troubleshooting section above
2. Verify VizionStreamer backend is running correctly
3. Test GStreamer pipelines manually with gst-launch-1.0