gstreamerViewer/README.md

# GStreamer Viewer

A Qt6-based GUI application for controlling and viewing video streams from cameras via the VizionStreamer backend. This application provides real-time camera control, GStreamer pipeline configuration, and video display capabilities.

## Features

- **Video Streaming Control**: Configure and start/stop camera streaming with GStreamer pipelines
- **Real-time Video Display**: View the streamed video with embedded display
- **Image Transformations**: Rotate (0°, 90°, 180°, 270°) and flip (horizontal/vertical) video
- **Zoom Support**: Zoom from 50% to 200%
- **Camera Parameter Control**: Adjust exposure, white balance, brightness, contrast, saturation, sharpness, gamma, and gain
- **Pipeline Presets**: Quick access to common pipeline configurations (MJPEG UDP, H.264 UDP, local display, etc.)
- **Format Detection**: Automatically fetch and select supported camera formats
- **Quick Start**: One-click auto-configuration and streaming
- **Unix Socket Communication**: Communicates with VizionStreamer backend via `/tmp/vizion_control.sock`

## System Requirements

- Linux (tested on Arch Linux)
- Qt6
- GStreamer 1.0
- VizionStreamer backend (not included)

## Installation

### Arch Linux

Install the required packages using pacman:

```bash
sudo pacman -S qt6-base gstreamer gst-plugins-base gst-plugins-good gst-plugins-bad cmake base-devel
```

**Package breakdown:**
- `qt6-base`: Qt6 framework (Widgets, Network, Core modules)
- `gstreamer`: GStreamer multimedia framework
- `gst-plugins-base`: Base GStreamer plugins (videoconvert, etc.)
- `gst-plugins-good`: Good quality plugins (JPEG encoding/decoding, RTP, UDP)
- `gst-plugins-bad`: Additional plugins (optional, for more formats)
- `cmake`: Build system
- `base-devel`: C++ compiler and build tools

### Debian/Ubuntu

Install the required packages using apt:

```bash
sudo apt update
sudo apt install qt6-base-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
                 gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-tools \
                 cmake build-essential
```

**Package breakdown:**
- `qt6-base-dev`: Qt6 development files
- `libgstreamer1.0-dev`: GStreamer development headers
- `libgstreamer-plugins-base1.0-dev`: Base plugins development files
- `gstreamer1.0-plugins-good`: Good quality plugins runtime
- `gstreamer1.0-plugins-bad`: Additional plugins (optional)
- `gstreamer1.0-tools`: GStreamer command-line tools (for debugging with gst-launch-1.0)
- `cmake`: Build system
- `build-essential`: C++ compiler and build tools

### Optional: H.264 Support

For H.264 streaming (requires additional codec):

**Arch Linux:**
```bash
sudo pacman -S gst-libav
```

**Debian/Ubuntu:**
```bash
sudo apt install gstreamer1.0-libav
```

## Building

### Quick Build (using build script)

The easiest way to build the project:

```bash
cd /home/maik/project/gstreamerViewer
./build.sh
```

### Manual Build

1. Clone or extract the project:
```bash
cd /home/maik/project/gstreamerViewer
```

2. Create a build directory:
```bash
mkdir -p build
cd build
```

3. Configure with CMake:
```bash
cmake ..
```

4. Build the project:
```bash
make -j$(nproc)
```

5. The executable will be located at:
```bash
./gstreamerViewer
```

## Usage

### Prerequisites

1. **VizionStreamer Backend**: Ensure the VizionStreamer backend is running and the Unix domain socket is available at `/tmp/vizion_control.sock`

2. **Camera Connection**: Connect your camera (e.g., VCI-AR0234-C) and ensure VizionStreamer has selected the correct camera device

### Quick Start Workflow

The easiest way to start streaming:

1. Launch the application:
   ```bash
   # Using the run script (checks for VizionStreamer)
   ./run.sh

   # Or directly from build directory
   cd build && ./gstreamerViewer
   ```

2. Navigate to the **"GStreamer Pipeline"** tab

3. Click the **"⚡ Quick Start (Auto Configure & Stream)"** button
   - This automatically:
     - Sets the camera format (1280x720@30fps UYVY or best available)
     - Configures the MJPEG UDP streaming pipeline
     - Starts the stream

4. Switch to the **"Video Viewer"** tab

5. Ensure the source type is set to **"UDP MJPEG Stream"**

6. Click **"Start Viewer"** to display the video

### Manual Configuration

For more control over the streaming setup:

#### Step 1: Configure Camera Format

1. Go to the **"Camera Control"** tab
2. Click **"Get Available Formats"** to fetch supported formats from the camera
3. Select your desired format from the dropdown
4. Click **"Set Format"**

#### Step 2: Configure Pipeline

1. Go to the **"GStreamer Pipeline"** tab
2. Select a pipeline preset from the dropdown, or enter a custom pipeline:
   - **MJPEG UDP Stream**: Best for raw formats (UYVY, YUY2), no additional plugins needed
   - **UDP H.264 Stream**: Requires gst-libav, better compression
   - **Local Display**: For testing (shows video on server side)
3. Click **"Set Pipeline"**

#### Step 3: Start Streaming

1. Click **"Start Stream"** in the GStreamer Pipeline tab
2. The status should show "Status: Streaming" with a green background

#### Step 4: View the Stream

1. Go to the **"Video Viewer"** tab
2. Select the appropriate source type (matches your pipeline):
   - **UDP MJPEG Stream** for MJPEG UDP pipeline
   - **UDP H.264 Stream** for H.264 UDP pipeline
3. Verify host/port settings (default: port 5000)
4. Click **"Start Viewer"**
5. Video will appear in a separate window

### Camera Control

The **"Camera Control"** tab provides real-time adjustment of camera parameters:

- **Exposure**: Auto or Manual mode with adjustable value
- **White Balance**: Auto or Manual with temperature control (2800-6500K)
- **Image Adjustments**:
  - Brightness (0-255)
  - Contrast (0-255)
  - Saturation (0-255)
  - Sharpness (0-255)
  - Gamma (72-500)
  - Gain (0-100)

All slider changes are applied immediately to the camera.

## Pipeline Presets Explained

### MJPEG UDP Stream
```
videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000
```
- **Best for**: Raw formats (UYVY, YUY2, RGB)
- **Pros**: No additional plugins needed, reliable
- **Cons**: Lower compression than H.264

### UDP H.264 Stream
```
videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=127.0.0.1 port=5000
```
- **Best for**: Higher compression, lower bandwidth
- **Pros**: Better compression
- **Cons**: Requires gst-libav plugin

### Local Display
```
videoconvert ! autovideosink
```
- **Best for**: Testing camera without network streaming
- **Shows**: Video on the server machine

## Troubleshooting

### Issue: "Failed to start streaming"

**Solution**: Ensure the camera format is set before starting the stream:
1. Click "Get Available Formats" in Camera Control tab
2. Select a supported format
3. Click "Set Format"
4. Try starting the stream again

### Issue: Video is black/not displaying

**Possible causes:**
1. **Pipeline mismatch**: Ensure the viewer source type matches the streaming pipeline
2. **No UDP packets**: Verify with tcpdump:
   ```bash
   sudo tcpdump -i lo udp port 5000 -nn
   ```
3. **Wrong camera selected**: Check VizionStreamer logs to ensure correct camera is active

### Issue: "No element 'avdec_h264'"

**Solution**: Install the gst-libav plugin or use the MJPEG UDP pipeline instead

**Arch Linux:**
```bash
sudo pacman -S gst-libav
```

**Debian/Ubuntu:**
```bash
sudo apt install gstreamer1.0-libav
```

### Issue: Connection error to VizionStreamer

**Solution**: Verify the backend is running:
```bash
ls -la /tmp/vizion_control.sock
```

If the socket doesn't exist, start VizionStreamer.

**Test the connection**:
```bash
./test_connection.sh
```

This script will verify the socket exists and test basic communication with VizionStreamer.

### Issue: X11 BadWindow errors with video display

**Note**: The application uses `autovideosink` which opens a separate video window. This is intentional due to X11 limitations with embedded video overlays in Qt.

### Debug: Test pipeline manually

Test if GStreamer can receive the stream:
```bash
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec ! autovideosink
```

## Supported Cameras

This application works with cameras supported by the VizionStreamer backend, including:
- VCI-AR0234-C (tested: UYVY at 1920x1200@60fps)
- Other V4L2-compatible cameras

## Architecture

```
┌─────────────────────┐
│  gstreamerViewer    │
│  (Qt6 GUI)          │
└──────────┬──────────┘
           │ Unix Socket (/tmp/vizion_control.sock)
           │ JSON Commands
           ▼
┌─────────────────────┐
│  VizionStreamer     │
│  Backend            │
└──────────┬──────────┘
           │ V4L2 / VizionSDK
           ▼
┌─────────────────────┐
│  Camera Hardware    │
│  (VCI-AR0234-C)     │
└─────────────────────┘

Video Stream Flow:
Camera → VizionStreamer → GStreamer Pipeline → UDP/Local → VideoViewer
```

## Socket API Commands

The application communicates with VizionStreamer using JSON commands. See `SOCKET_API.md` for full protocol documentation.

Example commands:
- `get_formats`: Retrieve available camera formats
- `set_format`: Set camera resolution, framerate, and pixel format
- `set_pipeline`: Configure GStreamer pipeline
- `start_stream`: Start camera streaming
- `stop_stream`: Stop camera streaming
- `get_status`: Query streaming status
- `set_exposure`, `set_brightness`, etc.: Camera parameter controls

## License

**CC BY-NC-SA 4.0** - Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International

Copyright (c) 2025 Maik Jurischka

This software is free to:
- **Use** for non-commercial purposes
- **Share** - copy and redistribute in any medium or format
- **Adapt** - remix, transform, and build upon the material

Under the following terms:
- **Attribution** - You must give appropriate credit and indicate if changes were made
- **NonCommercial** - You may not use the material for commercial purposes
- **ShareAlike** - Derivative works must be distributed under the same license

See [LICENSE](LICENSE) file for full license text.

Full license: https://creativecommons.org/licenses/by-nc-sa/4.0/

**Disclaimer**: This software is provided "as is", without warranty of any kind, express or implied.

## Credits

Built with:
- Qt6 Framework
- GStreamer Multimedia Framework
- VizionSDK

## Support

For issues or questions:
1. Check the Troubleshooting section above
2. Verify VizionStreamer backend is running correctly
3. Test GStreamer pipelines manually with `gst-launch-1.0`