These are the instructions that do not to work anymore: https://www.waveshare.com/wiki/HDMI_to_CSI_Adapter

Luckily it seems that the Waveshare adapter is pretty much the same hardware as the Geekworm C790. On their Wiki they describe the necessary steps to to get the adaptor running and how captured frames can be written to disk. This is all done using the v4l2-ctl (Video for Linux) command line tool.

https://wiki.geekworm.com/CSI_Manual_on_Pi_5

These are the steps that work for me.

# 01. Connect HDMI device, eg. camera

# 02. List all available v4l2 devices
# (important is the mediaX device under rp1-cfe, eg. media3)
 v4l2-ctl --list-devices

# 03. Show media device information
# important is the device that is listed under the entity named tc358743 
# eg. /dev/v4l-subdev2
media-ctl -d /dev/media3 -p

# 04. Get, extract and set edid files
wget https://files.waveshare.com/upload/8/89/TC358743-Driver.zip
unzip TC358743-Driver.zip
cd ~/TC358743-Driver
v4l2-ctl -d /dev/v4l-subdev2 --set-edid=file=./edid/1080p30edid

# 05. Query timings
v4l2-ctl -d /dev/v4l-subdev2 --query-dv-timings

# 06. Set timings
v4l2-ctl -d /dev/v4l-subdev2 --set-dv-bt-timings query

# 07. Initialize media device
media-ctl -d /dev/media3 -r

# 08. Connect CSI2's pad4 to rp1-cfe-csi2_ch0's pad0
media-ctl -d /dev/media3 -l ''\''csi2'\'':4 -> '\''rp1-cfe-csi2_ch0'\'':0 [1]'

# 09. Configure the media node
media-ctl -d /dev/media3 -V ''\''csi2'\'':0 [fmt:RGB888_1X24/1920x1080 field:none colorspace:srgb]'
media-ctl -d /dev/media3 -V ''\''csi2'\'':4 [fmt:RGB888_1X24/1920x1080 field:none colorspace:srgb]'

# 10. Set output format
v4l2-ctl -v width=1920,height=1080,pixelformat=RGB3

# 11. Capture test frames
v4l2-ctl --verbose -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat='RGB3' --stream-mmap=4 --stream-skip=3 --stream-count=2 --stream-to=hdmiin.yuv --stream-poll

# 12. View stream with ffplay
v4l2-ctl --verbose -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat='RGB3' --stream-mmap=4 --stream-skip=3 --stream-poll --stream-to - | ffplay -f rawvideo -pixel_format rgb24 -video_size 1920x1080 -i -

#vm1

This is what HfS Ernst Busch announced on their website (translation from german original):

Thanks to funding from the Aventis Foundation, the “Laboratory for Digitality” is now developing a video mixer for theater and media art. This mixer will enable a common scenario: mixing up to two video cameras with pre-produced videos and projecting them onto up to two video projectors. The VM-1 will also serve as an introduction to programming, allowing users to create their own video effects.

Designed to be cost-effective, user-friendly, and open-source, the VM-1 aims to provide an alternative to proprietary video systems, thereby enhancing user independence. Additionally, HfS Ernst Busch seeks to use this project to showcase the creative potential of video and film as mediums. Techniques of image composition will be taught and explored in an accessible manner using the VM-1. Ultimately, this project aims not only to drive technological innovation but also to open new avenues for creative theater-making among students.

Background

In 2023 Julian and I started this project by creating a proof-of-concept of an affordable and open-source video mixing device in 2023.

The POC was developed using Python and GStreamer on the Raspberry Pi 4 platform. It showed that the hardware is capable of decoding and displaying multiple 1080p30 (video and camera) streams at once. Although we were convinced that our idea is doable using the RPi4, we would have had to compromise on some features due to the limited performance of the platform.

As the powerful new Raspberry Pi 5 was released by the end of 2023, we felt like we are on the right track. So we started the development of a prototype using C++, Dear ImGui and OpenGL ES and also developed our own PCB for the hardware controller. This layed the foundation for the current project iteration.

#vm1