I heard you are not having fun enough with d today.

Do you know you can do things like this with dlang now? After some fiddling with it, my last commits made this possible.

how it looks like: https://www.youtube.com/watch?v=ACC_-TDAtqc
source code: https://github.com/aferust/oclcv/tree/main/examples/centroidtracking
DCV: https://github.com/libmir/dcv

Sorry for the potato-quality video. My art director is on vacation.

I am cheating a little with OpenCL since things are not fast enough at the moment.

Hope you like it.

Enjoy!

On Wednesday, 15 February 2023 at 17:32:33 UTC, Ferhat Kurtulmuş wrote:

Nice!

That's pretty cool, thanks for sharing!

On Wednesday, 15 February 2023 at 17:32:33 UTC, Ferhat Kurtulmuş wrote:

Hello everyone,

I was looking for ways to run pre-trained DCNN models (inference) using D. I then ran across onnxruntime, which has a c API. Luckily, it has a bindbc binding readily available. Nowadays, to run inference routines of CNN models, we only need some basic image processing to satisfy the input shape requirements of those models. We have mir.ndslice and dcv, and then we should be able to run, for instance, tinyYOLOv3 with video streams. I believe that such applications will attract more people's attention to d.

Here is how it looks like and the source code:

https://www.youtube.com/watch?v=m3ex9lDELfQ
https://github.com/aferust/dcv-tinyyolov3

On Wednesday, 15 February 2023 at 17:32:33 UTC, Ferhat Kurtulmuş wrote:

Hello everyone,

I was looking for ways to run pre-trained DCNN models (inference) using D. I then ran across onnxruntime, which has a c API. Luckily, it has a bindbc binding readily available. Nowadays, to run inference routines of CNN models, we only need some basic image processing to satisfy the input shape requirements of those models. We have mir.ndslice and dcv, and then we should be able to run, for instance, tinyYOLOv3 with video streams. I believe that such applications will attract more people's attention to d.

Here is how it looks like and the source code:

https://www.youtube.com/watch?v=m3ex9lDELfQ
https://github.com/aferust/dcv-tinyyolov3

On Tuesday, 28 February 2023 at 12:08:14 UTC, Ferhat Kurtulmuş wrote:

Great job. Could we have any comparison in the performance/memory usage versus original solution in Python?

On Tuesday, 28 February 2023 at 12:29:05 UTC, Sergey wrote:

I have not conducted any comparisons yet. There are a lot of factors affecting performance. My old laptop lacks good cuda support, so I disabled the CUDA acceleration. I cannot give you a strongly backed test result, but I can say that preprocessing is not so costly in my example. The FPS drop is primarily due to onnxruntime itself. The preprocessing step only takes 2 or 3 msecs. The newer versions of onnxruntime have various backend options for acceleration, such as CUDA, tensorrt, directML (uses directX).

On Wednesday, 15 February 2023 at 17:32:33 UTC, Ferhat Kurtulmuş wrote:

I am facing strange ffmpeg errors on Windows. It fails to read a frame. Do you have any ideas how to solve it?

Here is the error:

ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --prefix=/mingw64 --target-os=mingw32 --arch=x86_64 --cc=gcc --cxx=g++ --disable-debug --disable-stripping --disable-doc --enable-dxva2 --enable-d3d11va --enable-fontconfig --enable-frei0r --enable-gmp --enable-gnutls --enable-gpl --enable-iconv --enable-libaom --enable-libass --enable-libbluray --enable-libcaca --enable-libdav1d --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libplacebo --enable-librsvg --enable-librtmp --enable-libssh --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libx264 --enable-libx265 --enable-libxvid --enable-libvpx --enable-libwebp --enable-libxml2 --enable-libzimg --enable-openal --enable-pic --enable-postproc --enable-runtime-cpudetect --enable-swresample --enable-version3 --enable-vulkan --enable-zlib --enable-librav1e --enable-libsvtav1 --enable-libmfx --enable-amf --enable-nvenc --logfile=config.log --enable-shared
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video_640x480.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.27.100
  Duration: 00:01:00.01, start: 0.000000, bitrate: 833 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 640x480 [SAR 4:3 DAR 16:9], 822 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 2 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
Using auto hwaccel type dxva2 with new default device.
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
configure renderer for GL-31
Output #0, image2pipe, to 'pipe:':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.3.100
  Stream #0:0(und): Video: rawvideo (RGB[24] / 0x18424752), rgb24(pc, gbr/bt709/bt709, progressive), 640x480 [SAR 4:3 DAR 16:9], q=2-31, 147456 kb/s, 20 fps, 20 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.3.100 rawvideo
av_interleaved_write_frame(): Broken pipe time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A
[out#0/image2pipe @ 000001acc2c41d80] Error muxing a packet
[out#0/image2pipe @ 000001acc2c41d80] Error writing trailer: Broken pipe
frame=    1 fps=0.5 q=-0.0 Lsize=       0kB time=00:00:00.50 bitrate=   0.0kbits/s dup=0 drop=3 speed=0.231x
video:900kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Error Program exited with code -1073741819
Conversion failed!

Also, it does not work on MacOS as well due to glfw-d not supporting it. I will try to replace the gui part with Raylib... Do you have any better solutions?

On Wednesday, 15 February 2023 at 17:32:33 UTC, Ferhat Kurtulmuş wrote:

Cool stuff! I once contributed Randomized Hough transform to DCV.

On Friday, 23 June 2023 at 02:16:49 UTC, Ki Rill wrote:

I am typing those via my phone, and cannot test any code soon.

İf you are using pipe processes with ffmpeg executable, the terminal output can be a mix of outputs of ffmpeg and your main process. This makes difficult to search for errors. I think since your main process is interrupted due to an error (maybe an opencl error), ffmpeg stops too. Please note that the centroid tracking example does not use ffmpeg-d (you can modify the code to use it though).

For Mac you should use bindbc-glfw by deleting the subconf:

"subConfigurations": {
	"dcv:plot": "use-glfw-d"

You can also try using equivalent DCV functions instead of opencl to eliminate opencl errors.