Here is the result, the monitor works as expected, everything is running via the USB hub. The response times are not great, in fact the monitor would be unusable for actual work because the mouse and refresh rate lag quite badly, however for my purpose of using it as a picture frame, it was perfect.
So lets see how I got to this point. The first thing I did was make sure that I had a small SD card, in this case a microSD and an adapter. The boot up files do not need much space, so a 1Gb SD card is a complete overkill, but it's the smallest I could find. I then dug out an 8Gb USB key, this is where the operating system and the rest of my files would go. It's a lot quicker using USB, especially for things like building the kernel, which is required to get the DisplayLink USB monitor running.
With all that in my hands, I proceeded to download Raspbian. Once that finished, I followed this guide on how to transfer the image to the USB key and how to set up the SD card and then how to expand the file system on the USB key to fill the available space.
The next step was to compile the kernel. I followed instructions here. I opted for compiling directly on the Pi itself, it took overnight to complete the compilation.
Before I did the compilation, I went through the menuconfig and removed some things that I thought were not necessary, like all of the sound support, etc. In addition to this, and this is the most important step here, I enabled the option for Displaylink USB Framebuffer support. This option is found in:
Device Drivers -> Graphics Support -> Support for Frame buffer devices.
I left support for HDMI in place, but strictly speaking this is something that I could have disabled too.
Now I put the kernel on to compile and went to sleep. By morning this was completed, I got the new kernel and modules installed next.
So now my new kernel was working and I confirmed that I had a new framebuffer device at /dev/fb1. Now it was time to configure Xorg to use the new framebuffer. I found this page useful for this purpose.
I created a file named 60-plugable.conf in the /usr/share/X11/xorg.conf.d directory with the following content:
After this I cleaned up the file system a little by deleting the compilation directory and the firmware git checkout. This cleared out around 4.9Gb of space for me, which is significant considering there is only 8Gb of total storage for this setup.
After a restart, the DisplayLink monitor worked with Xorg, perfect! I've noticed that immediately after Xorg starts there are some artefacts on the monitor like in the photo below. These quickly disappear and the usual desktop comes to life.
So there you go, the only steps are to rebuild the kernel with DisplayLink support and to configure Xorg to use the newly created framebuffer.