The running kernel is not removed. Most (if not all?) distros will keep multiple previous kernels for fallback purposes. Some relatively recent projects will wait for system calls to complete and redirect new ones to the updated equivalent (see Live Kernel Patching).
The reason some things stop working is because the kernel modules directory is replaced. The running (old) kernel tries to access a module for a newer version and says nope.

Arch Linux has the package kernel-modules-hook which backs up the old modules and points the old kernel to them while it’s running. I’m not sure why it’s not installed by default.