I remember when it was necessary to to modify CFLAGS and HOSTCFLAGS in the make file to optimise the kernel for Pentium Processors but these kernels are now standard.
I'm never sure if the modules are hazards, and the logic, don't load it if you don't need it, seems compelling. The monolithic kernel locks you into a moment of time, and my luck always works as follows. The driver that I need gets updated 30 seconds after I build the kernel.