Ablate KAN and Fourier KAN vs. normal Linear Layers in LLMs.
Based on my package hlb-gpt-cli, which in turn is based on Fern's hlb-gpt.
Currently uses FourierKANs from FourierKAN, because the code is simpler to understand and therefore use than the original implementation of pykan. I do plan on testing KAN with splines at some point, though.
Results are coming soon.