I haven’t done any research into GPUs on Linux in awhile. I wasn’t aware nvidia finally released an open source driver. Looks like it’s recommended for the 3080. Seems like this was a fairly recent development so you may have more luck?

I switched over to GrapheneOS a couple months ago and couldn’t be happier. If you have a Pixel the switch is really easy. The biggest obstacle was exporting my contacts from my google account.