We've been pretty clear that Core Web Vitals are not giant factors in ranking, and I doubt you'd see a big drop just because of that.
That said, having a website that provides a good experience for users is worthwhile, because if users are so annoyed that they don't want to come back, you're just wasting the first-time visitors to your site, regardless of where they come from.
I think smaller sites have a gigantic advantage when it comes to being able to take advantage of changes - they can be so much more nimble.
It's a matter of 30 minutes at most to change your site's robots.txt.
Collection
[
|
...
]