I think the PR is sound, CI currently fails presumably because of the Azure debacle

We'll need #4548 merged before this can pass

As #4548 is merged, is this ok to merge now?

This patch appears to be reverting 2606727 from #3006, which means at least three things:

• If this revert is desirable, there may be a bunch of Generic and NFData instances that can also be cleaned up.
• Per that commit message, a consequence of this is that a logPerf measurement is no longer accurate, and we should figure out what to do about that.
• Also per that commit message, something about ‘actually populate the cache of f-thread’ may need attention (I don't know what that means).

@kritzcreek I know that commit was 7y ago, but do you have any recollection about this?

Oh my, I had to think for a bit...

‘actually populate the cache of f-thread’

Actually meant off-thread, which is about a worker thread that does the populateVolatileState logic in here in the background. I think at the time when measuring I figured out that leaving the value lazy meant the computation ended up being done on the thread that accepts and responds to requests. Which meant the first requests after a state reload would take a very long time. (That's also why most of the resolution logic is wrapped in STM)

I'm not sure if making things lazy here is going to help with peak memory usage, it might just delay it until you start sending some requests against the updated IdeState. Sorry, it's been a long time so I'm very fuzzy on the details...

fwiw, this change brings down what normally is 30 GBs of peak memory usage down to 10 GBs alongside bringing down memory pressure/time-to-diagnostics quite a bit on our codebase. We also use this with a custom IDE command that filters which externs files get loaded.

One side effect is that diagnostics are not immediately available until the first edit, but I'm not entirely sure if that's coming from this PR or the custom IDE command.

This LGTM aside from removing the redundant Generic/NFData instances.

Please don't take my comment as a blocker to merging. I haven't written Haskell in over 5(?) years, so I'm hardly an authority on the matter :D Just wanted to dump what context I could remember in case it was helpful.

One side effect is that diagnostics are not immediately available until the first edit, but I'm not entirely sure if that's coming from this PR or the custom IDE command.

I assume by diagnostics you mean type info etc. It would certainly seem worth checking this with a clean build because that sounds like a blocker to me (and presumably not at all necessary or deep)

This looks very easy to revert. If it was merged, it'd get released as a beta, right?
That could be a good way to gather feedback without having to dive extremely deep.

• If this revert is desirable, there may be a bunch of Generic and NFData instances that can also be cleaned up.

Good point. I've removed them.

• Per that commit message, a consequence of this is that a logPerf measurement is no longer accurate, and we should figure out what to do about that.

Also removed. I guess it would also be possible to only call force when logLevel == DEBUG

This looks ready to merge to me; any final objections?