softmax

October 12, 2023

chatgpt conversation #

this conversation arised from another conversation on RL (reinforcement learning)

remaining questions #

usefulness of MLE #

MLE is like asking what’s the most likely universe we are in, no it’s not it’s like what’s the most likely universe considering the probability of event only, and it only.

so if we want to pick which universe we want to go to, MLE gives the answer.

but if we ask which universe we might be in, MLE might not be a good answer.

Maximum entropy principle and Bias-variance tradeoff #

they seem to be related more than explored in the chat, i wanna go over it again

I haven’t digested the last part of converstaion #

further exploration #

Entropy definition #

entropy and statistics #

I thought entropy was from information theory. Seeing entropy is playing a rather big role in statistics, information theory might be a good topic to learn