LLM output is also not an unbiased PRNG; it will give you a result based on the probabilities of the next word according to the model, which will likely be heavily biased towards more common values.
Heavily biased towards more common values of a particular subculture of a particular culture, in fact.
Want to have some fun? Ask a culturally weighted question (say about gun ownership) in English. Keep the question as neutral as possible. Watch the pro-gun stance take hold, nine times out of ten. Now ask the same question in a new session translated to idiomatic German. Again make sure the question is as neutral as possible. Compare the answers.
LLM output is also not an unbiased PRNG; it will give you a result based on the probabilities of the next word according to the model, which will likely be heavily biased towards more common values.
Heavily biased towards more common values of a particular subculture of a particular culture, in fact.
Want to have some fun? Ask a culturally weighted question (say about gun ownership) in English. Keep the question as neutral as possible. Watch the pro-gun stance take hold, nine times out of ten. Now ask the same question in a new session translated to idiomatic German. Again make sure the question is as neutral as possible. Compare the answers.