openai.com/index/separating-signal-from-noise-coding-evaluations

evals are just a way to justify grades

but seriously though

haha

idk man i just want pizza

both?

or to improve?

to torture us?

what’s the point of evals anyway

ugh fine.

back to evals...

yes 😤

off-topic: anyone else hate mondays

now i’m more confused

great.

yep.

so it’s all about context

ohhh

then 20 tests is just noise

if the one-liner is obvious

no like...

wait what

overkill much

like 20 tests for a one-liner

sometimes too much signal is noise

but also...

like a symphony

it was beautiful

i’ve seen one of those

with comments and tests and everything

like a masterpiece