Here is my understanding of samples: Samples are used for determining the "accuracy" of the color. However there is a law of diminishing returns in effect for samples. Once your sample count reaches a certain level it becomes less likely you will see a change by increasing it higher. That level increases as scene complexity increases.
That is roughly true, though I think the term "color" here, while accurate, is perhaps misleading. Think of it like this - when an artist paints something, they often start with the big features first. The result looks kind of like what you're after, you can tell what it is. Then they keep adjusting, adding more detail, especially more small detail. Over time, the painting resolves to being "correct".
If you like the science, what's basically happening here is that a bunch of random rays are being cast throughout the scene from the camera, and along their journey they hit various things and get absorbed (matte materials) or reflected (depending on the roughness value) or transmitted (glass/wax/subsurface materials) etc. Once all the rays have been calculated, you get an idea of what the final image will be like. But you really need a loooot of
random rays flying around to get a final image. Active areas of research in general focus on ways to make rays that are smarter than random.
Since we need so many rays to hit everything, statistically, we do not want to ask the computer to send out 1000000000 rays at once - you would be sitting there forever, and for many scenes you don't need that many. So instead, it sends out say 10000 at a time and waits to see how good the result is, as defined by your human eyes. That's one Sample. Then it sends another 10000 (made up) rays randomly and waits to see what those hit. 2nd Sample. For some simple scenes, maybe 1,000,000 random rays is enough to make you say "that looks good enough". For other scenes, it may take 45,000,000. These are made up numbers but just illustrates what a sample is.
So long story short, the more samples you have, the more chance you have given these random rays to propagate the scene and hit whatever they will hit and eventually land on part of the model that represents a pixel. You're correct in that, at a certain point, there is no human-perceptible difference. For example, an image made from a sample count of 1000 will, under most circumstances encountered by scenes generated from Alibre, be no different than a sample count of 9,000,000, for a given image output size.
So the trick is to find the lowest samples that are "good enough" to give you want. Generally speaking, this number is under 1000 samples almost always, and is usually in the 100-400 range for most things, but really is model and material dependent, and depends on how eagle eyed you are. If your model is a novel series of lenses that shoots light through water to focus on a square of wax, you may be sitting at your computer for a long while to get a perfect result - it's going to take a long time for enough of the
random rays to pass through that lens system and hit the target wax. Most random rays wont even hit the first lens, for example. If your model is metal and paint, it's a lot faster to get accurate results with lower samples.