Since is has become obvious after speaking to 99% of the sig makers on this site that people rate without having any knowledge of what was done, here's the typical criteria:
Blending: How well the render/stock fits in with the background and effects. Good blending will make a sig look more like a scene out of a movie or something. Usually done with smudging, blurring, placing effects over the render, ect.
Flow/Focal: Not all sigs will have flow, but all should have a focal. Flow shows the direction the sig will make the viewer's eyes gradually move in. The focal would be how well someone uses all elements to establish which is the center point of attention of the sig. Usually done with blurring/sharpening, contrast/color, smudging, and utilizing c4d's and brushes to create flow.
Lighting: Try to make the light and shadows be correct so that it appears that the scene you have created is one out of some sort of production. Example: Sun in the top right and shadows on the bottom left areas of the sig and renders. Any part that is not receiving the light should be darker. Usually done with soft brushing black and white, lighting effect filters, and burn/dodge tool
Placement: This usually is just judged based on the rule of thirds which most people follow to keep from looking basic and uninteresting in sigs. http://en.wikipedia.org/wiki/Rule_of_thirds for reference
Text: How well it goes with the flow, while not taking away from the focal is usually what is taken into account when considering whether or not the text is good. Placement, font, size, and color are all important and should match the style and choices of your sig.
Overall: Giving a rating based on effort compounded with factors from all of these criteria.
If I forgot one, let me know.
Also, no one has to follow all of these, but they have been used to describe what a proper sig should have. In some cases, going against one of these will create a more powerful effect, so don't be afraid to stray from it.
Never rate based on what the render or stock used was. A better render doesn't mean a better signature rating.
