All of the small bars within each pattern are actually the same color (i.e., all of the blue segments are the same color blue, all of the red segments are the same color red, and all of the small grey bars are the same color grey). The lightness differences you are experiencing are massive illusions. A theory of what's underlying the lightness illusions in this figure is described in a paper to appear in as special issue of Perception on "Contextual effects on color appearance."