The Neural Control of Vision
O. Illusions

It appears that size does have an influence on the perceived direction of motion. In rows 1 and 3 motion is perceived mostly horizontally (zig/zag) and in rows 2 and 4 vertically (see/saw).

Now let us put together all three variables, color, shape and size in another display.