Clocks are based on sundials. The little hand roughly follows where a shadow would be. The rest is just what people agreed on made the most sense.