The differential rotation of compact magnetic elements during activity cycles 20 and 21 (1966 - 1986) is studied by using solar synoptic charts. For each hemisphere the compact magnetic elements with the polarity of the circumpolar magnetic field have larger rotation rates than the elements with the opposite polarity. This difference in rotation rates is present during the whole cycle except during the polarity reversal of the circumpolar field.